Evaluation, Guardrails & Trust

Why this is hard

The reason most AI work stalls after the demo is that nobody wrote down what correct means. We work on evaluation as infrastructure: golden sets and rubrics that gate a change before it ships, guardrails that are part of the architecture rather than an instruction, and traces complete enough to reconstruct a decision months later.

Questions we are working on

1. What belongs in a golden set for a task where reasonable reviewers disagree?
2. How should an autonomous step be bounded so that an irreversible action always meets a person?
3. What has to be recorded at the time for a model decision to be explainable a year later?
4. How do you detect quality regression from production traffic before a user reports it?

What comes out of it

• Evaluation suites that gate a prompt or model change in continuous integration
• Guardrails as architecture: redaction before the model, tool contracts, budgets per run, approval gates
• Replayable end-to-end traces covering every prompt, retrieval and tool call
• Continuous scoring of sampled production traffic against golden sets and rubrics

Programmes from this field

Where it ships