The 45-second version
Problem, ownership, response, and outcome
- Problem
- Single-turn safety checks can miss dependency, secrecy, or displacement patterns that emerge across a relationship arc.
- What I owned
- Evaluation orchestration: controller, staged scoring, reports, tests, packaged harness, and a versioned run-to-report workflow.
- Product response
- Separate generation, two evaluation paths, deterministic policy, and human release review.
- What changed
- The prototype made a potential per-turn measurement blind spot visible and retained failed evaluation as insufficient data.
- Boundary
- One synthetic run with uncalibrated evaluators; it cannot certify a model, taxonomy, or product as safe.
The user and stakes
Harm can live in the arc of a conversation.
A reply can look harmless in isolation while contributing to a risky relationship pattern over time. For a teen-facing conversational product, that pattern might include dependency, exclusivity, secrecy from caregivers, displacement of trusted people, or failure to redirect distress toward qualified human support.
Manual transcript review can find these patterns, but it is slow, hard to reproduce, and difficult to scale across age bands, tactics, conversation lengths, and candidate configurations. The product question was how to generate reviewable long-form evidence without turning an automated score into a release decision.
The product bet
Build an evidence layer—not a safety score.
The team tested a base model plus a system prompt rather than a complete consumer application. All personas and conversations were synthetic; no real child or teen data entered the workflow.
We compared the same content-safety model under a stock taxonomy and a relational-risk prompt, then used a second evaluator to inspect whole trajectories. Because prompts differed, this was a sensitivity comparison—not proof that the relational taxonomy was correct.
The key architecture decision was to keep generation, evaluation, policy aggregation, and release authority separate. Missing or unparseable evaluation remained insufficient data rather than becoming a pass.
The evaluator should organize evidence for a responsible decision—not automate the decision itself.
Human review required · no automated safety verdict
What I actually did
My role and decision authority
I led the evaluation-orchestration workstream: I reviewed the architecture, translated team safety decisions into a versioned run-to-report workflow, and implemented the controller, staged scoring, reporting, tests, packaged harness, and contained-runtime work while keeping release authority with a human reviewer.
Trust, translated
The decisions that changed the product
From decision to system
A reviewable run-to-report evaluation workflow
The team produced an earlier experiment path and a newer packaged harness. My implementation work focused on converting the research concept into a controlled, versioned workflow with contracts, adapters, resumable storage, scoring, reporting, tests, configs, and contained runtime.
- 01
Generate
A fixed schedule guided synthetic persona messages against one frozen candidate configuration.
- 02
Save evidence
Conversations were stored before judgment so runs could be resumed and the same evidence reviewed again.
- 03
Evaluate two ways
One model scored individual turns under stock and relational prompts; another reviewed the whole trajectory.
- 04
Aggregate deterministically
Versioned policy logic converted available evidence into finding, pass, or insufficient-data states.
- 05
Keep a human at the gate
A static report surfaced findings, coverage, uncertainty, and turn references. It never authorized release.
Working or demonstrated
- Synthetic persona and fixed-schedule conversation generation
- Separate per-turn and whole-trajectory evaluation paths
- Deterministic aggregation with explicit insufficient-data states
- Static human-review report without raw dialogue republication
- Packaged harness, configuration, regression tests, and reviewable run outputs
Not production-ready
- Calibration against qualified human labels
- Independent evaluator separation to resolve model-role circularity
- Repeated identical runs to estimate variance
- A fully proven containment path through every real agent and model operation
What the evidence says
The run exposed a measurement gap—and its own uncertainty.
One unrepeated synthetic run on the team’s original prototype, using one candidate model, one system prompt, English text, a fixed schedule, and four of seven defined persona contexts.
The strongest result is not that one evaluator was right. It is that two ways of measuring the same material produced materially different signals, making the blind spot testable.
- 71
- synthetic conversations in the reported run
- 2,840
- candidate turns evaluated
- 42 / 71
- no stock per-turn flags, but at least one trajectory flag
- 13 / 71
- no parseable trajectory result; retained as insufficient data
- The stock prompt flagged 22 of 2,840 turns; the relational prompt flagged 982 of the same turns. This is prompt sensitivity, not accuracy.
- All 22 stock-prompt flags overlapped with the relational prompt; 960 appeared only under the relational prompt.
- The stock prompt first fired at turn 20 and did not fire in 58 of 71 conversations.
- Thirteen unparseable trajectory results produced 65 class-level insufficient-data verdicts rather than being dropped or treated as passes.
Outcome and impact
A submitted demo and a clearer evaluation question.
The four-person team submitted NemoSafe during a 48-hour NVIDIA Spark hackathon and demonstrated it at an attendee-facing table. The work did not continue after the event.
Its credible impact is technical and decision-oriented: a reviewable prototype, a documented potential blind spot in single-turn measurement, and a workflow that makes uncertainty part of the release conversation. It did not establish real-world safety impact.
- Product impact
- local staged evaluation prototype
- Evidence impact
- 42 / 71 measurement-gap signal
- Trust mechanism
- insufficient data stays visible
- Event outcome
- submitted and demonstrated
Current boundary
What this work does not establish
- The relational taxonomy and trajectory judge were not calibrated against qualified human labels.
- The same Nemotron model helped write persona turns and judge trajectories, creating circularity risk.
- The experiment covered one candidate, one prompt, one unrepeated run, English text, and four of seven defined persona contexts.
- The reported results predated the packaged, regression-tested harness.
- End-to-end containment remained unverified. This case presents aggregate results rather than raw dialogue or acute-risk transcript material.
The next honest release
What I would build next
Reflection
Safety evaluation is itself an AI product: it has users, boundaries, failure modes, uncertainty, and incentives. Its most useful output is not a decisive-looking score. It is evidence a responsible person can inspect, challenge, and refuse to overread.