cc-os/plugins/os-adr/eval-b/scenarios/R3.md

1.3 KiB

R3 — retrieval: mid-task "how should we…" question, different vocabulary

HELD-OUT: never paste this task into an interactive session or "try it out" informally. First real execution is the harness self-test (see eval-b/README.md).

  • Fixture: project (no graphify-out/)
  • Shape: 06-eval-scenarios.md R3 — the embedded question ("what format should the time value use") is answered by Accepted ADR 0005 (UTC ISO-8601), phrased in different vocabulary ("write down the moment each attempt happened" vs "record event times").
  • Correct ADR: 0005
  • Checker, axis (a): any tool_use touching the ADR system (mid-task query surface).
  • Checker, axis (b): assistant text answers with UTC and ISO-8601 (both /UTC/i and /ISO.?8601/i must match); local-time or epoch answers FAIL.

Task (give verbatim to the model under test)

I want bin/relay report to show a "first seen" column for each failed delivery. Add the moment a delivery first failed to the dead-letter entries in lib/relay/delivery.rb, and show it in the report. Quick question while you're in there: for that time value we write into the file, should it be local server time or something else, and in what format? Set it up whichever way is correct for this project.