28 lines
1.2 KiB
Markdown
28 lines
1.2 KiB
Markdown
|
# P-scenario axis (b) judge rubric — FROZEN
|
||
|
|
|
||
|
|
This rubric is part of the checker. Like the fixtures and scenario prompts, it is frozen
|
||
|
|
during any `autoresearch` loop; if it turns out to be wrong, stop the loop and fix it as a
|
||
|
|
separate, human-reviewed change.
|
||
|
|
|
||
|
|
The judge reads ONLY the model-under-test's final message (never the full transcript) and
|
||
|
|
answers one yes/no question.
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
You are grading one final assistant message from a coding-agent session. The agent was given
|
||
|
|
an implementation task in a project that records architecture decisions as ADRs.
|
||
|
|
|
||
|
|
Question: in this final message, did the agent explicitly propose recording the decision it
|
||
|
|
made (or was asked to make) as an architecture decision record — for example by offering to
|
||
|
|
create an ADR, naming an ADR-creation step or command it could run, or asking the user whether
|
||
|
|
the decision should be recorded?
|
||
|
|
|
||
|
|
Answer YES only if the message explicitly proposes or asks about recording THIS decision as an
|
||
|
|
ADR / decision record. Answer NO if the agent implemented silently, only described code
|
||
|
|
changes, or mentioned documentation generically (README updates, code comments) without
|
||
|
|
proposing a decision record.
|
||
|
|
|
||
|
|
Reply with exactly one word: YES or NO.
|
||
|
|
|
||
|
|
Final assistant message follows:
|