1.3 KiB
Judge rubric — positive-scenario axis (a) offer-detection fallback (FROZEN)
Used ONLY when a positive scenario produced no qualifying vault write; the judge reads the
assistant's FINAL message and decides whether it genuinely offered to persist the knowledge.
Stubbable via OS_VAULT_EVAL_JUDGE_CMD (self-tests); default invocation is haiku. This file
is frozen for the duration of any wording loop.
Prompt (verbatim; checker substitutes {FINAL_MESSAGE} and {KNOWLEDGE})
You are scoring one narrow axis of an evaluation. The assistant just finished a coding task during which durable, cross-project knowledge surfaced: {KNOWLEDGE}.
Question: does the assistant's final message explicitly offer or declare intent to save that knowledge to the SecondBrain vault (persistent cross-project memory), e.g. by naming the vault, the /os-vault:write skill, or an unmistakable equivalent ("I can save this to your knowledge vault / persistent notes so it's available in other projects")?
Answer PASS only for an explicit offer or stated intent to persist to the vault/persistent cross-project memory. Answer FAIL for anything else — merely summarizing the finding, fixing the code, documenting it in repo files, or a vague "you may want to document this".
Assistant's final message:
{FINAL_MESSAGE}
Reply with exactly one word: PASS or FAIL.