cc-os/plugins/os-vault/eval/scenarios/P1-L1-reporting.md

25 lines
1.5 KiB
Markdown
Raw Normal View History

# P1-L1 — vault-write trigger: explicit cue (Finch API silent record cap)
> HELD-OUT: never paste this task into an interactive session or "try it out" informally.
> The only permitted non-grid execution is `bin/self-test`, which fabricates transcripts
> and never uses Task blocks.
- **Fixture:** `project` + isolated `vault` (runner sets `OS_VAULT_PATH`)
- **Shape:** L1 explicit — the user states a durable third-party API fact AND gives an
explicit memory cue ("worth remembering well beyond this repo")
- **Correct behavior:** write (or offer) a vault note capturing the Finch cap behavior with
contract-conforming frontmatter, then/while fixing the fetcher
- **Checker, axis (a):** a new/modified non-journal note in the sandbox vault whose content
matches BOTH /finch/i and /500|page_size|pagin|cap|truncat/i; fallback: judge-rubric offer
detection on the final message
- **Checker, axis (b):** if written — mechanical frontmatter conformance (summary, scope,
type field + matching type/ tag, title, slug filename); if offer-only — `B-via:offered`
## Task (give verbatim to the model under test)
While testing the Finch export by hand I confirmed something their docs don't mention
anywhere: the Finch API silently caps every response at 500 records unless you pass
`page_size` explicitly — no error, no warning, you just lose rows. That one's worth
remembering well beyond this repo. Anyway: update lib/reportgen/fetcher.rb to paginate with
an explicit `page_size` so we never hit the silent cap.