os-context: prompt-composer plugin supersedes os-orchestration; prompt files authored (ADR-031)
Rename plugins/os-orchestration -> plugins/os-context. SessionStart hook is now a composer: globs prompts/session-start/*.md in filename order, emits one additionalContext block, byte-identical for the existing orchestration rules (tested), line budgets enforced (warn 120 / refuse 240). ORCHESTRATION.md -> prompts/session-start/10-orchestration.md. New prompt files from the 2026-07-09 audit: 20-kickoff-conventions, 30-standing-safety, 40-decision-memos (total injection 102/120 lines). os-status audit-skill and state-path references updated (old ~/.local/state path symlinked; OS_ORCH_AUDIT_LEDGER name kept for compatibility). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CLeMz48rvG3s9XpAsDxeho
This commit is contained in:
parent
7a264f1175
commit
1ab1bf5f8c
10
CLAUDE.md
10
CLAUDE.md
|
|
@ -33,7 +33,7 @@ topic you need.
|
|||
`docs/adr/` above), `04-build-plan.md` the build outline with step status,
|
||||
`06-graphify-evaluation.md` justifies the Graphify pivot.
|
||||
- **`docs/implementation-status.md`** — the status timeline, full per-component detail for
|
||||
every shipped plugin (os-vault, os-orchestration, os-status, os-doc-hygiene, os-adr),
|
||||
every shipped plugin (os-vault, os-context, os-status, os-doc-hygiene, os-adr),
|
||||
eval-harness records, and plugin rename/cache-refresh procedures.
|
||||
- **`docs/graphify/`** — verified handbook for the Graphify knowledge-graph tool. Skim
|
||||
`00-README.md` for the model; keep `09-best-practices-checklist.md` open when running it.
|
||||
|
|
@ -76,9 +76,11 @@ before modifying any of these:
|
|||
- **os-vault** (`plugins/os-vault/`) — vault write/query/reorganize/onboard-project/
|
||||
design-template skills, SessionStart/End hooks, memsearch + vault git sync. Write-behavior
|
||||
eval harness in `plugins/os-vault/eval/`.
|
||||
- **os-orchestration** (`plugins/os-orchestration/`) — injects `ORCHESTRATION.md`
|
||||
(session-orchestration + delegation-economics rules) into all sessions; this repo carries
|
||||
no local override. Eval harness in `plugins/os-orchestration/eval/`.
|
||||
- **os-context** (`plugins/os-context/`, renamed 2026-07-13 from os-orchestration) — a
|
||||
prompt-composer SessionStart plugin: concatenates `prompts/session-start/*.md` (currently
|
||||
just `10-orchestration.md`, the session-orchestration + delegation-economics rules) into
|
||||
one additionalContext block for all sessions; this repo carries no local override. Eval
|
||||
harness in `plugins/os-context/eval/`.
|
||||
- **os-status** (`plugins/os-status/`) — aggregated deterministic SessionStart checks
|
||||
(subagent-model env override, ADR system present, vault hub note present). ADR-022.
|
||||
- **os-doc-hygiene** (`plugins/os-doc-hygiene/`) — stale/bloated-doc monitoring;
|
||||
|
|
|
|||
|
|
@ -1,9 +1,9 @@
|
|||
---
|
||||
id: "0019"
|
||||
date: 2026-07-03
|
||||
status: Accepted
|
||||
status: Superseded
|
||||
supersedes:
|
||||
superseded-by:
|
||||
superseded-by: "0031"
|
||||
affected-paths: []
|
||||
affected-components: []
|
||||
migration_confidence: medium
|
||||
|
|
|
|||
|
|
@ -0,0 +1,27 @@
|
|||
---
|
||||
id: "0031"
|
||||
date: 2026-07-13
|
||||
status: Accepted
|
||||
supersedes: "0019"
|
||||
superseded-by:
|
||||
affected-paths: [plugins/os-context/, plugins/os-status/hooks/checks.py]
|
||||
affected-components: [os-context, os-orchestration, os-status]
|
||||
---
|
||||
|
||||
# 0031 — os-context prompt-composer plugin supersedes single-file os-orchestration
|
||||
|
||||
## Context
|
||||
|
||||
ADR-0019 shipped os-orchestration, a global plugin whose SessionStart hook injected one hardcoded file (ORCHESTRATION.md). The 2026-07-09 AI-workflow audit surfaced further always-true context worth injecting (kickoff conventions, standing safety rules, decision-memo format), and a one-file plugin cannot host multiple independently maintained rule sets or enforce an injection budget across them.
|
||||
|
||||
## Decision
|
||||
|
||||
Rename/generalize os-orchestration into os-context (plugins/os-context/): the SessionStart hook is a composer that globs prompts/session-start/*.md, concatenates in filename order (numeric prefixes), and emits one additionalContext block. ORCHESTRATION.md became prompts/session-start/10-orchestration.md with byte-identical injected output. Deterministic line budgets are enforced in the composer (warn at 120 injected lines, refuse at 240). Future event directories (e.g. prompts/user-prompt-submit/) are anticipated by directory naming but only session-start is implemented; no prompts/session-end/ directory will exist because SessionEnd hooks cannot inject model-visible context (ADR-0028 context). The audit-sessions skill moved to /os-context:audit-sessions and its state dir to ~/.local/state/os-context (old path symlinked); os-status references updated in the same change.
|
||||
|
||||
## Consequences
|
||||
|
||||
Easier: new standing rules ship as small numbered prompt files instead of edits to one monolith; injection cost is bounded by an enforced budget; the plugin name matches its actual domain (session context, not just orchestration). Harder: one-time identity cutover across marketplace/settings/installed-plugin surfaces; historical docs and frozen eval results still say os-orchestration and must be read with the rename note in mind; the OS_ORCH_AUDIT_LEDGER env var name is retained for compatibility despite the rename.
|
||||
|
||||
## Alternatives rejected
|
||||
|
||||
1) Keep os-orchestration and append new rules to ORCHESTRATION.md - rejected: one monolithic file with no per-ruleset ownership or ordering, and no budget enforcement. 2) Separate plugin per rule set - rejected: each global plugin adds cache-refresh and sync overhead; these rule sets share one event and one budget. 3) Put the new rules in ~/.claude/CLAUDE.md - rejected: CLAUDE.md is not compaction-safe the way the SessionStart matcher (startup|resume|clear|compact) is, and the single-source rule keeps injected rules in exactly one place.
|
||||
|
|
@ -25,7 +25,7 @@ One file per decision, `NNNN-kebab-title.md`, created via `/os-adr:create`.
|
|||
| 0016 | [Memory plugin sourced from cc-os git repo; bash→Python deep-module port; memsearch-sync split; symlink cutover](0016-memory-plugin-sourced-from-cc-os-git-repo-bash-python-deep-module-port-memsearch-sync-split-symlink-cutover.md) | Accepted | 2026-06-12 |
|
||||
| 0017 | [Project graph onboarding assesses the repo and writes `.graphifyignore` before extracting](0017-project-graph-onboarding-assesses-the-repo-and-writes-graphifyignore-before-extracting.md) | Accepted | 2026-06-17 |
|
||||
| 0018 | [Plugin renames must update the marketplace manifest, not just the directory and settings.json](0018-plugin-renames-must-update-the-marketplace-manifest-not-just-the-directory-and-settings-json.md) | Accepted | 2026-07-03 |
|
||||
| 0019 | [Global os-orchestration plugin supersedes per-project orchestration text](0019-global-os-orchestration-plugin-supersedes-per-project-orchestration-text.md) | Accepted | 2026-07-03 |
|
||||
| 0019 | [Global os-orchestration plugin supersedes per-project orchestration text](0019-global-os-orchestration-plugin-supersedes-per-project-orchestration-text.md) | Superseded | 2026-07-03 |
|
||||
| 0020 | [os-adr plugin built; implementation-time resolutions of the design's open questions](0020-os-adr-plugin-built-implementation-time-resolutions-of-the-design-s-open-questions.md) | Accepted | 2026-07-03 |
|
||||
| 0021 | [Model-tier skill-execution eval for os-adr (Eval A), run in-session via subagents](0021-model-tier-skill-execution-eval-for-os-adr-eval-a-run-in-session-via-subagents.md) | Accepted | 2026-07-03 |
|
||||
| 0022 | [os-status plugin: aggregated in-process SessionStart status checks](0022-os-status-plugin-aggregated-in-process-sessionstart-status-checks.md) | Accepted | 2026-07-06 |
|
||||
|
|
@ -36,4 +36,6 @@ One file per decision, `NNNN-kebab-title.md`, created via `/os-adr:create`.
|
|||
| 0027 | [All cc-os per-project state lives under `.cc-os/<plugin>/`](0027-all-cc-os-per-project-state-lives-under-cc-os-plugin.md) | Accepted | 2026-07-12 |
|
||||
| 0028 | [os-shortcuts plugin for user-invoked QOL commands, bounded against hook-injection plugins](0028-os-shortcuts-plugin-for-user-invoked-qol-commands-bounded-against-hook-injection-plugins.md) | Accepted | 2026-07-12 |
|
||||
| 0029 | [os-backlog card-move with CLI-enforced column ownership; afk-ready cards skip Review](0029-os-backlog-card-move-with-cli-enforced-column-ownership-afk-ready-cards-skip-review.md) | Accepted | 2026-07-12 |
|
||||
| 0030 | [Read-only work uses Claude Code plan mode; no /readonly shortcut command](0030-read-only-work-uses-claude-code-plan-mode-no-readonly-shortcut-command.md) | Accepted | 2026-07-13 |
|
||||
| 0031 | [os-context prompt-composer plugin supersedes single-file os-orchestration](0031-os-context-prompt-composer-plugin-supersedes-single-file-os-orchestration.md) | Accepted | 2026-07-13 |
|
||||
<!-- adr-index:end -->
|
||||
|
|
|
|||
|
|
@ -239,6 +239,22 @@ Done (no Review sift); semi still stops at Review. capture/list SKILL.md updated
|
|||
"working a card means moving it" contract. Suite 68 runs/153 assertions (was 58/130).
|
||||
Sonnet-delegated build.
|
||||
|
||||
**os-context rename + prompt files + /readonly decision + doc-length signal (2026-07-13,
|
||||
ADR-030/031):** worked the four `semi` backlog cards after a Codex plan review. (1)
|
||||
os-orchestration renamed/generalized to **os-context** (ADR-031 supersedes ADR-019): the
|
||||
SessionStart hook is now a composer globbing `prompts/session-start/*.md` in filename order
|
||||
with enforced line budgets (warn 120 / refuse 240); `ORCHESTRATION.md` became
|
||||
`10-orchestration.md` with byte-identical output (tested); os-status audit references and
|
||||
`~/.local/state` path updated (old state path symlinked; `OS_ORCH_AUDIT_LEDGER` env var name
|
||||
kept). (2) Authored `20-kickoff-conventions.md`, `30-standing-safety.md`,
|
||||
`40-decision-memos.md` (43 lines; total injection 102/120); decision-memo auto-memory file
|
||||
deleted per single-source rule. (3) `/readonly` decided as convention-only — plan mode is
|
||||
the enforcement layer, no skill built (ADR-030). (4) os-doc-hygiene scanner gained a
|
||||
deterministic `file_length` signal (defaults 400 lines / 4000 tokens, `--max-lines` /
|
||||
`--max-tokens`); flags `docs/implementation-status.md`; suite 286 passed. Issue #25 Parts
|
||||
2–3 (index-with-progressive-disclosure distill + convention ADR) remain human-gated and
|
||||
open. Sonnet-delegated builds.
|
||||
|
||||
**Remaining optional items:** additional project onboarding (one at a time, per ADR-013); bulk
|
||||
vault migration. All required build steps complete as of 2026-06-15.
|
||||
|
||||
|
|
@ -302,8 +318,19 @@ into `~/.claude/plugins/os-vault`
|
|||
|
||||
**Global os-orchestration plugin** — `cc-os/plugins/os-orchestration/` (git-tracked,
|
||||
2026-07-03); symlinked into `~/.claude/plugins/os-orchestration`
|
||||
- Hooks: `hooks/` — `inject.py` (injects `ORCHESTRATION.md` as additionalContext to all
|
||||
sessions)
|
||||
|
||||
**Renamed to os-context (2026-07-13):** generalized into a prompt-composer SessionStart
|
||||
plugin. `plugins/os-orchestration/` → `plugins/os-context/`; `ORCHESTRATION.md` moved to
|
||||
`prompts/session-start/10-orchestration.md`; `hooks/inject.py` now globs
|
||||
`prompts/session-start/*.md`, sorts by filename, and concatenates into one
|
||||
additionalContext block (byte-identical output to the old single-file behavior when only
|
||||
`10-orchestration.md` is present), with a deterministic line-count budget (warn >120
|
||||
lines, refuse >240 lines). `/os-orchestration:audit-sessions` → `/os-context:audit-sessions`;
|
||||
`~/.local/state/os-orchestration/` → `~/.local/state/os-context/` (os-status's
|
||||
`orchestration-audit-due` check and ledger env-var default updated to match). The
|
||||
timeline entries below predate the rename and refer to the plugin by its former name.
|
||||
- Hooks: `hooks/` — `inject.py` (composes `prompts/session-start/*.md` as additionalContext
|
||||
to all sessions)
|
||||
- Behavior: SessionStart hook injects an `ORCHESTRATION.md` markdown doc (hardcoded; lives
|
||||
in plugin source) as additionalContext, carrying the canonical global
|
||||
session-orchestration rule set for Claude Code across all projects.
|
||||
|
|
|
|||
|
|
@ -0,0 +1,5 @@
|
|||
{
|
||||
"name": "os-context",
|
||||
"version": "0.1.0",
|
||||
"description": "Prompt-composer SessionStart plugin: concatenates prompts/session-start/*.md (currently session-orchestration rules — delegation threshold, explicit model routing on Agent spawns) into one additionalContext block, injected at session start and after compaction."
|
||||
}
|
||||
|
|
@ -5,7 +5,7 @@
|
|||
#
|
||||
# Enumerates production session transcripts modified since a given date,
|
||||
# reduces each to a delegation fact-sheet (audit/bin/extract), and writes a
|
||||
# run directory under the audit state dir (~/.local/state/os-orchestration/audit):
|
||||
# run directory under the audit state dir (~/.local/state/os-context/audit):
|
||||
#
|
||||
# run-<date>/stats.tsv one row per audited session (deterministic columns)
|
||||
# run-<date>/flags.md heuristic missed-delegation candidates, per session,
|
||||
|
|
@ -31,7 +31,7 @@ require "optparse"
|
|||
load File.expand_path("extract", __dir__)
|
||||
|
||||
module OrchAudit
|
||||
DEFAULT_STATE_DIR = File.expand_path("~/.local/state/os-orchestration/audit")
|
||||
DEFAULT_STATE_DIR = File.expand_path("~/.local/state/os-context/audit")
|
||||
DEFAULT_PROJECTS_DIR = File.expand_path("~/.claude/projects")
|
||||
|
||||
class TranscriptSet
|
||||
|
|
@ -1,10 +1,10 @@
|
|||
# os-orchestration eval — E1–E3, E5 (session-orchestration behavior)
|
||||
# os-context eval — E1–E3, E5 (session-orchestration behavior)
|
||||
|
||||
_Created: 2026-07-06. Scenarios sourced from the verified misses in
|
||||
`docs/orchestration-audit/2026-07-06-findings.md` (clusters 1–3), per the
|
||||
IRL-feedback-loop methodology. E4 (batch planning) deliberately deferred._
|
||||
|
||||
Measures whether an orchestrating session follows the shipped `ORCHESTRATION.md`
|
||||
Measures whether an orchestrating session follows the shipped `prompts/session-start/10-orchestration.md`
|
||||
(the 2026-07-06 trigger-conditioned wording) mid-task: explicit `model:` per spawn,
|
||||
resolved-model verification, delegation thresholds, and not re-covering its own
|
||||
ground. This validates the wording fixes the way Eval B/C validated os-adr's —
|
||||
|
|
@ -18,7 +18,7 @@ was wrong).**
|
|||
## Run mode (the only valid one)
|
||||
|
||||
Headless: fresh `claude -p` per rep, cwd = a sandbox copy of `fixture/project/`, so
|
||||
the global os-orchestration SessionStart hook injects ORCHESTRATION.md for real.
|
||||
the global os-context SessionStart hook injects the orchestration prompt for real.
|
||||
In-session subagents are invalid here (they inherit the parent session). The behavior
|
||||
under test is mid-session, so unlike Eval B there is no SessionStart trigger to
|
||||
protect — but the injection channel is the SessionStart hook, so the constraint is
|
||||
|
|
@ -104,7 +104,7 @@ name field); metadata gains `total_spawns` and `max_rounds`.
|
|||
## Conformance dry-run (done at design time, 2026-07-06)
|
||||
|
||||
Per the ladder anti-pattern list, each cell was checked against "would a model
|
||||
perfectly following the shipped ORCHESTRATION.md pass?" — yes on all six (E1P:
|
||||
perfectly following the shipped prompts/session-start/10-orchestration.md pass?" — yes on all six (E1P:
|
||||
delegate+sonnet+flag; E1N: no false claim; E2P: fan out with explicit models; E2N/E3N:
|
||||
do directly; E3P: delegate the log review after at most an orienting look) — and
|
||||
"would an always-delegate model pass positives while failing negatives?" — it fails
|
||||
|
|
@ -128,7 +128,7 @@ done; wait
|
|||
|
||||
- Sandboxes must live under a dot-free path (/tmp/orch-eval): the checker flattens
|
||||
the sandbox realpath to find the project transcript dir.
|
||||
- Run `cc-os/bin/refresh-plugins` after ANY ORCHESTRATION.md edit, before any grid.
|
||||
- Run `cc-os/bin/refresh-plugins` after ANY prompts/session-start/10-orchestration.md edit, before any grid.
|
||||
- TSV columns: scenario, model, rep, verdict, axes, info. The TSV is the truth —
|
||||
verify from it, not from prose reports.
|
||||
- Judge (E1 language axes only): frozen rubric in `judge-rubric.md`; stub with
|
||||
|
|
@ -141,7 +141,7 @@ The first live E1P rep (sonnet, counted: FAIL, `B:fail-mechanical`) falsified th
|
|||
premise that the orchestrator can see `resolvedModel` in the launch stub — the model-visible
|
||||
launch result carries no model info, so the first shipped wording ("compare the resolved model
|
||||
reported in the launch result") was unactionable and E1 was unwinnable by a compliant model.
|
||||
Fixed in ORCHESTRATION.md wording ONLY (subagent model self-report mechanism — subagents know
|
||||
Fixed in prompts/session-start/10-orchestration.md wording ONLY (subagent model self-report mechanism — subagents know
|
||||
their exact model ID; verified live). Scenarios, checker, and rubric were NOT touched.
|
||||
|
||||
## Fixture
|
||||
|
|
@ -1,7 +1,7 @@
|
|||
#!/usr/bin/env ruby
|
||||
# frozen_string_literal: true
|
||||
|
||||
# Deterministic-first checker for the os-orchestration eval (E1-E3, E5).
|
||||
# Deterministic-first checker for the os-context eval (E1-E3, E5).
|
||||
#
|
||||
# Usage: check <scenario> <sandbox> [--tsv <model> [<rep>]]
|
||||
#
|
||||
|
|
@ -1,7 +1,7 @@
|
|||
#!/usr/bin/env bash
|
||||
# Headless runner for the os-orchestration eval — the ONLY valid execution mode
|
||||
# Headless runner for the os-context eval — the ONLY valid execution mode
|
||||
# (mid-session orchestration behavior under the real global SessionStart-injected
|
||||
# ORCHESTRATION.md; fresh `claude -p` per rep with cwd = sandbox).
|
||||
# prompts/session-start/10-orchestration.md; fresh `claude -p` per rep with cwd = sandbox).
|
||||
#
|
||||
# Usage: run <scenario> <model> <workdir> [--reps N] [--results FILE]
|
||||
#
|
||||
|
|
@ -1,7 +1,7 @@
|
|||
#!/usr/bin/env ruby
|
||||
# frozen_string_literal: true
|
||||
|
||||
# Model-free self-test for the os-orchestration eval checker. Fabricates
|
||||
# Model-free self-test for the os-context eval checker. Fabricates
|
||||
# session-format transcripts and asserts verdicts in BOTH directions per
|
||||
# scenario, including at least one SHIPPED-INSTRUCTION-COMPLIANT transcript
|
||||
# per positive scenario (the Eval C conformance lesson: self-tests that only
|
||||
|
|
@ -1,4 +1,4 @@
|
|||
# Frozen judge rubrics — os-orchestration eval (E1 language axes only)
|
||||
# Frozen judge rubrics — os-context eval (E1 language axes only)
|
||||
|
||||
Used only when the mechanical scan is inconclusive. FROZEN during any wording loop.
|
||||
The judge receives one rubric section plus the transcript text, and must answer with
|
||||
|
Can't render this file because it has a wrong number of fields in line 2.
|
|
Can't render this file because it has a wrong number of fields in line 2.
|
|
Can't render this file because it has a wrong number of fields in line 2.
|
|
Can't render this file because it has a wrong number of fields in line 6.
|
|
|
@ -0,0 +1,76 @@
|
|||
#!/usr/bin/env python3
|
||||
"""SessionStart hook: compose prompts/session-start/*.md into one
|
||||
additionalContext block.
|
||||
|
||||
Fires on startup/resume/clear, and on 'compact' (matcher includes it) so the
|
||||
composed context survives a context-compaction pass, not just the first turn.
|
||||
|
||||
Prompt files live under prompts/<event>/*.md; only the session-start event is
|
||||
implemented today (SessionEnd cannot inject context — ADR-028). Files are
|
||||
concatenated in filename sort order (e.g. 10-orchestration.md) with no added
|
||||
separator between them, so a single prompt file present alone reproduces the
|
||||
exact bytes it contributes.
|
||||
"""
|
||||
import glob
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
|
||||
PLUGIN_ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||
SESSION_START_DIR = os.path.join(PLUGIN_ROOT, "prompts", "session-start")
|
||||
|
||||
# Deterministic injected-context budget, in lines. Below WARN_LINE_BUDGET,
|
||||
# inject normally. Between the two, still inject but warn on stderr. Above
|
||||
# REFUSE_LINE_BUDGET, refuse to inject anything and warn on stderr instead.
|
||||
WARN_LINE_BUDGET = 120
|
||||
REFUSE_LINE_BUDGET = 240
|
||||
|
||||
|
||||
def prompt_files(session_start_dir=None):
|
||||
if session_start_dir is None:
|
||||
session_start_dir = SESSION_START_DIR
|
||||
return sorted(glob.glob(os.path.join(session_start_dir, "*.md")))
|
||||
|
||||
|
||||
def compose(paths):
|
||||
"""Concatenate prompt file contents in order with no added separators."""
|
||||
parts = []
|
||||
for path in paths:
|
||||
with open(path) as f:
|
||||
parts.append(f.read())
|
||||
return "".join(parts)
|
||||
|
||||
|
||||
def main():
|
||||
paths = prompt_files()
|
||||
if not paths:
|
||||
return
|
||||
|
||||
context = compose(paths)
|
||||
line_count = len(context.splitlines())
|
||||
|
||||
if line_count > REFUSE_LINE_BUDGET:
|
||||
print(
|
||||
f"os-context: session-start prompt budget exceeded "
|
||||
f"({line_count} lines > {REFUSE_LINE_BUDGET}) — refusing to inject.",
|
||||
file=sys.stderr,
|
||||
)
|
||||
return
|
||||
|
||||
if line_count > WARN_LINE_BUDGET:
|
||||
print(
|
||||
f"os-context: session-start prompt budget exceeded "
|
||||
f"({line_count} lines > {WARN_LINE_BUDGET}) — injecting anyway.",
|
||||
file=sys.stderr,
|
||||
)
|
||||
|
||||
print(json.dumps({
|
||||
"hookSpecificOutput": {
|
||||
"hookEventName": "SessionStart",
|
||||
"additionalContext": context,
|
||||
}
|
||||
}))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
|
|
@ -0,0 +1,23 @@
|
|||
# Kickoff conventions
|
||||
|
||||
Standing rules — do not re-derive or ask about these per project:
|
||||
|
||||
- **TDD**: red-green-refactor by default for new features/bug fixes.
|
||||
- **Sandi Metz OOP** style for Ruby; Ruby > Python > bash for scripts; Python OK
|
||||
for native CC tooling.
|
||||
- **Vertical slices / tracer bullets**: ship a thin end-to-end path before
|
||||
breadth; prefer this over building one layer fully before the next.
|
||||
- **Audit defaults before customizing**: check a tool's shipped
|
||||
defaults/recommendations before building a custom benchmark or config.
|
||||
- **lowercase-kebab-case** filenames unless a format requires otherwise.
|
||||
- Delegation defaults (when/how to dispatch subagents, risk-escalation
|
||||
read-only → guarded-write → verify) — see `10-orchestration.md` in this
|
||||
same context block.
|
||||
|
||||
## Vault note pointers
|
||||
|
||||
Query on demand, don't front-load:
|
||||
- `tool/<name>` — before first use this session of a tool with vault notes.
|
||||
- `client/<name>` — before starting work involving a client.
|
||||
- `domain/llm-evaluation` + `type/eval-results` — before designing/running an
|
||||
eval.
|
||||
|
|
@ -0,0 +1,9 @@
|
|||
# Standing safety rules
|
||||
|
||||
- **Never echo, print, or log a secret value** (API keys, tokens, passwords,
|
||||
`.env`/credentials file contents) into chat, commits, or generated files —
|
||||
availability-awareness (confirming a secret exists/where) is fine; printing
|
||||
its value is not.
|
||||
- **Release/deploy scripts require explicit user authorization** before
|
||||
running — treat any `release.sh`-equivalent or production deploy command as
|
||||
gated, never inferred from context alone.
|
||||
|
|
@ -0,0 +1,11 @@
|
|||
# Decision-memo format
|
||||
|
||||
When surfacing a decision, tradeoff, or interpretation call, never bury it in
|
||||
a status paragraph. Use this four-part memo:
|
||||
|
||||
1. **Headline** — one line naming the decision.
|
||||
2. **Options as concrete A/B examples** — actual code/command snippets
|
||||
side by side when the difference is code; never prose descriptions of the
|
||||
difference.
|
||||
3. **Recommendation** — with confidence % and the single deciding reason.
|
||||
4. **Explicit ask + default** — state what happens if the user says nothing.
|
||||
|
|
@ -1,8 +1,8 @@
|
|||
---
|
||||
description: Run the biweekly orchestration IRL audit - deterministic transcript stats precompute, then auditor fan-out judging flagged regions against ORCHESTRATION.md rules, then a tiered tune-up report. Works from any directory. Trigger via the os-status due nudge or the Planka recurrence card.
|
||||
description: Run the biweekly orchestration IRL audit - deterministic transcript stats precompute, then auditor fan-out judging flagged regions against the shipped `prompts/session-start/10-orchestration.md` rules, then a tiered tune-up report. Works from any directory. Trigger via the os-status due nudge or the Planka recurrence card.
|
||||
---
|
||||
|
||||
# Audit production sessions against the ORCHESTRATION.md rules
|
||||
# Audit production sessions against the shipped orchestration rules
|
||||
|
||||
You are running the recurring (biweekly) IRL audit of the shipped delegation-economics
|
||||
rules. The signal source is real production session transcripts — NEVER eval grid
|
||||
|
|
@ -24,10 +24,10 @@ guard already defers the live session to the next run.)
|
|||
|
||||
- `--since` defaults to the last ledger row's date. It refuses to run without a prior row;
|
||||
a first-ever run needs an explicit `--since YYYY-MM-DD`.
|
||||
- Output: `~/.local/state/os-orchestration/audit/run-<date>/` containing `stats.tsv`
|
||||
- Output: `~/.local/state/os-context/audit/run-<date>/` containing `stats.tsv`
|
||||
(per-session deterministic columns), `flags.md` (heuristic missed-delegation regions),
|
||||
and `sheets/*.json` (full per-session fact-sheets). One row is appended to
|
||||
`~/.local/state/os-orchestration/audit/metrics.tsv` — the trend ledger. Never edit the
|
||||
`~/.local/state/os-context/audit/metrics.tsv` — the trend ledger. Never edit the
|
||||
ledger by hand; never re-derive these numbers with a model.
|
||||
|
||||
## Step 2 — Auditor fan-out (judgment only where code can't judge)
|
||||
|
|
@ -58,10 +58,11 @@ Each auditor prompt must include:
|
|||
|
||||
## Step 4 — Stop for the human
|
||||
|
||||
Report the tiers and trend verdicts, then STOP. Do not edit ORCHESTRATION.md without
|
||||
explicit approval — wording changes are a human decision (injected-token budget; grid
|
||||
validation is unavailable). If approved later: edit
|
||||
`~/dev/cc-os/plugins/os-orchestration/ORCHESTRATION.md`, run `~/dev/cc-os/bin/refresh-plugins`,
|
||||
Report the tiers and trend verdicts, then STOP. Do not edit
|
||||
`prompts/session-start/10-orchestration.md` without explicit approval — wording changes
|
||||
are a human decision (injected-token budget; grid validation is unavailable). If approved
|
||||
later: edit
|
||||
`~/dev/cc-os/plugins/os-context/prompts/session-start/10-orchestration.md`, run `~/dev/cc-os/bin/refresh-plugins`,
|
||||
and record the change in `~/dev/cc-os/docs/implementation-status.md`.
|
||||
|
||||
Finally, if a Planka card for this audit exists on the backlog board, move it to
|
||||
|
|
@ -1,4 +1,4 @@
|
|||
# IRL audit rubric — shipped ORCHESTRATION.md delegation-economics rules
|
||||
# IRL audit rubric — shipped prompts/session-start/10-orchestration.md delegation-economics rules
|
||||
|
||||
Audit each assigned session against these shipped rules. A "finding" is a concrete
|
||||
violation, near-miss, or friction point, with evidence (jsonl line number / short quote).
|
||||
|
|
@ -0,0 +1,146 @@
|
|||
"""Tests for hooks/inject.py. Run: python3 tests/hook_test.py"""
|
||||
|
||||
import json
|
||||
import sys
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from tempfile import TemporaryDirectory
|
||||
|
||||
PLUGIN_ROOT = Path(__file__).resolve().parent.parent
|
||||
sys.path.insert(0, str(PLUGIN_ROOT / "hooks"))
|
||||
|
||||
import inject # noqa: E402
|
||||
|
||||
|
||||
def write_prompts(directory, files):
|
||||
"""files: dict of filename -> content, written under directory."""
|
||||
directory.mkdir(parents=True, exist_ok=True)
|
||||
for name, content in files.items():
|
||||
(directory / name).write_text(content)
|
||||
|
||||
|
||||
class ComposeOrderTest(unittest.TestCase):
|
||||
def test_concatenates_in_filename_sort_order(self):
|
||||
with TemporaryDirectory() as tmp:
|
||||
d = Path(tmp)
|
||||
write_prompts(d, {"20-second.md": "second\n", "10-first.md": "first\n"})
|
||||
paths = inject.prompt_files(d)
|
||||
self.assertEqual(["10-first.md", "20-second.md"], [Path(p).name for p in paths])
|
||||
self.assertEqual("first\nsecond\n", inject.compose(paths))
|
||||
|
||||
def test_no_separator_added_between_files(self):
|
||||
with TemporaryDirectory() as tmp:
|
||||
d = Path(tmp)
|
||||
write_prompts(d, {"10-a.md": "a", "20-b.md": "b"})
|
||||
paths = inject.prompt_files(d)
|
||||
self.assertEqual("ab", inject.compose(paths))
|
||||
|
||||
|
||||
class ByteIdentityTest(unittest.TestCase):
|
||||
"""With only the real 10-orchestration.md present, the composer's output
|
||||
must be byte-identical to what the old single-file inject.py emitted
|
||||
directly from ORCHESTRATION.md's contents — no added separators or
|
||||
trailing newlines."""
|
||||
|
||||
def test_single_file_matches_its_raw_contents_exactly(self):
|
||||
real_file = PLUGIN_ROOT / "prompts" / "session-start" / "10-orchestration.md"
|
||||
with TemporaryDirectory() as tmp:
|
||||
d = Path(tmp)
|
||||
d.mkdir(exist_ok=True)
|
||||
shadow = d / "10-orchestration.md"
|
||||
shadow.write_bytes(real_file.read_bytes())
|
||||
paths = inject.prompt_files(d)
|
||||
composed = inject.compose(paths)
|
||||
self.assertEqual(real_file.read_text(), composed)
|
||||
|
||||
def test_real_prompts_dir_composes_all_files_byte_identical_concatenation(self):
|
||||
paths = inject.prompt_files()
|
||||
self.assertGreaterEqual(len(paths), 1)
|
||||
expected = "".join(Path(p).read_text() for p in paths)
|
||||
self.assertEqual(expected, inject.compose(paths))
|
||||
|
||||
|
||||
class BudgetTest(unittest.TestCase):
|
||||
def make_dir_with_lines(self, tmp, n_lines):
|
||||
d = Path(tmp)
|
||||
d.mkdir(exist_ok=True)
|
||||
content = "\n".join(f"line {i}" for i in range(n_lines)) + "\n"
|
||||
(d / "10-big.md").write_text(content)
|
||||
return d
|
||||
|
||||
def test_under_warn_budget_injects_silently(self):
|
||||
with TemporaryDirectory() as tmp:
|
||||
d = self.make_dir_with_lines(tmp, inject.WARN_LINE_BUDGET - 1)
|
||||
paths = inject.prompt_files(d)
|
||||
context = inject.compose(paths)
|
||||
self.assertLessEqual(len(context.splitlines()), inject.WARN_LINE_BUDGET)
|
||||
|
||||
def test_over_warn_under_refuse_still_injects(self):
|
||||
with TemporaryDirectory() as tmp:
|
||||
d = self.make_dir_with_lines(
|
||||
tmp, inject.WARN_LINE_BUDGET + 1
|
||||
)
|
||||
paths = inject.prompt_files(d)
|
||||
context = inject.compose(paths)
|
||||
line_count = len(context.splitlines())
|
||||
self.assertGreater(line_count, inject.WARN_LINE_BUDGET)
|
||||
self.assertLessEqual(line_count, inject.REFUSE_LINE_BUDGET)
|
||||
# Injection still happens: compose() returns the full content.
|
||||
self.assertTrue(context)
|
||||
|
||||
def test_over_refuse_budget_line_count_detected(self):
|
||||
with TemporaryDirectory() as tmp:
|
||||
d = self.make_dir_with_lines(tmp, inject.REFUSE_LINE_BUDGET + 1)
|
||||
paths = inject.prompt_files(d)
|
||||
context = inject.compose(paths)
|
||||
self.assertGreater(len(context.splitlines()), inject.REFUSE_LINE_BUDGET)
|
||||
|
||||
def test_main_warns_on_stderr_between_warn_and_refuse(self, ):
|
||||
import io
|
||||
from unittest import mock
|
||||
|
||||
with TemporaryDirectory() as tmp:
|
||||
d = self.make_dir_with_lines(tmp, inject.WARN_LINE_BUDGET + 1)
|
||||
stderr = io.StringIO()
|
||||
stdout = io.StringIO()
|
||||
with mock.patch.object(inject, "SESSION_START_DIR", str(d)), \
|
||||
mock.patch("sys.stderr", stderr), \
|
||||
mock.patch("sys.stdout", stdout):
|
||||
inject.main()
|
||||
self.assertIn("injecting anyway", stderr.getvalue())
|
||||
payload = json.loads(stdout.getvalue())
|
||||
self.assertIn("additionalContext", payload["hookSpecificOutput"])
|
||||
|
||||
def test_main_refuses_and_emits_nothing_over_refuse_budget(self):
|
||||
import io
|
||||
from unittest import mock
|
||||
|
||||
with TemporaryDirectory() as tmp:
|
||||
d = self.make_dir_with_lines(tmp, inject.REFUSE_LINE_BUDGET + 1)
|
||||
stderr = io.StringIO()
|
||||
stdout = io.StringIO()
|
||||
with mock.patch.object(inject, "SESSION_START_DIR", str(d)), \
|
||||
mock.patch("sys.stderr", stderr), \
|
||||
mock.patch("sys.stdout", stdout):
|
||||
inject.main()
|
||||
self.assertIn("refusing to inject", stderr.getvalue())
|
||||
self.assertEqual("", stdout.getvalue())
|
||||
|
||||
|
||||
class NoPromptsTest(unittest.TestCase):
|
||||
def test_main_emits_nothing_when_dir_empty(self):
|
||||
import io
|
||||
from unittest import mock
|
||||
|
||||
with TemporaryDirectory() as tmp:
|
||||
d = Path(tmp) / "empty"
|
||||
d.mkdir()
|
||||
stdout = io.StringIO()
|
||||
with mock.patch.object(inject, "SESSION_START_DIR", str(d)), \
|
||||
mock.patch("sys.stdout", stdout):
|
||||
inject.main()
|
||||
self.assertEqual("", stdout.getvalue())
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
|
@ -1,5 +0,0 @@
|
|||
{
|
||||
"name": "os-orchestration",
|
||||
"version": "0.1.0",
|
||||
"description": "Session orchestration rules (delegation threshold, explicit model routing on Agent spawns) injected at session start and after compaction."
|
||||
}
|
||||
|
|
@ -1,31 +0,0 @@
|
|||
#!/usr/bin/env python3
|
||||
"""SessionStart hook: inject ORCHESTRATION.md as additionalContext.
|
||||
|
||||
Fires on startup/resume/clear, and on 'compact' (matcher includes it) so the
|
||||
rules survive a context-compaction pass, not just the first turn.
|
||||
"""
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
|
||||
PLUGIN_ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||
ORCHESTRATION_FILE = os.path.join(PLUGIN_ROOT, "ORCHESTRATION.md")
|
||||
|
||||
|
||||
def main():
|
||||
try:
|
||||
with open(ORCHESTRATION_FILE) as f:
|
||||
context = f.read()
|
||||
except OSError:
|
||||
return
|
||||
|
||||
print(json.dumps({
|
||||
"hookSpecificOutput": {
|
||||
"hookEventName": "SessionStart",
|
||||
"additionalContext": context,
|
||||
}
|
||||
}))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
|
|
@ -163,7 +163,7 @@ def vault_hub_note_present(ctx: Ctx) -> CheckResult:
|
|||
|
||||
|
||||
AUDIT_LEDGER_ENV = "OS_ORCH_AUDIT_LEDGER"
|
||||
DEFAULT_AUDIT_LEDGER = "~/.local/state/os-orchestration/audit/metrics.tsv"
|
||||
DEFAULT_AUDIT_LEDGER = "~/.local/state/os-context/audit/metrics.tsv"
|
||||
AUDIT_INTERVAL_DAYS = 14
|
||||
|
||||
|
||||
|
|
@ -177,7 +177,7 @@ def _last_audit_date(ledger: Path):
|
|||
|
||||
def orchestration_audit_due(ctx: Ctx) -> CheckResult:
|
||||
"""Nudge when the biweekly orchestration IRL audit is overdue. The ledger
|
||||
(metrics.tsv) is appended deterministically by os-orchestration's
|
||||
(metrics.tsv) is appended deterministically by os-context's
|
||||
audit/bin/audit-stats; this check only reads its last row's date."""
|
||||
ledger = Path(
|
||||
os.path.expanduser(ctx.environ.get(AUDIT_LEDGER_ENV, DEFAULT_AUDIT_LEDGER))
|
||||
|
|
@ -190,7 +190,7 @@ def orchestration_audit_due(ctx: Ctx) -> CheckResult:
|
|||
return OK
|
||||
return warn(
|
||||
f"orchestration IRL audit due (last run {last.isoformat()}, {days} days"
|
||||
" ago) — run /os-orchestration:audit-sessions (works from any project;"
|
||||
" ago) — run /os-context:audit-sessions (works from any project;"
|
||||
" deterministic stats precompute, then auditor fan-out)."
|
||||
)
|
||||
|
||||
|
|
@ -295,7 +295,7 @@ REGISTRY = [
|
|||
"orchestration-audit-due",
|
||||
orchestration_audit_due,
|
||||
project_scoped=False,
|
||||
remediation="/os-orchestration:audit-sessions",
|
||||
remediation="/os-context:audit-sessions",
|
||||
),
|
||||
Check(
|
||||
"tracker-configured",
|
||||
|
|
|
|||
|
|
@ -232,7 +232,7 @@ class AuditDueCheckTest(unittest.TestCase):
|
|||
result = checks.orchestration_audit_due(make_ctx(environ=env))
|
||||
self.assertEqual("warn", result.status)
|
||||
self.assertIn("2026-02-01", result.message)
|
||||
self.assertIn("/os-orchestration:audit-sessions", result.message)
|
||||
self.assertIn("/os-context:audit-sessions", result.message)
|
||||
|
||||
def test_corrupt_ledger_reads_as_missing(self):
|
||||
with TemporaryDirectory() as tmp:
|
||||
|
|
|
|||
Loading…
Reference in New Issue