From a06431c5fdcce445f547d88fe1d6b332b741b2da Mon Sep 17 00:00:00 2001 From: jared Date: Wed, 8 Jul 2026 13:36:16 -0400 Subject: [PATCH] =?UTF-8?q?system-prompt=20profiles=20Wave=201:=20CLAUDE.m?= =?UTF-8?q?d=20split=20(~11k=E2=86=92~2.1k=20tokens),=20B3+C3=20findings,?= =?UTF-8?q?=20slimmed=20plan?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - B1: CLAUDE.md reduced to orientation only; status timeline, component detail, and plugin procedures moved to docs/implementation-status.md (loaded on demand). Fresh-session check: ~32k context vs 41.8k baseline (~9.7k/session saved), orientation probes pass. - B3: memsearch SessionStart injection measured (61 sessions, median ~11.7KB); size is hardcoded in the hook, no config knob — report in docs/plans/. - C3: --setting-sources must include "user" or all plugins silently drop; ~/.claude/profiles/*.json overlays are disjoint from bin/refresh-plugins. - Plan docs: Gate G0 cleared (WS2/WS4 complete); scope slimmed — A1/A2 deferred behind an interactive /context check, D dropped (pointer kept), C minimized. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01Uc1HZDxnpzTJ8EYLqruEnY --- CLAUDE.md | 299 ++++------- docs/implementation-status.md | 490 ++++++++++++++++++ docs/plans/b3-memsearch-injection-review.md | 91 ++++ docs/plans/c3-spike-findings.md | 114 ++++ .../system-prompt-profiles-assessment.md | 30 +- docs/plans/system-prompt-profiles-tasks.md | 69 +++ 6 files changed, 892 insertions(+), 201 deletions(-) create mode 100644 docs/implementation-status.md create mode 100644 docs/plans/b3-memsearch-injection-review.md create mode 100644 docs/plans/c3-spike-findings.md create mode 100644 docs/plans/system-prompt-profiles-tasks.md diff --git a/CLAUDE.md b/CLAUDE.md index 9012dcd..c4a4cf3 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1,230 +1,129 @@ # CLAUDE.md -This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. +This file provides guidance to Claude Code (claude.ai/code) when working with code in this +repository. It is orientation only — implementation history, component details, and +operational procedures live in [docs/implementation-status.md](docs/implementation-status.md) +(read it on demand, e.g. before touching a plugin or claiming something isn't built yet). ## What this repository is -`cc-os` is a **design + implementation repository** — it captures the design of a **personal, -cross-project memory system for Claude Code** (for a multi-client freelancer) plus the research -that informs it. The global memory plugin is now partially implemented; markdown specs, ADRs, -and the build plan remain the source of truth for what is being built and what remains. - -Everything is markdown-as-truth. When asked to "build," work from the staged tasks in -`docs/memory-system/04-build-plan.md`, not ad hoc. +`cc-os` is a **design + implementation repository** for a **personal, cross-project memory +system for Claude Code** (for a multi-client freelancer), plus a family of global Claude Code +plugins and the research/evals that inform them. Everything is markdown-as-truth: specs, +ADRs, and the build plan in `docs/` are the source of truth for what is being built. When +asked to "build," work from the staged tasks in `docs/memory-system/04-build-plan.md`, not ad +hoc. ## Directory layout -Numbered files within a directory are not a required reading order — read the one whose topic -you need. +Numbered files within a directory are not a required reading order — read the one whose +topic you need. -- **`docs/memory-system/`** — the design of the memory system itself. Go here to understand or - change *what is being built*. `02-system-design.md` is the architecture, `03-architecture- - decisions.md` is the ADR log (each decision + what was rejected/deferred and why), - `04-build-plan.md` is the build outline, `06-graphify-evaluation.md` justifies the Graphify - pivot. Read the specific file relevant to your task; read the whole set only when reworking - the design. -- **`docs/graphify/`** — a verified handbook for the Graphify knowledge-graph tool (the chosen - knowledge-layer engine). Go here when working with Graphify commands/behavior. Skim - `00-README.md` for the model, keep `09-best-practices-checklist.md` open while actually - running it. Claims are provenance-tagged (`[github]` trustworthy; `[interview]` / - `[unverified claim]` not). +- **`docs/memory-system/`** — the design of the memory system itself. `02-system-design.md` + is the architecture, `03-architecture-decisions.md` is the ADR log, `04-build-plan.md` the + build outline with step status, `06-graphify-evaluation.md` justifies the Graphify pivot. +- **`docs/implementation-status.md`** — the status timeline, full per-component detail for + every shipped plugin (os-vault, os-orchestration, os-status, os-doc-hygiene, os-adr), + eval-harness records, and plugin rename/cache-refresh procedures. +- **`docs/graphify/`** — verified handbook for the Graphify knowledge-graph tool. Skim + `00-README.md` for the model; keep `09-best-practices-checklist.md` open when running it. + Claims are provenance-tagged (`[github]` trustworthy; `[interview]` / `[unverified claim]` + not). - **`graphify-interview`, `memory-systems-compared060326`** (repo root) — raw source - transcripts (marketing / video). Only open these to trace where a claim came from; treat as - *intent*, not *fact* — they were already corrected against primary sources in `docs/`. -- **`openspec/`** — spec-driven change management (see workflow below). `changes/` holds live - changes, `changes/archive/` completed ones, `specs/` stable specs. -- **`.claude/`, `.codex/`, `.pi/`** — **identical** copies of the OpenSpec skills for three AI - assistants. Only open when changing a skill — and mirror any change across all three. + transcripts. Only open to trace a claim's origin; treat as *intent*, not *fact*. +- **`plugins/`** — source of the global plugins, symlinked into `~/.claude/plugins/`. +- **`openspec/`** — spec-driven change management (see workflow below). +- **`.claude/`, `.codex/`, `.pi/`** — **identical** copies of the OpenSpec skills for three + AI assistants. Only open when changing a skill — and mirror any change across all three. ## The current design in one paragraph -This is a **work in progress**, not a frozen spec. The paragraph below is the current approach; -treat it as the default you operate from, but **whenever a conversation with the user changes -the design, update this paragraph (and the relevant `docs/memory-system/` files + an ADR) to -match.** Keep it accurate, don't preserve it for its own sake. +This is a work in progress, not a frozen spec; **when a conversation changes the design, +update this paragraph (and the relevant `docs/memory-system/` files + an ADR) to match.** -Two memory types kept as **separate systems**: **episodic** ("what happened, when") handled by -**memsearch** (Milvus Lite, embedded), and **semantic/knowledge** ("how do we…") handled by the -existing **`~/Documents/SecondBrain` Obsidian vault** as the single source of truth. Notes keep -`summary` + six flat, parallel namespaced facets (`type/`/`client/`/`project/`/`domain/`/`tool/`/`convention/`) plus `scope/` as metadata; hierarchy and relationships are expressed via hub notes (`type/hub`), wikilinks, and Graphify graph edges — not nested tag paths. The vault is queried via a **Graphify knowledge graph** (local Ollama SLM for doc extraction, free -tree-sitter AST for code). Retrieval is hook-injected + on-demand so project repos stay thin; -freshness is lazy -(write-time hook + SessionStart reconcile, no daemon/cron); the vault syncs to a VPS while -indexes stay disposable and rebuildable. Ships as a global Claude Code plugin (`os-vault`) with skills. +Two memory types kept as **separate systems**: **episodic** ("what happened, when") handled +by **memsearch** (Milvus Lite, embedded, single global store for ALL clients by design — +ADR-015), and **semantic/knowledge** ("how do we…") handled by the existing +**`~/Documents/SecondBrain` Obsidian vault** as the single source of truth (ADR-012). Notes +keep `summary` + six flat namespaced facets +(`type/`/`client/`/`project/`/`domain/`/`tool/`/`convention/`) plus `scope/` (ADR-011); +hierarchy comes from hub notes, wikilinks, and Graphify graph edges — not nested tags +(Graphify is a structure extractor, not a topic clusterer; hubs are author-provided, +ADR-014). The vault is queried via a **Graphify knowledge graph** (local Ollama SLM for doc +extraction, tree-sitter AST for code); Graphify replaced the originally-planned Ruby/SQLite +tag-index CLI (2026-06-03 pivot). Retrieval is hook-injected + on-demand; freshness is lazy +(write-time hook + SessionStart reconcile, no daemon); the vault and memsearch store +auto-sync to private Forgejo repos via SessionEnd hooks. Ships as a global Claude Code +plugin (`os-vault`) with skills. Projects are onboarded one at a time; bulk vault migration +is deferred to last (ADR-013). -**Recent pivot (2026-06-03):** Graphify **replaces** the originally-planned Ruby/SQLite -tag-index CLI and also covers the deferred QMD semantic layer. `04-build-plan.md` and -`06-graphify-evaluation.md` reflect this; if an older doc still describes the Ruby CLI, defer -to those two and fix the stale doc. +## Implemented components (inventory) -**Decisions locked (2026-06-04):** Six-facet tag taxonomy + `scope/` (ADR-011); reuse `~/Documents/SecondBrain` vault rather than creating a new one (ADR-012); build-first / migrate-incrementally — build full system against a fixture set first, defer bulk vault migration to last, onboard projects one at a time (ADR-013). +All build-plan steps required for the memory system are complete (2026-06-15). Full detail +for each item is in [docs/implementation-status.md](docs/implementation-status.md) — read it +before modifying any of these: -**Empirical finding locked (2026-06-05):** Graphify is a structure extractor, not a topic clusterer — no emergent hub nodes appear even at `--mode deep`; hub notes + wikilinks must be author-provided during migration (not deferred). Migration scaffolding is now a first-class deliverable. Open question: do facet tags create graph edges? (ADR-014; findings: `docs/memory-system/07-graph-connectivity-findings.md`). +- **os-vault** (`plugins/os-vault/`) — vault write/query/reorganize/onboard-project/ + design-template skills, SessionStart/End hooks, memsearch + vault git sync. Write-behavior + eval harness in `plugins/os-vault/eval/`. +- **os-orchestration** (`plugins/os-orchestration/`) — injects `ORCHESTRATION.md` + (session-orchestration + delegation-economics rules) into all sessions; this repo carries + no local override. Eval harness in `plugins/os-orchestration/eval/`. +- **os-status** (`plugins/os-status/`) — aggregated deterministic SessionStart checks + (subagent-model env override, ADR system present, vault hub note present). ADR-022. +- **os-doc-hygiene** (`plugins/os-doc-hygiene/`) — stale/bloated-doc monitoring; + check/clean/status/sweep skills. +- **os-adr** (`plugins/os-adr/`) — ADR system: Ruby `lib/adr/` + CLIs, create/init/migrate/ + find skills; Eval A/B/C harnesses. Rollout order locked: pilot projects, then the cc-os + retrofit of `03-architecture-decisions.md`, then wider. +- **Graphify** v0.8.31 (`~/.local/bin/graphify`; PyPI package is `graphifyy`, double-y) — + vault graph at `~/Documents/SecondBrain/graphify-out/`, per-project graphs at + `/graphify-out/` (gitignored); both disposable/rebuildable. +- **memsearch** v0.4.6 (marketplace plugin) — `~/.memsearch/memory/YYYY-MM-DD.md` daily + files + Milvus index; `/memory-recall`, `/memory-config`. -**Implementation status (2026-06-09):** The global Claude Code plugin is live (`~/.claude/plugins/os-vault/`). Steps 1, 2a, 2b, 3, and 6 of the build plan are complete (including the `onboard-project` skill, previously TODO under Step 6/Part D). Step 1 completed 2026-06-09: vault-conventions.md exists, all 6 fixture notes seeded, and 14 Graphify handbook + memory-system design notes migrated to the vault as migration scaffolding. Step 4 (memsearch) completed 2026-06-09: plugin installed via marketplace, MEMSEARCH_DIR set global (~/.memsearch), Stop hook verified producing daily memory files, search confirmed working. Step 5a (memsearch episodic git sync) completed 2026-06-09: dedicated private Forgejo repo (`forgejo.swansoncloud.com/jared/memsearch`), whitelist `.gitignore` (memory/*.md only), auto-commit+push via cc-os `session-end.sh` hook (ADR-015; relocated to `memsearch_sync.py` by ADR-016, 2026-06-12). Step 5b (Obsidian vault → VPS sync) remains. See `docs/memory-system/04-build-plan.md` for full step status. - -**Decision (2026-06-09):** Single global memsearch store for ALL clients — cross-client commingling is the accepted, intended design (ADR-015 resolved). One private Forgejo repo (`forgejo.swansoncloud.com/jared/memsearch`), not per-client repos. Forward direction: minimize/eliminate dedicated per-client working directories; work projects locally or use a single general `clients/` dir for non-local client work — memory is captured globally regardless of cwd. No `clients/` directory structure designed yet; open item. - -**Implementation status (2026-06-12):** os-vault plugin source moved into git at `cc-os/plugins/os-vault/` and bash hooks ported to Python (deep-module architecture: shared `config.py`, `hook_io.py`, `session_state.py`; thin entry-point scripts). Cutover via symlink `~/.claude/plugins/os-vault → cc-os/plugins/os-vault/` and settings.json hook rewrite. memsearch sync split into dedicated `memsearch_sync.py` SessionEnd hook (relocation of ADR-015 behavior, not reversal). Fresh-session test passed 2026-06-12. See ADR-016. - -**Implementation status (2026-06-15):** Step 5b done and automated — vault (`~/Documents/SecondBrain`) initialized as git repo, pushed to private Forgejo (`ssh://git@forgejo.swansoncloud.com:2222/jared/SecondBrain.git`); 52 files, git chosen over Syncthing. Auto-commit+push wired via `vault_sync.py` SessionEnd hook (third SessionEnd hook, runs after `session_end.py` so the daily journal note is included; push-only — SessionStart pull is an optional future item for multi-machine). Step 2e pilot done — llf-schema (`/home/jared/dev/llf-schema`, PHP 8.2 WordPress plugin) onboarded via `/os-vault:onboard-project` skill: 605 nodes / 930 edges / 52 communities; Ollama doc pass lossy on WordPress docs but AST pass solid. Step 2d closed — live vault `graphify-out/` is the baseline; fixture-only build superseded. - -**Implementation status (2026-06-17):** `/os-vault:onboard-project` skill now uses assessment-first onboarding: surveys the repo structure, classifies exclude candidates by TYPE (11 categories: fetched deps, build output, caches, VCS internals, editor/AI-tooling dirs, lockfiles, coverage/logs, bulk data, binaries, secrets, graphify-out/ — illustrative names, not fixed templates), generates a per-project `.graphifyignore`, confirms with the user, then extracts the graph using `qwen25-coder-7b-16k` (config.yaml `ollama_model`; 16k-context build of qwen2.5-coder:7b — larger context window cuts chunk count per doc, the main speed lever). Fixes issue where repos with large dependency trees routed non-code files through the Ollama doc pass (see ADR-017). - -**Implementation status (2026-06-30):** Phase 1 of the SecondBrain Content Plan complete (`docs/memory-system/09-sb-content-plan.md`, issues #1–#6). `~/Documents/SecondBrain/vault-conventions.md` reconciled to a single authoritative typed frontmatter schema (removed the contradictory block; canonical filenames are slug-only, `source:` is a frontmatter field not a tag, `scope` is a field not a tag). New `/os-vault:design-template` skill added (`cc-os/plugins/os-vault/skills/design-template/SKILL.md`) — routes between template-design (4-step process + injection-economics filter) and new-type creation (9-step lifecycle). Three vault note templates created in `~/Documents/SecondBrain/_templates/` (`howto.md`, `convention.md`, `reference.md`; `reference` carries a four-subtype variant selector — pattern/framework, api-integration, role-definitions, design-rules), each dogfooded against a real note. Four proof-of-concept vault notes patched (cookbook subtype → pattern/framework, glossary de-duped to a wikilink, design-mode scope → project, tags + `last_reviewed` added). Phase 2 (issue #7) remains open as the steady-state migration/onboarding epic (no code). See `docs/memory-system/09-sb-content-plan.md`. - -## Implemented Components - -**Global os-vault plugin** — `cc-os/plugins/os-vault/` (git-tracked, 2026-06-12); symlinked into `~/.claude/plugins/os-vault` -- Hooks: `hooks/` — `session_start.py`, `session_context.py` (project graph path only), `post_tool_use_write.py`, `session_end.py` (vault journal), `memsearch_sync.py` (second SessionEnd hook; memsearch auto-commit+push, 30s timeout), `vault_sync.py` (third SessionEnd hook; vault auto-commit+push to forgejo.swansoncloud.com/jared/SecondBrain, 30s timeout; mirrors memsearch_sync.py) -- Shared modules: `config.py` (load_config → frozen Config dataclass), `hook_io.py` (read_input → HookInput dataclass), `session_state.py` (record_touch/read_touches; encapsulates `/tmp/claude-vault-touched-$SESSION_ID` contract) -- Skills: `skills/` — query, write, reorganize, onboard-project (assessment-first onboarding: surveys repo, classifies excludes by type (11 categories), writes per-project `.graphifyignore`, confirms with user, then extracts using `qwen25-coder-7b-16k` — per ADR-017), design-template (routes template-design ↔ new-type-creation: 4-step template-design process + injection-economics filter, 9-step new-type lifecycle; backs Phase 1 of the SB Content Plan — see `docs/memory-system/09-sb-content-plan.md`) -- Config: `config.yaml` — vault path, Ollama model (qwen25-coder-7b-16k), env vars -- Hook wiring: `~/.claude/settings.json` (hook entries invoke `/usr/bin/python3` with absolute paths into cc-os) -- Isolation seam (2026-07-06, WS2): `OS_VAULT_PATH` env var overrides `vault_path` for the write skill and every hook via `config.load_config()`; `OS_VAULT_SKIP_REBUILD` suppresses the SessionStart graphify rebuild. Added for the eval harness; zero production impact when unset. -- Write-behavior eval (2026-07-06, WS2): `plugins/os-vault/eval/` — held-out unprompted vault-write discrimination eval (Eval B/C playbook: ambiguity ladder L1 explicit → L3 conceptual, paired positives/negatives, 6 run-set scenarios on a new `reportgen` Ruby fixture + 6 frozen reserve twins in a different knowledge domain, isolated sandbox vault, headless-only runner, deterministic-first Ruby checker with narrow frozen offer-detection judge fallback stubbable via `OS_VAULT_EVAL_JUDGE_CMD`, model-free `bin/self-test` 21/21). **Scenario Task blocks are held-out — never run informally; the reserve set is never even read informally.** Procedure: `plugins/os-vault/eval/README.md`. -- Untuned baseline grid (2026-07-06, run-set × sonnet/haiku × 3 reps + 1 counted canary, 37 reps, zero harness errors): **positives 1/19, negatives 18/18**. Headline: at L1 (explicit cue) both tiers persist the knowledge every rep but route it to Claude Code's built-in auto-memory (`~/.claude/projects//memory/`) instead of the vault; L2/L3 mostly don't persist at all; zero over-triggering anywhere. The WS2 wording loop (step 4, NOT started; run-set still uncontaminated) therefore has two targets: trigger at L2/L3, and route to the vault over auto-memory at L1. TSV: `eval/results/2026-07-06-baseline-grid.tsv`; full analysis: vault note [[os-vault-write-eval-baseline-grid-results]]. -- WS2 wording loop COMPLETE (2026-07-07, 4 iterations + confirm + held-out reserve; run-set AND reserve now contaminated): shipped destination-ladder write SKILL.md + SessionStart USAGE_NOTE (mechanical triggers + before-final-reply checkpoint), mechanical vault-root resolution (`${OS_VAULT_PATH:-$HOME/Documents/SecondBrain}` — fixed real-vault breaches found on iteration 1), fixture CLAUDE.md "Cross-project memory" section (**the real-project adoption template**, required to reach haiku at L2/L3; includes ADR-vs-vault disambiguation), candidates 2–4 unscored, post-reserve YAML-colon quoting rule. Confirm grid sonnet 18/18 / haiku 16/18; reserve (held-out) positives 8/18 strict / trigger-axis 13/18, negatives 17/18 (vs baseline 1/19, 18/18). Residual gaps: YAML-hostile titles, haiku L2 no-trigger, generalization-laundering over-trigger. One checker false-positive fixed+rescored mid-loop (breach regex vs `2>/dev/null`); negative-criterion redesign pre-registered. TSVs: `eval/results/2026-07-07-wording-*.tsv`; loop log: `autoresearch/classic-260707-1253/loop-log.md`; analysis: vault note [[os-vault-write-eval-wording-loop-results]]. Next signal is production IRL audits. - -**Global os-orchestration plugin** — `cc-os/plugins/os-orchestration/` (git-tracked, 2026-07-03); symlinked into `~/.claude/plugins/os-orchestration` -- Hooks: `hooks/` — `inject.py` (injects `ORCHESTRATION.md` as additionalContext to all sessions) -- Behavior: SessionStart hook injects an `ORCHESTRATION.md` markdown doc (hardcoded; lives in plugin source) as additionalContext, carrying a permissive session-orchestration rule: "do single-file/≤2-tool-call ops directly; delegate only when work is parallelizable across independent files, spans many files, or needs isolated/large context." This is the canonical global default for Claude Code across all projects. -- Migration: migrated from a standalone repo (`~/dev/cc-plugins/orchestration/`, 2026-07-03) and integrated into cc-os. Supersedes the per-project copy-pasted orchestration text blocks that previously existed in individual project CLAUDE.md files (including a stricter local override that cc-os had carried — now removed; see ADR-019). -- Session audit (WS1, 2026-07-06): 10 stratified real sessions audited via `audit/bin/extract` (Ruby fact-sheet extractor, dual-use as future eval checker) + parallel sonnet auditors + verified synthesis. Findings + E1–E4 eval-scenario backlog: `docs/orchestration-audit/2026-07-06-findings.md`. Headlines: model param was silently overridden to haiku on all 23 verified spawns — root cause found post-audit: `CLAUDE_CODE_SUBAGENT_MODEL=haiku` in `~/.claude/settings.json` env block (set by an earlier session as a cost measure; removed 2026-07-06; the Fable-5 correlation was coincidental timing). Omitted model param inherits the main-loop model (opus/sonnet) — misses cluster in pre-rollout and ops sessions; no over-sharing/full-dump failures anywhere. -- Wording fixes shipped (2026-07-06, post-audit): ORCHESTRATION.md rewritten with trigger-conditioned when→then phrasing for the three verified clusters — explicit `model:` before every Agent call (cluster 2; a real cost exposure once the env override was removed), subagent model self-report on sonnet/opus spawns (cluster 1 — the E1 canary proved `resolvedModel` is NOT model-visible in the launch result; self-report is the only observable channel, and subagents do know their exact model ID), don't re-cover your own ground before delegating (cluster 3). Delegation thresholds unchanged. Cache refreshed via `bin/refresh-plugins`. Deterministic env-override detection (warn on `CLAUDE_CODE_SUBAGENT_MODEL`) is delegated to WS3's status-check design. -- Eval harness E1–E3 (2026-07-06): `plugins/os-orchestration/eval/` — scripted headless orchestration-behavior eval scored by the audit extractor (extended with per-segment read targets). 6 run-set scenarios (E1P/N downgrade detection with `CLAUDE_CODE_SUBAGENT_MODEL=haiku` runner stub; E2P/N explicit-model fan-out vs no-delegation-on-trivial; E3P/N delegate-investigation vs orienting-reads-correct) + 6 frozen reserve twins; new Node.js "relaystation" fixture with deterministic sandbox-time log generation; deterministic-first checker (`eval/bin/check`, narrow frozen haiku-judge fallback for E1 language axes only, stubbable via `ORCH_EVAL_JUDGE_CMD`); model-free `bin/self-test` 21/21 green incl. shipped-instruction-compliant conformance transcripts. FAIL axes map only to verified audit misses; tier choice informational. **Scenario Task blocks are held-out — never run informally.** Procedure: `plugins/os-orchestration/eval/README.md`. -- Baseline grid run (2026-07-06, run-set × sonnet/haiku × 3 reps + 2 counted canaries, all counted, zero harness errors): negatives 18/18 PASS both tiers (zero over-delegation, zero false downgrade claims); positives 1/18 on the delegate-at-all axis — but transcript verification shows most positive FAILs are *superior direct strategies* (E2P sonnet: one scripted bulk edit; E3P sonnet: surgical greps over 400KB logs), which the WS1 audit itself scored as justified non-delegation, vs genuine miss shapes (E3P haiku: whole 112KB logs into own context; E2P haiku: 20-call per-file grind). Conditional rules validated where engaged: 14/14 spawns explicit-model; the one delegating E1P rep flagged the stubbed downgrade via self-report. Criterion redesign for E2P/E3P A-axes is pre-registered (anchor thresholds independently, NOT to this grid) before any rescore/tuning; reserve stays frozen. ORCHESTRATION.md gained a scripted-bulk-edit carve-out (a uniform multi-file change one script covers is direct work). TSV: `eval/results/2026-07-06-baseline-grid.tsv`; full analysis: vault note [[os-orchestration-eval-baseline-grid-results]]. - -- WS4 economics loop COMPLETE + SHIPPED (2026-07-08; run-set AND reserve now contaminated for future wording tuning): ORCHESTRATION.md v3 shipped — cost-asymmetry + tier-conditional threshold opener, symmetric delegate-when/work-directly-when trigger lists (replaces "delegate only when"), batching rule (~5–8 items per grouped spawn, SendMessage reuse), async rule, Workflow effort dial; explicit-model/self-report/don't-re-cover-ground rules unchanged. Harness extended first (commit b226601): real token econ axes — subagent transcripts persist at `///subagents/agent-*.jsonl` with per-message usage blocks (solves the mini-audit's instrumentation gap; vault note [[claude-code-subagent-transcripts-and-token-accounting]]) — E5 batching pair + reserve twins (authored by subagent, held-out content never entered the tuning session), fable column, pre-registered E2P/E3P criterion redesign landed (scripted-direct PASS branch; 74KB ingestion anchor + id-keyed root-cause concepts). Results: E5P 0/6 → majority-PASS both tiers incl. reserve twin 5/5 (fable median mltok 43% of baseline, H3 met); negatives clean everywhere (36/36 across confirm+reserve). Residual gaps: E1P sonnet delegate-at-all (fixture too small — needs bigger fixture before meaningful), fable downgrade-flag ~1/4 judge flicker. 15 baseline reps were session-limit-truncated and mechanically excluded+replaced (limit-banner sweep now a standard step, promoted to the autoresearch howto). TSVs: `eval/results/2026-07-08-*.tsv`; loop log: `autoresearch/classic-260708-1039/`; vault note [[os-orchestration-ws4-econ-wording-results]]. IRL re-audit of ~5 real Fable sessions scheduled ~2026-07-22. - -**Global os-status plugin** — `cc-os/plugins/os-status/` (git-tracked, 2026-07-06); symlinked into `~/.claude/plugins/os-status` -- Purpose: aggregated deterministic SessionStart status checks for the cc-os plugin family — per-project artifacts (ADR system, vault hub note) and environment hazards (subagent model env override). Built via OpenSpec change `add-os-status-plugin` (supersedes the WS3 plan doc); decision record ADR-022. -- Architecture: checks are **in-process** plain Python functions in `hooks/checks.py` with the uniform signature `check(ctx) -> CheckResult(status, message)` plus one registry entry (no per-plugin subprocess protocol, no per-project code copies — both rejected in ADR-022 after perspective review). Three states: `ok` (silent), `note` (near-zero-token additionalContext line, never snoozed/suppressed), `warn` (aggregated into at most one banner; once-per-day snooze + permanent `suppress-` markers in gitignored per-project `.cc-os/` — state only, never code; `.cc-os/config` holds optional `hub`/`vault_path` key=value overrides). -- Initial checks: `subagent-model-env-override` (env + `~/.claude/settings.json` `env` block; runs outside git projects too — catches the WS1 Cluster 1 incident deterministically at session start), `adr-system-present` (verbatim port of os-adr's hook behavior incl. `.os-adr/suppress`), `vault-hub-note-present` (config slug first, else `type/hub` + `project/` facet-tag scan of the vault; missing → warn naming `/os-vault:write`). -- Tests: `python3 tests/hook_test.py` (36, model-free) incl. byte-identity of PRESENT_NOTE/ABSENT_NOTE against os-adr's source. Invariants in `invariants.md`. Smoke-tested 2026-07-06 (cache==source, real headless session fired the hook, env-override canary warns). - -**Global os-doc-hygiene plugin** — `cc-os/plugins/os-doc-hygiene/` (git-tracked, 2026-07-03); symlinked into `~/.claude/plugins/os-doc-hygiene` -- Hooks: `hooks/hooks.json` → SessionStart hook (matcher: startup|resume) runs `scripts/reminder.py` via `${CLAUDE_PLUGIN_ROOT}` (5s timeout), emitting a deterministic zero-token reminder banner -- Behavior: Monitors stale and bloated project documentation per-project under `.dochygiene/` state dir (gitignored). SessionStart reminder is deterministic (no AI tokens, once/day snooze). Skills (verb-first, no `commands/` dispatcher — invoked directly as `/os-doc-hygiene:`, per [[cc-os-plugin-skill-naming-convention]]): `check` (AI-assisted classification of staleness signals, emits machine+human reports), `clean` (AI-assisted or deterministic patch application with git-safe scoped cleanup), `status` (read-only lifecycle-timestamp read), `sweep` (check then clean in sequence). Reversion-protected via invariants.md + golden-example test fixtures. -- Migration: migrated from standalone repo (`~/dev/cc-plugins/doc-hygiene/`, 2026-07-03) and integrated into cc-os. No content conflicts detected (doc-hygiene scope does not overlap cc-os memory system). Renamed from `doc-hygiene` to `os-doc-hygiene` per cc-os plugin naming convention. Skills renamed `hygiene-check`/`hygiene-clean` → `check`/`clean`, and the `commands/hygiene.md` dispatcher removed in favor of two new skills (`status`, `sweep`), aligning with the `os-vault`/`os-orchestration` pattern of no `commands/` directory (2026-07-03). - -**Global os-adr plugin** — `cc-os/plugins/os-adr/` (git-tracked, 2026-07-03); symlinked into `~/.claude/plugins/os-adr` -- Purpose: Architecture Decision Records, consistently in every project — one customized-Nygard template (frontmatter `id`/`date`/`status`/`supersedes`/`superseded-by`/`affected-paths`/`affected-components`; Context/Decision/Consequences/Alternatives-rejected sections), `docs/adr/NNNN-kebab-title.md` one-file-per-decision + fully-regenerated `docs/adr/README.md` index. Built from `docs/adr-system/` (requirements `04`, PRD `05`, eval sketches `06`) via OpenSpec change `add-os-adr-plugin`. -- Core: Ruby OO library `lib/adr/` (`Record`, `Repository`, `Index`, `Template`, `Detector`, `Migrator`, `MigrationReport`, `Finder`; Sandi Metz style, injected paths) behind thin CLIs in `bin/` (`adr-new`, `adr-init`, `adr-detect`, `adr-migrate`, `adr-find`). Tests: `ruby tests/all.rb` (47) + `python3 tests/hook_test.py` (15), all model-free. Behavioral invariants in `invariants.md`. -- Hook: **moved to os-status (2026-07-06)** — the SessionStart existence check now runs as the `adr-system-present` check in the os-status plugin (`hooks.json` here is empty; `hooks/session_start.py` stays as the wording source of record, byte-copied into os-status). Behavior unchanged: present → near-zero-token usage note naming `/os-adr:create` + `/os-adr:find`; absent → once-per-day init/migrate suggestion, silenced permanently by `.os-adr/suppress` (still honored); silent outside git projects. -- Skills (verb-first, no `commands/`): `create` (one-invocation templated+numbered+indexed ADR with mechanical supersession), `init`, `migrate` (non-destructive: detect surveyed shapes → mechanical heuristic fill → LLM fills only manifest-listed interpretive fields via `--apply-fills` → `migration_confidence` frontmatter + `docs/adr/migration-report.md`; old-system deletion only as an explicit separate user-approved step), `find` (deterministic-first retrieval: affected-paths/components match → Accepted-only filter → one-hop Graphify query-path expansion with graceful degradation → AI judgment over the candidate set only). -- Migration pilot passed 2026-07-03 (sandboxed project copies; gate threshold 25% low-confidence flags): viking-warrior-training-log 0%, delta-refinery 0%, llf-schema 8.3%. Retrieval verified end-to-end against the real llf-schema project graph. Resolved: ADRs need no Graphify tag/edge convention — retrieval layer 3 expands the *query* paths via graph-node `source_file`s. -- Eval A harness (2026-07-03, ADR-021): `plugins/os-adr/eval/` — prompted skill-execution eval across model tiers (haiku/sonnet). Two fixtures (generated with the plugin's own CLIs), six scenarios (S1 create, S2 create+supersede, S3 find/conflict, S4 find/distractor, S5 init, S6 migrate+fills), deterministic Ruby checker (`eval/bin/check`, structural invariants, TSV mode for autoresearch), sandbox + headless-runner scripts, runner-prompt template. Primary run mode: in-session Agent-tool subagents with pinned `model:` (cheaper than `claude -p`); optimized via the `/autoresearch` Classic loop over SKILL.md *wording only* (checker/fixtures/scenarios frozen during a loop). Procedure: `plugins/os-adr/eval/README.md`. Self-tested both directions; grid run 2026-07-03, all scenarios × both tiers passing. Distinct from the held-out Eval B (unprompted behavior) — do not conflate. -- Eval B harness (2026-07-03, OpenSpec change `add-os-adr-eval-b-harness`): `plugins/os-adr/eval-b/` — held-out unprompted-behavior eval (requirements 4–5). 7 scenarios (W1–W3 write-trigger, R1–R4 retrieval) authored from the frozen shapes in `docs/adr-system/06-eval-scenarios.md`; dedicated Ruby webhook-relay fixture with a 6-ADR history (Superseded pair + near-miss distractors, generated via the plugin's own CLIs); R4's one-hop graph reach uses a real `graphify update` AST build (model-free, rebuilt via `eval-b/bin/build-fixture-graph`, never committed). **Headless-only runner** (`eval-b/bin/run` — fresh `claude -p` per rep, cwd = sandbox, so the real SessionStart hook fires; in-session subagents are invalid here, unlike Eval A) and a two-axis deterministic-first checker (`eval-b/bin/check`): axis (a) unprompted consultation, mechanical from transcript tool_use blocks; axis (b) correct-ADR citation (R1–R4) or new-ADR-file with a narrow frozen-rubric haiku judge fallback (W1–W3, `judge-rubric.md`, stubbable via `ADR_EVAL_B_JUDGE_CMD`). `R4-nograph` is the graph-degradation variant (expected FAIL). Self-tested both directions model-free via `eval-b/bin/self-test`. **Scenario Task blocks are held-out — never run them informally.** Procedure: `plugins/os-adr/eval-b/README.md`. Grid run 2026-07-03 (1 rep/cell): haiku 0/8 PASS (never unprompted-consults the ADR system in any scenario — a real gap, not a harness defect); sonnet 5/8 PASS (fails W3 — doesn't propose recording the decision; fails R1 — misses the direct-conflict retrieval). `R4-nograph` FAILed on both tiers as expected (degradation check, only meaningful paired with an R4 PASS — sonnet has one, haiku doesn't). Full results + observations (prompting-issue hypothesis, open question on whether in-session subagents could ever validly substitute for part of this measurement) written to the vault: [[os-adr-eval-b-grid-results-and-observations]]. -- **Initial iteration complete (2026-07-03):** plugin build, migration pilot, Eval A grid (clean pass), and Eval B grid (baseline captured above) are all done — this closes the first pass on os-adr. Follow-on work is deliberately deferred to future sessions, not in-flight. -- **Wording experiment complete (2026-07-04):** the follow-up `/autoresearch` loop over Eval B wording ran (5 iterations, checker/fixtures/scenarios/rubric frozen; fixture CLAUDE.md declared a wording surface upfront) and closed the gap — final full grid **sonnet 8/8, haiku 7/8** (from 5/8 / 0/8 baseline; haiku's one miss is a W3 axis-b judge-boundary flicker). Winning wording shipped in `hooks/session_start.py` (PRESENT_NOTE), find/create SKILL.mds, and `eval-b/fixture/project/CLAUDE.md` (new — the trigger-phrased "Architecture decisions" section, the template for real-project adoption). Confirmed mechanisms: trigger-conditioned when→then phrasing beats inventory phrasing on both tiers; each rule must live where its precondition is visible (the reversal→supersede rule in the find skill's act-on-findings step fixed W3); lower tiers need *mechanical* triggers ("before your first edit to any existing file → run `/os-adr:find` on those paths; additions count") — semantic triggers ("architecture-level choice") only reach sonnet. Open: channel ablation never run (hook vs CLAUDE.md redundancy unknown); R4-nograph now passes both tiers, so the graph-degradation check no longer differentiates. Full hypothesis→result mapping: vault note [[os-adr-eval-b-wording-experiment-hypotheses]]. **Before designing or running any autoresearch eval, Read `~/Documents/SecondBrain/howto/running-autoresearch-skill-evals.md`.** -- **Eval B W3 stability check (2026-07-06):** 7 headless reps of haiku × W3 (all reps counted, including two the runner initially excluded, to avoid peeking bias): axis (a) consultation 7/7 PASS; axis (b) recording-offer ~5/7 (~75–85%) — one FAIL plausibly infra, one a genuine behavioral miss (consulted, then asked a clarifying question instead of unconditionally proposing the superseding ADR; the conditional-phrasing failure mode iteration 3 fixed on sonnet, improved but not eliminated on haiku). Verdict: intermittent flicker, not a hard gap — grid claim stays **haiku 7/8**, W3 characterized as an ~1-in-4/5 axis-b miss. Haiku cleared for daily use with that caveat. Recorded in the vault good-enough gate: [[os-adr-eval-b-wording-experiment-hypotheses]]. -- **Eval C harness built, NOT yet run (2026-07-06):** `plugins/os-adr/eval-c/` — ambiguity-ladder DISCRIMINATION eval (held-out; Eval B is contaminated by the wording loop). 3 levels (explicit → moderate → conceptual framing) × paired positive/negative scenarios; run-set (6, job-execution domain) + frozen reserve-set (6, notifications domain — becomes the measurement set if anyone ever tunes wording against the run-set). New Python fixture (`taskq` async job queue, 6-ADR history generated via the plugin's own CLIs, trigger-phrased CLAUDE.md section copied from eval-b). Scoring: positives = both axes; negatives FAIL **only on unneeded ADR creation** — consultation and truthful ADR citation are informational (`A:yes/no`, `cited-adr:yes/no`), never FAILs, because the shipped wording endorses cheap finds and the negatives sit deliberately in ADR-covered domains (two earlier negative designs that punished instruction-compliant behavior were caught in review and redesigned/fixed). Model-free `bin/self-test` green, including a truthful-citation-must-PASS guard. **Scenario Task blocks are held-out — never run informally.** Methodology: vault notes [[eval-methodology-ladder]] (per-level pass bars, reserve discipline) and [[eval-methodology-irl-feedback-loop]] (post-rollout session-audit backlog); vault also gained an `eval-results` note type + `_templates/eval-results.md` (2026-07-06). -- **Eval C frozen grid run (2026-07-06):** run-set × {sonnet, haiku} × 3 reps/cell (36 headless reps, all counted) — **12/12 cells PASS at ≥2/3 majority; haiku 18/18 reps, sonnet 17/18**. Zero over-trigger ADR creation on any negative at any level; every positive pass was via governing-ADR citation (16) or judge-recognized proposal (1), never duplicate creation. Sonnet's one FAIL rep (P1-L1) is a final-message-citation boundary artifact (implemented the ADR-0002 policy, cited only 0005), not a behavioral miss. Two harness defects fixed on the first live rep, BEFORE the grid, with Task blocks/fixtures/rubric untouched: (1) `bin/run` pipefail bug aborted a cell's reps after the first FAIL; (2) positive axis-b required creation/proposal, contradicting the README's own "consulting and/or recording is correct" — the run-set positives sit in *already-decided* territory, so find→cite→comply is correct; fixed via a mechanical `cited-governing` PASS branch + two new self-test guards (`B-via:` now recorded in TSV). Verdict: the Eval B wording **generalizes** (new fixture/language/domain, decreasing cue explicitness, paired negatives) — grid saturated, ceiling not found; next signal is production (rollout + IRL session audits), not another lab rung. Run-set is now the baseline; wording tuning against it moves measurement to the reserve-set. Full results: vault note [[os-adr-eval-c-frozen-grid-results]]. -- **Remaining (locked rollout order):** real-project migration/adoption one at a time via `/os-adr:migrate` — pilot projects first, then the cc-os retrofit (its 19-ADR monolithic file was deliberately excluded from the pilot), then wider. When onboarding real projects, add the trigger-phrased "Architecture decisions" CLAUDE.md section (copy from `eval-b/fixture/project/CLAUDE.md`; candidate: emit it from `/os-adr:init`/`migrate`). -- **Resolved (2026-07-04):** os-adr's skills failed to register on first install due to stale plugin caches — the cache that was installed on 2026-07-03 17:21:48 was missing `hooks/hooks.json`, `bin/adr-detect`, `bin/adr-find`, `bin/adr-migrate`. Root cause: unknown, but the caches were restored via plugin uninstall/reinstall. A parallel issue affected os-doc-hygiene (cache retained deleted `commands/` directory). Fixed by manually refreshing both caches, then created `bin/refresh-plugins` automation to prevent future drift — see "Editing a local plugin (cache refresh)" subsection for refresh procedure. Investigation result: stale caches (not manifest-naming issues) were the cause; slash command registration works correctly once caches are fresh. - -**Graphify** — v0.8.31 at `/home/jared/.local/bin/graphify` -- Vault graph: `~/Documents/SecondBrain/graphify-out/` — disposable, structural index rebuilt by SessionStart hook; handles relational/graph queries. Distinct from vault-index.json (planned, Plan B Phase 1): a flat `{tag: [{path, title, summary}]}` lookup rebuilt at SessionEnd, queried by the `/memory-find` skill for fast tag-based SB note discovery. -- Project graph: `/graphify-out/` — same pattern; gitignore it in each project repo -- Vault conventions: `~/Documents/SecondBrain/vault-conventions.md` — frontmatter contract + tag taxonomy (canonical name decided 2026-06-09; formerly referred to as CONVENTIONS.md) - -**memsearch** — v0.4.6 via Claude Code plugin marketplace (`memsearch@memsearch-plugins`) -- Hooks: Stop, SessionStart, UserPromptSubmit, SessionEnd (ship with plugin) -- Memory store: `~/.memsearch/memory/YYYY-MM-DD.md` daily files (global, cross-project, all clients — one store by design) -- Index: `~/.memsearch/milvus.db` (Milvus Lite, local); embeddings via ONNX bge-m3 -- Config: `MEMSEARCH_DIR=~/.memsearch` in `~/.zshrc` for global scope -- Skills: `/memory-recall`, `/memory-config` (ship with plugin) -- Git sync: `~/.memsearch` is a dedicated private Forgejo repo (`forgejo.swansoncloud.com/jared/memsearch`); whitelist `.gitignore` commits only `memory/*.md` (excludes rebuildable `milvus.db`, model, config); auto-commit+push wired into the cc-os memory plugin's own `memsearch_sync.py` SessionEnd hook (relocated from `session-end.sh` by ADR-016, 2026-06-12; behavior preserved — see ADR-015) - -**Obsidian vault git sync** — `~/Documents/SecondBrain` (2026-06-15) -- Remote: `ssh://git@forgejo.swansoncloud.com:2222/jared/SecondBrain.git` (private Forgejo; web: `https://forgejo.swansoncloud.com/jared/SecondBrain`) -- 52 files tracked (notes, journal, templates, vault-conventions.md, CLAUDE.md, .obsidian config); `graphify-out/` excluded via `.gitignore` -- Auto-commit+push via `vault_sync.py` SessionEnd hook in cc-os memory plugin (mirrors `memsearch_sync.py`; push-only — SessionStart pull is optional future item for multi-machine) - -**Remaining optional items:** SessionStart vault pull (multi-machine sync; push-only is the current design); additional project onboarding (one at a time, per ADR-013); bulk vault migration. All required build steps complete as of 2026-06-15. - -**Plugin renamed `memory` → `os-vault` (2026-07-03):** directory renamed in git, `~/.claude/plugins/os-vault` symlink and `settings.json`'s `enabledPlugins` updated — but the `local-plugins` marketplace manifest (`~/.claude/plugins/.claude-plugin/marketplace.json`) still declared the old `memory`/`./memory` entry, and `installed_plugins.json` still had a cached `memory@local-plugins` install record. Skills (and therefore slash commands) never registered under the new name as a result; hooks kept working because they're wired by absolute path in `settings.json`, independent of plugin resolution. Fixed by editing the marketplace manifest to `"os-vault"`/`"./os-vault"`, then `claude plugin marketplace update local-plugins`, `claude plugin install os-vault@local-plugins`, `claude plugin uninstall memory@local-plugins`. See ADR-018 and the "Renaming or moving a local plugin" procedure below. - -### Renaming or moving a local plugin - -Renaming a plugin directory under a directory-source marketplace (like `local-plugins`, whose -source is `~/.claude/plugins` itself) requires updating **three** places, not just the directory -and `settings.json`: - -1. **Plugin directory / symlink** — the actual files (or symlink target, for plugins that live in - a git repo like `cc-os/plugins/os-vault/`). -2. **Marketplace manifest** — `~/.claude/plugins/.claude-plugin/marketplace.json` (for - `local-plugins`) lists each plugin's `name` and `source` path explicitly; this does **not** - auto-discover from the directory listing. Update the entry to match the new name/path. -3. **`settings.json` `enabledPlugins`** — the `@` key must match the manifest - entry from step 2. - -Then refresh the plugin manager's state so it re-resolves the marketplace and drops the stale -install record: - -```bash -claude plugin marketplace update # re-validates the manifest -claude plugin install @ # creates the new install record -claude plugin uninstall @ # drops the stale one -claude plugin list # verify: new name enabled, old name gone -claude plugin details @ # verify skills/agents/hooks resolved -``` - -Skipping step 2 is the failure mode to watch for: hooks (wired by absolute path in -`settings.json`) keep working, masking the fact that skills/slash-commands silently stopped -registering under the plugin's new name. - -### Editing a local plugin (cache refresh) - -Plugin source edits (including SKILL.md wording, hook scripts, or CLI code) do not reach running -sessions until the cache is refreshed. Claude Code caches plugins when installed, copying them -to `~/.claude/plugins/cache////`. To refresh a stale cache after -source edits: - -```bash -bin/refresh-plugins [marketplace-names...] # refresh all or specific local marketplaces -``` - -This uninstalls and reinstalls each plugin, forcing the cache to repopulate. See `bin/refresh-plugins --help` for usage. After refreshing, `claude plugin details @` should show updated skill descriptions and hook counts. Directory-source marketplaces (e.g., `local-plugins` sourced from `~/.claude/plugins`, `cc-plugins` sourced from `~/dev/cc-plugins`) are the only ones that need manual refresh — public marketplaces auto-update. +**Eval discipline (applies to every harness above):** scenario Task blocks are held-out — +never run them informally; reserve sets are never even read informally. Before designing or +running any autoresearch eval, Read +`~/Documents/SecondBrain/howto/running-autoresearch-skill-evals.md`. Wording loops for +os-adr Eval B, os-vault WS2, and os-orchestration WS4 are complete and shipped +(2026-07-04/07/08); their run-sets AND reserves are contaminated for future wording tuning — +next signal is production IRL session audits. ## Issue tracking -Issues (created via `/to-issues`) live on self-hosted Forgejo (`jared/cc-os`), queried with the `tea` CLI — not GitHub/`gh`. See `docs/issue-workflow.md` for token setup, listing, implementing, and closing issues. +Issues (created via `/to-issues`) live on self-hosted Forgejo (`jared/cc-os`), queried with +the `tea` CLI — not GitHub/`gh`. See `docs/issue-workflow.md`. ## OpenSpec workflow -Changes are managed spec-driven via OpenSpec. Use the matching skills rather than editing spec -files by hand: `openspec-explore` (think through an idea), `openspec-propose` (create a change -with design/specs/tasks), `openspec-apply-change` (implement tasks), `openspec-verify-change` -(validate before archiving), `openspec-archive-change`. Slash commands mirror these under -`/opsx:*`. Live changes live in `openspec/changes/`, completed ones in -`openspec/changes/archive/`, stable specs in `openspec/specs/`. - -`openspec/config.yaml` uses `schema: spec-driven`; its `context` block is currently empty — -project context for OpenSpec artifacts comes from `docs/` and this file. +Changes are managed spec-driven via OpenSpec. Use the matching skills rather than editing +spec files by hand: `openspec-explore`, `openspec-propose`, `openspec-apply-change`, +`openspec-verify-change`, `openspec-archive-change` (slash commands under `/opsx:*`). Live +changes in `openspec/changes/`, completed in `openspec/changes/archive/`, stable specs in +`openspec/specs/`. `openspec/config.yaml` uses `schema: spec-driven`; project context for +OpenSpec artifacts comes from `docs/` and this file. ## Conventions specific to this repo - **Provenance discipline:** when writing about Graphify or anything sourced from the interview transcripts, keep the inline source tags and never promote an `[interview]` / `[unverified claim]` to fact without checking a primary source (the GitHub repo is the - authority; it was anchored to Graphify **v0.8.30**). -- **Dates are absolute** (e.g. `2026-06-03`), and design docs carry a `_Last updated:_` / - status line — keep these current when editing. -- **Decisions live in ADRs.** Don't silently reverse a locked decision; add or amend an ADR in - `03-architecture-decisions.md` with the reasoning. -- The package on PyPI is `graphifyy` (double-y) but the command is `graphify`. -- **Keep this file current:** When a build step completes, (a) mark it done in `docs/memory-system/04-build-plan.md`, (b) add or update a component pointer in the Implemented Components section above, and (c) update the current design paragraph if the design changed. This file is the AI's orientation entry point — accuracy matters more than brevity. -- **Plugin and skill naming:** Before naming ANY new cc-os plugin, skill, or slash command, Read - `~/Documents/SecondBrain/cc-os-plugin-skill-naming-convention.md` and follow it — do NOT invent - a new convention. Rules in brief: plugins are `os-[domain]`; skills are verb-first kebab-case - (`query`, `onboard-project`), invoked as `/os-[domain]:[verb]`; no `commands/` dispatcher - directories; **never set a `name:` field in SKILL.md frontmatter** — the directory name is the - skill name, and an explicit `name:` collapses the slash command to a bare unnamespaced form - (`/find` instead of `/os-adr:find`; found and fixed 2026-07-04). After editing any SKILL.md, - run `bin/refresh-plugins` (installs cache plugin files; source edits don't reach sessions until - refreshed). Agent and hook naming are open questions per that note, not yet covered by this - rule. - -**Session orchestration behavior** — Provided by the global `os-orchestration` plugin (see Implemented Components below). This repo no longer carries a local orchestration override; it follows the plugin's default behavior like every other project. + authority; anchored to Graphify v0.8.30). +- **Dates are absolute** (e.g. `2026-06-03`); design docs carry a `_Last updated:_` / status + line — keep these current when editing. +- **Decisions live in ADRs.** Don't silently reverse a locked decision; add or amend an ADR + in `docs/memory-system/03-architecture-decisions.md` with the reasoning. +- **Keep records current:** when a build step completes, (a) mark it done in + `docs/memory-system/04-build-plan.md`, (b) record it in + `docs/implementation-status.md` (timeline entry + component pointer), and (c) update the + design paragraph in THIS file only if the design itself changed. Never append status + history to this file — it goes in implementation-status.md. +- **Plugin and skill naming:** before naming ANY new cc-os plugin, skill, or slash command, + Read `~/Documents/SecondBrain/cc-os-plugin-skill-naming-convention.md` and follow it. In + brief: plugins are `os-[domain]`; skills are verb-first kebab-case, invoked as + `/os-[domain]:[verb]`; no `commands/` dispatcher directories; **never set a `name:` field + in SKILL.md frontmatter** (it collapses the slash command to a bare unnamespaced form). +- **After editing any plugin source** (SKILL.md, hooks, CLIs), run `bin/refresh-plugins` — + source edits don't reach sessions until the cache is refreshed. Rename/move procedure and + cache details: docs/implementation-status.md → "Operational procedures". diff --git a/docs/implementation-status.md b/docs/implementation-status.md new file mode 100644 index 0000000..29d5f3e --- /dev/null +++ b/docs/implementation-status.md @@ -0,0 +1,490 @@ +# cc-os implementation status & changelog + +_Last updated: 2026-07-08. This is the append-only implementation-status record moved out of +`CLAUDE.md` (2026-07-08, task B1 of +[system-prompt-profiles-tasks.md](plans/system-prompt-profiles-tasks.md)). CLAUDE.md keeps +orientation only; when a build step completes, record it HERE (plus +`docs/memory-system/04-build-plan.md`), and touch CLAUDE.md only if the current design or +conventions changed._ + +## Status timeline + +**Recent pivot (2026-06-03):** Graphify **replaces** the originally-planned Ruby/SQLite +tag-index CLI and also covers the deferred QMD semantic layer. `04-build-plan.md` and +`06-graphify-evaluation.md` reflect this; if an older doc still describes the Ruby CLI, defer +to those two and fix the stale doc. + +**Decisions locked (2026-06-04):** Six-facet tag taxonomy + `scope/` (ADR-011); reuse +`~/Documents/SecondBrain` vault rather than creating a new one (ADR-012); build-first / +migrate-incrementally — build full system against a fixture set first, defer bulk vault +migration to last, onboard projects one at a time (ADR-013). + +**Empirical finding locked (2026-06-05):** Graphify is a structure extractor, not a topic +clusterer — no emergent hub nodes appear even at `--mode deep`; hub notes + wikilinks must be +author-provided during migration (not deferred). Migration scaffolding is now a first-class +deliverable. Open question: do facet tags create graph edges? (ADR-014; findings: +`docs/memory-system/07-graph-connectivity-findings.md`). + +**Implementation status (2026-06-09):** The global Claude Code plugin is live +(`~/.claude/plugins/os-vault/`). Steps 1, 2a, 2b, 3, and 6 of the build plan are complete +(including the `onboard-project` skill, previously TODO under Step 6/Part D). Step 1 +completed 2026-06-09: vault-conventions.md exists, all 6 fixture notes seeded, and 14 +Graphify handbook + memory-system design notes migrated to the vault as migration +scaffolding. Step 4 (memsearch) completed 2026-06-09: plugin installed via marketplace, +MEMSEARCH_DIR set global (~/.memsearch), Stop hook verified producing daily memory files, +search confirmed working. Step 5a (memsearch episodic git sync) completed 2026-06-09: +dedicated private Forgejo repo (`forgejo.swansoncloud.com/jared/memsearch`), whitelist +`.gitignore` (memory/*.md only), auto-commit+push via cc-os `session-end.sh` hook (ADR-015; +relocated to `memsearch_sync.py` by ADR-016, 2026-06-12). Step 5b (Obsidian vault → VPS +sync) remained at that date. + +**Decision (2026-06-09):** Single global memsearch store for ALL clients — cross-client +commingling is the accepted, intended design (ADR-015 resolved). One private Forgejo repo +(`forgejo.swansoncloud.com/jared/memsearch`), not per-client repos. Forward direction: +minimize/eliminate dedicated per-client working directories; work projects locally or use a +single general `clients/` dir for non-local client work — memory is captured globally +regardless of cwd. No `clients/` directory structure designed yet; open item. + +**Implementation status (2026-06-12):** os-vault plugin source moved into git at +`cc-os/plugins/os-vault/` and bash hooks ported to Python (deep-module architecture: shared +`config.py`, `hook_io.py`, `session_state.py`; thin entry-point scripts). Cutover via symlink +`~/.claude/plugins/os-vault → cc-os/plugins/os-vault/` and settings.json hook rewrite. +memsearch sync split into dedicated `memsearch_sync.py` SessionEnd hook (relocation of +ADR-015 behavior, not reversal). Fresh-session test passed 2026-06-12. See ADR-016. + +**Implementation status (2026-06-15):** Step 5b done and automated — vault +(`~/Documents/SecondBrain`) initialized as git repo, pushed to private Forgejo +(`ssh://git@forgejo.swansoncloud.com:2222/jared/SecondBrain.git`); 52 files, git chosen over +Syncthing. Auto-commit+push wired via `vault_sync.py` SessionEnd hook (third SessionEnd +hook, runs after `session_end.py` so the daily journal note is included; push-only — +SessionStart pull is an optional future item for multi-machine). Step 2e pilot done — +llf-schema (`/home/jared/dev/llf-schema`, PHP 8.2 WordPress plugin) onboarded via +`/os-vault:onboard-project` skill: 605 nodes / 930 edges / 52 communities; Ollama doc pass +lossy on WordPress docs but AST pass solid. Step 2d closed — live vault `graphify-out/` is +the baseline; fixture-only build superseded. + +**Implementation status (2026-06-17):** `/os-vault:onboard-project` skill now uses +assessment-first onboarding: surveys the repo structure, classifies exclude candidates by +TYPE (11 categories: fetched deps, build output, caches, VCS internals, editor/AI-tooling +dirs, lockfiles, coverage/logs, bulk data, binaries, secrets, graphify-out/ — illustrative +names, not fixed templates), generates a per-project `.graphifyignore`, confirms with the +user, then extracts the graph using `qwen25-coder-7b-16k` (config.yaml `ollama_model`; +16k-context build of qwen2.5-coder:7b — larger context window cuts chunk count per doc, the +main speed lever). Fixes issue where repos with large dependency trees routed non-code files +through the Ollama doc pass (see ADR-017). + +**Implementation status (2026-06-30):** Phase 1 of the SecondBrain Content Plan complete +(`docs/memory-system/09-sb-content-plan.md`, issues #1–#6). +`~/Documents/SecondBrain/vault-conventions.md` reconciled to a single authoritative typed +frontmatter schema (removed the contradictory block; canonical filenames are slug-only, +`source:` is a frontmatter field not a tag, `scope` is a field not a tag). New +`/os-vault:design-template` skill added +(`cc-os/plugins/os-vault/skills/design-template/SKILL.md`) — routes between template-design +(4-step process + injection-economics filter) and new-type creation (9-step lifecycle). +Three vault note templates created in `~/Documents/SecondBrain/_templates/` (`howto.md`, +`convention.md`, `reference.md`; `reference` carries a four-subtype variant selector — +pattern/framework, api-integration, role-definitions, design-rules), each dogfooded against +a real note. Four proof-of-concept vault notes patched (cookbook subtype → +pattern/framework, glossary de-duped to a wikilink, design-mode scope → project, tags + +`last_reviewed` added). Phase 2 (issue #7) remains open as the steady-state +migration/onboarding epic (no code). See `docs/memory-system/09-sb-content-plan.md`. + +**Plugin renamed `memory` → `os-vault` (2026-07-03):** directory renamed in git, +`~/.claude/plugins/os-vault` symlink and `settings.json`'s `enabledPlugins` updated — but +the `local-plugins` marketplace manifest +(`~/.claude/plugins/.claude-plugin/marketplace.json`) still declared the old +`memory`/`./memory` entry, and `installed_plugins.json` still had a cached +`memory@local-plugins` install record. Skills (and therefore slash commands) never +registered under the new name as a result; hooks kept working because they're wired by +absolute path in `settings.json`, independent of plugin resolution. Fixed by editing the +marketplace manifest to `"os-vault"`/`"./os-vault"`, then +`claude plugin marketplace update local-plugins`, +`claude plugin install os-vault@local-plugins`, +`claude plugin uninstall memory@local-plugins`. See ADR-018 and the "Renaming or moving a +local plugin" procedure below. + +**Resolved (2026-07-04):** os-adr's skills failed to register on first install due to stale +plugin caches — the cache that was installed on 2026-07-03 17:21:48 was missing +`hooks/hooks.json`, `bin/adr-detect`, `bin/adr-find`, `bin/adr-migrate`. Root cause: +unknown, but the caches were restored via plugin uninstall/reinstall. A parallel issue +affected os-doc-hygiene (cache retained deleted `commands/` directory). Fixed by manually +refreshing both caches, then created `bin/refresh-plugins` automation to prevent future +drift — see "Editing a local plugin (cache refresh)" below. Investigation result: stale +caches (not manifest-naming issues) were the cause; slash command registration works +correctly once caches are fresh. + +**Remaining optional items:** SessionStart vault pull (multi-machine sync; push-only is the +current design); additional project onboarding (one at a time, per ADR-013); bulk vault +migration. All required build steps complete as of 2026-06-15. + +## Implemented components + +**Global os-vault plugin** — `cc-os/plugins/os-vault/` (git-tracked, 2026-06-12); symlinked +into `~/.claude/plugins/os-vault` +- Hooks: `hooks/` — `session_start.py`, `session_context.py` (project graph path only), + `post_tool_use_write.py`, `session_end.py` (vault journal), `memsearch_sync.py` (second + SessionEnd hook; memsearch auto-commit+push, 30s timeout), `vault_sync.py` (third + SessionEnd hook; vault auto-commit+push to forgejo.swansoncloud.com/jared/SecondBrain, 30s + timeout; mirrors memsearch_sync.py) +- Shared modules: `config.py` (load_config → frozen Config dataclass), `hook_io.py` + (read_input → HookInput dataclass), `session_state.py` (record_touch/read_touches; + encapsulates `/tmp/claude-vault-touched-$SESSION_ID` contract) +- Skills: `skills/` — query, write, reorganize, onboard-project (assessment-first + onboarding: surveys repo, classifies excludes by type (11 categories), writes per-project + `.graphifyignore`, confirms with user, then extracts using `qwen25-coder-7b-16k` — per + ADR-017), design-template (routes template-design ↔ new-type-creation: 4-step + template-design process + injection-economics filter, 9-step new-type lifecycle; backs + Phase 1 of the SB Content Plan — see `docs/memory-system/09-sb-content-plan.md`) +- Config: `config.yaml` — vault path, Ollama model (qwen25-coder-7b-16k), env vars +- Hook wiring: `~/.claude/settings.json` (hook entries invoke `/usr/bin/python3` with + absolute paths into cc-os) +- Isolation seam (2026-07-06, WS2): `OS_VAULT_PATH` env var overrides `vault_path` for the + write skill and every hook via `config.load_config()`; `OS_VAULT_SKIP_REBUILD` suppresses + the SessionStart graphify rebuild. Added for the eval harness; zero production impact when + unset. +- Write-behavior eval (2026-07-06, WS2): `plugins/os-vault/eval/` — held-out unprompted + vault-write discrimination eval (Eval B/C playbook: ambiguity ladder L1 explicit → L3 + conceptual, paired positives/negatives, 6 run-set scenarios on a new `reportgen` Ruby + fixture + 6 frozen reserve twins in a different knowledge domain, isolated sandbox vault, + headless-only runner, deterministic-first Ruby checker with narrow frozen offer-detection + judge fallback stubbable via `OS_VAULT_EVAL_JUDGE_CMD`, model-free `bin/self-test` 21/21). + **Scenario Task blocks are held-out — never run informally; the reserve set is never even + read informally.** Procedure: `plugins/os-vault/eval/README.md`. +- Untuned baseline grid (2026-07-06, run-set × sonnet/haiku × 3 reps + 1 counted canary, 37 + reps, zero harness errors): **positives 1/19, negatives 18/18**. Headline: at L1 (explicit + cue) both tiers persist the knowledge every rep but route it to Claude Code's built-in + auto-memory (`~/.claude/projects//memory/`) instead of the vault; L2/L3 mostly don't + persist at all; zero over-triggering anywhere. TSV: + `eval/results/2026-07-06-baseline-grid.tsv`; full analysis: vault note + [[os-vault-write-eval-baseline-grid-results]]. +- WS2 wording loop COMPLETE (2026-07-07, 4 iterations + confirm + held-out reserve; run-set + AND reserve now contaminated): shipped destination-ladder write SKILL.md + SessionStart + USAGE_NOTE (mechanical triggers + before-final-reply checkpoint), mechanical vault-root + resolution (`${OS_VAULT_PATH:-$HOME/Documents/SecondBrain}` — fixed real-vault breaches + found on iteration 1), fixture CLAUDE.md "Cross-project memory" section (**the + real-project adoption template**, required to reach haiku at L2/L3; includes ADR-vs-vault + disambiguation), candidates 2–4 unscored, post-reserve YAML-colon quoting rule. Confirm + grid sonnet 18/18 / haiku 16/18; reserve (held-out) positives 8/18 strict / trigger-axis + 13/18, negatives 17/18 (vs baseline 1/19, 18/18). Residual gaps: YAML-hostile titles, + haiku L2 no-trigger, generalization-laundering over-trigger. One checker false-positive + fixed+rescored mid-loop (breach regex vs `2>/dev/null`); negative-criterion redesign + pre-registered. TSVs: `eval/results/2026-07-07-wording-*.tsv`; loop log: + `autoresearch/classic-260707-1253/loop-log.md`; analysis: vault note + [[os-vault-write-eval-wording-loop-results]]. Next signal is production IRL audits. + +**Global os-orchestration plugin** — `cc-os/plugins/os-orchestration/` (git-tracked, +2026-07-03); symlinked into `~/.claude/plugins/os-orchestration` +- Hooks: `hooks/` — `inject.py` (injects `ORCHESTRATION.md` as additionalContext to all + sessions) +- Behavior: SessionStart hook injects an `ORCHESTRATION.md` markdown doc (hardcoded; lives + in plugin source) as additionalContext, carrying the canonical global + session-orchestration rule set for Claude Code across all projects. +- Migration: migrated from a standalone repo (`~/dev/cc-plugins/orchestration/`, 2026-07-03) + and integrated into cc-os. Supersedes the per-project copy-pasted orchestration text + blocks that previously existed in individual project CLAUDE.md files (including a stricter + local override that cc-os had carried — now removed; see ADR-019). +- Session audit (WS1, 2026-07-06): 10 stratified real sessions audited via + `audit/bin/extract` (Ruby fact-sheet extractor, dual-use as future eval checker) + + parallel sonnet auditors + verified synthesis. Findings + E1–E4 eval-scenario backlog: + `docs/orchestration-audit/2026-07-06-findings.md`. Headlines: model param was silently + overridden to haiku on all 23 verified spawns — root cause found post-audit: + `CLAUDE_CODE_SUBAGENT_MODEL=haiku` in `~/.claude/settings.json` env block (set by an + earlier session as a cost measure; removed 2026-07-06; the Fable-5 correlation was + coincidental timing). Omitted model param inherits the main-loop model (opus/sonnet) — + misses cluster in pre-rollout and ops sessions; no over-sharing/full-dump failures + anywhere. +- Wording fixes shipped (2026-07-06, post-audit): ORCHESTRATION.md rewritten with + trigger-conditioned when→then phrasing for the three verified clusters — explicit + `model:` before every Agent call (cluster 2), subagent model self-report on sonnet/opus + spawns (cluster 1 — the E1 canary proved `resolvedModel` is NOT model-visible in the + launch result; self-report is the only observable channel), don't re-cover your own ground + before delegating (cluster 3). Delegation thresholds unchanged. Cache refreshed via + `bin/refresh-plugins`. Deterministic env-override detection (warn on + `CLAUDE_CODE_SUBAGENT_MODEL`) is delegated to WS3's status-check design. +- Eval harness E1–E3 (2026-07-06): `plugins/os-orchestration/eval/` — scripted headless + orchestration-behavior eval scored by the audit extractor (extended with per-segment read + targets). 6 run-set scenarios (E1P/N downgrade detection with + `CLAUDE_CODE_SUBAGENT_MODEL=haiku` runner stub; E2P/N explicit-model fan-out vs + no-delegation-on-trivial; E3P/N delegate-investigation vs orienting-reads-correct) + 6 + frozen reserve twins; new Node.js "relaystation" fixture with deterministic sandbox-time + log generation; deterministic-first checker (`eval/bin/check`, narrow frozen haiku-judge + fallback for E1 language axes only, stubbable via `ORCH_EVAL_JUDGE_CMD`); model-free + `bin/self-test` 21/21 green incl. shipped-instruction-compliant conformance transcripts. + FAIL axes map only to verified audit misses; tier choice informational. **Scenario Task + blocks are held-out — never run informally.** Procedure: + `plugins/os-orchestration/eval/README.md`. +- Baseline grid run (2026-07-06, run-set × sonnet/haiku × 3 reps + 2 counted canaries, all + counted, zero harness errors): negatives 18/18 PASS both tiers (zero over-delegation, zero + false downgrade claims); positives 1/18 on the delegate-at-all axis — but transcript + verification shows most positive FAILs are *superior direct strategies* (E2P sonnet: one + scripted bulk edit; E3P sonnet: surgical greps over 400KB logs), which the WS1 audit + itself scored as justified non-delegation, vs genuine miss shapes (E3P haiku: whole 112KB + logs into own context; E2P haiku: 20-call per-file grind). Conditional rules validated + where engaged: 14/14 spawns explicit-model; the one delegating E1P rep flagged the stubbed + downgrade via self-report. Criterion redesign for E2P/E3P A-axes pre-registered (anchor + thresholds independently, NOT to this grid) before any rescore/tuning; reserve stays + frozen. ORCHESTRATION.md gained a scripted-bulk-edit carve-out. TSV: + `eval/results/2026-07-06-baseline-grid.tsv`; full analysis: vault note + [[os-orchestration-eval-baseline-grid-results]]. +- WS4 economics loop COMPLETE + SHIPPED (2026-07-08; run-set AND reserve now contaminated + for future wording tuning): ORCHESTRATION.md v3 shipped — cost-asymmetry + + tier-conditional threshold opener, symmetric delegate-when/work-directly-when trigger + lists (replaces "delegate only when"), batching rule (~5–8 items per grouped spawn, + SendMessage reuse), async rule, Workflow effort dial; explicit-model/self-report/ + don't-re-cover-ground rules unchanged. Harness extended first (commit b226601): real token + econ axes — subagent transcripts persist at + `///subagents/agent-*.jsonl` with per-message usage blocks + (solves the mini-audit's instrumentation gap; vault note + [[claude-code-subagent-transcripts-and-token-accounting]]) — E5 batching pair + reserve + twins (authored by subagent, held-out content never entered the tuning session), fable + column, pre-registered E2P/E3P criterion redesign landed (scripted-direct PASS branch; + 74KB ingestion anchor + id-keyed root-cause concepts). Results: E5P 0/6 → majority-PASS + both tiers incl. reserve twin 5/5 (fable median mltok 43% of baseline, H3 met); negatives + clean everywhere (36/36 across confirm+reserve). Residual gaps: E1P sonnet delegate-at-all + (fixture too small — needs bigger fixture before meaningful), fable downgrade-flag ~1/4 + judge flicker. 15 baseline reps were session-limit-truncated and mechanically + excluded+replaced (limit-banner sweep now a standard step, promoted to the autoresearch + howto). TSVs: `eval/results/2026-07-08-*.tsv`; loop log: + `autoresearch/classic-260708-1039/`; vault note + [[os-orchestration-ws4-econ-wording-results]]. IRL re-audit of ~5 real Fable sessions + scheduled ~2026-07-22. + +**Global os-status plugin** — `cc-os/plugins/os-status/` (git-tracked, 2026-07-06); +symlinked into `~/.claude/plugins/os-status` +- Purpose: aggregated deterministic SessionStart status checks for the cc-os plugin family — + per-project artifacts (ADR system, vault hub note) and environment hazards (subagent model + env override). Built via OpenSpec change `add-os-status-plugin` (supersedes the WS3 plan + doc); decision record ADR-022. +- Architecture: checks are **in-process** plain Python functions in `hooks/checks.py` with + the uniform signature `check(ctx) -> CheckResult(status, message)` plus one registry entry + (no per-plugin subprocess protocol, no per-project code copies — both rejected in ADR-022 + after perspective review). Three states: `ok` (silent), `note` (near-zero-token + additionalContext line, never snoozed/suppressed), `warn` (aggregated into at most one + banner; once-per-day snooze + permanent `suppress-` markers in gitignored + per-project `.cc-os/` — state only, never code; `.cc-os/config` holds optional + `hub`/`vault_path` key=value overrides). +- Initial checks: `subagent-model-env-override` (env + `~/.claude/settings.json` `env` + block; runs outside git projects too — catches the WS1 Cluster 1 incident deterministically + at session start), `adr-system-present` (verbatim port of os-adr's hook behavior incl. + `.os-adr/suppress`), `vault-hub-note-present` (config slug first, else `type/hub` + + `project/` facet-tag scan of the vault; missing → warn naming `/os-vault:write`). +- Tests: `python3 tests/hook_test.py` (36, model-free) incl. byte-identity of + PRESENT_NOTE/ABSENT_NOTE against os-adr's source. Invariants in `invariants.md`. + Smoke-tested 2026-07-06 (cache==source, real headless session fired the hook, + env-override canary warns). + +**Global os-doc-hygiene plugin** — `cc-os/plugins/os-doc-hygiene/` (git-tracked, +2026-07-03); symlinked into `~/.claude/plugins/os-doc-hygiene` +- Hooks: `hooks/hooks.json` → SessionStart hook (matcher: startup|resume) runs + `scripts/reminder.py` via `${CLAUDE_PLUGIN_ROOT}` (5s timeout), emitting a deterministic + zero-token reminder banner +- Behavior: Monitors stale and bloated project documentation per-project under + `.dochygiene/` state dir (gitignored). SessionStart reminder is deterministic (no AI + tokens, once/day snooze). Skills (verb-first, no `commands/` dispatcher — invoked directly + as `/os-doc-hygiene:`, per [[cc-os-plugin-skill-naming-convention]]): `check` + (AI-assisted classification of staleness signals, emits machine+human reports), `clean` + (AI-assisted or deterministic patch application with git-safe scoped cleanup), `status` + (read-only lifecycle-timestamp read), `sweep` (check then clean in sequence). + Reversion-protected via invariants.md + golden-example test fixtures. +- Migration: migrated from standalone repo (`~/dev/cc-plugins/doc-hygiene/`, 2026-07-03) and + integrated into cc-os. No content conflicts detected. Renamed from `doc-hygiene` to + `os-doc-hygiene` per cc-os plugin naming convention. Skills renamed + `hygiene-check`/`hygiene-clean` → `check`/`clean`, and the `commands/hygiene.md` + dispatcher removed in favor of two new skills (`status`, `sweep`), aligning with the + `os-vault`/`os-orchestration` pattern of no `commands/` directory (2026-07-03). + +**Global os-adr plugin** — `cc-os/plugins/os-adr/` (git-tracked, 2026-07-03); symlinked into +`~/.claude/plugins/os-adr` +- Purpose: Architecture Decision Records, consistently in every project — one + customized-Nygard template (frontmatter + `id`/`date`/`status`/`supersedes`/`superseded-by`/`affected-paths`/`affected-components`; + Context/Decision/Consequences/Alternatives-rejected sections), + `docs/adr/NNNN-kebab-title.md` one-file-per-decision + fully-regenerated + `docs/adr/README.md` index. Built from `docs/adr-system/` (requirements `04`, PRD `05`, + eval sketches `06`) via OpenSpec change `add-os-adr-plugin`. +- Core: Ruby OO library `lib/adr/` (`Record`, `Repository`, `Index`, `Template`, `Detector`, + `Migrator`, `MigrationReport`, `Finder`; Sandi Metz style, injected paths) behind thin + CLIs in `bin/` (`adr-new`, `adr-init`, `adr-detect`, `adr-migrate`, `adr-find`). Tests: + `ruby tests/all.rb` (47) + `python3 tests/hook_test.py` (15), all model-free. Behavioral + invariants in `invariants.md`. +- Hook: **moved to os-status (2026-07-06)** — the SessionStart existence check now runs as + the `adr-system-present` check in the os-status plugin (`hooks.json` here is empty; + `hooks/session_start.py` stays as the wording source of record, byte-copied into + os-status). Behavior unchanged: present → near-zero-token usage note naming + `/os-adr:create` + `/os-adr:find`; absent → once-per-day init/migrate suggestion, silenced + permanently by `.os-adr/suppress` (still honored); silent outside git projects. +- Skills (verb-first, no `commands/`): `create` (one-invocation templated+numbered+indexed + ADR with mechanical supersession), `init`, `migrate` (non-destructive: detect surveyed + shapes → mechanical heuristic fill → LLM fills only manifest-listed interpretive fields + via `--apply-fills` → `migration_confidence` frontmatter + `docs/adr/migration-report.md`; + old-system deletion only as an explicit separate user-approved step), `find` + (deterministic-first retrieval: affected-paths/components match → Accepted-only filter → + one-hop Graphify query-path expansion with graceful degradation → AI judgment over the + candidate set only). +- Migration pilot passed 2026-07-03 (sandboxed project copies; gate threshold 25% + low-confidence flags): viking-warrior-training-log 0%, delta-refinery 0%, llf-schema 8.3%. + Retrieval verified end-to-end against the real llf-schema project graph. Resolved: ADRs + need no Graphify tag/edge convention — retrieval layer 3 expands the *query* paths via + graph-node `source_file`s. +- Eval A harness (2026-07-03, ADR-021): `plugins/os-adr/eval/` — prompted skill-execution + eval across model tiers (haiku/sonnet). Two fixtures (generated with the plugin's own + CLIs), six scenarios (S1 create, S2 create+supersede, S3 find/conflict, S4 + find/distractor, S5 init, S6 migrate+fills), deterministic Ruby checker (`eval/bin/check`, + structural invariants, TSV mode for autoresearch), sandbox + headless-runner scripts, + runner-prompt template. Primary run mode: in-session Agent-tool subagents with pinned + `model:`; optimized via the `/autoresearch` Classic loop over SKILL.md *wording only* + (checker/fixtures/scenarios frozen during a loop). Procedure: + `plugins/os-adr/eval/README.md`. Self-tested both directions; grid run 2026-07-03, all + scenarios × both tiers passing. Distinct from the held-out Eval B (unprompted behavior) — + do not conflate. +- Eval B harness (2026-07-03, OpenSpec change `add-os-adr-eval-b-harness`): + `plugins/os-adr/eval-b/` — held-out unprompted-behavior eval (requirements 4–5). 7 + scenarios (W1–W3 write-trigger, R1–R4 retrieval) authored from the frozen shapes in + `docs/adr-system/06-eval-scenarios.md`; dedicated Ruby webhook-relay fixture with a 6-ADR + history (Superseded pair + near-miss distractors, generated via the plugin's own CLIs); + R4's one-hop graph reach uses a real `graphify update` AST build (model-free, rebuilt via + `eval-b/bin/build-fixture-graph`, never committed). **Headless-only runner** + (`eval-b/bin/run` — fresh `claude -p` per rep, cwd = sandbox, so the real SessionStart + hook fires; in-session subagents are invalid here, unlike Eval A) and a two-axis + deterministic-first checker (`eval-b/bin/check`): axis (a) unprompted consultation, + mechanical from transcript tool_use blocks; axis (b) correct-ADR citation (R1–R4) or + new-ADR-file with a narrow frozen-rubric haiku judge fallback (W1–W3, `judge-rubric.md`, + stubbable via `ADR_EVAL_B_JUDGE_CMD`). `R4-nograph` is the graph-degradation variant + (expected FAIL). Self-tested both directions model-free via `eval-b/bin/self-test`. + **Scenario Task blocks are held-out — never run them informally.** Procedure: + `plugins/os-adr/eval-b/README.md`. Grid run 2026-07-03 (1 rep/cell): haiku 0/8 PASS; + sonnet 5/8 PASS (fails W3, R1). `R4-nograph` FAILed on both tiers as expected. Full + results + observations: vault note [[os-adr-eval-b-grid-results-and-observations]]. +- **Initial iteration complete (2026-07-03):** plugin build, migration pilot, Eval A grid + (clean pass), and Eval B grid (baseline) all done — closes the first pass on os-adr. +- **Wording experiment complete (2026-07-04):** the follow-up `/autoresearch` loop over + Eval B wording ran (5 iterations, checker/fixtures/scenarios/rubric frozen; fixture + CLAUDE.md declared a wording surface upfront) and closed the gap — final full grid + **sonnet 8/8, haiku 7/8** (from 5/8 / 0/8 baseline; haiku's one miss is a W3 axis-b + judge-boundary flicker). Winning wording shipped in `hooks/session_start.py` + (PRESENT_NOTE), find/create SKILL.mds, and `eval-b/fixture/project/CLAUDE.md` (new — the + trigger-phrased "Architecture decisions" section, the template for real-project adoption). + Confirmed mechanisms: trigger-conditioned when→then phrasing beats inventory phrasing on + both tiers; each rule must live where its precondition is visible (the reversal→supersede + rule in the find skill's act-on-findings step fixed W3); lower tiers need *mechanical* + triggers — semantic triggers only reach sonnet. Open: channel ablation never run; + R4-nograph now passes both tiers, so the graph-degradation check no longer differentiates. + Full hypothesis→result mapping: vault note [[os-adr-eval-b-wording-experiment-hypotheses]]. + **Before designing or running any autoresearch eval, Read + `~/Documents/SecondBrain/howto/running-autoresearch-skill-evals.md`.** +- **Eval B W3 stability check (2026-07-06):** 7 headless reps of haiku × W3 (all reps + counted): axis (a) consultation 7/7 PASS; axis (b) recording-offer ~5/7 (~75–85%) — one + FAIL plausibly infra, one a genuine behavioral miss. Verdict: intermittent flicker, not a + hard gap — grid claim stays **haiku 7/8**, W3 characterized as an ~1-in-4/5 axis-b miss. + Haiku cleared for daily use with that caveat. Recorded in the vault good-enough gate: + [[os-adr-eval-b-wording-experiment-hypotheses]]. +- **Eval C harness built, NOT yet run as designed (2026-07-06):** `plugins/os-adr/eval-c/` — + ambiguity-ladder DISCRIMINATION eval (held-out; Eval B is contaminated by the wording + loop). 3 levels (explicit → moderate → conceptual framing) × paired positive/negative + scenarios; run-set (6, job-execution domain) + frozen reserve-set (6, notifications + domain). New Python fixture (`taskq` async job queue, 6-ADR history generated via the + plugin's own CLIs, trigger-phrased CLAUDE.md section copied from eval-b). Scoring: + positives = both axes; negatives FAIL **only on unneeded ADR creation**. Model-free + `bin/self-test` green, including a truthful-citation-must-PASS guard. **Scenario Task + blocks are held-out — never run informally.** Methodology: vault notes + [[eval-methodology-ladder]] and [[eval-methodology-irl-feedback-loop]]; vault also gained + an `eval-results` note type + `_templates/eval-results.md` (2026-07-06). +- **Eval C frozen grid run (2026-07-06):** run-set × {sonnet, haiku} × 3 reps/cell (36 + headless reps, all counted) — **12/12 cells PASS at ≥2/3 majority; haiku 18/18 reps, + sonnet 17/18**. Zero over-trigger ADR creation on any negative at any level; every + positive pass was via governing-ADR citation (16) or judge-recognized proposal (1), never + duplicate creation. Sonnet's one FAIL rep (P1-L1) is a final-message-citation boundary + artifact, not a behavioral miss. Two harness defects fixed on the first live rep, BEFORE + the grid, with Task blocks/fixtures/rubric untouched: (1) `bin/run` pipefail bug; (2) + positive axis-b contradiction fixed via a mechanical `cited-governing` PASS branch + two + new self-test guards (`B-via:` now recorded in TSV). Verdict: the Eval B wording + **generalizes** — grid saturated, ceiling not found; next signal is production (rollout + + IRL session audits), not another lab rung. Run-set is now the baseline; wording tuning + against it moves measurement to the reserve-set. Full results: vault note + [[os-adr-eval-c-frozen-grid-results]]. +- **Remaining (locked rollout order):** real-project migration/adoption one at a time via + `/os-adr:migrate` — pilot projects first, then the cc-os retrofit (its 19-ADR monolithic + file was deliberately excluded from the pilot), then wider. When onboarding real projects, + add the trigger-phrased "Architecture decisions" CLAUDE.md section (copy from + `eval-b/fixture/project/CLAUDE.md`; candidate: emit it from `/os-adr:init`/`migrate`). + +**Graphify** — v0.8.31 at `/home/jared/.local/bin/graphify` +- Vault graph: `~/Documents/SecondBrain/graphify-out/` — disposable, structural index + rebuilt by SessionStart hook; handles relational/graph queries. Distinct from + vault-index.json (planned, Plan B Phase 1): a flat `{tag: [{path, title, summary}]}` + lookup rebuilt at SessionEnd, queried by the `/memory-find` skill for fast tag-based SB + note discovery. +- Project graph: `/graphify-out/` — same pattern; gitignore it in each project + repo +- Vault conventions: `~/Documents/SecondBrain/vault-conventions.md` — frontmatter contract + + tag taxonomy (canonical name decided 2026-06-09; formerly referred to as CONVENTIONS.md) + +**memsearch** — v0.4.6 via Claude Code plugin marketplace (`memsearch@memsearch-plugins`) +- Hooks: Stop, SessionStart, UserPromptSubmit, SessionEnd (ship with plugin) +- Memory store: `~/.memsearch/memory/YYYY-MM-DD.md` daily files (global, cross-project, all + clients — one store by design) +- Index: `~/.memsearch/milvus.db` (Milvus Lite, local); embeddings via ONNX bge-m3 +- Config: `MEMSEARCH_DIR=~/.memsearch` in `~/.zshrc` for global scope +- Skills: `/memory-recall`, `/memory-config` (ship with plugin) +- Git sync: `~/.memsearch` is a dedicated private Forgejo repo + (`forgejo.swansoncloud.com/jared/memsearch`); whitelist `.gitignore` commits only + `memory/*.md`; auto-commit+push wired into `memsearch_sync.py` SessionEnd hook (relocated + from `session-end.sh` by ADR-016, 2026-06-12; behavior preserved — see ADR-015) + +**Obsidian vault git sync** — `~/Documents/SecondBrain` (2026-06-15) +- Remote: `ssh://git@forgejo.swansoncloud.com:2222/jared/SecondBrain.git` (private Forgejo; + web: `https://forgejo.swansoncloud.com/jared/SecondBrain`) +- 52 files tracked (notes, journal, templates, vault-conventions.md, CLAUDE.md, .obsidian + config); `graphify-out/` excluded via `.gitignore` +- Auto-commit+push via `vault_sync.py` SessionEnd hook (mirrors `memsearch_sync.py`; + push-only — SessionStart pull is optional future item for multi-machine) + +## Operational procedures + +### Renaming or moving a local plugin + +Renaming a plugin directory under a directory-source marketplace (like `local-plugins`, whose +source is `~/.claude/plugins` itself) requires updating **three** places, not just the +directory and `settings.json`: + +1. **Plugin directory / symlink** — the actual files (or symlink target, for plugins that + live in a git repo like `cc-os/plugins/os-vault/`). +2. **Marketplace manifest** — `~/.claude/plugins/.claude-plugin/marketplace.json` (for + `local-plugins`) lists each plugin's `name` and `source` path explicitly; this does + **not** auto-discover from the directory listing. Update the entry to match the new + name/path. +3. **`settings.json` `enabledPlugins`** — the `@` key must match the + manifest entry from step 2. + +Then refresh the plugin manager's state so it re-resolves the marketplace and drops the +stale install record: + +```bash +claude plugin marketplace update # re-validates the manifest +claude plugin install @ # creates the new install record +claude plugin uninstall @ # drops the stale one +claude plugin list # verify: new name enabled, old name gone +claude plugin details @ # verify skills/agents/hooks resolved +``` + +Skipping step 2 is the failure mode to watch for: hooks (wired by absolute path in +`settings.json`) keep working, masking the fact that skills/slash-commands silently stopped +registering under the plugin's new name. + +### Editing a local plugin (cache refresh) + +Plugin source edits (including SKILL.md wording, hook scripts, or CLI code) do not reach +running sessions until the cache is refreshed. Claude Code caches plugins when installed, +copying them to `~/.claude/plugins/cache////`. To refresh a +stale cache after source edits: + +```bash +bin/refresh-plugins [marketplace-names...] # refresh all or specific local marketplaces +``` + +This uninstalls and reinstalls each plugin, forcing the cache to repopulate. See +`bin/refresh-plugins --help` for usage. After refreshing, +`claude plugin details @` should show updated skill descriptions and hook +counts. Directory-source marketplaces (e.g., `local-plugins` sourced from +`~/.claude/plugins`, `cc-plugins` sourced from `~/dev/cc-plugins`) are the only ones that +need manual refresh — public marketplaces auto-update. diff --git a/docs/plans/b3-memsearch-injection-review.md b/docs/plans/b3-memsearch-injection-review.md new file mode 100644 index 0000000..9045fc0 --- /dev/null +++ b/docs/plans/b3-memsearch-injection-review.md @@ -0,0 +1,91 @@ +# B3 — memsearch SessionStart injection cost review + +_Reviewed 2026-07-08. memsearch v0.4.6 (installed) / v0.4.11 (also cached, not yet installed). Plugin: `memsearch@memsearch-plugins`._ + +## What gets injected + +`hooks/session-start.sh` (both cached versions, logic unchanged 0.4.6→0.4.11) builds a +`# Recent Memory` block on every SessionStart and returns it as `additionalContext`: + +- Finds the daily memory files under `$MEMORY_DIR` (`~/.memsearch/memory/YYYY-MM-DD.md`), + sorted by filename descending, and takes the **2 most recent**. +- For each of those 2 files, greps lines matching `^(#{2,4} |- )` (headings + bullets) and + takes the **first 40 matching lines** (`head -40`). +- Concatenates both files' extracts under one `# Recent Memory` heading and returns it. + +This is entirely hardcoded (`head -2`, `head -40`) in the shell script — there is no +per-file/per-line/byte cap that's user-configurable. + +## Measurements (cc-os project, last ~7 days of session transcripts) + +Extracted the `additionalContext` string from every SessionStart hook event whose payload +contained `# Recent Memory`, across `~/.claude/projects/-home-jared-dev-cc-os/*.jsonl` +(61 SessionStart events found in that window — heavy usage during the WS2/WS4 eval loops, +so this sample is skewed toward busy days, not a quiet baseline). + +| Stat | chars | approx tokens (÷4) | +|---|---|---| +| min | 5,976 | ~1,494 | +| median | 11,721 | ~2,930 | +| avg | 11,978 | ~2,995 | +| max | 15,407 | ~3,852 | + +This roughly confirms the ~13 KB figure already observed. Size scales with how many +`##`/`###`/bullet lines the 2 most recent daily files accumulated that day — on a day with a +lot of sessions/subagent activity (like the recent eval-loop days), each daily file alone can +hit the 40-line cap, so the injection saturates near its structural ceiling +(2 files × 40 lines × ~150–200 chars/line ≈ 12,000–16,000 chars), matching the observed range. + +Caveat: this is a per-project sample from one heavy-usage repo (cc-os) over 7 days; other +projects' daily files will differ, and this sampling method (string-matching the raw jsonl, +not a documented API) is the best available without deeper memsearch internals — reasonably +solid since the hook's exact output string was recovered from real transcripts, not estimated. + +## Config surface found + +Read `skills/memory-config/SKILL.md` (both cached versions) and `hooks/session-start.sh` +directly. Findings: + +- The **documented** TOML config surface (`plugins.claude-code.summarize`, `.project_review`, + `.user_profile`, `.memory_to_skill`, `[llm.providers.*]`, `[prompts]`) controls + summarization, maintenance artifacts (PROJECT.md/USER.md), skill distillation, and provider + routing — **none of it touches the SessionStart recall injection**. +- No `[plugins.claude-code.session_start]` or equivalent section exists in the documented + schema, and none of the constants used by the injection logic (`head -2` file count, + `head -40` line count, the `^(#{2,4} |- )` grep pattern) appear anywhere in `config.toml`, + `memsearch config list --resolved/--global/--project`, or the SKILL.md. They are hardcoded + in `hooks/session-start.sh` itself. +- The only adjacent, real lever is **memory file density**: since the cap is by line count + (not token count) over headings+bullets, whatever produces terse vs. verbose daily-file + bullets (the Stop-hook summarizer, governed by `plugins.claude-code.summarize.model`, + defaults to haiku) indirectly controls how much of the 40-line budget per file is "real + signal" vs. how many days of content get squeezed out. + +## Recommendation + +**No first-class cap/trim setting exists in memsearch's config schema for this injection.** +The two concrete options, in preference order: + +1. **Workaround, no code change (recommended to try first):** the injection is driven purely + by memory-file content density, not by any setting. If daily files are noisy (e.g. many + short low-signal bullets from busy multi-session days, as in the cc-os sample), the + Stop-hook summarizer producing those bullets is the actual place to add discipline — + tightening what counts as bullet-worthy would shrink the 40-line/file budget's real size + without touching the plugin. This is a memsearch summarizer/prompt-quality issue, not a + session-cost cap. +2. **Direct edit, unsupported (only if (1) is insufficient):** hand-edit the cached + `hooks/session-start.sh` at + `~/.claude/plugins/cache/memsearch-plugins/memsearch//hooks/session-start.sh`, + lowering `head -2` → `head -1` (halves injection size, roughly ~6–8 KB/day instead of + ~12–15 KB) and/or `head -40` → a smaller value (e.g. 20). Expected savings: cutting to 1 + file × 20 lines would drop the typical injection from ~11–15 KB (~2,900–3,850 tokens) to + roughly ~3 KB (~750 tokens), a ~75% reduction. **Caveat:** this is an unmanaged edit to + plugin-cache source — it will be silently overwritten on the next + `claude plugin update memsearch` / cache refresh, is not upstream-supported, and isn't a + "setting" in the config-surface sense the task asked to identify. Treat it as a stopgap, + not a durable fix; if this repo wants a durable cap, the change belongs upstream in + `zilliztech/memsearch` (file an issue/PR to add a + `[plugins.claude-code.session_start]` `max_files` / `max_lines_per_file` config section) + rather than as a local hack here. + +No setting was applied — this is a review/recommendation only per the task. diff --git a/docs/plans/c3-spike-findings.md b/docs/plans/c3-spike-findings.md new file mode 100644 index 0000000..a1ae998 --- /dev/null +++ b/docs/plans/c3-spike-findings.md @@ -0,0 +1,114 @@ +# C3 spike findings: `--setting-sources` interaction + overlay file placement + +_Task: C3 in [system-prompt-profiles-tasks.md](system-prompt-profiles-tasks.md). Feeds B2._ + +## Method note (important correction mid-spike) + +Initial attempts asked the model to self-report whether ORCHESTRATION.md/skills were +injected ("ORCH=yes/no SKILL=yes/no"). This was **unreliable** — a default run that the +transcript later proved *did* receive the orchestration injection had the model answer +`ORCH=no`. Model self-report is not a valid signal for this spike. + +Switched to a deterministic signal: the session transcript JSONL at +`~/.claude/projects//.jsonl` records hook execution as +`attachment.type == "hook_additional_context"` entries (one per SessionStart hook that +returned `additionalContext`) and the full injected skill list as +`attachment.type == "skill_listing"`. Both are populated *before* the model ever sees the +turn, so they're a ground-truth substitute for `/context`. + +## Spike (a): `--setting-sources` vs symlinked local plugins + absolute-path hooks + +4 headless haiku runs from an empty scratch dir +(`/tmp/claude-1000/.../scratchpad/c3spike`), `claude -p --model haiku --output-format json +"reply OK"`, varying `--setting-sources`. For each, inspected the resulting transcript +JSONL for hook-injected additionalContext count and whether plugin-specific skills +(`os-adr`, `os-vault`) appear in the injected skill listing (built-in skills like +`autoresearch`/`caveman` appear regardless — only plugin skill presence is diagnostic). + +| `--setting-sources` value | additionalContext entries (hooks fired) | os-adr/os-vault skills present | +|---|---|---| +| (no flag, default) | 1 (confirmed in first exploratory run) | yes | +| `""` (none) | 0 | no | +| `user` | 1 | yes | +| `project,local` | 0 | no | + +Interpretation: + +- Hooks in this setup are wired **only** in `~/.claude/settings.json` (a **user**-scope + settings file), by absolute path into `cc-os/`. They fire if and only if the `user` + settings source is included. `--setting-sources ""` and `--setting-sources + project,local` both silently drop them — no error, no warning, just no + `additionalContext`. This is a real footgun for a future `cyolo` profile wrapper: any + profile that passes `--setting-sources` explicitly must include `user` or it silently + loses ORCHESTRATION.md injection, os-status checks, os-vault hooks, and every other + hook in the plugin family. +- Plugin skill registration (the `os-adr:find` style slash-command/skill listing) tracks + the same axis: present with `user` included, absent without it. This makes sense — + `enabledPlugins` lives in the user-scope `~/.claude/settings.json`, so a + `--setting-sources` value that excludes `user` also excludes plugin enablement, not + just hooks. +- Symlinks themselves are a non-issue: once the `user` settings source is included, the + symlinked plugin (`~/.claude/plugins/os-adr -> cc-os/plugins/os-adr`) resolves and + loads identically to the default case — no special-casing needed for the symlink or + for the hooks' absolute paths into `cc-os/`. The failure mode is entirely about which + settings **source** is included, not about symlinks or path resolution. + +**Verdict (a):** `--setting-sources` interacts cleanly with symlinked plugins and +absolute-path hooks in the sense that nothing breaks or errors — but it is an all-or- +nothing gate: excluding `user` from `--setting-sources` silently disables every cc-os +plugin (hooks AND skills) with no error surfaced. Any profile wrapper must always include +`user` in `--setting-sources` (or simply not pass the flag) to keep the plugin family +alive; only `project`/`local` inclusion should be varied per profile. + +## Spike (b): where should per-profile settings overlay files live? + +Read `bin/refresh-plugins` (Ruby). It touches exactly these paths, all read-only except +for the plugin cache itself: + +- Reads `~/.claude/plugins/known_marketplaces.json` (marketplace list/config) +- Reads `~/.claude/plugins/installed_plugins.json` (which plugins are installed per + marketplace) +- For each installed plugin under a `directory`-source marketplace, shells out to + `claude plugin uninstall @` / `claude plugin install + @` — this rewrites `~/.claude/plugins/cache// + //` (via Claude Code's own plugin manager, not directly) and updates + `installed_plugins.json`'s install record for that plugin. +- Verifies via `diff -rq` between the cache dir and the plugin's source dir under + `source_path` (from marketplace config, e.g. `~/dev/cc-os/plugins` or + `~/dev/cc-plugins`). + +It never touches `~/.claude/settings.json`, never touches any file under a +`~/.claude/profiles/` style directory, and never touches arbitrary `--settings ` +overlays — its blast radius is strictly `known_marketplaces.json`, +`installed_plugins.json`, and the `plugins/cache///` tree. + +**Recommendation:** put per-profile settings overlay JSONs at `~/.claude/profiles/*.json` +(e.g. `~/.claude/profiles/cc-os-design.json`, `client-dev.json`, `brainstorm.json`), +passed via `claude --settings ~/.claude/profiles/.json`. This path is structurally +disjoint from everything `refresh-plugins` reads or writes (`plugins/cache/`, +`plugins/known_marketplaces.json`, `plugins/installed_plugins.json`) — there is no +plausible future edit to `refresh-plugins` that would need to reach into +`~/.claude/profiles/`, since that script's whole job is cache-vs-source verification for +marketplace-installed plugins, not settings resolution. + +Interaction with `enabledPlugins`: `--settings ` is merged with (not a full +replacement for) the resolved settings from the sources selected by `--setting-sources` +(per spike (a), keep `user` in the source list so the base `~/.claude/settings.json` +`enabledPlugins`/hooks/env still apply). The profile overlay should therefore only need +to express a *diff* — an `enabledPlugins` map with the profile's unwanted plugins set to +`false` (or, if this repo prefers positive lists, the full set with only the +wanted ones `true`) — rather than duplicating the whole settings file. This diff-only +overlay is exactly what B2 should generate per profile. One caution to carry into B2: +verify at build time whether CC deep- or shallow-merges the `enabledPlugins` object +across `--settings` sources (not tested in this spike) — either way, a plugin disabled +via the overlay's `enabledPlugins: {"name": false}` does not require touching +`known_marketplaces.json` or `installed_plugins.json`, since `enabledPlugins` toggles are +independent of the marketplace/install-record identity those files hold (the files +`refresh-plugins` owns). + +**Verdict (b):** `~/.claude/profiles/*.json` (passed via `--settings`) is safe — it lives +entirely outside the three paths `refresh-plugins` reads/writes/rewrites +(`known_marketplaces.json`, `installed_plugins.json`, `plugins/cache/`), and +`enabledPlugins` toggles in an overlay are orthogonal to the marketplace/cache identity +`refresh-plugins` verifies, so cache refreshes and profile overlays can never clobber +each other. diff --git a/docs/plans/system-prompt-profiles-assessment.md b/docs/plans/system-prompt-profiles-assessment.md index 1e879aa..bedfee4 100644 --- a/docs/plans/system-prompt-profiles-assessment.md +++ b/docs/plans/system-prompt-profiles-assessment.md @@ -1,6 +1,34 @@ # System-prompt profiles: assessment & recommended approach -_Last updated: 2026-07-08 · Status: assessment only, no build started_ +_Last updated: 2026-07-08 · Status: assessment + next-actions agreed; deferral lifted — +WS2/WS4 evals confirmed complete 2026-07-08 (Gate G0 cleared), execution may begin_ + +> Task breakdown for subagent implementation: +> [system-prompt-profiles-tasks.md](system-prompt-profiles-tasks.md) (2026-07-08). + +## Agreed next actions (2026-07-08, deferred) + +Two tracks, both approved by the user: + +1. **Top-down diet** (first, biggest win): + - Split cc-os `CLAUDE.md` (~11k tokens) to a ~2–3k orientation + on-demand status/changelog + doc, using `@`-import / pointer style per + `~/servers/proxmox-hermes/docs/claude-md-maintenance.md` (keep it short; prefer pointers + to copies; never auto-append). + - Then per-profile plugin enablement (~4.8k max, second priority) and a memsearch + injection review. + - **Do not start while the other session's evals are running** — CLAUDE.md is live input + to those sessions. +2. **Bottom-up audit** (build anytime, read-only): + - One-file logging proxy (`ANTHROPIC_BASE_URL` pass-through) that dumps the `system` and + `tools` arrays from a real request — the audit basis for lean profiles. Expect savings + to come from per-profile `--tools` lists (~18k of schemas), not prompt rewriting (~6.3k). + +Related future idea (out of scope, captured in vault note +[[claude-md-budget-linter-plugin-concept]]): a deterministic CLAUDE.md budget linter — +hook-driven, JSON per-project config with quantified user-agreed exceptions, silent in range, +maintenance agent only when out of range; possible extension to whole-project weekly hygiene +scans. Overlaps os-doc-hygiene/os-status — resolve at build time. Companion to the WS4 orchestration-economics work. Goal: reduce per-session context cost and improve focus by controlling what reaches the model's context — the built-in system prompt, diff --git a/docs/plans/system-prompt-profiles-tasks.md b/docs/plans/system-prompt-profiles-tasks.md new file mode 100644 index 0000000..50a940d --- /dev/null +++ b/docs/plans/system-prompt-profiles-tasks.md @@ -0,0 +1,69 @@ +# System-prompt profiles: task breakdown for subagent implementation + +_Last updated: 2026-07-08 · Source plan: [system-prompt-profiles-assessment.md](system-prompt-profiles-assessment.md)_ + +Groups are ordered by dependency, not priority. Group A is read-only and can start +immediately, in parallel. Group B is the biggest win but is **gated**. Groups C and D are +sequential follow-ons. + +**Slimmed scope (2026-07-08, user-approved):** execute quick/high-value wins only — +B1 → B3 ∥ C3 → B2, then a minimal C (settings-overlay aliases, no auto-detection). +A1/A2 are deferred until a live interactive `/context` check shows tool schemas still +dominate despite ToolSearch deferred loading (one user-assisted check replaces Group A; +A3's headless matrix is already substantially in the assessment doc). D is dropped from +the plan — revisit only if CC economics change; max upside ~6.3k with fork-maintenance +downside. + +**Gate G0 — CLEARED (2026-07-08):** user confirmed the os-orchestration (WS4) and os-vault +(WS2) eval loops are complete; no eval runs remain in flight. Group B is unblocked. + +--- + +## Group A — Bottom-up audit (read-only, start anytime, parallelizable) + +| ID | Task | Depends on | Suggested agent | +|---|---|---|---| +| A1 | **Logging proxy.** One-file `ANTHROPIC_BASE_URL` pass-through that forwards to `api.anthropic.com` and dumps the `system` and `tools` arrays of each request to a local file. Ruby preferred (Sandi Metz style not required for a one-file tool). No response mutation, streaming must pass through untouched. Deliverable: `bin/` or `tools/` script + 3-line usage note. | — | sonnet | +| A2 | **Live prompt capture + audit.** Run one headless and (user-assisted) one interactive session through the A1 proxy from cc-os and from an empty dir; save the dumped `system`/`tools` arrays; produce an audit doc: token cost per system-prompt section, per tool schema, and a "safely droppable per profile" table. Expect savings in `--tools` lists (~18k), not prompt text (~6.3k). | A1 | sonnet | +| A3 | **Phase 0 measurement.** Script the headless baseline matrix (already partially done in the plan: 4 configs measured) and extend to 3–4 representative session types (cc-os design, client dev, brainstorm) via `--output-format json` usage fields; record one interactive `/context` check (user-assisted) to validate the headless numbers against a Fable session. Deliverable: a measurements table appended to the assessment doc. | — | haiku (scripted runs) + user for the interactive check | + +A1 and A3 run in parallel; A2 follows A1. + +## Group B — Top-down diet (**gated on G0**, sequential) + +| ID | Task | Depends on | Suggested agent | +|---|---|---|---| +| B1 | **Split cc-os CLAUDE.md.** ~11k tokens → ~2–3k orientation file + `docs/implementation-status.md` (the append-only changelog moved out, linked not `@`-imported so it loads on demand). Follow `~/servers/proxmox-hermes/docs/claude-md-maintenance.md`: keep it short, pointers over copies, never auto-append. Must preserve: the "keep this file current" contract (redirected to the new doc), plugin naming rule, ADR discipline, OpenSpec workflow. Verify a fresh session still orients correctly. | G0 | sonnet (judgment about what is orientation vs changelog); main-loop review before commit | +| B2 | **Per-profile plugin enablement.** Design + create settings overlay files (one per profile: e.g. `cc-os-design`, `client-dev`, `brainstorm`) listing `enabledPlugins`; a brainstorm profile drops rails-ui-component / api-wrapper / invoice-ninja / creative-team etc. Resolve where overlays live so they don't fight `bin/refresh-plugins` (open question #3). | B1 (so measured savings are attributable) | sonnet | +| B3 | **memsearch injection review.** Measure the SessionStart recall injection size across a week of sessions (~13 KB observed once); review memsearch plugin config options for capping/trimming it; recommend (not apply) a setting. | — (parallel with B1/B2 once G0 clears) | haiku | + +## Group C — `cyolo` profile wrapper (Phase 2, after B) + +| ID | Task | Depends on | Suggested agent | +|---|---|---|---| +| C1 | **Profile prompt components.** Create `~/.claude/profiles/` with `dev.md`, `ruby.md`, `planning.md`, `brainstorm.md`. Objective profiles state the session endpoint explicitly ("this session ends when the plan is written…"). Small files, ~0.5k tokens each. | B2 (profiles reference the settings overlays) | sonnet | +| C2 | **Wrapper script.** `cyolo` shell wrapper: resolve profile from flag (`--dev`, `--planning`) + project detection (pwd, Gemfile…), compose append-file, launch `claude --append-system-prompt-file --settings [--tools …]`. | C1 | sonnet | +| C3 | **Open-question spikes** (can run before/parallel with C1): (a) does `--setting-sources` interact cleanly with symlinked local plugins + absolute-path hooks? (b) confirmed overlay location from B2 survives `bin/refresh-plugins`. Small experiments, written up as findings notes. | — | haiku | + +## Group D — Eval-gated replacement experiment (optional, last) + +| ID | Task | Depends on | Suggested agent | +|---|---|---|---| +| D1 | **Eval design.** Read `~/Documents/SecondBrain/howto/running-autoresearch-skill-evals.md` FIRST (standing rule). Design the grid: scenario set × {default, dieted stack, profile stack, full-replacement lean prompt} × model tiers; score task success + tokens/turn + wrong-tool proxies. Design only — do not run without user sign-off. | C2 (profile stack must exist to be a grid column) | sonnet | +| D2 | **Run + verdict.** Execute the grid headless-only; promote full replacement only if it wins on *outcome*, not just token count. | D1 | per playbook | + +## Out of scope (captured, do not implement) + +- CLAUDE.md budget linter plugin — vault note [[claude-md-budget-linter-plugin-concept]]; + overlaps os-doc-hygiene/os-status, resolve at build time. +- Per-model profile variants — deferred by user decision; revisit after profiles exist. + +## Suggested execution waves + +Superseded by the slimmed scope above. Current order: + +1. **Wave 1 (parallel):** B1 (main-loop), B3 + C3 (subagents). +2. **Wave 2:** B2 (needs C3's overlay-location answer). +3. **Wave 3 (minimal C):** settings-overlay aliases only; full C1/C2 wrapper deferred + until the friction is felt. +4. **Deferred:** A1/A2 pending the interactive `/context` check; D dropped (pointer kept).