cc-os/CLAUDE.md

229 lines
42 KiB
Markdown
Raw Normal View History

# CLAUDE.md
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
## What this repository is
`cc-os` is a **design + implementation repository** — it captures the design of a **personal,
cross-project memory system for Claude Code** (for a multi-client freelancer) plus the research
that informs it. The global memory plugin is now partially implemented; markdown specs, ADRs,
and the build plan remain the source of truth for what is being built and what remains.
Everything is markdown-as-truth. When asked to "build," work from the staged tasks in
`docs/memory-system/04-build-plan.md`, not ad hoc.
## Directory layout
Numbered files within a directory are not a required reading order — read the one whose topic
you need.
- **`docs/memory-system/`** — the design of the memory system itself. Go here to understand or
change *what is being built*. `02-system-design.md` is the architecture, `03-architecture-
decisions.md` is the ADR log (each decision + what was rejected/deferred and why),
`04-build-plan.md` is the build outline, `06-graphify-evaluation.md` justifies the Graphify
pivot. Read the specific file relevant to your task; read the whole set only when reworking
the design.
- **`docs/graphify/`** — a verified handbook for the Graphify knowledge-graph tool (the chosen
knowledge-layer engine). Go here when working with Graphify commands/behavior. Skim
`00-README.md` for the model, keep `09-best-practices-checklist.md` open while actually
running it. Claims are provenance-tagged (`[github]` trustworthy; `[interview]` /
`[unverified claim]` not).
- **`graphify-interview`, `memory-systems-compared060326`** (repo root) — raw source
transcripts (marketing / video). Only open these to trace where a claim came from; treat as
*intent*, not *fact* — they were already corrected against primary sources in `docs/`.
- **`openspec/`** — spec-driven change management (see workflow below). `changes/` holds live
changes, `changes/archive/` completed ones, `specs/` stable specs.
- **`.claude/`, `.codex/`, `.pi/`** — **identical** copies of the OpenSpec skills for three AI
assistants. Only open when changing a skill — and mirror any change across all three.
## The current design in one paragraph
This is a **work in progress**, not a frozen spec. The paragraph below is the current approach;
treat it as the default you operate from, but **whenever a conversation with the user changes
the design, update this paragraph (and the relevant `docs/memory-system/` files + an ADR) to
match.** Keep it accurate, don't preserve it for its own sake.
Two memory types kept as **separate systems**: **episodic** ("what happened, when") handled by
**memsearch** (Milvus Lite, embedded), and **semantic/knowledge** ("how do we…") handled by the
existing **`~/Documents/SecondBrain` Obsidian vault** as the single source of truth. Notes keep
`summary` + six flat, parallel namespaced facets (`type/`/`client/`/`project/`/`domain/`/`tool/`/`convention/`) plus `scope/` as metadata; hierarchy and relationships are expressed via hub notes (`type/hub`), wikilinks, and Graphify graph edges — not nested tag paths. The vault is queried via a **Graphify knowledge graph** (local Ollama SLM for doc extraction, free
tree-sitter AST for code). Retrieval is hook-injected + on-demand so project repos stay thin;
freshness is lazy
(write-time hook + SessionStart reconcile, no daemon/cron); the vault syncs to a VPS while
indexes stay disposable and rebuildable. Ships as a global Claude Code plugin (`os-vault`) with skills.
**Recent pivot (2026-06-03):** Graphify **replaces** the originally-planned Ruby/SQLite
tag-index CLI and also covers the deferred QMD semantic layer. `04-build-plan.md` and
`06-graphify-evaluation.md` reflect this; if an older doc still describes the Ruby CLI, defer
to those two and fix the stale doc.
**Decisions locked (2026-06-04):** Six-facet tag taxonomy + `scope/` (ADR-011); reuse `~/Documents/SecondBrain` vault rather than creating a new one (ADR-012); build-first / migrate-incrementally — build full system against a fixture set first, defer bulk vault migration to last, onboard projects one at a time (ADR-013).
**Empirical finding locked (2026-06-05):** Graphify is a structure extractor, not a topic clusterer — no emergent hub nodes appear even at `--mode deep`; hub notes + wikilinks must be author-provided during migration (not deferred). Migration scaffolding is now a first-class deliverable. Open question: do facet tags create graph edges? (ADR-014; findings: `docs/memory-system/07-graph-connectivity-findings.md`).
**Implementation status (2026-06-09):** The global Claude Code plugin is live (`~/.claude/plugins/os-vault/`). Steps 1, 2a, 2b, 3, and 6 of the build plan are complete (including the `onboard-project` skill, previously TODO under Step 6/Part D). Step 1 completed 2026-06-09: vault-conventions.md exists, all 6 fixture notes seeded, and 14 Graphify handbook + memory-system design notes migrated to the vault as migration scaffolding. Step 4 (memsearch) completed 2026-06-09: plugin installed via marketplace, MEMSEARCH_DIR set global (~/.memsearch), Stop hook verified producing daily memory files, search confirmed working. Step 5a (memsearch episodic git sync) completed 2026-06-09: dedicated private Forgejo repo (`forgejo.swansoncloud.com/jared/memsearch`), whitelist `.gitignore` (memory/*.md only), auto-commit+push via cc-os `session-end.sh` hook (ADR-015; relocated to `memsearch_sync.py` by ADR-016, 2026-06-12). Step 5b (Obsidian vault → VPS sync) remains. See `docs/memory-system/04-build-plan.md` for full step status.
**Decision (2026-06-09):** Single global memsearch store for ALL clients — cross-client commingling is the accepted, intended design (ADR-015 resolved). One private Forgejo repo (`forgejo.swansoncloud.com/jared/memsearch`), not per-client repos. Forward direction: minimize/eliminate dedicated per-client working directories; work projects locally or use a single general `clients/` dir for non-local client work — memory is captured globally regardless of cwd. No `clients/` directory structure designed yet; open item.
**Implementation status (2026-06-12):** os-vault plugin source moved into git at `cc-os/plugins/os-vault/` and bash hooks ported to Python (deep-module architecture: shared `config.py`, `hook_io.py`, `session_state.py`; thin entry-point scripts). Cutover via symlink `~/.claude/plugins/os-vault → cc-os/plugins/os-vault/` and settings.json hook rewrite. memsearch sync split into dedicated `memsearch_sync.py` SessionEnd hook (relocation of ADR-015 behavior, not reversal). Fresh-session test passed 2026-06-12. See ADR-016.
**Implementation status (2026-06-15):** Step 5b done and automated — vault (`~/Documents/SecondBrain`) initialized as git repo, pushed to private Forgejo (`ssh://git@forgejo.swansoncloud.com:2222/jared/SecondBrain.git`); 52 files, git chosen over Syncthing. Auto-commit+push wired via `vault_sync.py` SessionEnd hook (third SessionEnd hook, runs after `session_end.py` so the daily journal note is included; push-only — SessionStart pull is an optional future item for multi-machine). Step 2e pilot done — llf-schema (`/home/jared/dev/llf-schema`, PHP 8.2 WordPress plugin) onboarded via `/os-vault:onboard-project` skill: 605 nodes / 930 edges / 52 communities; Ollama doc pass lossy on WordPress docs but AST pass solid. Step 2d closed — live vault `graphify-out/` is the baseline; fixture-only build superseded.
**Implementation status (2026-06-17):** `/os-vault:onboard-project` skill now uses assessment-first onboarding: surveys the repo structure, classifies exclude candidates by TYPE (11 categories: fetched deps, build output, caches, VCS internals, editor/AI-tooling dirs, lockfiles, coverage/logs, bulk data, binaries, secrets, graphify-out/ — illustrative names, not fixed templates), generates a per-project `.graphifyignore`, confirms with the user, then extracts the graph using `qwen25-coder-7b-16k` (config.yaml `ollama_model`; 16k-context build of qwen2.5-coder:7b — larger context window cuts chunk count per doc, the main speed lever). Fixes issue where repos with large dependency trees routed non-code files through the Ollama doc pass (see ADR-017).
**Implementation status (2026-06-30):** Phase 1 of the SecondBrain Content Plan complete (`docs/memory-system/09-sb-content-plan.md`, issues #1#6). `~/Documents/SecondBrain/vault-conventions.md` reconciled to a single authoritative typed frontmatter schema (removed the contradictory block; canonical filenames are slug-only, `source:` is a frontmatter field not a tag, `scope` is a field not a tag). New `/os-vault:design-template` skill added (`cc-os/plugins/os-vault/skills/design-template/SKILL.md`) — routes between template-design (4-step process + injection-economics filter) and new-type creation (9-step lifecycle). Three vault note templates created in `~/Documents/SecondBrain/_templates/` (`howto.md`, `convention.md`, `reference.md`; `reference` carries a four-subtype variant selector — pattern/framework, api-integration, role-definitions, design-rules), each dogfooded against a real note. Four proof-of-concept vault notes patched (cookbook subtype → pattern/framework, glossary de-duped to a wikilink, design-mode scope → project, tags + `last_reviewed` added). Phase 2 (issue #7) remains open as the steady-state migration/onboarding epic (no code). See `docs/memory-system/09-sb-content-plan.md`.
## Implemented Components
**Global os-vault plugin** — `cc-os/plugins/os-vault/` (git-tracked, 2026-06-12); symlinked into `~/.claude/plugins/os-vault`
- Hooks: `hooks/``session_start.py`, `session_context.py` (project graph path only), `post_tool_use_write.py`, `session_end.py` (vault journal), `memsearch_sync.py` (second SessionEnd hook; memsearch auto-commit+push, 30s timeout), `vault_sync.py` (third SessionEnd hook; vault auto-commit+push to forgejo.swansoncloud.com/jared/SecondBrain, 30s timeout; mirrors memsearch_sync.py)
- Shared modules: `config.py` (load_config → frozen Config dataclass), `hook_io.py` (read_input → HookInput dataclass), `session_state.py` (record_touch/read_touches; encapsulates `/tmp/claude-vault-touched-$SESSION_ID` contract)
- Skills: `skills/` — query, write, reorganize, onboard-project (assessment-first onboarding: surveys repo, classifies excludes by type (11 categories), writes per-project `.graphifyignore`, confirms with user, then extracts using `qwen25-coder-7b-16k` — per ADR-017), design-template (routes template-design ↔ new-type-creation: 4-step template-design process + injection-economics filter, 9-step new-type lifecycle; backs Phase 1 of the SB Content Plan — see `docs/memory-system/09-sb-content-plan.md`)
- Config: `config.yaml` — vault path, Ollama model (qwen25-coder-7b-16k), env vars
- Hook wiring: `~/.claude/settings.json` (hook entries invoke `/usr/bin/python3` with absolute paths into cc-os)
- Isolation seam (2026-07-06, WS2): `OS_VAULT_PATH` env var overrides `vault_path` for the write skill and every hook via `config.load_config()`; `OS_VAULT_SKIP_REBUILD` suppresses the SessionStart graphify rebuild. Added for the eval harness; zero production impact when unset.
- Write-behavior eval (2026-07-06, WS2): `plugins/os-vault/eval/` — held-out unprompted vault-write discrimination eval (Eval B/C playbook: ambiguity ladder L1 explicit → L3 conceptual, paired positives/negatives, 6 run-set scenarios on a new `reportgen` Ruby fixture + 6 frozen reserve twins in a different knowledge domain, isolated sandbox vault, headless-only runner, deterministic-first Ruby checker with narrow frozen offer-detection judge fallback stubbable via `OS_VAULT_EVAL_JUDGE_CMD`, model-free `bin/self-test` 21/21). **Scenario Task blocks are held-out — never run informally; the reserve set is never even read informally.** Procedure: `plugins/os-vault/eval/README.md`.
- Untuned baseline grid (2026-07-06, run-set × sonnet/haiku × 3 reps + 1 counted canary, 37 reps, zero harness errors): **positives 1/19, negatives 18/18**. Headline: at L1 (explicit cue) both tiers persist the knowledge every rep but route it to Claude Code's built-in auto-memory (`~/.claude/projects/<slug>/memory/`) instead of the vault; L2/L3 mostly don't persist at all; zero over-triggering anywhere. The WS2 wording loop (step 4, NOT started; run-set still uncontaminated) therefore has two targets: trigger at L2/L3, and route to the vault over auto-memory at L1. TSV: `eval/results/2026-07-06-baseline-grid.tsv`; full analysis: vault note [[os-vault-write-eval-baseline-grid-results]].
- WS2 wording loop COMPLETE (2026-07-07, 4 iterations + confirm + held-out reserve; run-set AND reserve now contaminated): shipped destination-ladder write SKILL.md + SessionStart USAGE_NOTE (mechanical triggers + before-final-reply checkpoint), mechanical vault-root resolution (`${OS_VAULT_PATH:-$HOME/Documents/SecondBrain}` — fixed real-vault breaches found on iteration 1), fixture CLAUDE.md "Cross-project memory" section (**the real-project adoption template**, required to reach haiku at L2/L3; includes ADR-vs-vault disambiguation), candidates 24 unscored, post-reserve YAML-colon quoting rule. Confirm grid sonnet 18/18 / haiku 16/18; reserve (held-out) positives 8/18 strict / trigger-axis 13/18, negatives 17/18 (vs baseline 1/19, 18/18). Residual gaps: YAML-hostile titles, haiku L2 no-trigger, generalization-laundering over-trigger. One checker false-positive fixed+rescored mid-loop (breach regex vs `2>/dev/null`); negative-criterion redesign pre-registered. TSVs: `eval/results/2026-07-07-wording-*.tsv`; loop log: `autoresearch/classic-260707-1253/loop-log.md`; analysis: vault note [[os-vault-write-eval-wording-loop-results]]. Next signal is production IRL audits.
**Global os-orchestration plugin** — `cc-os/plugins/os-orchestration/` (git-tracked, 2026-07-03); symlinked into `~/.claude/plugins/os-orchestration`
- Hooks: `hooks/``inject.py` (injects `ORCHESTRATION.md` as additionalContext to all sessions)
- Behavior: SessionStart hook injects an `ORCHESTRATION.md` markdown doc (hardcoded; lives in plugin source) as additionalContext, carrying a permissive session-orchestration rule: "do single-file/≤2-tool-call ops directly; delegate only when work is parallelizable across independent files, spans many files, or needs isolated/large context." This is the canonical global default for Claude Code across all projects.
- Migration: migrated from a standalone repo (`~/dev/cc-plugins/orchestration/`, 2026-07-03) and integrated into cc-os. Supersedes the per-project copy-pasted orchestration text blocks that previously existed in individual project CLAUDE.md files (including a stricter local override that cc-os had carried — now removed; see ADR-019).
- Session audit (WS1, 2026-07-06): 10 stratified real sessions audited via `audit/bin/extract` (Ruby fact-sheet extractor, dual-use as future eval checker) + parallel sonnet auditors + verified synthesis. Findings + E1E4 eval-scenario backlog: `docs/orchestration-audit/2026-07-06-findings.md`. Headlines: model param was silently overridden to haiku on all 23 verified spawns — root cause found post-audit: `CLAUDE_CODE_SUBAGENT_MODEL=haiku` in `~/.claude/settings.json` env block (set by an earlier session as a cost measure; removed 2026-07-06; the Fable-5 correlation was coincidental timing). Omitted model param inherits the main-loop model (opus/sonnet) — misses cluster in pre-rollout and ops sessions; no over-sharing/full-dump failures anywhere.
- Wording fixes shipped (2026-07-06, post-audit): ORCHESTRATION.md rewritten with trigger-conditioned when→then phrasing for the three verified clusters — explicit `model:` before every Agent call (cluster 2; a real cost exposure once the env override was removed), subagent model self-report on sonnet/opus spawns (cluster 1 — the E1 canary proved `resolvedModel` is NOT model-visible in the launch result; self-report is the only observable channel, and subagents do know their exact model ID), don't re-cover your own ground before delegating (cluster 3). Delegation thresholds unchanged. Cache refreshed via `bin/refresh-plugins`. Deterministic env-override detection (warn on `CLAUDE_CODE_SUBAGENT_MODEL`) is delegated to WS3's status-check design.
- Eval harness E1E3 (2026-07-06): `plugins/os-orchestration/eval/` — scripted headless orchestration-behavior eval scored by the audit extractor (extended with per-segment read targets). 6 run-set scenarios (E1P/N downgrade detection with `CLAUDE_CODE_SUBAGENT_MODEL=haiku` runner stub; E2P/N explicit-model fan-out vs no-delegation-on-trivial; E3P/N delegate-investigation vs orienting-reads-correct) + 6 frozen reserve twins; new Node.js "relaystation" fixture with deterministic sandbox-time log generation; deterministic-first checker (`eval/bin/check`, narrow frozen haiku-judge fallback for E1 language axes only, stubbable via `ORCH_EVAL_JUDGE_CMD`); model-free `bin/self-test` 21/21 green incl. shipped-instruction-compliant conformance transcripts. FAIL axes map only to verified audit misses; tier choice informational. **Scenario Task blocks are held-out — never run informally.** Procedure: `plugins/os-orchestration/eval/README.md`.
- Baseline grid run (2026-07-06, run-set × sonnet/haiku × 3 reps + 2 counted canaries, all counted, zero harness errors): negatives 18/18 PASS both tiers (zero over-delegation, zero false downgrade claims); positives 1/18 on the delegate-at-all axis — but transcript verification shows most positive FAILs are *superior direct strategies* (E2P sonnet: one scripted bulk edit; E3P sonnet: surgical greps over 400KB logs), which the WS1 audit itself scored as justified non-delegation, vs genuine miss shapes (E3P haiku: whole 112KB logs into own context; E2P haiku: 20-call per-file grind). Conditional rules validated where engaged: 14/14 spawns explicit-model; the one delegating E1P rep flagged the stubbed downgrade via self-report. Criterion redesign for E2P/E3P A-axes is pre-registered (anchor thresholds independently, NOT to this grid) before any rescore/tuning; reserve stays frozen. ORCHESTRATION.md gained a scripted-bulk-edit carve-out (a uniform multi-file change one script covers is direct work). TSV: `eval/results/2026-07-06-baseline-grid.tsv`; full analysis: vault note [[os-orchestration-eval-baseline-grid-results]].
**Global os-status plugin** — `cc-os/plugins/os-status/` (git-tracked, 2026-07-06); symlinked into `~/.claude/plugins/os-status`
- Purpose: aggregated deterministic SessionStart status checks for the cc-os plugin family — per-project artifacts (ADR system, vault hub note) and environment hazards (subagent model env override). Built via OpenSpec change `add-os-status-plugin` (supersedes the WS3 plan doc); decision record ADR-022.
- Architecture: checks are **in-process** plain Python functions in `hooks/checks.py` with the uniform signature `check(ctx) -> CheckResult(status, message)` plus one registry entry (no per-plugin subprocess protocol, no per-project code copies — both rejected in ADR-022 after perspective review). Three states: `ok` (silent), `note` (near-zero-token additionalContext line, never snoozed/suppressed), `warn` (aggregated into at most one banner; once-per-day snooze + permanent `suppress-<check>` markers in gitignored per-project `.cc-os/` — state only, never code; `.cc-os/config` holds optional `hub`/`vault_path` key=value overrides).
- Initial checks: `subagent-model-env-override` (env + `~/.claude/settings.json` `env` block; runs outside git projects too — catches the WS1 Cluster 1 incident deterministically at session start), `adr-system-present` (verbatim port of os-adr's hook behavior incl. `.os-adr/suppress`), `vault-hub-note-present` (config slug first, else `type/hub` + `project/<name>` facet-tag scan of the vault; missing → warn naming `/os-vault:write`).
- Tests: `python3 tests/hook_test.py` (36, model-free) incl. byte-identity of PRESENT_NOTE/ABSENT_NOTE against os-adr's source. Invariants in `invariants.md`. Smoke-tested 2026-07-06 (cache==source, real headless session fired the hook, env-override canary warns).
**Global os-doc-hygiene plugin** — `cc-os/plugins/os-doc-hygiene/` (git-tracked, 2026-07-03); symlinked into `~/.claude/plugins/os-doc-hygiene`
- Hooks: `hooks/hooks.json` → SessionStart hook (matcher: startup|resume) runs `scripts/reminder.py` via `${CLAUDE_PLUGIN_ROOT}` (5s timeout), emitting a deterministic zero-token reminder banner
- Behavior: Monitors stale and bloated project documentation per-project under `.dochygiene/` state dir (gitignored). SessionStart reminder is deterministic (no AI tokens, once/day snooze). Skills (verb-first, no `commands/` dispatcher — invoked directly as `/os-doc-hygiene:<skill>`, per [[cc-os-plugin-skill-naming-convention]]): `check` (AI-assisted classification of staleness signals, emits machine+human reports), `clean` (AI-assisted or deterministic patch application with git-safe scoped cleanup), `status` (read-only lifecycle-timestamp read), `sweep` (check then clean in sequence). Reversion-protected via invariants.md + golden-example test fixtures.
- Migration: migrated from standalone repo (`~/dev/cc-plugins/doc-hygiene/`, 2026-07-03) and integrated into cc-os. No content conflicts detected (doc-hygiene scope does not overlap cc-os memory system). Renamed from `doc-hygiene` to `os-doc-hygiene` per cc-os plugin naming convention. Skills renamed `hygiene-check`/`hygiene-clean` → `check`/`clean`, and the `commands/hygiene.md` dispatcher removed in favor of two new skills (`status`, `sweep`), aligning with the `os-vault`/`os-orchestration` pattern of no `commands/` directory (2026-07-03).
**Global os-adr plugin** — `cc-os/plugins/os-adr/` (git-tracked, 2026-07-03); symlinked into `~/.claude/plugins/os-adr`
- Purpose: Architecture Decision Records, consistently in every project — one customized-Nygard template (frontmatter `id`/`date`/`status`/`supersedes`/`superseded-by`/`affected-paths`/`affected-components`; Context/Decision/Consequences/Alternatives-rejected sections), `docs/adr/NNNN-kebab-title.md` one-file-per-decision + fully-regenerated `docs/adr/README.md` index. Built from `docs/adr-system/` (requirements `04`, PRD `05`, eval sketches `06`) via OpenSpec change `add-os-adr-plugin`.
- Core: Ruby OO library `lib/adr/` (`Record`, `Repository`, `Index`, `Template`, `Detector`, `Migrator`, `MigrationReport`, `Finder`; Sandi Metz style, injected paths) behind thin CLIs in `bin/` (`adr-new`, `adr-init`, `adr-detect`, `adr-migrate`, `adr-find`). Tests: `ruby tests/all.rb` (47) + `python3 tests/hook_test.py` (15), all model-free. Behavioral invariants in `invariants.md`.
- Hook: **moved to os-status (2026-07-06)** — the SessionStart existence check now runs as the `adr-system-present` check in the os-status plugin (`hooks.json` here is empty; `hooks/session_start.py` stays as the wording source of record, byte-copied into os-status). Behavior unchanged: present → near-zero-token usage note naming `/os-adr:create` + `/os-adr:find`; absent → once-per-day init/migrate suggestion, silenced permanently by `.os-adr/suppress` (still honored); silent outside git projects.
- Skills (verb-first, no `commands/`): `create` (one-invocation templated+numbered+indexed ADR with mechanical supersession), `init`, `migrate` (non-destructive: detect surveyed shapes → mechanical heuristic fill → LLM fills only manifest-listed interpretive fields via `--apply-fills``migration_confidence` frontmatter + `docs/adr/migration-report.md`; old-system deletion only as an explicit separate user-approved step), `find` (deterministic-first retrieval: affected-paths/components match → Accepted-only filter → one-hop Graphify query-path expansion with graceful degradation → AI judgment over the candidate set only).
- Migration pilot passed 2026-07-03 (sandboxed project copies; gate threshold 25% low-confidence flags): viking-warrior-training-log 0%, delta-refinery 0%, llf-schema 8.3%. Retrieval verified end-to-end against the real llf-schema project graph. Resolved: ADRs need no Graphify tag/edge convention — retrieval layer 3 expands the *query* paths via graph-node `source_file`s.
- Eval A harness (2026-07-03, ADR-021): `plugins/os-adr/eval/` — prompted skill-execution eval across model tiers (haiku/sonnet). Two fixtures (generated with the plugin's own CLIs), six scenarios (S1 create, S2 create+supersede, S3 find/conflict, S4 find/distractor, S5 init, S6 migrate+fills), deterministic Ruby checker (`eval/bin/check`, structural invariants, TSV mode for autoresearch), sandbox + headless-runner scripts, runner-prompt template. Primary run mode: in-session Agent-tool subagents with pinned `model:` (cheaper than `claude -p`); optimized via the `/autoresearch` Classic loop over SKILL.md *wording only* (checker/fixtures/scenarios frozen during a loop). Procedure: `plugins/os-adr/eval/README.md`. Self-tested both directions; grid run 2026-07-03, all scenarios × both tiers passing. Distinct from the held-out Eval B (unprompted behavior) — do not conflate.
- Eval B harness (2026-07-03, OpenSpec change `add-os-adr-eval-b-harness`): `plugins/os-adr/eval-b/` — held-out unprompted-behavior eval (requirements 45). 7 scenarios (W1W3 write-trigger, R1R4 retrieval) authored from the frozen shapes in `docs/adr-system/06-eval-scenarios.md`; dedicated Ruby webhook-relay fixture with a 6-ADR history (Superseded pair + near-miss distractors, generated via the plugin's own CLIs); R4's one-hop graph reach uses a real `graphify update` AST build (model-free, rebuilt via `eval-b/bin/build-fixture-graph`, never committed). **Headless-only runner** (`eval-b/bin/run` — fresh `claude -p` per rep, cwd = sandbox, so the real SessionStart hook fires; in-session subagents are invalid here, unlike Eval A) and a two-axis deterministic-first checker (`eval-b/bin/check`): axis (a) unprompted consultation, mechanical from transcript tool_use blocks; axis (b) correct-ADR citation (R1R4) or new-ADR-file with a narrow frozen-rubric haiku judge fallback (W1W3, `judge-rubric.md`, stubbable via `ADR_EVAL_B_JUDGE_CMD`). `R4-nograph` is the graph-degradation variant (expected FAIL). Self-tested both directions model-free via `eval-b/bin/self-test`. **Scenario Task blocks are held-out — never run them informally.** Procedure: `plugins/os-adr/eval-b/README.md`. Grid run 2026-07-03 (1 rep/cell): haiku 0/8 PASS (never unprompted-consults the ADR system in any scenario — a real gap, not a harness defect); sonnet 5/8 PASS (fails W3 — doesn't propose recording the decision; fails R1 — misses the direct-conflict retrieval). `R4-nograph` FAILed on both tiers as expected (degradation check, only meaningful paired with an R4 PASS — sonnet has one, haiku doesn't). Full results + observations (prompting-issue hypothesis, open question on whether in-session subagents could ever validly substitute for part of this measurement) written to the vault: [[os-adr-eval-b-grid-results-and-observations]].
- **Initial iteration complete (2026-07-03):** plugin build, migration pilot, Eval A grid (clean pass), and Eval B grid (baseline captured above) are all done — this closes the first pass on os-adr. Follow-on work is deliberately deferred to future sessions, not in-flight.
- **Wording experiment complete (2026-07-04):** the follow-up `/autoresearch` loop over Eval B wording ran (5 iterations, checker/fixtures/scenarios/rubric frozen; fixture CLAUDE.md declared a wording surface upfront) and closed the gap — final full grid **sonnet 8/8, haiku 7/8** (from 5/8 / 0/8 baseline; haiku's one miss is a W3 axis-b judge-boundary flicker). Winning wording shipped in `hooks/session_start.py` (PRESENT_NOTE), find/create SKILL.mds, and `eval-b/fixture/project/CLAUDE.md` (new — the trigger-phrased "Architecture decisions" section, the template for real-project adoption). Confirmed mechanisms: trigger-conditioned when→then phrasing beats inventory phrasing on both tiers; each rule must live where its precondition is visible (the reversal→supersede rule in the find skill's act-on-findings step fixed W3); lower tiers need *mechanical* triggers ("before your first edit to any existing file → run `/os-adr:find` on those paths; additions count") — semantic triggers ("architecture-level choice") only reach sonnet. Open: channel ablation never run (hook vs CLAUDE.md redundancy unknown); R4-nograph now passes both tiers, so the graph-degradation check no longer differentiates. Full hypothesis→result mapping: vault note [[os-adr-eval-b-wording-experiment-hypotheses]]. **Before designing or running any autoresearch eval, Read `~/Documents/SecondBrain/howto/running-autoresearch-skill-evals.md`.**
- **Eval B W3 stability check (2026-07-06):** 7 headless reps of haiku × W3 (all reps counted, including two the runner initially excluded, to avoid peeking bias): axis (a) consultation 7/7 PASS; axis (b) recording-offer ~5/7 (~7585%) — one FAIL plausibly infra, one a genuine behavioral miss (consulted, then asked a clarifying question instead of unconditionally proposing the superseding ADR; the conditional-phrasing failure mode iteration 3 fixed on sonnet, improved but not eliminated on haiku). Verdict: intermittent flicker, not a hard gap — grid claim stays **haiku 7/8**, W3 characterized as an ~1-in-4/5 axis-b miss. Haiku cleared for daily use with that caveat. Recorded in the vault good-enough gate: [[os-adr-eval-b-wording-experiment-hypotheses]].
Record Eval C frozen grid run: 12/12 cells PASS, Eval B wording generalizes First full run of the Eval C discrimination harness (3-level ambiguity ladder, 6-scenario run-set, 3 reps/cell, both sonnet+haiku): 36 headless reps, 12/12 cells PASS at ≥2/3 majority. Haiku 18/18 reps, sonnet 17/18 (P1-L1 rep 1: final message citing only 0005, not governing 0002 — a boundary artifact, not a behavioral miss). Zero over-trigger ADR creation on any negative at any level; all positive passes via governing-ADR citation (16 reps) or judge fallback (1). Two harness defects fixed before running the grid: (1) bin/run pipefail bug that aborted remaining reps after first FAIL (changed checker invocation to || true); (2) positive axis-b previously required ADR creation only, contradicting the README's design statement that "consulting and/or recording is correct" — run-set positives sit in already-decided territory, so find→cite→comply is the correct behavior. Implemented three-branch axis-b check: created (new ADR file), cited-governing (final message cites the governing Accepted ADR), or judge fallback. Added two self-test guards to verify the compliance path. Scenario metadata updated to document the corrected axis-b checker behavior (cited-governing as a valid pass branch). TSV reasons column now records the pass path via `B-via:` field for positives. Verdict: Eval B wording generalizes to a new fixture, new language, new domain, and decreasing cue explicitness. Grid ceiling not found; next signal is production (rollout + IRL session audits). Run-set baseline locked; wording tuning moves measurement to reserve-set.
2026-07-06 16:49:42 +00:00
- **Eval C harness built, NOT yet run (2026-07-06):** `plugins/os-adr/eval-c/` — ambiguity-ladder DISCRIMINATION eval (held-out; Eval B is contaminated by the wording loop). 3 levels (explicit → moderate → conceptual framing) × paired positive/negative scenarios; run-set (6, job-execution domain) + frozen reserve-set (6, notifications domain — becomes the measurement set if anyone ever tunes wording against the run-set). New Python fixture (`taskq` async job queue, 6-ADR history generated via the plugin's own CLIs, trigger-phrased CLAUDE.md section copied from eval-b). Scoring: positives = both axes; negatives FAIL **only on unneeded ADR creation** — consultation and truthful ADR citation are informational (`A:yes/no`, `cited-adr:yes/no`), never FAILs, because the shipped wording endorses cheap finds and the negatives sit deliberately in ADR-covered domains (two earlier negative designs that punished instruction-compliant behavior were caught in review and redesigned/fixed). Model-free `bin/self-test` green, including a truthful-citation-must-PASS guard. **Scenario Task blocks are held-out — never run informally.** Methodology: vault notes [[eval-methodology-ladder]] (per-level pass bars, reserve discipline) and [[eval-methodology-irl-feedback-loop]] (post-rollout session-audit backlog); vault also gained an `eval-results` note type + `_templates/eval-results.md` (2026-07-06).
- **Eval C frozen grid run (2026-07-06):** run-set × {sonnet, haiku} × 3 reps/cell (36 headless reps, all counted) — **12/12 cells PASS at ≥2/3 majority; haiku 18/18 reps, sonnet 17/18**. Zero over-trigger ADR creation on any negative at any level; every positive pass was via governing-ADR citation (16) or judge-recognized proposal (1), never duplicate creation. Sonnet's one FAIL rep (P1-L1) is a final-message-citation boundary artifact (implemented the ADR-0002 policy, cited only 0005), not a behavioral miss. Two harness defects fixed on the first live rep, BEFORE the grid, with Task blocks/fixtures/rubric untouched: (1) `bin/run` pipefail bug aborted a cell's reps after the first FAIL; (2) positive axis-b required creation/proposal, contradicting the README's own "consulting and/or recording is correct" — the run-set positives sit in *already-decided* territory, so find→cite→comply is correct; fixed via a mechanical `cited-governing` PASS branch + two new self-test guards (`B-via:` now recorded in TSV). Verdict: the Eval B wording **generalizes** (new fixture/language/domain, decreasing cue explicitness, paired negatives) — grid saturated, ceiling not found; next signal is production (rollout + IRL session audits), not another lab rung. Run-set is now the baseline; wording tuning against it moves measurement to the reserve-set. Full results: vault note [[os-adr-eval-c-frozen-grid-results]].
- **Remaining (locked rollout order):** real-project migration/adoption one at a time via `/os-adr:migrate` — pilot projects first, then the cc-os retrofit (its 19-ADR monolithic file was deliberately excluded from the pilot), then wider. When onboarding real projects, add the trigger-phrased "Architecture decisions" CLAUDE.md section (copy from `eval-b/fixture/project/CLAUDE.md`; candidate: emit it from `/os-adr:init`/`migrate`).
- **Resolved (2026-07-04):** os-adr's skills failed to register on first install due to stale plugin caches — the cache that was installed on 2026-07-03 17:21:48 was missing `hooks/hooks.json`, `bin/adr-detect`, `bin/adr-find`, `bin/adr-migrate`. Root cause: unknown, but the caches were restored via plugin uninstall/reinstall. A parallel issue affected os-doc-hygiene (cache retained deleted `commands/` directory). Fixed by manually refreshing both caches, then created `bin/refresh-plugins` automation to prevent future drift — see "Editing a local plugin (cache refresh)" subsection for refresh procedure. Investigation result: stale caches (not manifest-naming issues) were the cause; slash command registration works correctly once caches are fresh.
**Graphify** — v0.8.31 at `/home/jared/.local/bin/graphify`
- Vault graph: `~/Documents/SecondBrain/graphify-out/` — disposable, structural index rebuilt by SessionStart hook; handles relational/graph queries. Distinct from vault-index.json (planned, Plan B Phase 1): a flat `{tag: [{path, title, summary}]}` lookup rebuilt at SessionEnd, queried by the `/memory-find` skill for fast tag-based SB note discovery.
- Project graph: `<project-root>/graphify-out/` — same pattern; gitignore it in each project repo
- Vault conventions: `~/Documents/SecondBrain/vault-conventions.md` — frontmatter contract + tag taxonomy (canonical name decided 2026-06-09; formerly referred to as CONVENTIONS.md)
**memsearch** — v0.4.6 via Claude Code plugin marketplace (`memsearch@memsearch-plugins`)
- Hooks: Stop, SessionStart, UserPromptSubmit, SessionEnd (ship with plugin)
- Memory store: `~/.memsearch/memory/YYYY-MM-DD.md` daily files (global, cross-project, all clients — one store by design)
- Index: `~/.memsearch/milvus.db` (Milvus Lite, local); embeddings via ONNX bge-m3
- Config: `MEMSEARCH_DIR=~/.memsearch` in `~/.zshrc` for global scope
- Skills: `/memory-recall`, `/memory-config` (ship with plugin)
- Git sync: `~/.memsearch` is a dedicated private Forgejo repo (`forgejo.swansoncloud.com/jared/memsearch`); whitelist `.gitignore` commits only `memory/*.md` (excludes rebuildable `milvus.db`, model, config); auto-commit+push wired into the cc-os memory plugin's own `memsearch_sync.py` SessionEnd hook (relocated from `session-end.sh` by ADR-016, 2026-06-12; behavior preserved — see ADR-015)
**Obsidian vault git sync** — `~/Documents/SecondBrain` (2026-06-15)
- Remote: `ssh://git@forgejo.swansoncloud.com:2222/jared/SecondBrain.git` (private Forgejo; web: `https://forgejo.swansoncloud.com/jared/SecondBrain`)
- 52 files tracked (notes, journal, templates, vault-conventions.md, CLAUDE.md, .obsidian config); `graphify-out/` excluded via `.gitignore`
- Auto-commit+push via `vault_sync.py` SessionEnd hook in cc-os memory plugin (mirrors `memsearch_sync.py`; push-only — SessionStart pull is optional future item for multi-machine)
**Remaining optional items:** SessionStart vault pull (multi-machine sync; push-only is the current design); additional project onboarding (one at a time, per ADR-013); bulk vault migration. All required build steps complete as of 2026-06-15.
**Plugin renamed `memory``os-vault` (2026-07-03):** directory renamed in git, `~/.claude/plugins/os-vault` symlink and `settings.json`'s `enabledPlugins` updated — but the `local-plugins` marketplace manifest (`~/.claude/plugins/.claude-plugin/marketplace.json`) still declared the old `memory`/`./memory` entry, and `installed_plugins.json` still had a cached `memory@local-plugins` install record. Skills (and therefore slash commands) never registered under the new name as a result; hooks kept working because they're wired by absolute path in `settings.json`, independent of plugin resolution. Fixed by editing the marketplace manifest to `"os-vault"`/`"./os-vault"`, then `claude plugin marketplace update local-plugins`, `claude plugin install os-vault@local-plugins`, `claude plugin uninstall memory@local-plugins`. See ADR-018 and the "Renaming or moving a local plugin" procedure below.
### Renaming or moving a local plugin
Renaming a plugin directory under a directory-source marketplace (like `local-plugins`, whose
source is `~/.claude/plugins` itself) requires updating **three** places, not just the directory
and `settings.json`:
1. **Plugin directory / symlink** — the actual files (or symlink target, for plugins that live in
a git repo like `cc-os/plugins/os-vault/`).
2. **Marketplace manifest**`~/.claude/plugins/.claude-plugin/marketplace.json` (for
`local-plugins`) lists each plugin's `name` and `source` path explicitly; this does **not**
auto-discover from the directory listing. Update the entry to match the new name/path.
3. **`settings.json` `enabledPlugins`** — the `<name>@<marketplace>` key must match the manifest
entry from step 2.
Then refresh the plugin manager's state so it re-resolves the marketplace and drops the stale
install record:
```bash
claude plugin marketplace update <marketplace-name> # re-validates the manifest
claude plugin install <new-name>@<marketplace-name> # creates the new install record
claude plugin uninstall <old-name>@<marketplace-name> # drops the stale one
claude plugin list # verify: new name enabled, old name gone
claude plugin details <new-name>@<marketplace-name> # verify skills/agents/hooks resolved
```
Skipping step 2 is the failure mode to watch for: hooks (wired by absolute path in
`settings.json`) keep working, masking the fact that skills/slash-commands silently stopped
registering under the plugin's new name.
### Editing a local plugin (cache refresh)
Plugin source edits (including SKILL.md wording, hook scripts, or CLI code) do not reach running
sessions until the cache is refreshed. Claude Code caches plugins when installed, copying them
to `~/.claude/plugins/cache/<marketplace>/<plugin>/<version>/`. To refresh a stale cache after
source edits:
```bash
bin/refresh-plugins [marketplace-names...] # refresh all or specific local marketplaces
```
This uninstalls and reinstalls each plugin, forcing the cache to repopulate. See `bin/refresh-plugins --help` for usage. After refreshing, `claude plugin details <name>@<marketplace>` should show updated skill descriptions and hook counts. Directory-source marketplaces (e.g., `local-plugins` sourced from `~/.claude/plugins`, `cc-plugins` sourced from `~/dev/cc-plugins`) are the only ones that need manual refresh — public marketplaces auto-update.
## Issue tracking
Issues (created via `/to-issues`) live on self-hosted Forgejo (`jared/cc-os`), queried with the `tea` CLI — not GitHub/`gh`. See `docs/issue-workflow.md` for token setup, listing, implementing, and closing issues.
## OpenSpec workflow
Changes are managed spec-driven via OpenSpec. Use the matching skills rather than editing spec
files by hand: `openspec-explore` (think through an idea), `openspec-propose` (create a change
with design/specs/tasks), `openspec-apply-change` (implement tasks), `openspec-verify-change`
(validate before archiving), `openspec-archive-change`. Slash commands mirror these under
`/opsx:*`. Live changes live in `openspec/changes/`, completed ones in
`openspec/changes/archive/`, stable specs in `openspec/specs/`.
`openspec/config.yaml` uses `schema: spec-driven`; its `context` block is currently empty —
project context for OpenSpec artifacts comes from `docs/` and this file.
## Conventions specific to this repo
- **Provenance discipline:** when writing about Graphify or anything sourced from the
interview transcripts, keep the inline source tags and never promote an `[interview]` /
`[unverified claim]` to fact without checking a primary source (the GitHub repo is the
authority; it was anchored to Graphify **v0.8.30**).
- **Dates are absolute** (e.g. `2026-06-03`), and design docs carry a `_Last updated:_` /
status line — keep these current when editing.
- **Decisions live in ADRs.** Don't silently reverse a locked decision; add or amend an ADR in
`03-architecture-decisions.md` with the reasoning.
- The package on PyPI is `graphifyy` (double-y) but the command is `graphify`.
- **Keep this file current:** When a build step completes, (a) mark it done in `docs/memory-system/04-build-plan.md`, (b) add or update a component pointer in the Implemented Components section above, and (c) update the current design paragraph if the design changed. This file is the AI's orientation entry point — accuracy matters more than brevity.
- **Plugin and skill naming:** Before naming ANY new cc-os plugin, skill, or slash command, Read
`~/Documents/SecondBrain/cc-os-plugin-skill-naming-convention.md` and follow it — do NOT invent
a new convention. Rules in brief: plugins are `os-[domain]`; skills are verb-first kebab-case
(`query`, `onboard-project`), invoked as `/os-[domain]:[verb]`; no `commands/` dispatcher
directories; **never set a `name:` field in SKILL.md frontmatter** — the directory name is the
skill name, and an explicit `name:` collapses the slash command to a bare unnamespaced form
(`/find` instead of `/os-adr:find`; found and fixed 2026-07-04). After editing any SKILL.md,
run `bin/refresh-plugins` (installs cache plugin files; source edits don't reach sessions until
refreshed). Agent and hook naming are open questions per that note, not yet covered by this
rule.
**Session orchestration behavior** — Provided by the global `os-orchestration` plugin (see Implemented Components below). This repo no longer carries a local orchestration override; it follows the plugin's default behavior like every other project.