cc-os/docs/plans/system-prompt-profiles-assessment.md

System-prompt profiles: assessment & recommended approach

Last updated: 2026-07-08 · Status: assessment + next-actions agreed; deferral lifted — WS2/WS4 evals confirmed complete 2026-07-08 (Gate G0 cleared), execution may begin

Task breakdown for subagent implementation: system-prompt-profiles-tasks.md (2026-07-08).

Agreed next actions (2026-07-08, deferred)

Two tracks, both approved by the user:

1. Top-down diet (first, biggest win):
   • Split cc-os CLAUDE.md (~11k tokens) to a ~2–3k orientation + on-demand status/changelog doc, using @-import / pointer style per ~/servers/proxmox-hermes/docs/claude-md-maintenance.md (keep it short; prefer pointers to copies; never auto-append).
   • Then per-profile plugin enablement (~4.8k max, second priority) and a memsearch injection review.
   • Do not start while the other session's evals are running — CLAUDE.md is live input to those sessions.
2. Bottom-up audit (build anytime, read-only):
   • One-file logging proxy (ANTHROPIC_BASE_URL pass-through) that dumps the system and tools arrays from a real request — the audit basis for lean profiles. Expect savings to come from per-profile --tools lists (~18k of schemas), not prompt rewriting (~6.3k).

Related future idea (out of scope, captured in vault note claude-md-budget-linter-plugin-concept): a deterministic CLAUDE.md budget linter — hook-driven, JSON per-project config with quantified user-agreed exceptions, silent in range, maintenance agent only when out of range; possible extension to whole-project weekly hygiene scans. Overlaps os-doc-hygiene/os-status — resolve at build time.

Companion to the WS4 orchestration-economics work. Goal: reduce per-session context cost and improve focus by controlling what reaches the model's context — the built-in system prompt, and (more importantly) the layers we add on top of it.

What is mechanically possible today

Verified against claude --help (current install) and https://code.claude.com/docs/en/agent-sdk/modifying-system-prompts.md (fetched 2026-07-08):

Mechanism	What it does	Risk
--append-system-prompt[-file]	Adds instructions after the full default prompt	None — nothing removed
--system-prompt[-file]	Fully replaces the built-in prompt for the session	High — loses tool guidance, safety rules, env context; you must re-provide them
--exclude-dynamic-system-prompt-sections	Moves per-machine sections (cwd, git status, env, memory paths) into first user message → cross-session cache reuse	Low; env context carries slightly less weight
--tools <list> / --disallowedTools	Restricts the built-in toolset (removes tool schemas from context)	Low
--settings / --setting-sources user,project,local	Controls which settings layers (and therefore CLAUDE.md files, enabled plugins, hooks) load	Low
Output styles (~/.claude/output-styles/*.md, `keep-coding-instructions: true	false`)	Replace or extend the coding-instruction section of the default prompt; persistent, reusable across projects
Plugin enable/disable per session (--settings overlay or profile-specific settings file)	Each enabled plugin's skills/agents/hooks descriptions are injected every session	Low — biggest immediate lever
Agent SDK (systemPrompt: {type:'preset', append, excludeDynamicSections} or custom string)	Same controls programmatically; SDK default is a minimal prompt unless the preset is requested	—

Deferred-tool loading (ToolSearch) already ships in current CC — most tool schemas are no longer paid up front. That reduced the value of "gut the system prompt" since much of the old bloat was tool definitions.

Measured baseline (2026-07-08, headless haiku -p runs, --output-format json usage)

Configuration	Context tokens	Delta
--setting-sources "" --tools "" (system prompt text only)	6,347	—
--setting-sources "" (prompt + full built-in tool schemas)	24,277	+17.9k = tool schemas
Default settings, empty dir (user stack: 17 plugins, global CLAUDE.md, hooks)	29,042	+4.8k = user stack
Full stack, cc-os dir	41,765	+12.7k = project layer (CLAUDE.md ~11k dominates)

Interpretation:

• The system prompt text is only ~6.3k tokens. The ~24k harness floor is dominated by tool schemas (~18k) — so the bottom-up lever is --tools restriction per profile, not prompt rewriting. Full --system-prompt replacement can save at most ~6k and carries the fork-maintenance risk.
• The 17-plugin user stack costs ~4.8k, not tens of k — cheaper than estimated.
• The single largest single item we control is the cc-os CLAUDE.md (~11k).
• An interactive session that reads ~80k after one exchange is ~42k fixed overhead plus ~38k of turn-1 work product (web fetches, file reads) — conversation content, not harness bloat. Caveat: these baselines are headless haiku; an interactive Fable session's skill/agent listings and model-specific sections may differ somewhat — worth one /context check interactively.

Key finding: our bloat is mostly self-inflicted, not Anthropic's

Measured 2026-07-08 for a cc-os session:

• cc-os CLAUDE.md: ~43 KB ≈ ~11k tokens, injected into every session. Much of it is an append-only implementation-status changelog, not orientation the session needs.
• 17 enabled plugins — every skill description (~80 skills), agent type, and hook usage note is injected into every session regardless of relevance (creative-team, invoice-ninja, rails-ui-component, api-wrapper… in a cc-os design session).
• memsearch SessionStart recall injection: ~13 KB this session.
• Multiple SessionStart hooks (orchestration, os-vault, os-status) each add context.

The built-in system prompt is a static, cached prefix (cache reads are ~10% of input price after the first turn within the 5-min TTL). The layers above are the same cost class and they're the ones we control. The "bloat → hallucination" link is context rot: attention degrades as context grows, whatever the source. Cutting 20k tokens of our own always-on injection beats fighting Anthropic's prompt.

Pi reality check: pi.dev makes no speed/cost/hallucination claims on its site. Its lean-ness comes from being a different harness (no MCP, no subagents, no permission system, minimal prompt + extensions) — not from stripping Claude Code. Fully replacing CC's system prompt while keeping CC's harness means maintaining a fork of prompt content the model was tuned around, re-broken by every CC update.

Recommended approach: subtract-first modular profiles

A cyolo <flags> shell wrapper that composes existing mechanisms per objective/project. Profiles mostly subtract (plugins, tools, setting sources) and lightly add (one append-file per objective). Full system-prompt replacement is a later, eval-gated experiment on headless sessions only.

1. Phase 0 — measure. Run /context in 3–4 representative session types (cc-os design, client dev, brainstorm) and record the actual breakdown. No optimizing before baselining.
2. Phase 1 — diet (no new machinery).
   • Split cc-os CLAUDE.md: keep orientation (~2–3k tokens), move the implementation-status changelog to a linked doc the AI reads on demand (progressive disclosure).
   • Per-profile plugin enablement: a settings overlay per profile listing enabledPlugins; a brainstorm profile doesn't load rails-ui-component/api-wrapper/invoice-ninja.
3. Phase 2 — cyolo profile wrapper.
   • Central evergreen component prompts in one source dir (e.g. ~/.claude/profiles/): dev.md, ruby.md, planning.md (objective-focused: starting point → process → definition of done), brainstorm.md.
   • Wrapper resolves profile from flag (--dev, --planning) + project detection (pwd, Gemfile, etc.), then launches claude --append-system-prompt-file <composed> --settings <profile-settings> [--tools …].
   • Objective profiles state the session's endpoint explicitly ("this session ends when the plan is written and shared; implementation belongs to a later session").
4. Phase 3 — eval-gated replacement experiment (optional). Arize-style loop, matching the autoresearch playbook: scenario set × {default prompt, dieted stack, profile stack, full-replacement lean prompt} × model tiers; score task success + tokens/turn + wrong-tool/hallucination proxies from transcripts. Only promote full replacement if it wins on outcome, not just token count. Read ~/Documents/SecondBrain/howto/running-autoresearch-skill-evals.md first, per standing rule.

Bottom-up track note: to audit the real shipped prompt, capture a live request body via a local logging proxy (ANTHROPIC_BASE_URL pointed at a one-file pass-through that dumps the system and tools arrays) — don't rely on the model to self-report its prompt. Given the 6.3k measurement, audit it to find what a lean profile can safely drop, but expect the --tools list to be where the real savings are.

Open questions

• Per-model profile variants: deferred (user agrees it's too granular now); the wording-loop evals already show tier-dependent phrasing needs, so revisit after profiles exist.
• Whether --setting-sources interacts cleanly with symlinked local plugins + hook absolute paths in settings.json (hooks are wired by absolute path and would still fire).
• Where profile settings overlays live so they don't fight bin/refresh-plugins.