cc-os/docs/plans/system-prompt-profiles-assessment.md

# System-prompt profiles: assessment & recommended approach

_Last updated: 2026-07-08 · Status: assessment only, no build started_

Companion to the WS4 orchestration-economics work. Goal: reduce per-session context cost and
improve focus by controlling what reaches the model's context — the built-in system prompt,
and (more importantly) the layers we add on top of it.

## What is mechanically possible today

Verified against `claude --help` (current install) and
https://code.claude.com/docs/en/agent-sdk/modifying-system-prompts.md (fetched 2026-07-08):

| Mechanism | What it does | Risk |
|---|---|---|
| `--append-system-prompt[-file]` | Adds instructions after the full default prompt | None — nothing removed |
| `--system-prompt[-file]` | **Fully replaces** the built-in prompt for the session | High — loses tool guidance, safety rules, env context; you must re-provide them |
| `--exclude-dynamic-system-prompt-sections` | Moves per-machine sections (cwd, git status, env, memory paths) into first user message → cross-session cache reuse | Low; env context carries slightly less weight |
| `--tools <list>` / `--disallowedTools` | Restricts the built-in toolset (removes tool schemas from context) | Low |
| `--settings` / `--setting-sources user,project,local` | Controls which settings layers (and therefore CLAUDE.md files, enabled plugins, hooks) load | Low |
| Output styles (`~/.claude/output-styles/*.md`, `keep-coding-instructions: true|false`) | Replace or extend the *coding-instruction section* of the default prompt; persistent, reusable across projects | Medium — partial replacement, officially supported |
| Plugin enable/disable per session (`--settings` overlay or profile-specific settings file) | Each enabled plugin's skills/agents/hooks descriptions are injected every session | Low — biggest immediate lever |
| Agent SDK (`systemPrompt: {type:'preset', append, excludeDynamicSections}` or custom string) | Same controls programmatically; SDK default is a *minimal* prompt unless the preset is requested | — |

Deferred-tool loading (ToolSearch) already ships in current CC — most tool schemas are no
longer paid up front. That reduced the value of "gut the system prompt" since much of the old
bloat was tool definitions.

## Measured baseline (2026-07-08, headless haiku `-p` runs, `--output-format json` usage)

| Configuration | Context tokens | Delta |
|---|---|---|
| `--setting-sources "" --tools ""` (system prompt text only) | **6,347** | — |
| `--setting-sources ""` (prompt + full built-in tool schemas) | **24,277** | +17.9k = tool schemas |
| Default settings, empty dir (user stack: 17 plugins, global CLAUDE.md, hooks) | **29,042** | +4.8k = user stack |
| Full stack, cc-os dir | **41,765** | +12.7k = project layer (CLAUDE.md ~11k dominates) |

Interpretation:
- **The system prompt *text* is only ~6.3k tokens.** The ~24k harness floor is dominated by
  **tool schemas (~18k)** — so the bottom-up lever is `--tools` restriction per profile, not
  prompt rewriting. Full `--system-prompt` replacement can save at most ~6k and carries the
  fork-maintenance risk.
- The 17-plugin user stack costs ~4.8k, not tens of k — cheaper than estimated.
- The single largest single item we control is the cc-os CLAUDE.md (~11k).
- An interactive session that reads ~80k after one exchange is ~42k fixed overhead plus
  ~38k of *turn-1 work product* (web fetches, file reads) — conversation content, not
  harness bloat. Caveat: these baselines are headless haiku; an interactive Fable session's
  skill/agent listings and model-specific sections may differ somewhat — worth one `/context`
  check interactively.

## Key finding: our bloat is mostly self-inflicted, not Anthropic's

Measured 2026-07-08 for a cc-os session:

- **cc-os `CLAUDE.md`: ~43 KB ≈ ~11k tokens**, injected into every session. Much of it is an
  append-only implementation-status changelog, not orientation the session needs.
- **17 enabled plugins** — every skill description (~80 skills), agent type, and hook usage
  note is injected into every session regardless of relevance (`creative-team`,
  `invoice-ninja`, `rails-ui-component`, `api-wrapper`… in a cc-os design session).
- **memsearch SessionStart recall injection**: ~13 KB this session.
- Multiple SessionStart hooks (orchestration, os-vault, os-status) each add context.

The built-in system prompt is a static, cached prefix (cache reads are ~10% of input price
after the first turn within the 5-min TTL). The layers above are the same cost class *and*
they're the ones we control. The "bloat → hallucination" link is context rot: attention
degrades as context grows, whatever the source. Cutting 20k tokens of our own always-on
injection beats fighting Anthropic's prompt.

Pi reality check: pi.dev makes **no** speed/cost/hallucination claims on its site. Its
lean-ness comes from being a *different harness* (no MCP, no subagents, no permission
system, minimal prompt + extensions) — not from stripping Claude Code. Fully replacing CC's
system prompt while keeping CC's harness means maintaining a fork of prompt content the
model was tuned around, re-broken by every CC update.

## Recommended approach: subtract-first modular profiles

A `cyolo <flags>` shell wrapper that composes **existing** mechanisms per objective/project.
Profiles mostly *subtract* (plugins, tools, setting sources) and lightly *add*
(one append-file per objective). Full system-prompt replacement is a later, eval-gated
experiment on headless sessions only.

1. **Phase 0 — measure.** Run `/context` in 3–4 representative session types (cc-os design,
   client dev, brainstorm) and record the actual breakdown. No optimizing before baselining.
2. **Phase 1 — diet (no new machinery).**
   - Split cc-os CLAUDE.md: keep orientation (~2–3k tokens), move the implementation-status
     changelog to a linked doc the AI reads on demand (progressive disclosure).
   - Per-profile plugin enablement: a settings overlay per profile listing `enabledPlugins`;
     a brainstorm profile doesn't load rails-ui-component/api-wrapper/invoice-ninja.
3. **Phase 2 — `cyolo` profile wrapper.**
   - Central evergreen component prompts in one source dir (e.g. `~/.claude/profiles/`):
     `dev.md`, `ruby.md`, `planning.md` (objective-focused: starting point → process →
     definition of done), `brainstorm.md`.
   - Wrapper resolves profile from flag (`--dev`, `--planning`) + project detection (pwd,
     Gemfile, etc.), then launches
     `claude --append-system-prompt-file <composed> --settings <profile-settings> [--tools …]`.
   - Objective profiles state the session's endpoint explicitly ("this session ends when the
     plan is written and shared; implementation belongs to a later session").
4. **Phase 3 — eval-gated replacement experiment (optional).** Arize-style loop, matching the
   autoresearch playbook: scenario set × {default prompt, dieted stack, profile stack,
   full-replacement lean prompt} × model tiers; score task success + tokens/turn +
   wrong-tool/hallucination proxies from transcripts. Only promote full replacement if it
   wins on outcome, not just token count. Read
   `~/Documents/SecondBrain/howto/running-autoresearch-skill-evals.md` first, per standing rule.

Bottom-up track note: to audit the real shipped prompt, capture a live request body via a
local logging proxy (`ANTHROPIC_BASE_URL` pointed at a one-file pass-through that dumps the
`system` and `tools` arrays) — don't rely on the model to self-report its prompt. Given the
6.3k measurement, audit it to find what a lean profile can safely drop, but expect the
`--tools` list to be where the real savings are.

## Open questions

- Per-model profile variants: deferred (user agrees it's too granular now); the wording-loop
  evals already show tier-dependent phrasing needs, so revisit after profiles exist.
- Whether `--setting-sources` interacts cleanly with symlinked local plugins + hook absolute
  paths in `settings.json` (hooks are wired by absolute path and would still fire).
- Where profile settings overlays live so they don't fight `bin/refresh-plugins`.