cc-os/plugins/os-sdlc/reference/self-improvement-loops.md

self improvement loops

Status: direction — as of 2026-07-16 Connects to: never-ask-twice, plugin-factory, pipeline-observability, standards-and-conventions, debugger, overview

Purpose

Names the four periodic audit loops that keep the factory (and the plugins it builds) honest over time, and which of them already exist vs. are new. Load this before designing any audit skill, before scoping ADR-audit work, or before wiring an improvement-ticket generator into the factory.

Design

Four loops, each catching a different kind of drift. None of them fire mid-run — all are periodic, out-of-band audits, deliberately kept separate from the pipeline itself so an audit's own cost/latency never blocks a ticket.

• a. Session audits (exists) — os-context's biweekly IRL audit. Catches behavior drift in how sessions actually run. Needs rework per cc-os Forgejo issue #71: current criteria don't catch expensive-model overuse where delegation or determinism would cut cost with no quality drop — a gap directly relevant to os-sdlc's actor-choice discipline (see deterministic-gates's code/engineer/agent framing).
• b. ADR audits (new, proposed 2026-07-16) — catches corpus drift, the counterpart to session audits but aimed at the decision record itself rather than session transcripts. Periodically audits the ADR corpus for: stale decisions, missed promotions to the vault (a local ADR that should have graduated per never-ask-twice's promotion-by-audit step but didn't), and scope-classification judgment calls that slipped through at mint time. A wrong mint-time call is treated as a delay, not a loss — anything missed gets caught in the next ADR audit, so the loop is self-correcting rather than requiring the mint-time call to be perfect. Proposed mechanics (2026-07-16, user): the copy-tag-audit pattern from pipeline-observability — a hook copies every new ADR into the vault at write time; the audit reads only virgin (untagged) copies, keeps the durable ones, revises for clarity, tags them audited; the next round checks only new virgin ADRs. This makes the capture side deterministic and the audit's workload bounded.
• c. Eval harnesses (exists — autoresearch pattern, already running for os-vault, os-adr, os-context) — catches capability regression. Functions as "the AI version of TDD" for plugin-type projects: a plugin's eval suite is the thing that must stay green across changes, the same role a test suite plays for application code (see plugin-factory for the eval-as-green-command connection).
• d. Skill-lint (new, cc-os Forgejo issue #72) — catches skill-authoring drift. os-context lints SKILL.md files for progressive disclosure on complex tasks and reasonable file sizes, keeping agent context windows lean as the skill corpus grows.

Two agent-tuning applications of loop (c)'s autoresearch pattern, both post-tracer-bullet:

• Standards minimization (standards-and-conventions) — the production agents (programmer/test-writer/reviewer) run with minimal standards in prompt; a full-standards Judge critiques the output and returns clues about which standard was missed. Keep/discard on those clues admits a prompt line only when it fixes an observed failure category — same coverage as front-loading everything, far less standing context. Discipline: baseline at zero standards first; judge clues feed the tuning loop only, never a production run (the runtime feedback channel stays the deterministic gate); judge per stage (so you know whose prompt to tune) with one end-to-end score as a guardrail against stages locally optimizing against each other.
• Lint-explanation copy-formula — the debugger's lint-thrash signal (a programmer failing the same rule repeatedly) says a gate message is failing to direct the fix. Autoresearch a copy formula (structure + perspective) that maximizes fix-rate per token of explanation — the linter messages are effectively the agent's prompt now, so their quality is worth tuning.

The debugger is a fifth improvement input: its "why the stuck team stalled" diagnosis turns recurring stall causes into harness/prompt/gate fixes.

Together, these four loops are:

• the promotion engine for never-ask-twice — (b) is the mechanism that actually performs the audit-driven promotion that mechanism depends on;
• the self-evolution channel for plugin-factory — (a), (c), and (d) are exactly the inputs that turn into improvement tickets fed back into factory intake.

Open questions

• ADR-audit criteria: what specifically counts as "stale" and "durable," and cadence. (Mechanical detection is now sketched via the copy-tag-audit pattern above — remaining question is the judgment rubric the audit applies to each virgin ADR.)
• Whether ADR audits live in os-context (alongside the existing session-audit skill) or in os-adr (alongside find/create/migrate) — an ownership question, not yet settled.
• Whether (a)-(d) should share a common audit-report format so plugin-factory's improvement-ticket generator can consume all four uniformly, or whether each stays bespoke.

Sources

• os-context audit-sessions skill (existing biweekly IRL audit)
• cc-os Forgejo issue #71 (session-audit rework, expensive-model overuse)
• cc-os Forgejo issue #72 (skill-lint)
• plugins/os-vault/eval/, plugins/os-adr/eval/, plugins/os-context/eval/ (autoresearch eval harnesses)
• ADR-0037
• 2026-07-16 design session (this doc's origin)