cc-os/plugins/os-doc-hygiene/scripts/CONTEXT.md

scripts/

Deterministic Python scripts for the doc-hygiene plugin. All scripts are model-free (no API calls), produce structured JSON output, and exit with meaningful codes. Each script is a small, single-responsibility class designed for isolated testing with injected clocks/filesystems.

Contents

File	Purpose
scanner.py	Deterministic doc scanner. Walks scoped files under the resolved project root, applies the ordered short-circuiting exclusion pipeline (dir-prune incl. .cc-os/ and legacy .dochygiene/ self-exclusion → .dochygiene-ignore match → hygiene: frozen frontmatter → append-only-log detection; invariant #9), and computes objective per-path signals (broken refs, version skew, edit-recency vs git churn, location, archive ratio, frontmatter markers). Emits the intermediate artifact {project_root, scope_globs, excluded_dirs, files_scanned, shortlist, signals} — NOT a machine report (no entries/category/op/token_estimate). No model. Injected git-log / clock for testability. Default excludes (echoed verbatim in excluded_dirs): bare names — build, vendor, archive, graphify-out, .cc-os (shared cc-os state dir, ADR-027), .dochygiene (legacy state dir, invariant self-exclusion), fixtures — are matched on the directory name segment at any depth (any dir named archive is pruned, not just docs/archive); plus one path-aware entry examples/golden (any entry containing / is a parent/child name pair, pruned only where the child's immediate parent matches parent, depth-independent). fixtures + examples/golden prevent false-positive flagging of test-fixture trees and classifier golden inputs (deliberately-stale docs). golden is deliberately path-aware — a bare golden name-match would wrongly skip legitimate golden/ dirs in unrelated projects (doc-hygiene installs globally). The golden-test harness is unaffected: it roots the scanner at …/examples/golden/classifier/N/input/, so the examples→golden pair sits in the ancestry above the walk root and is never re-encountered as a child. Additive to this hardcoded-exclude prune, the scanner also consults rulebook.py for lifecycle rule matches during the walk (directory-rule matches prune the walk and emit one aggregate entry; IGNORE-surface rules prune with zero emission; file-rule matches attach a lifecycle signal alongside existing signals) — two independent prune mechanisms: the hardcoded DEFAULT_EXCLUDED_DIRS list is the safety net (fixed, not project-configurable), while the rulebook is the configurable lifecycle layer (global + per-project rules, tiered by lifetime taxonomy per ADR-0039).
state_store.py	State store. resolve_project_root(start_dir, fs) (pure: git root, fallback cwd) plus StateStore confining all writes to <root>/.cc-os/dochygiene/ (invariant #3, ADR-027, never edits .gitignore). Reads fall back to the legacy <root>/.dochygiene/ when the canonical dir has no state; the first write migrates legacy files in and removes the legacy dir. Lifecycle timestamps (last_check/last_clean/last_reminded), atomic write-temp→fsync→os.replace (D9), and report rollover keeping exactly one .json+.md pair (invariant #4). Injected clock.
reminder.py	SessionStart reminder. Pure Reminder.decide(last_check, last_reminded, now) decision (stale-and-not-snoozed ⇒ banner; calendar-day snooze keyed on last_reminded, invariants #1/#2). ReminderRunner wires it to the StateStore, emits a zero-token systemMessage JSON banner on stdout, and writes last_reminded as the only mutation, only on banner. Treats missing/corrupt state as never-checked (stale); always exits 0 — never blocks the session. No-ops unless cwd is inside a real git project (_is_git_project guard) so the user-scoped hook never creates .cc-os/dochygiene/ in an arbitrary dir like ~ (invariant #3) — flagged for human review. No scan, no model. DEFAULT_THRESHOLD_DAYS is an implementation constant, not a spec'd contract. Invoked by hooks/hooks.json via ${CLAUDE_PLUGIN_ROOT}/scripts/reminder.py.
validate_report.py	Schema-validator for machine-report JSON files. Enforces the frozen report contract: envelope fields, closed category enum, closed exact_edit.kind enum with per-kind required sub-fields, op_type↔exact_edit biconditional (invariant #11), and safety_tier derivation match (invariant #10). Collects all violations. Exit 0 = valid, exit 1 = invalid, exit 2 = usage error.
report_builder.py	Deterministic finalize pass (no model). Consumes the scanner artifact + the subagent's slim proposals and assembles a schema-valid machine report plus a human-report skeleton. Fills the four guardrail fields the model must not author: computes each entry's expected_sha256 over the live file, looks up is_destructive/is_reversible from the KIND_TABLE (keyed on exact_edit.kind), derives safety_tier via validate_report's derive_safety_tier (invariant #10 — single source of truth), and sources raw_tokens from token_estimator (over the reducible_range/anchor text). Rejects malformed proposals with a structured stderr error ({index, field, message}, exit 1); --out-json/--out-md write files and suppress the stdout bundle. An empty proposals array yields a valid empty-entries report.
token_estimator.py	Deterministic local token estimator (no model, no network — invariant #6). Produces the token_estimate.raw_tokens count for report entries. Pluggable TokenEstimator ABC with two backends: HeuristicEstimator (ALWAYS available, zero deps, ceil(len/4)) and TiktokenEstimator (optional accuracy upgrade, o200k_base, disallowed_special=()). default_estimator(cache_dir=…) selects tiktoken only when it is importable AND the vocab is vendored locally (detect-before-fetch: checks the sha1-named cache file exists before loading, so a cold cache never triggers a network download); otherwise returns the heuristic. Never raises; active backend is introspectable via .name. estimate(text)→int, estimate_file(path)→int (utf-8, errors="replace"), estimate_for_report(text)→{raw_tokens, injection_frequency:null, weighted_tokens:null} (v1 emits only raw_tokens; weighting is the v2 bonus). CLI: python token_estimator.py <path> prints JSON {path, backend, token_estimate}, exit 1 on missing file. tiktoken vocab is NOT committed (~2 MB; gitignored at scripts/tiktoken_cache/). Pre-warm once with: TIKTOKEN_CACHE_DIR=scripts/tiktoken_cache python -c "import tiktoken; tiktoken.get_encoding('o200k_base')". Cache-filename gotcha: tiktoken names the cache file sha1(blobpath_url) = fb374d419588a4632f3f557e76b4b70aebbca790 (NOT the sha256 content hash); a vendored file under any other name never hits and falls back to network.
rulebook.py	Deterministic rulebook loader (no model). Parses the global rulebook.json plus an optional per-project .dochygiene-rules.json envelope ({"schema_version": 1, "rules": [...]}), hard-failing on unparseable JSON/unknown schema_version and skip-and-warning on invalid/unconfirmed rules (missing confirmed_by → inactive). Compiles globs via glob.translate(recursive=True, include_hidden=True) (Python ≥3.13 required, clear error otherwise). Merges add-only across a two-axis precedence — project file > project dir > global file > global dir; within a tier, longest pattern then last-defined — with keep-shadowing neutralization; unmatched paths resolve to None; file-vs-directory match are distinguished; IGNORE-sentinel rules (no lifetime) prune with zero emission. Single query surface consumed by scanner.py's lifecycle signal pass.
calibrate_helpers.py	Deterministic helpers (no model) backing the /os-doc-hygiene:calibrate skill: clustering/sampling of scanner shortlist candidates for haiku nomination, rule-report assembly for the human-approval step, and class-not-path + prefer-narrower checks that keep proposed rules generalized rather than single-path. The skill itself drives the AI nomination/judge steps; this module is the deterministic scaffolding around them.
patch_applier.py	Deterministic patch-applier: consumes the machine report's deterministic entries, applies each exact_edit.kind mechanically via a per-file transaction (read-once → verify whole-file sha256 against expected_sha256 → apply edits descending by line, insert-frontmatter last → write once), skips files on content-guard mismatch or incompatible-op combinations, uses git mv for move-to-archive; structured JSON output, no model. CLI: python patch_applier.py --report <path> --apply-indices <idx,idx,...> emits {applied, skipped, failed, staged_paths} to stdout, exit 0 (partial ok), exit 1 (some applied + skipped), exit 2 (hard failure — rollback recommended). Invariant #8 (mtime guard).

Planned additions (future changes)

• token-estimator v2 — injection-frequency weighting + bottom-up rollup (weighted_tokens)