os-doc-hygiene: deterministic file-length signal + implementation-status.md distill to index-with-progressive-disclosure #25

New issue

Closed

opened 2026-07-12 22:28:46 +00:00 by jared · 2 comments

jared commented

2026-07-12 22:28:46 +00:00

Owner

This was generated by AI during triage.

Type: Semi · Triage: ready

Problem

AI-read markdown files grow unbounded: docs/implementation-status.md is 670 lines and read routinely at main-loop rates. The os-doc-hygiene scanner (scripts/scanner.py) emits objective signals (broken refs, version skew, churn, archive ratio, frontmatter markers) but has no length/size signal — the bloat/distill/split taxonomy exists downstream in the classifier, so oversized files are only caught if some other signal fires.

Part 1 — length signal (afk-ready)

Add a deterministic file_length signal to the scanner: line count + estimated tokens (token_estimator.py already exists) over a configurable threshold (default suggestion: 400 lines or ~4k tokens). Objective fact only, no classification — consistent with the doc-scanner spec. Tests alongside existing scanner tests. This makes /os-doc-hygiene:check flag every oversized doc from now on, cheaply.

Part 2 — distill implementation-status.md (semi: structure decisions are human gates)

Restructure to index-with-progressive-disclosure:

Main file keeps: status timeline headline entries + per-component one-liners with pointers (the 'issue and status' view).
Details move to per-component files (e.g. docs/implementation-status/.md) read on demand.
Stale/superseded implementation detail is removed, not moved where an ADR already records the decision — ADRs are the durable record; ask 'do we even need this?' per section.
CLAUDE.md pointer text updated accordingly.

Part 3 — convention (human gate)

Record the resulting rule as an ADR: AI-maintained knowledge files are indexes with progressive disclosure; details live in leaf files; the length signal enforces it. Applies later to CLAUDE.md cleanup (see os-context prompt-files card).

Acceptance

Scanner emits file_length signal with tests; threshold configurable
/os-doc-hygiene:check on cc-os flags implementation-status.md
implementation-status.md under threshold, functioning as index; detail files created only where still needed
ADR recorded for the index+PD convention (human approved)
refresh-plugins run; docs updated per repo convention

> *This was generated by AI during triage.* **Type:** Semi · **Triage:** ready ## Problem AI-read markdown files grow unbounded: docs/implementation-status.md is 670 lines and read routinely at main-loop rates. The os-doc-hygiene scanner (scripts/scanner.py) emits objective signals (broken refs, version skew, churn, archive ratio, frontmatter markers) but has **no length/size signal** — the bloat/distill/split taxonomy exists downstream in the classifier, so oversized files are only caught if some other signal fires. ## Part 1 — length signal (afk-ready) Add a deterministic `file_length` signal to the scanner: line count + estimated tokens (token_estimator.py already exists) over a configurable threshold (default suggestion: 400 lines or ~4k tokens). Objective fact only, no classification — consistent with the doc-scanner spec. Tests alongside existing scanner tests. This makes /os-doc-hygiene:check flag every oversized doc from now on, cheaply. ## Part 2 — distill implementation-status.md (semi: structure decisions are human gates) Restructure to index-with-progressive-disclosure: - Main file keeps: status timeline headline entries + per-component one-liners with pointers (the 'issue and status' view). - Details move to per-component files (e.g. docs/implementation-status/<component>.md) read on demand. - Stale/superseded implementation detail is **removed, not moved** where an ADR already records the decision — ADRs are the durable record; ask 'do we even need this?' per section. - CLAUDE.md pointer text updated accordingly. ## Part 3 — convention (human gate) Record the resulting rule as an ADR: AI-maintained knowledge files are indexes with progressive disclosure; details live in leaf files; the length signal enforces it. Applies later to CLAUDE.md cleanup (see os-context prompt-files card). ## Acceptance - [ ] Scanner emits file_length signal with tests; threshold configurable - [ ] /os-doc-hygiene:check on cc-os flags implementation-status.md - [ ] implementation-status.md under threshold, functioning as index; detail files created only where still needed - [ ] ADR recorded for the index+PD convention (human approved) - [ ] refresh-plugins run; docs updated per repo convention

jared added the

ready-for-agent

label

2026-07-12 22:28:47 +00:00

jared commented

2026-07-13 12:54:30 +00:00

Author

Owner

Codex review adjustments (2026-07-13), adopted:

Part 2 is a convention migration, not just a reorg: the file's append-only header language and CLAUDE.md's record-keeping rule must be updated in the same change.
Acceptance made numeric: main file under BOTH 400 lines and ~4k estimated tokens (matching the shipped scanner thresholds).
Operational procedures (e.g. plugin cache-refresh runbook) must land in a stable leaf file — never silently dropped.
Add a link-integrity pass after the split (no broken refs into the new per-component files).
bin/refresh-plugins only required if plugin source is actually touched; a docs-only change doesn't need it.

Codex review adjustments (2026-07-13), adopted: - Part 2 is a convention migration, not just a reorg: the file's append-only header language and CLAUDE.md's record-keeping rule must be updated in the same change. - Acceptance made numeric: main file under BOTH 400 lines and ~4k estimated tokens (matching the shipped scanner thresholds). - Operational procedures (e.g. plugin cache-refresh runbook) must land in a stable leaf file — never silently dropped. - Add a link-integrity pass after the split (no broken refs into the new per-component files). - bin/refresh-plugins only required if plugin source is actually touched; a docs-only change doesn't need it.

jared commented

2026-07-13 13:19:09 +00:00

Author

Owner

Parts 2-3 shipped in commit 15807ef (part 1 was ef8b7e4). Index is 110 lines / ~1.8k tokens; 8 component leaves + operational-procedures leaf; ADR-restated detail removed; CLAUDE.md updated; ADR-0032 Accepted. All acceptance criteria met.

Parts 2-3 shipped in commit 15807ef (part 1 was ef8b7e4). Index is 110 lines / ~1.8k tokens; 8 component leaves + operational-procedures leaf; ADR-restated detail removed; CLAUDE.md updated; ADR-0032 Accepted. All acceptance criteria met.

jared closed this issue

2026-07-13 13:19:10 +00:00

jared referenced this issue from a commit

2026-07-13 20:13:46 +00:00

os-doc-hygiene: deterministic file_length scanner signal (issue #25 part 1)

jared referenced this issue from a commit

2026-07-13 20:13:46 +00:00

docs: distill implementation-status.md to index+PD; ADR-0032 records the convention (issue #25 parts 2-3)

jared referenced this issue

2026-11-12 16:14:53 +00:00

Use Decision::Model::Batch in os-vault judge.rb and os-me; remove the second retry loop #1711

jared referenced this issue