jared/cc-os

Watch

Fork 0

Copy a pre-migrated template SQLite DB per test instead of running the schema migration per test #628

New issue

Open

opened 2026-09-22 21:14:19 +00:00 by jared · 1 comment

jared commented

2026-09-22 21:14:19 +00:00

Owner

Copy a pre-migrated template SQLite DB per test instead of running the schema migration per test

Problem

Follow-up to #624 (merged main cf017e0c 2026-09-22), which took the suite from 115 s to 45-53 s by removing every stay_serial! mark. The user asked for this ticket after reviewing the remaining profile. The user's assessment: the CLI endpoints already have direct unit tests; driving the Thor layer through a subprocess per test is a waste.

Observed

After #624 the suite runs at 8.5x parallelism on 12 cores (424 s summed test time / 49.8 s wall), so the parallel floor is about 35 s and the remaining cost is per-test work, not serialization. Three classes drive the real bin/os-sdlc-runner through Open3.capture3 once per test. Each call pays a fresh Ruby boot: interpreter, requires, Zeitwerk eager load, SQLite schema migration. Under load that is 5-6 s per test. Summed: CliNextInstructionTest 43 s, CliImplementationOpenTest 42 s, CliSubagentStopTest 22 s. IssueSourceTest shows 12 s per test while doing no IO; it is starved by the GVL while neighbor threads wait on subprocesses. Profile of one CLI test in the original #624 body put schema recreation at about 12% and TeaTracker#fetch_body at 9%.

Reproduce

cd plugins/os-sdlc && ruby tests/all.rb -v 2>&1 | sort -t= -k2 -rn | head -20

Expected

A test that needs a pipeline DB gets a copy of one pre-migrated template file (schema SCHEMA_VERSION, stamped user_version) instead of running Model::Db.create_tables from scratch.

Acceptance criteria

Model::Db.create_tables runs at most once per suite process for file DBs created by tests (measure with a counter or a profile).
ruby tests/all.rb passes with 0 failures, three runs.
Summed test time drops against the post-#638 baseline; record both numbers in the leaf History row.

Scope note

Narrowed 2026-09-23. The in-process CLI test conversion and the per-binary smoke test moved to #638 (child of #631). This ticket keeps only the template-DB copy.
Removed from this ticket and not reassigned yet (a user decision is needed): (1) rerun IssueSourceTest alone to confirm GVL contention; (2) lower the OS_SDLC_SUITE_BUDGET_S default in tests/all.rb.
Open decision before ready-for-agent: does the template copy live in production (Model::Db.recreate copies a per-process template) or in the test helper? A test-helper-only change has no red round in the pipeline. Add ## Owner sketch and ## Cases after the decision.

Origin

Trigger: profile run after #624, session c15fe063 (2026-09-22).
Improvised this session: none
Chain: slow suite ← per-test process boot in Cli*Test ← DESIGN (tests exercise the binary instead of the command object).
Root candidate: this ticket.
Where: plugins/os-sdlc/tests/pipeline/cli_next_instruction_test.rb, cli_implementation_open_test.rb, cli_subagent_stop_test.rb; lib/os_sdlc/pipeline/cli.rb.
Session: c15fe063
Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os/c15fe063-c64c-417b-b4e1-53a52ffca319.jsonl

# Copy a pre-migrated template SQLite DB per test instead of running the schema migration per test ## Problem Follow-up to #624 (merged main cf017e0c 2026-09-22), which took the suite from 115 s to 45-53 s by removing every stay_serial! mark. The user asked for this ticket after reviewing the remaining profile. The user's assessment: the CLI endpoints already have direct unit tests; driving the Thor layer through a subprocess per test is a waste. ## Observed After #624 the suite runs at 8.5x parallelism on 12 cores (424 s summed test time / 49.8 s wall), so the parallel floor is about 35 s and the remaining cost is per-test work, not serialization. Three classes drive the real bin/os-sdlc-runner through Open3.capture3 once per test. Each call pays a fresh Ruby boot: interpreter, requires, Zeitwerk eager load, SQLite schema migration. Under load that is 5-6 s per test. Summed: CliNextInstructionTest 43 s, CliImplementationOpenTest 42 s, CliSubagentStopTest 22 s. IssueSourceTest shows 12 s per test while doing no IO; it is starved by the GVL while neighbor threads wait on subprocesses. Profile of one CLI test in the original #624 body put schema recreation at about 12% and TeaTracker#fetch_body at 9%. ## Reproduce ```bash cd plugins/os-sdlc && ruby tests/all.rb -v 2>&1 | sort -t= -k2 -rn | head -20 ``` ## Expected - [ ] A test that needs a pipeline DB gets a copy of one pre-migrated template file (schema SCHEMA_VERSION, stamped user_version) instead of running `Model::Db.create_tables` from scratch. ## Acceptance criteria - [ ] `Model::Db.create_tables` runs at most once per suite process for file DBs created by tests (measure with a counter or a profile). - [ ] `ruby tests/all.rb` passes with 0 failures, three runs. - [ ] Summed test time drops against the post-#638 baseline; record both numbers in the leaf History row. ## Scope note Narrowed 2026-09-23. The in-process CLI test conversion and the per-binary smoke test moved to #638 (child of #631). This ticket keeps only the template-DB copy. Removed from this ticket and not reassigned yet (a user decision is needed): (1) rerun IssueSourceTest alone to confirm GVL contention; (2) lower the OS_SDLC_SUITE_BUDGET_S default in tests/all.rb. Open decision before ready-for-agent: does the template copy live in production (`Model::Db.recreate` copies a per-process template) or in the test helper? A test-helper-only change has no red round in the pipeline. Add `## Owner sketch` and `## Cases` after the decision. ## Origin - Trigger: profile run after #624, session c15fe063 (2026-09-22). - Improvised this session: none - Chain: slow suite ← per-test process boot in Cli*Test ← DESIGN (tests exercise the binary instead of the command object). - Root candidate: this ticket. - Where: plugins/os-sdlc/tests/pipeline/cli_next_instruction_test.rb, cli_implementation_open_test.rb, cli_subagent_stop_test.rb; lib/os_sdlc/pipeline/cli.rb. - Session: c15fe063 - Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os/c15fe063-c64c-417b-b4e1-53a52ffca319.jsonl

jared added the

plugin/os-sdlc

filed-by/user

labels

2026-09-22 21:14:19 +00:00

jared referenced this issue

2026-09-23 12:03:43 +00:00

CLI invocation seam + in-process CLI tests #631

jared referenced this issue

2026-09-23 13:04:18 +00:00

os-sdlc-runner: Pipeline::CLI receives an Invocation; bin/os-sdlc-runner is the only reader of process globals #638

jared changed title from ~~Run os-sdlc CLI tests in-process and copy a template DB; bin/os-sdlc-runner boots per test cost 107 s of 424 s~~ to Copy a pre-migrated template SQLite DB per test instead of running the schema migration per test

2026-09-23 13:06:11 +00:00

jared referenced this issue

2026-09-23 13:06:24 +00:00

CLI invocation seam + in-process CLI tests #631

jared referenced this issue

2026-09-23 13:10:45 +00:00

os-sdlc test suite regressed from 21 s to 112-129 s; three causes, fix and add a guard against reversion #624

jared commented

2026-09-23 13:14:28 +00:00

Author

Owner

What remains uncovered: the template-DB copy itself. plugins/os-sdlc/lib/os_sdlc/model/db.rb recreate (line 76) still calls create_tables(db) fresh every time; there is no pre-migrated template file or copy path anywhere in lib/os_sdlc/model/ or tests/support/. The ticket's own "Open decision before ready-for-agent" (production Model::Db.recreate vs. test-helper-only) is still unresolved, and the two removed-scope items (IssueSourceTest GVL re-check, OS_SDLC_SUITE_BUDGET_S default) are unassigned.

Leaving this open: it is not covered by #638/#639-#642 and nothing on main implements it. No branch or active session found for #628 (git branch -a / worktree list show none). Needs the production-vs-test-helper decision recorded (Owner sketch + Cases) before it can move to ready-for-agent.

Reviewed after the #628/#631 split. #638 (child of #631) absorbed the in-process CLI test conversion and per-binary smoke tests; #639-#642 cover the other plugin bin/ Invocation conversions. None of those touch DB setup cost. What remains uncovered: the template-DB copy itself. plugins/os-sdlc/lib/os_sdlc/model/db.rb `recreate` (line 76) still calls `create_tables(db)` fresh every time; there is no pre-migrated template file or copy path anywhere in lib/os_sdlc/model/ or tests/support/. The ticket's own "Open decision before ready-for-agent" (production Model::Db.recreate vs. test-helper-only) is still unresolved, and the two removed-scope items (IssueSourceTest GVL re-check, OS_SDLC_SUITE_BUDGET_S default) are unassigned. Leaving this open: it is not covered by #638/#639-#642 and nothing on main implements it. No branch or active session found for #628 (git branch -a / worktree list show none). Needs the production-vs-test-helper decision recorded (Owner sketch + Cases) before it can move to ready-for-agent.

jared referenced this issue

2026-09-23 16:46:03 +00:00

Ticket sections have no shared model, and pipeline roles work from silos of the ticket #643

jared referenced this issue

2026-09-23 17:08:48 +00:00

Ticket sections have no shared model, and pipeline roles work from silos of the ticket #643

jared referenced this issue

2026-09-24 15:44:24 +00:00

CLI invocation seam + in-process CLI tests #631

jared added the

category/owner-ruling

label