Harden stay_serial! executor shutdown/start cycle against orphaned workers and leftover sentinels #503

New issue

Closed

opened 2026-08-25 01:01:48 +00:00 by jared · 1 comment

jared commented

2026-08-25 01:01:48 +00:00

Owner

Context

plugins/os-sdlc/tests/test_helper.rb run_test_exclusively shuts down and restarts Minitest's parallel executor per serial test. In minitest-6.0.6 parallel.rb, Executor#start replaces @pool unconditionally: if shutdown raises or is interrupted, orphaned threads keep popping the shared @queue, and surplus nil sentinels pushed per shutdown cycle can kill the next pool's threads on their first pop. Observed during #502 verification: 1 fatal "No live threads left. Deadlock?" hang in 22 suite runs, main thread at parallel.rb:54 Thread#join under run_test_exclusively; no thread held the SerializedChdir Monitor, ruling out the new chdir lock.

Ticket-skeptic verdict: CREATE (root location is run_test_exclusively itself; provenance is an observed hang plus minitest source reading).

Tasks

Harden the cycle: restart only after successful shutdown, drain leftover sentinels, or replace shutdown/start with a read-write lock (parallel tests read, serial tests write).
Reassess whether HandRolledFetchTest still needs stay_serial! now that test_helper serializes every Dir.chdir behind a Monitor.

Acceptance criteria

Repeated full parallel suite runs show no executor deadlock/hang and stay at the known baseline failures.

Origin

Trigger: 10-consecutive-run acceptance check for #502 (ruby tests/all.rb loops)
Improvised this session: none (hang did not recur; no workaround applied)
Chain: fatal deadlock hang ← executor shutdown/start hole in run_test_exclusively (test_helper.rb) ← UPSTREAM-TOOL (minitest 6.0.6 parallel.rb Executor#start/#shutdown semantics)
Root candidate: this ticket

## Context plugins/os-sdlc/tests/test_helper.rb run_test_exclusively shuts down and restarts Minitest's parallel executor per serial test. In minitest-6.0.6 parallel.rb, Executor#start replaces @pool unconditionally: if shutdown raises or is interrupted, orphaned threads keep popping the shared @queue, and surplus nil sentinels pushed per shutdown cycle can kill the next pool's threads on their first pop. Observed during #502 verification: 1 fatal "No live threads left. Deadlock?" hang in 22 suite runs, main thread at parallel.rb:54 Thread#join under run_test_exclusively; no thread held the SerializedChdir Monitor, ruling out the new chdir lock. Ticket-skeptic verdict: CREATE (root location is run_test_exclusively itself; provenance is an observed hang plus minitest source reading). ## Tasks - [ ] Harden the cycle: restart only after successful shutdown, drain leftover sentinels, or replace shutdown/start with a read-write lock (parallel tests read, serial tests write). - [ ] Reassess whether HandRolledFetchTest still needs stay_serial! now that test_helper serializes every Dir.chdir behind a Monitor. ## Acceptance criteria - [ ] Repeated full parallel suite runs show no executor deadlock/hang and stay at the known baseline failures. ## Origin - Trigger: 10-consecutive-run acceptance check for #502 (ruby tests/all.rb loops) - Improvised this session: none (hang did not recur; no workaround applied) - Chain: fatal deadlock hang ← executor shutdown/start hole in run_test_exclusively (test_helper.rb) ← UPSTREAM-TOOL (minitest 6.0.6 parallel.rb Executor#start/#shutdown semantics) - Root candidate: this ticket

jared added the

plugin/os-sdlc

label

2026-08-25 01:01:48 +00:00

jared referenced this issue

2026-08-25 01:02:00 +00:00

Absolutize ResolverScenario root, drop its Dir.chdir, retire stay_serial! on resolver_test #502

jared commented

2026-09-23 20:23:31 +00:00

Author

Owner

Resolution

Done: Closed in the 2026-09-23 relevance pass of retired-template tickets: OBSOLETE.

Evidence: stay_serial! fully removed: 0 hits in tests/*.rb; commits 7e526f83 (#502) + 8615acc9 (#624) retired all serial marks

Follow-ups: none

## Resolution **Done:** Closed in the 2026-09-23 relevance pass of retired-template tickets: OBSOLETE. **Evidence:** stay_serial! fully removed: 0 hits in tests/*.rb; commits 7e526f83 (#502) + 8615acc9 (#624) retired all serial marks **Follow-ups:** none

jared closed this issue

2026-09-23 20:23:31 +00:00