lsp_probe_bin_test daemon-reuse timing assertion flakes under high load average #648

New issue

Closed

opened 2026-09-23 17:42:22 +00:00 by jared · 1 comment

jared commented

2026-09-23 17:42:22 +00:00

Owner

lsp-probe daemon-reuse test uses a fixed timing bound that flakes under load

Problem

#645 merge verification on 2026-09-23 found timing flakes under load average ~28. plugins/os-sdlc/tests/lsp_probe_bin_test.rb#test_the_daemon_is_warmed_once_then_reused_by_later_calls_until_stop asserts wall-clock elapsed < 2.0 at line ~81. It failed with 8.0s elapsed on both main and the ticket-645 branch under load; it passed in isolation.

Cases

Given the test run at load average 28+, when the warm-call elapsed time is compared to the cold-start elapsed time measured in the same test, then the warm call is still clearly faster.
Given the test run in isolation (low load), when it runs, then it still passes.

Approach

Replace the fixed 2.0s bound with a relative bound against the measured cold-start time in the same test, keeping the assertion meaningful.

Required work:

Measure cold-start elapsed time in the test and assert warm-call elapsed is under a fraction of it (for example cold_start_elapsed * 0.5), removing the fixed < 2.0 bound — check: rake test plugins/os-sdlc/tests/lsp_probe_bin_test.rb
Confirm the assertion still fails if daemon reuse is broken (sanity check by temporarily forcing a cold spawn on every call)

Acceptance criteria

Given the test run at load average 28+, when it executes, then it passes without relaxing the check to always pass.
Given daemon reuse broken so every call cold-spawns, when the test runs, then it fails.

Non-goals

Land together with #651 (same file, related daemon lifecycle); do not solve #651's leak here.

Origin

Trigger: #645 merge verification, 2026-09-23
Chain: test flake ← fixed 2.0s timing assumption ← no load tolerance
Where: plugins/os-sdlc/tests/lsp_probe_bin_test.rb
Session: 073fd763-259f-4218-95e6-e8449caae354

# lsp-probe daemon-reuse test uses a fixed timing bound that flakes under load ## Problem #645 merge verification on 2026-09-23 found timing flakes under load average ~28. `plugins/os-sdlc/tests/lsp_probe_bin_test.rb#test_the_daemon_is_warmed_once_then_reused_by_later_calls_until_stop` asserts wall-clock `elapsed < 2.0` at line ~81. It failed with 8.0s elapsed on both main and the ticket-645 branch under load; it passed in isolation. ## Cases 1. Given the test run at load average 28+, when the warm-call elapsed time is compared to the cold-start elapsed time measured in the same test, then the warm call is still clearly faster. 2. Given the test run in isolation (low load), when it runs, then it still passes. ## Approach Replace the fixed 2.0s bound with a relative bound against the measured cold-start time in the same test, keeping the assertion meaningful. Required work: 1. Measure cold-start elapsed time in the test and assert warm-call elapsed is under a fraction of it (for example `cold_start_elapsed * 0.5`), removing the fixed `< 2.0` bound — check: `rake test plugins/os-sdlc/tests/lsp_probe_bin_test.rb` 2. Confirm the assertion still fails if daemon reuse is broken (sanity check by temporarily forcing a cold spawn on every call) ## Acceptance criteria 1. Given the test run at load average 28+, when it executes, then it passes without relaxing the check to always pass. 2. Given daemon reuse broken so every call cold-spawns, when the test runs, then it fails. ## Non-goals Land together with #651 (same file, related daemon lifecycle); do not solve #651's leak here. ## Origin - Trigger: #645 merge verification, 2026-09-23 - Chain: test flake ← fixed 2.0s timing assumption ← no load tolerance - Where: plugins/os-sdlc/tests/lsp_probe_bin_test.rb - Session: 073fd763-259f-4218-95e6-e8449caae354

jared added the

plugin/os-sdlc

filed-by/agent

labels

2026-09-23 17:42:22 +00:00

jared referenced this issue

2026-09-23 18:21:42 +00:00

os-sdlc test suite leaves ruby-lsp daemons running after full-suite runs #651

jared added the

ready-for-agent

label

2026-09-24 20:21:14 +00:00

jared referenced this issue

2026-09-25 15:00:53 +00:00

Route the ruby-lsp daemon lifecycle through the procman gem (ADR-0219 slice 1) #712

jared commented

2026-09-25 16:58:00 +00:00

Author

Owner

Resolution

Done: The daemon-reuse timing assertion is gone with the class it tested; lsp_probe_bin_test now runs only under PROCMAN_REAL=1 and reuse is procman's lease semantics (ADR-0219, #712).

Evidence: Commit cd265726 on branch ticket-712: tests/lsp_probe/daemon_test.rb and four daemon unit tests retired per the ticket's Retired tests section; suite green 1423 runs.

Follow-ups: none

## Resolution **Done:** The daemon-reuse timing assertion is gone with the class it tested; lsp_probe_bin_test now runs only under PROCMAN_REAL=1 and reuse is procman's lease semantics (ADR-0219, #712). **Evidence:** Commit cd265726 on branch ticket-712: tests/lsp_probe/daemon_test.rb and four daemon unit tests retired per the ticket's Retired tests section; suite green 1423 runs. **Follow-ups:** none

jared closed this issue

2026-09-25 16:58:00 +00:00

jared referenced this issue

2026-09-25 16:58:43 +00:00

Route the ruby-lsp daemon lifecycle through the procman gem (ADR-0219 slice 1) #712