SecondBrain/convention/tdd-methodology.md

type	title	summary	tags	scope	last_updated	date	related	source
convention	TDD and Test Isolation Methodology	Rules for writing isolated, non-flaky tests — setup-based test data over fixtures, ENV cleanup, and avoiding config caching that breaks test overrides. Answers "why is this test flaky / coupled to other tests?"	type/convention domain/testing	global	2026-07-13	2026-07-13	phlex-component-design ai-agent-rules	hyperthrive_dev/conventions/testing.md

TDD and Test Isolation Methodology

Purpose

Governs how to set up test data and handle process-global state (ENV, constants) so tests stay isolated, parallel-safe, and non-flaky.

Core Principles

1. Build test data explicitly in setup, not via fixtures. Fixtures hide what data exists behind a separate file, creating coupling between unrelated tests that happen to share a fixture. Explicit setup blocks make dependencies visible in the test itself, isolate each test's data, and work cleanly under parallel test execution.

2. Never cache dynamic/environment values in constants. A constant like COMPANY_NAME = ENV.fetch(...).freeze is captured once at class-load time — tests can't override it afterward because it's frozen before the test process even starts. Read ENV (or any config source that a test might need to vary) dynamically via a method call, not a frozen constant, so tests can verify fallback and override behavior.

3. Any test that mutates process-global state must restore it, unconditionally. Tests run in parallel processes; a leaked ENV change or global mutation from one test can flake an unrelated test. Always restore in an ensure block, not just at the end of the happy path — restoration must run even when the test body raises.

Patterns

Setup-based fixtures.

setup do
  @company = Company.create!(name: "Test Company")
  @contact = Contact.create!(name: "John Doe", company: @company)
end

ENV mutation with guaranteed restore.

test "with custom ENV" do
  original = ENV["MY_VAR"]
  ENV["MY_VAR"] = "test value"
  # ...
ensure
  original ? ENV["MY_VAR"] = original : ENV.delete("MY_VAR")
end

Dynamic config reads.

# GOOD — tests can override via ENV at runtime
def self.company_name = ENV.fetch("COMPANY_NAME", "Default")

4. CLI-contract tests that shell out inherit the workstation's live env — blank service credentials in the subprocess env. A test that runs a binary via system/backticks/Open3 passes the parent process's full ENV through, including any live service credentials on the machine (PLANKA_*, API keys, etc.). If the CLI under test talks to a service, the test can silently mutate live infrastructure. Discovered 2026-07-13: an os-backlog CLI test invoked board-ensure with live PLANKA_BASE_URL/PLANKA_USERNAME/PLANKA_PASSWORD set and created a real board on the production Planka instance. Build the subprocess env explicitly, blanking every service variable:

# GOOD — subprocess can never reach the live service
CLEAN_ENV = { "PLANKA_BASE_URL" => nil, "PLANKA_USERNAME" => nil, "PLANKA_PASSWORD" => nil }
def run_cli(*args) = Open3.capture3(CLEAN_ENV, BIN, *args)

Anti-Patterns

• Shelling out to a CLI under test without a scrubbed env → live credentials leak into the subprocess; the test can mutate real infrastructure
• Fixture files for test data → hidden coupling across tests; use setup blocks instead
• CONST = ENV.fetch(...).freeze in a service/class body → frozen at load time, untestable; wrap in a method
• ENV mutation without ensure → leaks into parallel test runs, causes flaky failures unrelated to the actual change
• Testing an ActiveJob without ActiveJob::TestHelper → can't assert assert_enqueued_with/perform behavior correctly

Related

• phlex-component-design — the ≥90% coverage bar these test patterns support
• ai-agent-rules — the "test after every change" rule this methodology implements