cc-os/plugins/os-orchestration/eval/fixture/project
jared fe83fdb290 os-orchestration: E1-E3 eval harness + cluster-1 wording v2 (self-report)
Harness: 6 run-set + 6 frozen-reserve paired scenarios from the WS1 verified
misses, Node relaystation fixture (deterministic sandbox-time logs), headless
runner with CLAUDE_CODE_SUBAGENT_MODEL stub for E1P, deterministic-first
checker driving audit/bin/extract (extended with per-segment read targets),
narrow frozen haiku-judge fallback for E1 language axes, model-free self-test
21/21 incl. shipped-instruction-compliant conformance transcripts.

Wording v2: the E1 canary (counted) proved resolvedModel is not model-visible
in the launch result, so the launch-stub-comparison rule was unactionable;
replaced with subagent model self-report on sonnet/opus spawns (subagents know
their exact model ID - verified live). Scenarios/checker/rubric untouched.
Deterministic env-override detection routed to WS3 (plan updated).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-06 18:04:15 -04:00
..
bin os-orchestration: E1-E3 eval harness + cluster-1 wording v2 (self-report) 2026-07-06 18:04:15 -04:00
docs os-orchestration: E1-E3 eval harness + cluster-1 wording v2 (self-report) 2026-07-06 18:04:15 -04:00
services os-orchestration: E1-E3 eval harness + cluster-1 wording v2 (self-report) 2026-07-06 18:04:15 -04:00
src os-orchestration: E1-E3 eval harness + cluster-1 wording v2 (self-report) 2026-07-06 18:04:15 -04:00
README.md os-orchestration: E1-E3 eval harness + cluster-1 wording v2 (self-report) 2026-07-06 18:04:15 -04:00
package.json os-orchestration: E1-E3 eval harness + cluster-1 wording v2 (self-report) 2026-07-06 18:04:15 -04:00

README.md

Relaystation

Relaystation is a small webhook and event relay service. Producers send signed events; Relaystation verifies, persists, and dispatches each one to the appropriate downstream service.

Overview

Events flow through four stages: ingest (src/relay.js), auth (src/auth.js), routing (src/router.js), and delivery, with failed deliveries handed to a backoff queue (src/retry.js). Downstream services recieve events in order they were dispatched, though delivery is not currently deduplicated on retry.

Layout

  • src/ — relay, router, retry, auth, metrics, and store modules.
  • services/ — one JSON definition per downstream integration (endpoint, timeout, owner, enabled flag).
  • docs/ — architecture overview, deployment guide, and on-call runbook.
  • bin/gen-logs — generates sample log data for local testing.

Getting started

See docs/architecture.md for how the pieces fit together and docs/deploy.md for environment variables and rollout steps. If you're on call, start with docs/runbook.md.

Status

Version 1.3.0. Internal service — not published externally.