cc-os/plugins/os-orchestration/eval/fixture/project/docs/handler-spec.md

Per-service delivery handlers

Relaystation currently delivers every event through the single generic path in src/relay.js (deliver → fetchLike, with failures handed to src/retry.js). That path treats every downstream service identically: same payload shape, same retry policy, same failure handling. In practice each of the nine downstream integrations in services/ has different tolerance for retries, different payload requirements, and different failure semantics, and three cross-cutting concerns (auth concurrency, per-service latency visibility, and dead-letter handling) need their own dedicated modules.

This spec defines twelve new handler modules under src/handlers/. Each module is self-contained and does not require changes to src/relay.js, src/router.js, src/retry.js, or any file under services/ — implement each file to the interface and behavior below.

Conventions

• 'use strict'; at the top, matching the rest of src/.
• Each module exports a plain object via module.exports = { ... } (same style as src/metrics.js / src/store.js), not a class.
• Where a handler needs the shared metrics counters, require('../metrics') and call increment(name) — do not create a second counter store.
• Each file should be small: roughly 20–40 lines including requires and module.exports.
• No new npm dependencies. Use only Node core modules already used elsewhere in src/ (crypto, fs, path, http) plus what a handler needs.
• Tests are not required for this pass — implementation only.

Part A — one handler per downstream service (9 files)

Each service handler lives at src/handlers/<service-name>.js and exports two functions:

module.exports = {
  prepare(event),   // returns the outbound payload object for this service
  isRetryable(err), // returns true/false given an error object with a
                     // `statusCode` field (may be undefined for network errors)
};

err.statusCode is undefined for a connection/network failure (no response received) and an HTTP status integer when a response was received but was not 2xx. Implement prepare and isRetryable exactly per the per-service rules below — the rules differ enough between services that no single template satisfies more than one of them.

Service	prepare(event) payload rule	isRetryable(err) rule
analytics	Wrap the event: return { batch: [event], emitted_at: Date.now() }.	Retry when statusCode is undefined (network) or >= 500. Never retry on any 4xx.
audit	Return the event plus an integrity field: { ...event, hash: sha256-hex of JSON.stringify(event) } (use crypto.createHash('sha256')).	Retry only on a network failure (statusCode === undefined). Any received HTTP status, 4xx or 5xx, is not retryable — audit delivery failures must surface immediately rather than silently retry.
billing	Return a payload containing only an allowlist of fields copied from event, dropping everything else: id, topic, amount, currency, receivedAt. Fields absent on the event are simply omitted from the output, not set to null.	Retry only on statusCode === 429 or statusCode === 503. Nothing else (not network failures, not other 5xx) is retryable.
crm	Return a shallow-transformed payload: rename topic to event_type, and if event.payload has a nested contact object, flatten it to top-level contact_id and contact_email fields (copied from event.payload.contact.id / .email if present) instead of nesting it.	Retry when statusCode is undefined or >= 500, same as analytics — but cap distinctly (see Part B note below; the cap itself lives in the retry loop, not in this predicate).
email	If event.payload is missing a to field, do not build a payload at all — call metrics.increment('handlers.email.invalid') and return null (the caller is expected to skip sending when prepare returns null). Otherwise return the event unchanged.	Retry when statusCode is undefined, 429, or >= 500.
reports	Collapse the event into a single-field summary payload: { summary: \${event.topic} at ${event.receivedAt}` }` (use the actual template literal).	Never retryable — always return false, regardless of err. Reports tolerates loss; a single attempt is sufficient.
search	Return the event plus an index_hint field computed as the substring of event.topic before its first . (or the whole topic if there is no .).	Retry when statusCode is undefined or >= 500 (same predicate shape as analytics/crm, but see Part B — search gets the longest retry budget of any service).
sms	Return the event with payload.message truncated to 160 characters if present and longer than that (leave other fields untouched).	Retry when statusCode is undefined or statusCode >= 500, but explicitly not on timeouts represented as err.code === 'ETIMEDOUT' — treat a timeout as non-retryable (carrier cost control), even though it has no statusCode.
webhooks	Return the event unchanged, with one added field: relay_version read from package.json's version field (require('../../package.json').version).	Retry on any non-2xx status, i.e. `statusCode === undefined

Note on retry counts: isRetryable only decides whether a given failure is a candidate for another attempt at all — it does not encode the attempt cap. Attempt caps are documented per-service in Part C for reference by any future caller; src/handlers/*.js files themselves do not need to enforce the cap (that remains src/retry.js's job when it is later wired up — not part of this pass).

Part B — three core handlers (3 files)

These are structurally different from the service handlers above — each addresses a different cross-cutting concern, not a per-service payload/retry rule.

src/handlers/auth.js — per-tenant concurrency limiter

Tracks how many deliveries are currently in flight for each tenant and enforces a maximum concurrency of 4 simultaneous in-flight deliveries per tenant. Export:

module.exports = {
  acquire(tenant), // returns true if the tenant is under its concurrency cap
                    // (and increments its in-flight count), false if the
                    // tenant is already at the cap (does not increment)
  release(tenant),  // decrements the tenant's in-flight count, floored at 0
  inFlight(tenant),  // returns the current in-flight count for a tenant (0 if unseen)
};

Keep the per-tenant counts in a plain object keyed by tenant name, module-scoped (same pattern as tokenCache in src/auth.js or counters in src/metrics.js). The concurrency cap (4) should be a named constant at the top of the file.

src/handlers/metrics.js — per-service latency histogram

Records delivery latency (milliseconds) per service and can report p50/p95. Export:

module.exports = {
  record(service, latencyMs), // append a sample for that service
  p50(service),                 // median of recorded samples for that service, or null if none
  p95(service),                 // 95th percentile (nearest-rank) for that service, or null if none
  reset(),                       // clear all recorded samples
};

Store samples per service in an array (module-scoped object keyed by service name). Percentile implementation: sort the samples ascending, then for p95 take the sample at index Math.ceil(0.95 * n) - 1 (nearest-rank method, clamped to a valid index); p50 uses 0.5 the same way. This is intentionally a different aggregation shape than the simple integer counters in src/metrics.js — do not just wrap the existing counters module.

src/handlers/store.js — dead-letter compaction

When an event has exhausted its retry attempts (mirrors MAX_ATTEMPTS in src/retry.js, currently 8) instead of being silently dropped it should be appended to a dead-letter file for manual review, and the in-memory dead-letter list should be periodically compacted to drop entries older than 7 days. Export:

module.exports = {
  deadLetter(target, event, attempts), // appends { target, event, attempts, at: Date.now() }
                                         // to data/deadletter.log (JSON line, same
                                         // append-on-write pattern as src/store.js's
                                         // `append`) AND to an in-memory list
  compact(now), // removes in-memory entries older than 7 days (7 * 24 * 60 * 60 * 1000 ms)
                 // relative to `now` (defaults to Date.now() if not passed);
                 // does not touch the on-disk log
  list(),        // returns the current in-memory dead-letter list
};

Use path.join(__dirname, '..', '..', 'data', 'deadletter.log') for the file path (mirrors DATA_DIR/LOG_FILE in src/store.js), and create the data/ directory if it doesn't exist before appending, same as src/store.js does.

Part C — reference: attempt caps (for future wiring, not required this pass)

Service	Max attempts
analytics	5
audit	1 (no retry)
billing	4
crm	3
email	6
reports	1 (no retry)
search	10
sms	2
webhooks	8 (matches src/retry.js MAX_ATTEMPTS)

Expected files

Implement all twelve files listed below. Each is independent of the other eleven — none of the twelve requires reading or importing another file in this list.

• src/handlers/analytics.js
• src/handlers/audit.js
• src/handlers/billing.js
• src/handlers/crm.js
• src/handlers/email.js
• src/handlers/reports.js
• src/handlers/search.js
• src/handlers/sms.js
• src/handlers/webhooks.js
• src/handlers/auth.js
• src/handlers/metrics.js
• src/handlers/store.js