171 lines
9.7 KiB
Markdown
171 lines
9.7 KiB
Markdown
# Per-service delivery handlers
|
||
|
||
Relaystation currently delivers every event through the single generic path in
|
||
`src/relay.js` (`deliver` → `fetchLike`, with failures handed to `src/retry.js`).
|
||
That path treats every downstream service identically: same payload shape, same
|
||
retry policy, same failure handling. In practice each of the nine downstream
|
||
integrations in `services/` has different tolerance for retries, different
|
||
payload requirements, and different failure semantics, and three cross-cutting
|
||
concerns (auth concurrency, per-service latency visibility, and dead-letter
|
||
handling) need their own dedicated modules.
|
||
|
||
This spec defines twelve new handler modules under `src/handlers/`. Each module
|
||
is self-contained and does not require changes to `src/relay.js`, `src/router.js`,
|
||
`src/retry.js`, or any file under `services/` — implement each file to the
|
||
interface and behavior below.
|
||
|
||
## Conventions
|
||
|
||
- `'use strict';` at the top, matching the rest of `src/`.
|
||
- Each module exports a plain object via `module.exports = { ... }` (same style
|
||
as `src/metrics.js` / `src/store.js`), not a class.
|
||
- Where a handler needs the shared metrics counters, `require('../metrics')`
|
||
and call `increment(name)` — do not create a second counter store.
|
||
- Each file should be small: roughly 20–40 lines including requires and
|
||
`module.exports`.
|
||
- No new npm dependencies. Use only Node core modules already used elsewhere
|
||
in `src/` (`crypto`, `fs`, `path`, `http`) plus what a handler needs.
|
||
- Tests are not required for this pass — implementation only.
|
||
|
||
## Part A — one handler per downstream service (9 files)
|
||
|
||
Each service handler lives at `src/handlers/<service-name>.js` and exports two
|
||
functions:
|
||
|
||
```js
|
||
module.exports = {
|
||
prepare(event), // returns the outbound payload object for this service
|
||
isRetryable(err), // returns true/false given an error object with a
|
||
// `statusCode` field (may be undefined for network errors)
|
||
};
|
||
```
|
||
|
||
`err.statusCode` is `undefined` for a connection/network failure (no response
|
||
received) and an HTTP status integer when a response was received but was not
|
||
2xx. Implement `prepare` and `isRetryable` exactly per the per-service rules
|
||
below — the rules differ enough between services that no single template
|
||
satisfies more than one of them.
|
||
|
||
| Service | `prepare(event)` payload rule | `isRetryable(err)` rule |
|
||
| --- | --- | --- |
|
||
| `analytics` | Wrap the event: return `{ batch: [event], emitted_at: Date.now() }`. | Retry when `statusCode` is undefined (network) or `>= 500`. Never retry on any 4xx. |
|
||
| `audit` | Return the event plus an integrity field: `{ ...event, hash: sha256-hex of JSON.stringify(event) }` (use `crypto.createHash('sha256')`). | Retry **only** on a network failure (`statusCode === undefined`). Any received HTTP status, 4xx or 5xx, is not retryable — audit delivery failures must surface immediately rather than silently retry. |
|
||
| `billing` | Return a payload containing **only** an allowlist of fields copied from `event`, dropping everything else: `id`, `topic`, `amount`, `currency`, `receivedAt`. Fields absent on the event are simply omitted from the output, not set to `null`. | Retry only on `statusCode === 429` or `statusCode === 503`. Nothing else (not network failures, not other 5xx) is retryable. |
|
||
| `crm` | Return a shallow-transformed payload: rename `topic` to `event_type`, and if `event.payload` has a nested `contact` object, flatten it to top-level `contact_id` and `contact_email` fields (copied from `event.payload.contact.id` / `.email` if present) instead of nesting it. | Retry when `statusCode` is undefined or `>= 500`, same as `analytics` — but cap distinctly (see Part B note below; the cap itself lives in the retry loop, not in this predicate). |
|
||
| `email` | If `event.payload` is missing a `to` field, do not build a payload at all — call `metrics.increment('handlers.email.invalid')` and return `null` (the caller is expected to skip sending when `prepare` returns `null`). Otherwise return the event unchanged. | Retry when `statusCode` is undefined, `429`, or `>= 500`. |
|
||
| `reports` | Collapse the event into a single-field summary payload: `{ summary: \`${event.topic} at ${event.receivedAt}\` }` (use the actual template literal). | Never retryable — always return `false`, regardless of `err`. Reports tolerates loss; a single attempt is sufficient. |
|
||
| `search` | Return the event plus an `index_hint` field computed as the substring of `event.topic` before its first `.` (or the whole topic if there is no `.`). | Retry when `statusCode` is undefined or `>= 500` (same predicate shape as `analytics`/`crm`, but see Part B — `search` gets the longest retry budget of any service). |
|
||
| `sms` | Return the event with `payload.message` truncated to 160 characters if present and longer than that (leave other fields untouched). | Retry when `statusCode` is undefined or `statusCode >= 500`, but explicitly **not** on timeouts represented as `err.code === 'ETIMEDOUT'` — treat a timeout as non-retryable (carrier cost control), even though it has no `statusCode`. |
|
||
| `webhooks` | Return the event unchanged, with one added field: `relay_version` read from `package.json`'s `version` field (`require('../../package.json').version`). | Retry on any non-2xx status, i.e. `statusCode === undefined || statusCode < 200 || statusCode >= 300`. |
|
||
|
||
Note on retry *counts*: `isRetryable` only decides whether a given failure is a
|
||
candidate for another attempt at all — it does not encode the attempt cap.
|
||
Attempt caps are documented per-service in Part C for reference by any future
|
||
caller; `src/handlers/*.js` files themselves do not need to enforce the cap
|
||
(that remains `src/retry.js`'s job when it is later wired up — not part of
|
||
this pass).
|
||
|
||
## Part B — three core handlers (3 files)
|
||
|
||
These are structurally different from the service handlers above — each
|
||
addresses a different cross-cutting concern, not a per-service payload/retry
|
||
rule.
|
||
|
||
### `src/handlers/auth.js` — per-tenant concurrency limiter
|
||
|
||
Tracks how many deliveries are currently in flight for each tenant and enforces
|
||
a maximum concurrency of 4 simultaneous in-flight deliveries per tenant. Export:
|
||
|
||
```js
|
||
module.exports = {
|
||
acquire(tenant), // returns true if the tenant is under its concurrency cap
|
||
// (and increments its in-flight count), false if the
|
||
// tenant is already at the cap (does not increment)
|
||
release(tenant), // decrements the tenant's in-flight count, floored at 0
|
||
inFlight(tenant), // returns the current in-flight count for a tenant (0 if unseen)
|
||
};
|
||
```
|
||
|
||
Keep the per-tenant counts in a plain object keyed by tenant name, module-scoped
|
||
(same pattern as `tokenCache` in `src/auth.js` or `counters` in
|
||
`src/metrics.js`). The concurrency cap (4) should be a named constant at the
|
||
top of the file.
|
||
|
||
### `src/handlers/metrics.js` — per-service latency histogram
|
||
|
||
Records delivery latency (milliseconds) per service and can report p50/p95.
|
||
Export:
|
||
|
||
```js
|
||
module.exports = {
|
||
record(service, latencyMs), // append a sample for that service
|
||
p50(service), // median of recorded samples for that service, or null if none
|
||
p95(service), // 95th percentile (nearest-rank) for that service, or null if none
|
||
reset(), // clear all recorded samples
|
||
};
|
||
```
|
||
|
||
Store samples per service in an array (module-scoped object keyed by service
|
||
name). Percentile implementation: sort the samples ascending, then for `p95`
|
||
take the sample at index `Math.ceil(0.95 * n) - 1` (nearest-rank method,
|
||
clamped to a valid index); `p50` uses `0.5` the same way. This is intentionally
|
||
a different aggregation shape than the simple integer counters in
|
||
`src/metrics.js` — do not just wrap the existing counters module.
|
||
|
||
### `src/handlers/store.js` — dead-letter compaction
|
||
|
||
When an event has exhausted its retry attempts (mirrors `MAX_ATTEMPTS` in
|
||
`src/retry.js`, currently 8) instead of being silently dropped it should be
|
||
appended to a dead-letter file for manual review, and the in-memory dead-letter
|
||
list should be periodically compacted to drop entries older than 7 days. Export:
|
||
|
||
```js
|
||
module.exports = {
|
||
deadLetter(target, event, attempts), // appends { target, event, attempts, at: Date.now() }
|
||
// to data/deadletter.log (JSON line, same
|
||
// append-on-write pattern as src/store.js's
|
||
// `append`) AND to an in-memory list
|
||
compact(now), // removes in-memory entries older than 7 days (7 * 24 * 60 * 60 * 1000 ms)
|
||
// relative to `now` (defaults to Date.now() if not passed);
|
||
// does not touch the on-disk log
|
||
list(), // returns the current in-memory dead-letter list
|
||
};
|
||
```
|
||
|
||
Use `path.join(__dirname, '..', '..', 'data', 'deadletter.log')` for the file
|
||
path (mirrors `DATA_DIR`/`LOG_FILE` in `src/store.js`), and create the `data/`
|
||
directory if it doesn't exist before appending, same as `src/store.js` does.
|
||
|
||
## Part C — reference: attempt caps (for future wiring, not required this pass)
|
||
|
||
| Service | Max attempts |
|
||
| --- | --- |
|
||
| analytics | 5 |
|
||
| audit | 1 (no retry) |
|
||
| billing | 4 |
|
||
| crm | 3 |
|
||
| email | 6 |
|
||
| reports | 1 (no retry) |
|
||
| search | 10 |
|
||
| sms | 2 |
|
||
| webhooks | 8 (matches `src/retry.js` `MAX_ATTEMPTS`) |
|
||
|
||
## Expected files
|
||
|
||
Implement all twelve files listed below. Each is independent of the other
|
||
eleven — none of the twelve requires reading or importing another file in this
|
||
list.
|
||
|
||
- `src/handlers/analytics.js`
|
||
- `src/handlers/audit.js`
|
||
- `src/handlers/billing.js`
|
||
- `src/handlers/crm.js`
|
||
- `src/handlers/email.js`
|
||
- `src/handlers/reports.js`
|
||
- `src/handlers/search.js`
|
||
- `src/handlers/sms.js`
|
||
- `src/handlers/webhooks.js`
|
||
- `src/handlers/auth.js`
|
||
- `src/handlers/metrics.js`
|
||
- `src/handlers/store.js`
|