cc-os/docs/memory-system/benchmark/extraction-spec.md

# Graphify Document-Extraction Benchmark — Shared Extraction Spec

_Last updated: 2026-06-04 — rules tightened after first benchmark run (bare-slash facets and undeclared relationship entities)_
_Status: active — used as the shared schema for the Claude-tier reference-set benchmark_

## Purpose

This spec defines the **minimal structured fragment** that every benchmarking subagent (one per
Claude tier) must emit when given a raw vault note. The fragment mimics what Graphify's local SLM
doc-extraction step produces: a list of **entities**, a list of **typed relationships** (edges
between entities), and per-item **confidence tags** where applicable.

The schema is intentionally minimal. It exists so that:
1. Per-model output files are **diffable entity-by-entity and edge-by-edge** across tiers.
2. The same schema is used unchanged when local Ollama models are later scored against this
   reference set — making the comparison apples-to-apples.

Subagents receive only the raw note text plus this spec. No repository files, no vault structure,
no project context. If any such context is injected by the environment, it must be ignored.

---

## Facet Vocabulary (closed)

Notes in the SecondBrain vault carry six flat, namespaced facets plus one scope tag. These are the
**only** valid values for the optional `facet` field on entities. Do not invent new namespaces.

| Prefix       | Meaning                                 |
|--------------|-----------------------------------------|
| `type/`      | What kind of thing the note is about    |
| `client/`    | A client identity                       |
| `project/`   | A project name                          |
| `domain/`    | A knowledge or technical domain         |
| `tool/`      | A specific tool, library, or CLI        |
| `convention/`| A working convention or practice        |
| `scope/`     | Applicability scope (cross-cutting tag) |

An entity should carry a `facet` value only when the note text directly supports mapping it to one
of the above. Absence of a `facet` field is correct when no mapping is warranted.

**Sub-value required (STRICT):** A `facet` value MUST include a non-empty sub-value after the
namespace slash — e.g. `tool/cursor`, `convention/tdd`, `domain/distributed-systems`. A bare
namespace with nothing after the slash (e.g. `tool/`, `domain/`) is **invalid** and must never
appear in output. If a suitable sub-value cannot be identified from the note text, omit the
`facet` field entirely.

---

## Output Schema

The required output format. Every subagent MUST emit this structure and no other top-level keys.

```yaml
# --- Graphify extraction fragment ---
# One block per fixture note. Repeat this structure for each note.

note_slug: <kebab-case-identifier-for-the-note>   # operator fills this in at run time

entities:
  - name: <string>          # exact or near-exact name as it appears in the note
    type: <string>          # e.g. Person, Tool, Project, Concept, Convention, Client, Domain
    facet: <string>         # OPTIONAL — must be one of the seven prefixes above, e.g. "tool/graphify"
    confidence: <string>    # OPTIONAL — omit if the entity is directly stated
                            # Values: INFERRED | AMBIGUOUS

relationships:
  - source: <entity name>   # must match a name in the entities list above
    type: <string>          # free-text verb phrase, e.g. "uses", "depends_on", "implements", "replaces"
    target: <entity name>   # must match a name in the entities list above
    confidence: <string>    # OPTIONAL — omit if the relationship is directly stated
                            # Values: INFERRED | AMBIGUOUS
```

### Confidence tag semantics

| Tag        | Meaning                                                                              |
|------------|--------------------------------------------------------------------------------------|
| _(absent)_ | The entity or relationship is **directly stated** in the note text.                  |
| `INFERRED` | Not stated literally but **reasonably deducible** from the note text alone.          |
| `AMBIGUOUS`| Supportable by the text but **uncertain or admits multiple readings**.               |

### Rules

- **Entities only:** extract entities that are named (not vague category references).
- **No invented names:** entity names must be grounded in the note text.
- **Relationship types:** free-text verb phrases; do not normalize to a fixed vocabulary.
- **Facet mapping:** use the closed vocabulary above; only assign when the note text supports it.
- **No extra keys:** do not add summaries, scores, embeddings, or metadata fields.
- **One block per note:** if processing multiple fixture notes, emit one `note_slug:` block per note,
  separated by `---`.
- **Referential integrity (STRICT):** every relationship `source` and `target` MUST exactly match
  the `name` of an entity declared in the `entities` list of the same block. Do not reference
  undeclared concept strings, free-form text, or the note's own `note_slug` as a `source` or
  `target`. If a relationship needs a concept that is not yet in the entity list, declare it as an
  entity first (with an appropriate `confidence` tag if it is inferred), then reference it.