On this page· 4

Specification

5. Kind registry

5.1 Typed kinds (19)

The rank is the canonical sort position. The primary id is the record's identity within its kind; pad20(n) means n in decimal, left-padded with 0 to 20 digits. The field list is the v1 schema; ? marks an OPTIONAL member, all others are REQUIRED. Every kind also carries v? and extra.

Table 5-1. Typed kinds (19)
Rank k Primary id v1 members
0 meta created_at "|" source created_at, source, tool?, description?, ngdb_generation? (u64)
1 payload empty string embedded (bool), csdt_file_checksum? (u32, always emitted, default 0), sections? (array of {index u32, section_type u16, record_count u64, dtype? string}, always emitted, default []), refs? (array of string, always emitted, default [])
2 csdt_ref ref_id ref_id, path, shard_set_id? (u32), file_checksum (u32), source_hash, section_type (u16), section_index (u32), record_range? ([u64, u64])
3 ent id id, t, name?, salience? (f32), stub? (bool, always emitted, default false), properties?, prov?
4 rel src "|" tgt "|" kind src, tgt, kind, properties?, prov?
5 hyper id id, members (array of string), kind, properties?
6 embref entity_id "|" target-key entity_id, target (§5.4), annotations?
7 prov subject "|" method "|" pad20(unix_secs) subject, agent, method, unix_secs (u64), inputs?, notes?
8 grounded_answer answer_id Application kind.
9 citation source_answer_id "|" entity_id "|" pad20(span.start) Application kind.
10 clause clause_id Application kind.
11 knob_set scope "|" key "|" pad20(unix_secs) Application kind.
12 discovery gid Application kind.
13 promo content_hash "|" pad20(order) Application kind.
14 codebook_manifest codebook_id Application kind.
15 session session_id Application kind.
16 sev pad20(step) "|" t Application kind.
17 session_end pad20(end_unix) Application kind.
18 outcome identity form (§10.3) of outcome Application kind.

Members without a stated type are JSON strings; u32, u64 are non-negative JSON integers in range.

Changing a typed kind's schema incompatibly requires raising its v (§12.2). Adding an OPTIONAL member at the same v is compatible, because older readers keep it in extra.

5.2 Reserved kinds

These kind strings are reserved, but have no typed parser in this revision. Readers MUST treat them as opaque (§4.4).

mut

Ten further kind names are reserved and are specified separately. A reader keeps any record whose kind it does not know as an opaque record (§4.4).

A reserved kind is promoted to the typed registry only by a spec revision that adds it to §5.1 with the same v1 schema.

Kind names that begin with x- belong to applications: no revision of this specification adds one to the typed kinds (§5.1) or the reserved kinds (§5.2).

5.2.1 mut v1 (frozen)

mut is one entry of an ordered mutation log. The schema below is the wire form of mut v1.

Table 5-2. Members of a `mut` record, version 1 (10)
Member Type Rule
k string "mut"
v integer 1
id string the LSN in decimal, left-padded with 0 to 20 digits. MUST equal pad20(lsn)
gen u64 publication generation
lsn u64 log sequence number within gen
op string one of put_record, delete_record, or one of the op names reserved for future use
rk string, OPTIONAL kind of the affected record. REQUIRED for put_record and delete_record
rid string, OPTIONAL primary id (§5.1) of the affected record. REQUIRED for put_record and delete_record
rec object, OPTIONAL put_record only: the complete affected record as its own JSONL object, including its k. Its kind and primary id MUST equal rk and rid. delete_record MUST NOT carry rec
other any members of the reserved ops, preserved. None of them may be named id

Emitted member order (under §10.1): k, v, id, gen, lsn, op, rk, rid, rec, then other members.

Rules the member table cannot express:

  • One file holds one gen. LSNs restart when gen changes, so two generations in one file would share ids and merge would drop records.
  • LSNs within a gen are gap-free and are applied in LSN order. Because mut is opaque, it sorts after every typed kind, by id, which is LSN order.
  • A mutation log is ordered, not a set. Producers MUST NOT combine logs with merge (§8).
  • rk/rid identify records, never id: an opaque record's identity reads id first (§5.3), so an entity id there would collapse a put and a delete of the same record.

5.3 Identity, sort order and dedup key

For every record:

  • kind = the k string.
  • rank = §5.1 rank when k is a typed kind string (this includes an opaque record whose k is a typed kind at a higher v), otherwise the maximum rank, placed after all typed kinds.
  • primary id = §5.1 for typed records. For an opaque record: the value of the first of id, ref_id, session_id, answer_id, clause_id that is present as a JSON string; if none is, the identity form (§10.3) of the whole object.
  • sort key = (rank, kind, primary id), compared rank numerically, then kind and primary id by UTF-8 byte order.
  • dedup key = (kind, primary id).

A writer MUST emit records in ascending sort key order. Records with equal sort keys keep their input order (the sort is stable). A writer SHOULD NOT emit two records with the same dedup key; readers MUST accept such files, and diff reports them (§8.2).

Note: an opaque record whose k is a typed kind at a higher v has the same dedup key as a v1 record with the same primary id. Merge treats the two as the same record (§8.1).

5.4 Payload binding targets

embref.target, session.trajectory, codebook_manifest.target and promo.curvature_refs[] are one of two JSON shapes:

  • {"local":{"section_index":S,"record_index":R}}: record R of section S of this file's own appendix. target-key = "local|" pad10(S) "|" pad20(R).
  • {"ext":{"ref_id":F,"record_index":R}}: record R of the section named by the csdt_ref record whose ref_id is F. R counts from the section start, not from any record_range. target-key = "ext|" F "|" pad20(R).

(pad10 pads to 10 digits.)

© 2020-2026 Cintile Inc. All Rights Reserved.

Anyone may implement this format. Copying or republishing the text of this specification requires permission from Cintile Inc.

Sections