How conformance works

The conformance corpus is a set of small input files and, for each test case, the result a correct implementation produces. Corpus version 1 has 111 cases. It is written against specification revision 6.0 and error-kind list version 1.

corpus version 1
revision 6.0
error-kind list version 1

Results

The Rust SDK passes 111 of 111 cases. Implementations

What a case is

A case names its input files, one operation to run on them, the specification section it tests, the conformance level it proves, and the expected result: a pass with values to compare, or a failure with exactly one error kind.

Each input is listed with its SHA-256, so a runner can show it read the bytes the case names.

Two further members, notes for the people who maintain the corpus, record how each input was made and where its expected value came from. They are not compared.

A caseJSON
{
  "id": "neg-footer-reserved-nonzero",
  "group": "negative",
  "since": 1,
  "level": "Reader-Core",
  "op": "open",
  "inputs": ["corpus/negative/footer-reserved-nonzero.plxi"],
  "input_sha256": [
    "61cdfccbc11b9cdcaca9b7be2fcbc6aa6b941fe59e55157448c5fc757564f5bb"
  ],
  "args": {"verify": true},
  "spec_ref": "spec §10.0, §9 step 3",
  "expect": {"ok": false, "error_kind": "invalid_footer"}
}

A case from group negative, with the members a runner compares. The file must fail to open with the error kind invalid_footer, because a reserved footer byte is not zero.

Rule zero: expected values do not come from the code under test

No expected value was produced by running a reader, merger or verifier under test. Each comes from one of four sources:

a value written by hand;
a SHA-256 of values written by hand;
a computation of a rule of the specification, in code that shares nothing with the implementation;
a count made by the corpus generator’s own byte scan of a stored file it did not write.

Input files may be produced with the reference writer: producing an input is not predicting an output. Each such input is checked byte for byte against an assembly of the hand-written values.

The corpus is regenerated in full and compared byte for byte with the stored copy.

Levels

An implementation claims one or more conformance levels. Section 14 defines what each level requires; each case proves exactly one level. Section 14

**Table 1.** Cases per level, corpus version 1 (111)
Level	Builds on	Cases
Reader-Core	—	42
Reader-Appendix	Reader-Core	27
Writer	—	27
Merger	—	11
Differ	—	4
Sharder	—	0

Of the levels with cases, the requirements section 14 lists for Reader-Core, Merger and Differ name no part that this publication leaves out; those for Reader-Appendix and Writer include sections 7.2 to 7.4.

Reader-Appendix and the appendix parts of Writer, Merger and Differ also rely on the CSDT container specification (17 cases cite sections 7.2 to 7.4). By level: Reader-Appendix 16, Merger 1.

Operations a runner calls

Each case names one of these operations. Merge and diff are section 8, verify is section 9; the others are the Rust SDK calls a runner uses to read and write packs.

**Table 2.** Operations of corpus version 1 (9)
Operation	What it does	Arguments	Cases
`open`	Reads a pack from its bytes. Unless `verify` is false, runs verify steps 1 to 8.	`verify` (true unless given); `max_input_bytes`, a cap the caller sets: an input larger than this, or for `.plxi.gz` an inflated input larger than this, fails with `limit_exceeded`; `by_path`, open from a file path instead of bytes.	39
`records`	Returns the records as the lines a reader emits (sections 4.3 and 4.4), in file order.	`verify`	25
`filter`	Returns the records whose kind is in `kinds` and whose primary id (section 5.3) starts with `id_prefix`. An argument that is not given matches every record. The specification does not define this operation.	`kinds`, `id_prefix`	2
`verify`	Runs section 9 on a pack opened with `verify` false and returns the verify report.	none	5
`appendix`	Reads the appendix: its list of sections, the bytes of one section, or the values of one named tensor.	`call`, `index`, `name`, `verify`	15
`bind`	Resolves every `embref` target under the Compact embedding profile (verify step 10).	`verify`	8
`write`	Writes the given records, in any order, as a new pack, and reads it back.	`records`, `appendix`	2
`merge`	Merges two packs (section 8.1).	`strategy`: `higher_salience`, `latest`, `union` or `manual`	11
`diff`	Compares two packs (section 8.2).	none	4

The input-size cap is the caller’s choice, not a rule of the specification, so the two cases that test it name no section.

How results are compared

The expected results of all cases are in one file, the manifest. A result is compared with it by these rules.

Expected objects are compared member by member, at every depth. A member the case does not give is not checked. Arrays are compared whole and in order.
sha256 is the digest of section 6: SHA-256 of bytes 257 up to L − 96, as lowercase hex. For a write case it is the SHA-256 of the whole output file.
canonical_jsonl_sha256 is the SHA-256 of the identity form of each record, each followed by a line feed, in file order. With no records it is the SHA-256 of the empty string.
lines are the records as a reader emits them, in file order. A case with more than 8 lines gives lines_sha256 instead: the SHA-256 of those lines, each followed by a line feed.
A merge result is compared without its payload records, plus a count of them, which section 8.1 makes 1.
Diff lists are sorted by sort key; a changed entry is {key, before, after} with key = [kind, primary id].
output_sha256_equals names another case whose output must be byte-identical to this one. It tests that an operation gives the same bytes for the same inputs.
A failing case is judged by its error kind only. The message is not compared.

What a level claim requires

A level claim rests on running every case of that level, from the case files, through the implementation’s public interface. A Reader-Appendix claim also runs every Reader-Core case: Reader-Appendix is Reader-Core plus section 7 and verify steps 9 and 10. A conforming runner:

Reads the manifest, stops if its corpus version or error-kind list version is higher than it knows, and never skips members it does not recognize.
Checks the SHA-256 of every input before each case. A mismatch is a fault in the copy of the corpus, reported apart from the case result. An input listed with no hash is a path that must not exist.
Runs the case’s operation with its arguments. Opening verifies by default, as section 9 says; an argument may turn that off or set an input size cap.
Compares the result with the case by the rules above; for a failing case, the error kind only.
Treats any internal result as a failure of the run, even in a case that expects an error.
Writes one result line per case, in manifest order, with the case id, pass or fail, and the error kind. A case skipped because the platform lacks something it needs is written as skipped, with the reason.

One result line per caseJSON
{"case":"neg-json-syntax","ok":false,"error_kind":"json"}
{"case":"graph-gzip","ok":true,"error_kind":null}

The case files of corpus version 1 are not published. A conformance claim therefore covers only implementations that Cintilé has run against the cases; today that is the Rust SDK.

Limits

The case files are not part of this publication. This page specifies the corpus method, and the cases page lists every case.

Shard. Version 1 has no case at the Sharder level.
Three error kinds have no case: shard_unsupported_spec, shard_quality_below_threshold and internal. Each of the other 19 has at least one.
Decoding the values of tensor sections in the appendix, writing a pack that has an appendix, and merging two appendices into their union. Section 14 lists these under the Reader-Appendix, Writer and Merger levels, so a pass on version 1 does not cover those parts of the three levels.
Duplicate member names inside a JSON object. Section 4.1
The meaning of the data inside the embedded container. The corpus tests its structure and checksums only.
Nineteen cases read packs kept outside the corpus directory. The cases page marks each of them “Outside the corpus”. Cases, corpus version 1
Section 9 lets a reader decompress .plxi.gz input (MAY), but graph-gzip and neg-limit-gzip-inflate-cap are Reader-Core cases that read it. Revision 6.0 does not say whether a reader without gzip may skip them and still claim Reader-Core.