Evaluation Administration Receipt: settling who ran the test and when the test set was fixed

The problem

A model or an agent is evaluated. A number comes out. The number goes into a datasheet, a procurement decision, an insurance submission, a press release.

Everybody who has run an evaluation knows the number is close to meaningless without the setup, and almost nobody records the setup in a way anyone else can check. Two questions decide whether a score means anything, and neither is usually answerable after the fact.

The first is contamination. Did the subject know what was on the test before it was tested. This is not exotic. The most common way an evaluation becomes worthless is that the items, or a close paraphrase, were in the training data or were handed over during setup. The distinction between a fair test and a leaked one is entirely a question of ordering: was the test set fixed before the subject learned about it. If both facts are rows in the evaluating party's own database, that ordering cannot be established.

The second is independence. Who selected the items, who ran the evaluation, who scored the outputs. A vendor evaluating its own product and a laboratory with no commercial relationship produce numbers with the same shape and completely different weight. Procurement teams know this and ask about it. What they get is prose in a slide: "independently evaluated". That phrase covers a range from a genuinely unaffiliated party who selected items the vendor never saw, all the way to a wholly owned subsidiary.

Name the parties and what each asserts without evidence. The subject asserts the evaluation was fair and independent. The administrator, who is often paid by the subject, asserts the same. A buyer asserts nothing and discounts the number by an unknown factor. An insurer asked to price a capability warranty on the strength of the number does the same, and the discount shows up in the premium. A regulator or standards body reading the datasheet has no mechanism to distinguish a self run test from an arms length one.

Then there is the failure that comes from disclosure rather than concealment. Evaluation results are commercially sensitive and often contractually confidential. An artifact that proves administration quality by restating the score cannot be shared with the people who most need it. The buyer's own procurement team can see the score, but the buyer's insurer might not be permitted to, and the reinsurer certainly is not. So the administration facts and the score travel together and get locked up together.

When the subject is autonomous the contamination question gets sharper still. An agent that can browse, that retains context across sessions, or that is retrained continuously has many routes to the test items that no one is tracking. Fixing the ordering does not solve that. It does establish the one thing that can be established: that the party who fixed the set did so before it told the subject anything.

What the receipt binds

The schema is at https://thehiveryiq.com/.well-known/schemas/eval-administration-v1.json.

The separation object is the substance. Three legs, four possible actors each. Splitting item selection from scoring is the single structural fact that distinguishes a real evaluation from a demonstration, and this receipt makes it a signed field rather than a claim in prose.

What is deliberately not in the signed body

No score. No accuracy, no pass rate, no latency, no ranking. The evaluation attestation holds the results and is referenced by digest. This receipt holds only the arrangement.

The absence is enforced by NO_SCORE_RESTATEMENT, which walks the entire signed body recursively. Any JSON number at any path other than signed_body.testset_ref.item_count fails. Any string that is a bare decimal, matching zero with optional decimals or one with optional trailing zeroes, fails. Any string that is a percentage, digits with optional decimals followed by a percent sign, fails. The gate reports the exact path. Since item_count is the only numeric field allowed to exist, there is nowhere for a score to hide, including inside a free text role field.

The test set items are also absent. What appears is testset_ref.digest_sha256, a 64 character lowercase hex digest of a set this service never receives. The party who holds the items can prove later which set was committed. Nobody else learns what the items were, which matters because publishing the items destroys the test set for every future subject.

The receipt envelope uses the shared typed receipt construction in src/typed/canon.js. The signed body is canonicalised with keys sorted recursively, arrays kept in order, no whitespace, and every code unit above U+007F escaped as a lowercase \uXXXX sequence so the bytes match Python json.dumps with ensure_ascii=True. payload_sha256 is lowercase hex SHA-256 over the UTF-8 bytes of that string. The Ed25519 signature covers the ASCII string

hive-receipt <receipt_id> <payload_sha256> <ts>

The prefix hive-receipt is the domain separation label. The same digest function produces evar_ref.payload_sha256, so this record commits to the exact bytes of the evaluation attestation while restating none of its contents.

The gates

Verification runs in this order and stops at the first failure.

1. SCHEMA. Validates the envelope against the eval administration v1 schema, including the closed enums on relationship, separation legs and independence class, and the boundary constant. A failure means this is not a well formed instrument of this type.
2. ISSUER_KEY_MATCH. Resolves envelope.key_id in the trusted keyring. A failure means the key does not resolve there, or no resolver was supplied.
3. PAYLOAD_DIGEST. Recomputes the canonical digest of signed_body against payload_sha256. A failure means the body was altered after signing.
4. RECEIPT_ID. Requires the identifier to parse and its type segment to equal signed_body.receipt_type. A failure means the identifier and the body disagree.
5. SIGNATURE. Verifies Ed25519 over the signing string against the trusted key, then a caller supplied key if one is offered. A failure means no available key produced this signature.
6. SIGNER_IDENTITY. Fails when only a caller supplied key verified, reporting internally consistent bytes from an untrusted signer.
7. EVAR_LINK. Requires an evaluation attestation envelope to be supplied and requires evar_ref.receipt_id and evar_ref.payload_sha256 to match the supplied envelope's identifier and recomputed body digest. A failure means the record names an attestation that was not produced, or a different one.
8. EVAR_INTEGRITY. Runs the full evaluation attestation verifier over the supplied envelope and reports its failed gate and reason. A failure means the attestation this record accompanies is itself broken.
9. TENANT_MATCH. Requires the administration tenant_id to equal the attestation tenant_id. A failure means an administration record from one tenant is being attached to another tenant's evaluation.
10. TESTSET_PRECOMMITMENT. Requires testset_ref.committed_at, testset_ref.disclosed_to_subject_at and the attestation's evaluation_window.opens_at to be valid instants and to satisfy committed_at at or before disclosed_to_subject_at at or before evaluation_window.opens_at. A failure means the test set was fixed after the subject was told about it, or after the evaluation window had already opened.
11. ITEM_COUNT_POSITIVE. Requires testset_ref.item_count to be at least the floor implied by the linked evaluation, with a default floor of one. A failure names both the declared count and the required minimum, and means the declared set is too small to support the evaluation it accompanies.
12. SEPARATION_COHERENCE. Rejects a record where relationship.administrator_to_subject is same_legal_entity while any separation leg claims third_party. A failure means the declared arrangement contradicts itself.
13. INDEPENDENCE_RECOMPUTE. Derives the class and compares it to independence_class. The derivation is fixed: same_legal_entity, or any leg performed by subject, gives self_administered; affiliate gives affiliated; contracted_vendor gives contracted; unaffiliated with no leg performed by the subject and with selected_items_by different from scored_results_by gives independent; everything else gives undisclosed. A failure means the caller asserted a class the declared facts do not produce.
14. NO_SCORE_RESTATEMENT. Rejects any numeric value, bare decimal or percentage outside testset_ref.item_count, naming the path. A failure means an evaluation result reached a record that must not carry one.

Read the independence derivation once more, because it is where the instrument earns its keep. Claiming independent is not enough. You have to declare unaffiliated, keep the subject out of all three legs, and have a different actor select the items from the one who scores the results. Fail any of those and the recomputed class drops, and if you wrote independent anyway the receipt does not verify.

The boundary

This receipt attests that a named evaluation administration arrangement was recorded alongside one specific evaluation attestation, that the test set digest was committed at or before the point the subject was told what the test set contained and at or before the evaluation window opened, and that the independence class was recomputed by this service from the declared relationship and separation of duties rather than supplied by the caller. The declared relationship, the declared separation of duties, and the identity of the administrator are recorded as asserted and are not independently verified here. This receipt does not contain, restate, or attest to any evaluation score or result. It does not attest that the test set was appropriate, that its items were representative, that the subject did not obtain the items by another route, that the evaluation method was adequate, or that the administrator was competent. It is not an accreditation, a certification, or an audit opinion, and no standards body, regulator, or insurer currently recognises it as one.

That string is a schema constant inside the signed bytes and is checked at verification.

The sentence about the subject obtaining items by another route is the honest one, and it is the one a marketing department would delete. The receipt fixes an ordering between a commitment and a disclosure. It cannot see a scraped copy of the benchmark, a leaked file, or training data that happened to contain the items. Saying so tells a buyer exactly what residual contamination risk it still owns, which is the information a buyer needs in order to decide whether to run its own held out test.

The sentence about no standards body recognising this as an accreditation is the second one that would get cut. Leaving it in is what lets a standards body engage with the instrument at all. An organisation that certifies things is acutely sensitive to artifacts that imply certification, and an artifact that disclaims it in its own signed bytes is one they can look at without a conflict.

There is a general shape here. An enumerated non attestation converts an open ended question into a closed one. Without the enumeration, a buyer holding this receipt has to ask what else it might mean and will assume the worst. With it, the buyer knows the receipt covers ordering and independence classification, knows it covers nothing about method quality, and can direct diligence at the gap. Narrower claims that are actually checkable settle disputes faster than broad claims that are not.

Adversarial cases

These come from test/eval-administration.test.mjs.

A test set committed after disclosure fails TESTSET_PRECOMMITMENT. This is contamination in its recorded form. Somebody tells the subject what will be tested and fixes the set afterward, possibly adjusting it in light of what the subject can do. The gate compares the two instants directly and also requires both to precede the evaluation window opening.

Same entity third party separation fails SEPARATION_COHERENCE. The forgery declares same_legal_entity for honesty on the relationship, then claims a third_party leg to recover some credibility. The two statements cannot both be true and the gate says so before the class is even derived.

A caller asserted class fails INDEPENDENCE_RECOMPUTE. The direct attack is to write independent regardless of the declared facts. The stated class is never an input to the derivation, only a value compared against it. Note also that a subject who selects its own items gets self_administered even under an unaffiliated relationship, so restructuring the relationship label alone does not buy a better class.

A zero item set fails ITEM_COUNT_POSITIVE. An evaluation over an empty set produces a number that means nothing. The gate rejects a count below the floor implied by the linked evaluation.

A wrong linked evar digest fails EVAR_LINK. Attaching a favourable administration record to a different attestation than the one it was built for changes the recomputed digest.

A tenant mismatch with the linked evaluation fails TENANT_MATCH. One organisation's independent administration record cannot be used to dress up another organisation's evaluation.

A bare decimal in a schema permitted role fails NO_SCORE_RESTATEMENT. The role field is free text and schema validation accepts it. Writing 0.94 there does not survive, because the gate treats any bare decimal anywhere as a restated score.

Two positive cases matter. A valid independent arrangement verifies, and a valid self administered arrangement verifies as recorded. The second is the important one. A truthfully declared self administered evaluation produces a valid receipt with independence_class of self_administered. The instrument is not a badge that only exists when the answer flatters you. It records the arrangement that happened, including the unflattering one, and lets the reader weigh it.

Performance

From hive-verifier-api/benchmarks/results-latest.json, measured at 2026-08-09T22:28:01.099Z at commit e167719f7d815809c6f55c5dde4495edf1c5615f over 200 iterations:

Local harness measurement on one machine and one Node version. The run recorded Node v20.20.1 on linux/x64 with two Intel Xeon cores at 2.90 GHz. It is a reproducible measurement of this code, not a production latency guarantee.

How to run it

• Host: https://thehiveryiq.com/v1
• Verify route: /verify/eval-administration, no credentials required
• Mint route: /mint/eval/administration, requires Authorization: Bearer $HIVE_TYPED_MINT_TOKEN

curl -sS -X POST https://thehiveryiq.com/v1/verify/eval-administration -H 'content-type: application/json' -d '{"receipt": <a receipt of this type>}'

Mint fails closed with 401 when the token is missing or wrong.

Where it sits in the canon

Family: evaluation. Category: typed receipt contract.

It attaches to an evaluation attestation and verifies it rather than trusting the reference. The two travel as a pair: the attestation carries the result, this record carries the conditions. Because it carries no score, it can be shared with parties who are not permitted to see the result at all, which is the point.

It connects sideways to the Portfolio Exposure Receipt, which counts eval_administrator as one of its five concentration dimensions. A book where most subjects were evaluated by the same administrator has a concentration that exposure receipt can measure and this receipt can explain.

It does not replace the evaluation attestation, an accreditation, or an audit opinion, and the boundary rules all three out in the signed bytes.

Status

production_deployed.