[Theme] Reading evaluation at population scale

Theme Name

Reading evaluation at population scale

Proposer(s)

Justin Philip Flores, Himalogic Software Pvt. Ltd. (Nepal). Raised as Proposer; co-development welcome, particularly from the measurement and evaluation-methodology side. This theme consumes the outputs of #1, #5, #6 and #7 and proposes no change to any of them.

Summary

Several claims settled into #6 over the past two weeks are, by the settling parties' own statements, not decidable from a single record. The issuer's declared relationship to the evaluated party is a claim whose truth the record cannot establish. The confidence value is an attributed engine claim whose honesty the record cannot establish. Open question 7 asks which evaluator-directed attacks, including a captured engine issuing attributed false verdicts, are detectable from the record at all. In each case the answer given is the same: not from the record.

The proposer of #6 has since drawn the boundary explicitly. Asked who recognizes when several locally valid determinations together indicate that the operating frame no longer holds, he answered that #6 cannot answer it and should not try, that what #6 can do is make each determination readable through scope, attribution and the issuer field so that a set of them can be read as a rate over a defined domain instead of as noise, and that who holds the cross-scope view is unsettled between #16, #13, and an external evaluator under #18. He has since put the reason more sharply, on FG-TIDA/use-cases#5: "At the moment of action there is one observer per record, so that diversity has to come from outside the record, which is where FG-TIDA/themes#6 hands off to FG-TIDA/themes#21."

This theme proposes to specify that layer. The unit of analysis is the population, not the record. A declared independence value that is false is invisible in any single record and is a distribution anomaly across many. So is a confidence threshold set to pass everything. So is a captured engine. The question is what a party with no relationship to any of the parties in a record can conclude from a population of them, and what the records, the taxonomy, and the evaluators must satisfy for that conclusion to be sound. Taxonomy here means a reference in #5's sense: an authored, versioned set of flag types with definitions and a change history.

Two independence conditions are in play and the theme treats them separately. One is who performs the evaluation, which is the issuer-relationship question. The other is who authors the reference. For a large class of obligations the reference is not authored by the operator and never will be, because jurisdictions write requirements and a provider does not declare what counts as a violation in a jurisdiction it serves. Where the applicable set is published under jurisdictional authority, the evaluator resolves it directly, without receiving it from the party under evaluation. Both conditions are declarable, neither is verifiable per record, and both are readable at rate.

On placement in the landscape presented to the 2 September call: the stack runs from foundations up through identity, attestation, authority, policy and conformance, to record and oversight, and then to governance at the top. This theme is the rung between the last two. #1 and #16 produce and act on records one at a time; #3 writes governance frameworks that have to be written against something. What a body of records entitles a governance framework, a procurement process, or a jurisdiction to conclude is not specified anywhere in the stack as drawn. The three-column schema in the same presentation, reference to evaluation to evidence with feedback into reference, is likewise per action. This theme does not add a column to it. It operates on the aggregate of the evidence column across many instances, which is a different axis.

This matters most for the relying party who is least equipped. An enterprise integrator can audit its counterparties by contract, and a regulator can compel disclosure. The general case is a party with neither: an end user of a service, or a jurisdiction that hosts the delivery of systems it did not build and cannot inspect. For that party a population of records read at rate is the only instrument available, and whether one exists is a design decision this group is making now.

Terms of Reference. This theme serves objective 3.4, criteria and benchmarks for assessing trustworthiness, accountability and traceability, and deliverable 4.5, assessment criteria and metrics for evaluating trustworthiness, with the behavioral trust signals of 4.3. Its closest Annex example is A.2.8: anomaly and deviation detection from observed behavior, a taxonomy of behavioral signal types, and a data model for communicating behavioral evidence. The Rationale states the need directly: across organizational and jurisdictional boundaries, trust "cannot be established or enforced by a single authority." The theme specifies evaluation evidence and metrics. It specifies no protocol and no governance policy, both of which the ToR places out of scope.

Scope / Open Questions

Four tasks. They are numbered in the order they were raised, which is not the order they depend on one another: task 4 defines the taxonomy a rate is measured against, task 3 establishes what can be assumed about the evaluators, task 1 classifies which claims those two together can support, and task 2 specifies what is published. The numbers are kept stable so that references in this thread still resolve. The theme adds nothing to the record format, the verdict vocabulary, or the appraisal checks; it specifies what may be computed over them.

Task 1, falsification of undecidable claims at population scale. For each per-record undecidable claim now named in #6, state what evidence a population provides that a record does not. This begins with a classification step ahead of any question about population size, distinguishing sampling uncertainty, where a claim is identifiable and more records narrow it, from structural indeterminacy, where a claim is not identifiable under the stated observation architecture and no volume of records changes that. The classification is made under stated evaluator-family assumptions; task 3 specifies how those assumptions are established, not merely declared. The three initial cases are the issuer-relationship declaration, the declared confidence threshold and value, and the captured-engine case in open question 7. The output is a statement of what is detectable, at what population size, with what false-positive behavior, and, importantly, what remains undetectable at any scale. A claim that population reading dissolves all three would be wrong and the theme should say where it fails.

Task 2, rates, not composite scores. What is published about an evaluated party should be per-flag-type rates over a declared taxonomy, not a composite score. A composite is a single number whose weighting is an unstated policy judgment, which makes it a capture surface: whoever sets the weights sets the outcome, and the weighting is invisible in the published figure. Per-type rates leave the cross-type judgment with the party bearing the loss. This task specifies what a published rate must carry to be interpretable: the assessment claim stated separately from the rate, the taxonomy and policy the rate is defined against with version, the population it was computed over, the observation period, the characteristics of the evaluator or evaluator family, what the rate is capable of establishing, and what residual indeterminacy remains under the stated observation architecture.

Rates are published per evaluator by default. Pooling several evaluators' rates into a single figure against the same taxonomy and policy is optional for a reporter, and it is the step that assumes the evaluators fail differently. The pooled figure cannot itself support that assumption: correlated evaluators pool into a number that looks more robust than its components and is not. An un-pooled publication carries no such assumption and leaves the spread visible to the reader, which is the composite-score argument of this task applied one level down.

Task 3, evaluator diversity as a design property. A single authoritative evaluator, however well accredited, produces errors that are perfectly correlated across everything it observes, and its blind spots become the population's. This is the same structural objection as trusting the declarer, moved one level up. The task addresses the apparent conflict between diversity and reproducibility: what an individual evaluator must satisfy to be independently reproducible, and what a family of evaluators must satisfy to be diverse in a way that is measurable, not asserted. There is an existence proof already in this repository: the distillation measurement reported on #18 was scored by four independent judges from four different organizations against a reference declared in advance, and was reproducible by a party with no relationship to the operator. The per-judge scoring is published, so the between-judge spread is readable directly. Four judges separately operated is not the same as four that fail differently, so the spread that measurement yields is only a lower bound on honest variance, which is itself the task 3 question. Agreement between evaluators is also not the same as correctness: a family can agree completely and be wrong together, which is the failure this task exists for. Diversity therefore matters more where a taxonomy's definitions are loose, but does not become unnecessary where they are tight, since shared systematic error survives a precise definition. Measuring it needs a test that catches shared mistakes, not one that counts disagreement.

Task 4, the scope of comparability. Rates are comparable only within a population sharing a taxonomy. This task specifies what defines a sharing set, what a taxonomy must carry to be a shared reference (version, definitions, change history), and what may and may not be concluded when comparing across taxonomies. The initial position is that a cross-taxonomy comparison is a comparison of what two jurisdictions chose to measure, and is not a comparison of scores. The absence of a flag type from a taxonomy's version history is itself readable. Where the taxonomy derives from published jurisdictional requirements, the derivation into evaluable form is an authored artifact with standing under #5, and this task states what it must carry.

Open questions for this thread:

1. What is the minimum population size at which a false independence declaration becomes distinguishable from honest variance, and what does that number depend on? The statistical machinery is standard; the quantity nobody has measured is how much honest evaluators vary from one another on identical traffic, which bounds the answer more tightly than sample size does. Three sources are available without new infrastructure: the existing power and inter-rater reliability literature, the four-judge measurement reported on #18, which is a between-judge variance observation already in this repository, and replication across checkpoints of a single evaluator against a frozen suite, which the proposer can run.
2. The benchmark literature handles the disclosure tension with held-out items, rotating probes, published methodology and delayed release, all of which assume an evaluator the evaluated party cannot inspect. A client-side evaluator does not have that property: it runs on the delivery path, and an adversary who possesses it can characterize it offline at leisure. So the question is what an evaluator whose weights and prompts are in the adversary's hands can publish that still makes its errors characterizable by a third party, and whether the answer is disclosure at all, or diversity across the family.
3. Diversity and reproducibility appear to conflict. Does a family of individually reproducible evaluators, diverse across the family, resolve it, and how is family-level diversity measured instead of asserted?
4. Who computes a published rate, and what stops that party becoming the single point of control the diversity requirement exists to avoid? This is #13's operator question arriving at this layer, and it needs an answer that does not reintroduce an authority.
   One candidate, raised by @zhaoxinghua09-cell: the computing party holds the records but neither authorship over the reference nor judgment over the taxonomy. That relocates the question without dissolving it, since computing a rate still involves aggregation choices; the un-pooled default in task 2 keeps those small, because pooling is where most of them enter.
5. Due process for statistical evidence has settled answers where the subject can see the basis and submit counter-evidence. A published rate does not fit that shape: it is computed over records the evaluated party never held, by evaluators it did not select, against a taxonomy it did not author. So the question is what it is entitled to see in order to contest the rate, and whether that disclosure defeats the answer to question 2. If contesting a rate requires handing the contesting party the evaluators' inputs, the two requirements are in direct tension and the theme should say which yields.
6. Which of the initial undecidable claims remains undecidable at population scale, and is that set stable as the population grows?

Related Work

draft-flores-airp-provenance-00 (IETF), an accountability layer for AI inference in which a client-side evaluator loyal to the user, and structurally not the provider, evaluates responses against declared policy and publishes per-flag-type rates and no composite score. Reference implementation and local evaluator running at tryairp.com, Apache 2.0. The decision to eliminate composite scores from the shared layer was made on the argument in task 2 above.

Empirical input from operating a small evaluator against a frozen held-out suite, offered as measurement, not as a result to adopt. The evaluator is a fine-tuned language model classifying inference output against a versioned taxonomy of flag types, a learned instrument, not a rules engine. Two findings bear on task 3. First, a single training checkpoint's per-class performance is one draw and not a measurement: across checkpoints, several classes move in and out of passing while aggregate performance is flat, which means an evaluator characterized at one checkpoint has been characterized at a moment, not as an instrument. Second, on the classes examined, the binding constraint on evaluator behavior was the definition of the category being evaluated, not model capacity, and no volume of additional training data corrected a category whose definition was too broad. Both findings argue that evaluator error is a property that has to be published and compared, not certified once.

RATS AR4SI for the evidence-side vocabulary, and the distinction @tyche-dev draws between an evidence-side gap and an action-side indeterminate, adopted here unchanged. AVS-0.2 for the appraisal checks this theme consumes. ISO/IEC 17025 and the accreditation model generally, as the contrast case: accreditation certifies an instrument once against a standard, where this theme proposes characterizing instruments continuously against each other.

Related Themes

#6 (Intent-based security policies): the primary upstream, and the source of this theme's mandate. #6 supplies the five-state verdict vocabulary, the issuer-relationship field, the declared confidence, and the requirement that every indeterminate carry scope and attribution. @cgorlla has stated that #6 makes each determination readable so that a set of them can be read as a rate, and that #6 cannot answer who reads them. This theme takes the reading side and proposes no change to anything on the record side.

#7 (Verifier-side requirements and failure semantics): consumed by version. AVS-0.2 decides whether a record is entitled to carry a state, so this theme operates only on records that have passed appraisal, and rejection classes are an input it does not adjudicate. The rejection rate across a population is itself a readable signal, and a distinctive one: a producer controls the verdicts its engine issues but does not control whether its records pass another party's appraisal, so the rejection rate is a measurement it cannot manage. This theme reads it without altering a class or a check.

#1 (Accountability and attribution): supplies the record and the properties any carried claim needs. This theme reads many such records and proposes no change to what one carries.

#5 (Provenance of authority): verification of an issuer-relationship declaration is a provenance question about the issuer and belongs there. This theme covers the case where that verification is unavailable to the relying party, which is the common case. The derivation of a published jurisdictional requirement set into evaluable form is an authored artifact and its standing is #5's question.

#16 (Operational human oversight): consumer, not upstream. The boundary is settled with #16's proposer: population-level evaluation stays here, operational oversight stays there, with an explicit handoff instead of a merge. The handoff carries the rate, the defined population, the observation period, the taxonomy and policy version, the residual indeterminacy, the characteristics of the evaluator or evaluator family, and what the rate is capable of establishing. Trigger qualification and everything downstream of it is #16's.

#13 (Ecosystem-level agent defense), including Ecosystem Awareness: closest neighbor on method, and the interface that most needs stating. #13 includes reputation scoring and shared signal exchange, and asks who operates such an infrastructure and what stops the operator becoming a single point of control. The position here is that the question follows from centralizing the computation, and that per-type rates over a declared taxonomy, computed within a population and published by no authority, do not require an operator to trust.

The Ecosystem Awareness work @dakleyer is developing now sits in #13's direction, with a shared external signaling interface there. The boundary between it and this theme is the question each answers, not who holds authority: this theme establishes what a population of observations supports, and Ecosystem Awareness establishes whether that is sufficient for a particular decision in its context. The distinction holds whether the receiving party is internal, external, distributed, or without authority to act. Findings pass from here to there without being recomputed. In the other direction, a report that a result was insufficient for a decision, naming the gap, is accepted. Decision context traveling toward the measurement, such as the target hypothesis or why it matters to the receiver, is not, since an evaluator that knows what its client needs is producing evidence for a purpose. A result returned as unidentifiable or unsupported under the available observation architecture is a first-class answer.

#18 (Trustworthiness of multi-objective agentic AI): the identifiability result there constrains this theme and is accepted. What a population changes is not the identifiability of the effective operator from one trace, which stands as stated, but what is readable when observations are generated through channels that do not share a failure mode.

#3 (Global trust and accountability): the governance layer. Per-type rates are the form of published evidence a liability or procurement framework can be written against, and the reason this theme resists composites is that a composite hands a governance framework a number whose policy content is hidden inside it.

Discussion

Open. The proposer is interested in the specification, not the chair, and the theme is deliberately narrow: it consumes four upstream themes and modifies none of them. If the group's judgment is that this belongs inside #13 alongside Ecosystem Awareness, that is a reasonable outcome and worth settling before the 30 November boundary.