Evidence Grading for Verifier Decisions: Separating Evidence Quality from Authorization Validity

Proposer(s)

LGD-Eval — Independent AI Evaluation

Summary

Verifier-side appraisal of agent evidence currently mixes two distinct questions into a single pass: how good is the evidence (its provenance, freshness, and machine-checkability) and is the presented authorization valid (who issued it, under what scope, for how long). Blending the two produces failure modes that are hard to reason about: a verifier that rejects high-quality evidence on a scope detail, or accepts a well-formed credential that was never actually authorized for the operation. This theme proposes that the two dimensions be declared and evaluated separately, with an explicit evidence grade on one axis and an authorization-validity state on the other, so that a relying party can state — and reproduce — why it accepted or rejected a given claim.

The proposal builds on the verifier-side requirements discussed in #7 (verifier obligations as first-class content; negative conformance vectors) and on the discussion of connection decisions in #29. It contributes a concrete, implementation-neutral mechanism for the appraisal side of both: an evidence grading scale, a separated authorization-validity dimension, explicit failure classes, a chained verification path, and a worked example set that is machine-runnable.

What is proposed

1. Evidence grading scale (evidence axis). A small, ordered set of evidence grades that a verifier can assign to a presented claim based on how it was obtained and how checkable it is — e.g. machine-verified from a registered source, self-attested by the agent, indirect/corroborated, or lower-confidence/unavailable. The grade is about the claim's quality, not about the actor's authorization.

2. Authorization-validity dimension, evaluated separately (authorization axis). A separate state for whether the presented authorization is valid, out of scope, expired, or unverifiable at verification time. This separation lets a verifier say "the evidence is strong, but this operation was not authorized" or "the authorization is valid, but the supporting evidence is weak" — instead of collapsing both into one verdict.

3. Explicit failure classes with normative semantics. Verifier outcomes should distinguish at least: MUST-reject cases (a valid-looking but unauthorized operation), distinguishable failure classes (scope violation vs. expiry vs. unverifiable source), and log-and-proceed boundaries — so that implementations fail in the same observable way, addressing the interoperability concern raised in #7 (implementations that fail differently break interoperability faster than implementations that verify differently).

4. Chained verification path. For multi-hop claims (A delegated to B, B presents to C), each hop carries its own evidence grade and authorization state; the verifier combines them along the chain and can report where a chain breaks rather than only that it failed.

5. Implementation-neutral worked examples. A machine-runnable example set demonstrating the mechanism against realistic cases — a valid registered credential, an out-of-scope issuer, an unavailable status source — with structured inputs and expected verdicts, so the mechanism can be tested independently of any particular implementation or schema.

Scope / Open Questions

• Whether the evidence grading scale should be a single shared scale or per-work-item scales with a common ordering.
• How authorization scope is expressed in a form that is machine-checkable across organizational boundaries (alignment with the "should problem" in #29).
• Where the boundary sits between verifier MUST-reject obligations and policy choices that remain with the relying party.
• Whether the worked example set should be contributed as a conformance corpus aligned to whatever schema the group adopts (as offered in #7).

Why it matters for FG-TIDA

The FG's framing asks whether, and under what conditions, an entity should be trusted to act. A verifier that cannot separate evidence quality from authorization validity cannot give a legible, reproducible answer. Making the two axes explicit turns the appraisal step into a testable, declarable artifact — and gives implementers a shared vocabulary for reporting why a claim was accepted or rejected.

Reference material

• Related theme: Verifier-side requirements and failure semantics for agent evidence (#7)
• Related theme: The Should Problem: governed connection decisions across organisational boundaries (#29)
• Worked examples (machine-runnable, published): FG-TIDA/use-cases discussion #27 — evidence grading and authorization separation, example set in LGD-Eval/evaluation-protocol/worked-examples/cases
• Tooling used to produce the examples: evidence grading reference implementation (see worked examples README for reproduction steps)