Contribution: Evidence-Graded Authorization Verdicts for Agentic AI (refined from themes#42)

Theme Proposal — Formally Refined

Following the open discussion in themes#42 (including the B1/B2 split suggested by Iván), this contribution is the refined theme proposal submitted as a standalone theme issue.

Theme Name

Evidence-Graded Authorization Verdicts for Agentic AI — a mechanism-oriented theme: how a human or an automated principal can verify an agent's authorization claim, at what grade of evidence, and what each grade means operationally.

Proposer(s)

LGD-Eval — independent third-party evaluation group. Evaluation protocol: https://github.com/LGD-Eval/evaluation-protocol. Conflict-of-interest disclosure: https://github.com/LGD-Eval/evaluation-protocol/blob/main/CONFLICT_OF_INTEREST_DISCLOSURE.md

Summary

When an agentic system acts under a delegated authority, the delegating party needs to know not just whether the agent's action was authorized, but how verifiable that fact is: what evidence exists, who can observe it, and what happens when evidence is missing, measured-and-wrong, or not yet decidable. This theme proposes a grading vocabulary — verified / unresolved / rejected / quarantined — plus a payload-digest convention, so that authorization claims can be stated, compared, and re-checked across implementations without trusting the reporter. It sits within FG-TIDA's scope: trust and identity mechanisms for agentic AI, at the level of evidence and protocol mechanics.

What is proposed

1. Evidence grading scale (evidence axis). A small, ordered set of evidence grades a verifier can assign to a presented claim based on how it was obtained and how checkable it is — e.g. machine-verified from a registered source, self-attested by the agent, indirect/corroborated, or lower-confidence/unavailable. The grade concerns the claim's quality, not the actor's authorization.

2. Authorization-validity dimension, evaluated separately (authorization axis). A separate state for whether the presented authorization is valid, out of scope, expired, or unverifiable at verification time. The separation lets a verifier state "the evidence is strong, but this operation was not authorized" or "the authorization is valid, but the supporting evidence is weak" — instead of collapsing both into one verdict.

3. Explicit failure classes with normative semantics. Verifier outcomes distinguish at least: MUST-reject cases (a valid-looking but unauthorized operation), distinguishable failure classes (scope violation vs. expiry vs. unverifiable source), and log-and-proceed boundaries — so that implementations fail in the same observable way. This addresses the interoperability concern that implementations which fail differently break interoperability faster than implementations which verify differently.

4. Chained verification path. For multi-hop claims (A delegated to B, B presents to C), each hop carries its own evidence grade and authorization state; the verifier combines them along the chain and can report where a chain breaks rather than only that it failed.

5. Implementation-neutral worked examples. A machine-runnable example set demonstrating the mechanism against realistic cases — a valid registered credential, an unverifiable issuer scope, a confirmed out-of-scope issuer, an unavailable status source — with structured inputs and expected verdicts, so the mechanism can be tested independently of any particular implementation or schema.

Scope / Open Questions

• What should a minimum verifiable authorization record contain (decision, evidence grade, reason codes, observed-behavior snapshot, payload digest)?
• How should "evidence absent" (unresolved) be distinguished operationally from "evidence found wrong" (rejected)? The two demand different operator responses; the B1/B2 case split in themes#42 is our concrete worked example.
• How should spec-undecided fields be handled — a quarantine category that excludes vectors from formal results until normative semantics settle?
• Who can observe satisfaction (witness scope) and on what basis (verification basis)? We propose the two axes be kept separate rather than collapsed into one strength value.
• How do these records compose with supply-chain attestation (e.g. in-toto-style eval-result predicates), so a conformance result can be independently re-verified rather than self-reported?

Related Work

• FG-TIDA themes#42 discussion (this thread), including the B1/B2 split discussion.
• IETF RATS / attestation evidence direction (evidence vs. appraisal).
• in-toto attestation predicates (evaluation results as signed evidence).
• LGD-Eval evaluation protocol (public): four-case worked examples (A/B1/B2/C), evidence grades, payload digests, selftest with 15 invariants.

Worked examples and verification

• Machine-runnable cases: https://github.com/LGD-Eval/evaluation-protocol/tree/main/worked-examples/cases
• Interactive demo (three cases, two-axis diagram, per-case verdict): https://github.com/LGD-Eval/evaluation-protocol/blob/main/demo/evidence-grading-demo.html
• Reproducibility: python tools/evidence-grader.py --selftest → SELFTEST OK (15 invariants); all four case digests reproduce end-to-end — A 640db270e69c9f79, B1 8d81c5d559914ec5, B2 c19e1781ebd4f690, C 8333b571827a1c01.