[Theme] Provision of information about AI Agent to support risk assessment

Theme Name

The identity of the AI Agent is not sufficient to ensure the trust of an operation it carries out. Identity of AI Agent only supports traceability and accountability. In order to ensure trust, the AI Agent identity shall be enhanced by information allowing to assess its trustworthiness and the corresponding risk.

Proposer(s)

A.Feraud - IN Groupe - alban.feraud@ingroupe.com

Summary

The identity of the AI Agent is not sufficient to ensure the trust of an operation it carries out. Identity of AI Agent only supports traceability and accountability. In order to ensure trust, the AI Agent identity shall be enhanced by information allowing to assess its trustworthiness and the corresponding risk.
Information allowing to assess the trustworthiness of an operation to be carried out by an AI Agent may be available and originating from various sources or entities such as (1) the AI Agent supplier, (2) the supervision authority, or (3) the certification authority (not limitative).
Such information may be manifold (not limitative):
• Security assessment or conformity assessment of a given operation carried out by the AI Agent, possibly with the corresponding operational measures to be enforced by the AI Agent owner;
• Level of confidence of the supplier for a given operation carried out by the AI Agent;
• Guidance from the supplier for a given operation carried out by the AI Agent;
• Possible risks and consequences
• Etc…

This information could usefully be exploited for risk assessment e.g.
• By the AI Agent owner to take a decision regarding authorizing or delegating some operation to an AI Agent;
• By the entity in charge of issuing a credential to an AI Agent representing an authorization or a delegation for some operation, to decide (1) whether or not such credentials should be issued and (2) under which conditions;
• By the Relying party or another AI Agent interacting with that AI Agent to assess the reliability and the trust to place in its operation;
This information is critical as it ultimately supports risk management by stakeholders (1) accepting an operation from an AI Agent or (2) giving authorizations or delegation to an AI Agent.

Scope / Open Questions

As part of this theme, the following aspects could be further analysed:
1. Taxonomy of the various sources and entities which could provide relevant information about AI Agent and which could further support risk assessment;
2. Taxonomy of the various entities which could benefit from this information about AI Agent to carry out risk assessment;
3. Protocol(s) for the retrieval of this information by entities willing to carry out a risk assessment. This should consider both the cases where the information is found in the AI Agent itself or remotely (e.g. website provided by the supplier, repository maintained by national authorities);
4. Content of the aforementioned information
a. Taxonomy and encoding of content of the aforementioned information;
b. Format and structure of the aforementioned information;
The aforementioned information should be designed to be machine readable;
5. Trust framework allowing verification of the authenticity, integrity and validity of the aforementioned information by the entities using it for risk assessment purposes;

Related Work

To be identified

Related Themes

This theme is related several other proposed themes including:
• https://github.com/FG-TIDA/themes/issues/7 as regards to actions carried out by an Agent verifier ahead of allowing the requested operation.
• https://github.com/FG-TIDA/themes/issues/10 as regards to governance functions to be carried out once an Agent has been identified.
• https://github.com/FG-TIDA/themes/issues/23 as regards to authorization decisions.
• https://github.com/FG-TIDA/themes/issues/27 and 28 as regards to verifier obligations.
• https://github.com/FG-TIDA/themes/issues/29 as regards to available evidence for a verifier.

Discussion