Platform OverviewOne evidence trail, from intake to Jira.See how it fits together
See it on your documentsA 30-minute call. Bring a vendor you are reviewing now.Book a demo
AI vendor riskPublished Updated

Explainable AI finding

Also called: Explainable finding, Evidence-linked finding

Definition

An explainable AI finding is a result produced with AI assistance that shows its basis: the rule it was checked against, the evidence excerpt and location it relies on, and enough context for a human reviewer to verify or reject it.

What Explainable AI finding means

AI can read long vendor documents faster than people, but a conclusion without its basis cannot be checked. An explainable finding states what was tested, what the document says and where, and how confident the result is, so a reviewer can confirm it quickly.

The idea reflects wider principles. The NIST AI RMF lists explainable and interpretable among the characteristics of trustworthy AI. The EU AI Act requires human oversight for high-risk systems. In compliance work the same standard applies as for any reviewer's work: a conclusion must trace back to evidence.

Explainable findings also make disagreement useful. When a reviewer overrides a finding, the recorded reason shows whether the evidence was misread, out of scope or outweighed by other facts. Over time, the pattern of overrides shows where rules or documents need attention.

What to check in vendor documents

  • Each AI-generated conclusion cites the rule or requirement it was tested against.
  • The excerpt and page or location in the source document are recorded.
  • A human reviewer can accept or override the result, and the decision is logged.
  • Overrides require a reason that stays with the record.
  • The system shows when evidence was not found, rather than filling the gap.

In ProcureCortex

Every ProcureCortex finding names its rule and keeps the evidence excerpt and page, and a reviewer accepts or overrides it with a recorded reason.

How an assessment works

Related terms

Sources

  1. NIST AI 100-1, Artificial Intelligence Risk Management Framework (AI RMF 1.0)doi.org/10.6028/NIST.AI.100-1
  2. Regulation (EU) 2024/1689 (AI Act)eur-lex.europa.eu/eli/reg/2024/1689/oj

Summarised for practitioners, not legal advice. Check the source text for your own obligations.

Explainable AI finding: questions

Short, direct answers.

Yes, if a human reviews them and each one shows the evidence it rests on, so the decision is the reviewer's and can be audited.

See it on your own vendor documents.

Open a demo workspace with a sample vendor, its documents and its findings. Or bring your own vendor to a 30-minute call.

Procurement, compliance, security and auditOne workspace, one record of every decision