Back to Dashboard
8.1 Clarification of Associated Risks
Original Problem in the Paper
Paper motivation/open problem: “Understanding risks associated with the development and deployment of AI systems enables policymakers to prioritize governance efforts, allocate resources effectively, and determine the urgency of addressing specific risks.” The paper identifies several related open problems: developing better threat models for AI risks because detailed threat models are “relatively underexplored”; applying standardized risk-management approaches such as causal mapping; improving incident reporting and monitoring because OECD/PAI-style databases rely on public sources, likely cover only a subset of incidents, and often lack details such as model specifics or deployed guardrails; and determining how non-public incidents can be reported and what technical information should be included for meaningful analysis.
July 2026 Update & Trajectory
By July 2026, several public risk-taxonomy and incident-monitoring resources are active or recently updated: OECD AIM, the MIT AI Risk Initiative’s Repository/Navigator/Incident Tracker, the International AI Safety Report 2026, and NIST’s GenAI Profile. Public incident-monitoring is operational in narrow forms, especially OECD AIM and MIT’s AI Incident Tracker, but both depend on public-source or voluntary reporting pipelines and have coverage, depth, and validation limits. The original problem remains open because public-source and voluntary incident datasets are structurally incomplete for non-public incidents; detailed causal threat models remain underdeveloped relative to broad risk taxonomies; existing incident reports often lack consistent model/version and safeguard or guardrail metadata; and severity, vulnerability, and responsibility ratings remain judgment-sensitive rather than standardized across domains. The sources reviewed here do not establish a mandatory global non-public AI incident-reporting regime as of July 2026.
Deployed / Operationalized
- OECD AI Incidents and Hazards Monitor documents incidents and hazards from public sources for policymakers and other stakeholders, with classifications that include AI principles, industries, affected stakeholders, harm types, severity, autonomy level, business function, and AI-system task.
- MIT AI Risk Initiative maintains the AI Risk Repository, AI Risk Navigator, AI Incident Tracker, AI Governance Map, and Priority AI Risks expert-survey materials; its site lists a June 30, 2026 Incident Tracker pilot-validation update and April 2026 Navigator/Governance Map updates.
- MIT’s AI Incident Tracker classifies more than 1,400 AI Incident Database reports by risk, cause, harm, severity, EU AI Act risk level, causal attributes, national-security impacts, and related dimensions, while warning that source reports vary in quality and depth, are voluntary, have sampling bias, and should be treated as indicative until further validated.
- The International AI Safety Report 2026, published 3 February 2026, is a review of scientific research on general-purpose AI capabilities, risks, impacts, and risk-management or mitigation approaches, led by Yoshua Bengio, authored by 100+ experts, and backed by over 30 countries and international organizations.
- NIST GenAI Profile defines 12 risks unique to or exacerbated by generative AI, including CBRN information or capabilities, confabulation, data privacy, environmental impacts, information integrity, information security, and value-chain/component integration, and notes that risk estimation is difficult because training-data visibility is limited and AI measurement/safety science is immature.
- The EU AI Act defines risk tiers and includes Article 73 serious-incident reporting duties for providers of high-risk AI systems placed on the Union market, with application dates depending on system category; providers of GPAI models with systemic risk have separate serious-incident tracking, documentation, and reporting duties under Article 55 that the Commission says became effective in August 2025. Separately, the OECD Hiroshima AI Process Reporting Framework standardizes voluntary organizational transparency reporting on advanced-AI risk-management practices, not mandatory incident reporting.
New Tractable Vectors
- Build shared mappings among risk taxonomies and governance frameworks, including NIST, MIT, OECD, the International AI Safety Report, and the EU AI Act, so incidents and governance coverage can be compared without treating the taxonomies as interchangeable.
- Pilot LLM-assisted first-pass incident classification with expert-human validation and held-out review sets, building on MIT’s June 2026 10-incident pilot showing that selected frontier models can approach human-review baselines on several taxonomies after prompt refinement.
- Develop structured incident/flaw-reporting forms that capture model/version where known, access modality, safeguards or guardrails, deployment context, affected stakeholders, severity, and causal-chain evidence, while allowing confidential or security-sensitive details to be protected or routed appropriately.
- Use stable public incident-corpus snapshots plus expert surveys to identify and validate priority monitoring areas, with candidate areas including non-consensual or deceptive synthetic media, health and safety automation, AI-assisted cyber misuse, autonomous-vehicle failures, financial fraud or misinformation, and AI infrastructure impacts.
Key Open Questions
- Trusted channels and legal protections for non-public incident reporting without exposing confidential, personal, or security-sensitive details.
- Causal attribution: separating an AI system’s contribution from human misuse, organizational failure, and ordinary software failure.
- Severity calibration across physical, financial, rights, psychological, environmental, and national-security harms.
- Coverage bias from dependence on news, public submissions, and public web sources, including possible language and geographic skew, remains an open measurement problem.
- Threat models for multi-agent and tool-using AI systems, human emotional dependence on AI assistants, correlated failures from common model dependencies or systemic monoculture, and possible strategic interaction among AI systems remain underdeveloped.
Evidence & Primary Sources
- Source paper: the paper states that understanding risks associated with AI-system development and deployment helps policymakers prioritize governance, allocate resources, and determine urgency; it identifies better threat models, standardized causal mapping, improved incident monitoring, public-source database limitations, missing model/guardrail details, and non-public incident reporting as open problems. (arXiv: 2407.14981): https://arxiv.org/abs/2407.14981
- OECD AIM documents AI incidents and hazards from public sources for policymakers and other stakeholders; the page describes AIM as an automated beta monitor using Event Registry and Microsoft Azure data processing, and displays incident/hazard fields such as AI principles, industries, affected stakeholders, harm types, severity, autonomy level, business function, and AI-system task. Because displayed examples are dynamic, individual examples should not be treated as stable evidence. (Accessed 7 July 2026; page copyright 2026): https://oecd.ai/en/incidents
- MIT AI Risk Initiative provides the AI Risk Repository, AI Risk Navigator, AI Incident Tracker, AI Governance Map, and Priority AI Risks materials; the homepage describes the repository as a living database of 1,700+ AI risks from 65 frameworks and lists June 2026 Incident Tracker updates plus April/June 2026 governance and navigator updates. (Website copyright 2026; June 2026 update listed): https://airisk.mit.edu/
- MIT AI Incident Tracker says it classifies more than 1,400 real-world reported incidents from the AI Incident Database by risk, cause, harm, severity, and other dimensions using an LLM pipeline; it also states that AIID reports rely on public and expert submissions, vary in quality, reliability, and depth, are voluntary, have sampling bias, and that observed patterns should be treated as indicative and validated through further analysis. (Website copyright 2026): https://airisk.mit.edu/ai-incident-tracker
- MIT’s “AI Incident Tracker June 2026 Update” describes a June 30, 2026 pilot validation study comparing eight LLMs against two expert reviewers on 10 incidents across five taxonomies. It reports that selected frontier models met or exceeded human-human baselines on several taxonomies, that EU AI Act risk-level classification remained difficult before prompt refinement, and that the work is a proof-of-concept needing larger validation. (30 June 2026): https://airisk.mit.edu/blog/ai-incident-tracker-june-2026-update
- International AI Safety Report 2026 says the second report was published 3 February 2026; it is a comprehensive review of scientific research on capabilities and risks of general-purpose AI systems, led by Yoshua Bengio, authored by 100+ experts, and backed/nominated by over 30 countries and intergovernmental organizations; it covers risks, impacts, mitigations, and risk management. (3 February 2026): https://internationalaisafetyreport.org/
- NIST GenAI Profile defines risks unique to or exacerbated by generative AI and identifies 12 risk categories: CBRN Information or Capabilities; Confabulation; Dangerous, Violent, or Hateful Content; Data Privacy; Environmental Impacts; Harmful Bias or Homogenization; Human-AI Configuration; Information Integrity; Information Security; Intellectual Property; Obscene, Degrading, and/or Abusive Content; and Value Chain and Component Integration. It also notes that risk estimation is difficult due to limited visibility into training data and immature AI measurement/safety science. (July 2024): https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf
- OECD Hiroshima AI Process Reporting Framework, launched in February 2025, provides a standardized structure for organizations to report on alignment with the Hiroshima AI Process Code of Conduct; Version 2.0 broadens participation across the AI ecosystem, organizations can submit reports on a rolling basis, submissions by 1 September 2026 feed the next analytical review, and submitted reports will be published. This is voluntary organizational transparency/practices reporting, not mandatory incident reporting. (Page copyright 2026): https://oecd.ai/en/transparency/overview
- European Commission AI Act explainer states that the AI Act defines four risk levels: unacceptable, high, transparency, and minimal/no risk; prohibited-practices rules became effective in February 2025, GPAI rules became effective in August 2025, transparency rules come into effect in August 2026, and providers/deployers report serious incidents and malfunctioning once systems are on the market. (Includes 2026 implementation updates): https://digital-strategy.ec.europa.eu/en/policies/regulatory-framework-ai
- EUR-Lex Regulation (EU) 2024/1689 provides the legal text for serious-incident reporting. Article 73 requires providers of high-risk AI systems placed on the Union market to report serious incidents to market surveillance authorities where the incident occurred, generally after establishing a causal link or reasonable likelihood and within specified deadlines. Article 55 requires providers of GPAI models with systemic risk to keep track of, document, and report relevant information about serious incidents and possible corrective measures to the AI Office and, as appropriate, national competent authorities. Article 3(49) defines “serious incident.” (Regulation (EU) 2024/1689): https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A32024R1689