AI Incident

WHAT ARE AI INCIDENTS?

An AI incident is broadly understood as any event, circumstance, or series of events in which the development, use, or malfunction of one or more AI systems — directly or indirectly — leads to harm. Such harms include physical injury, disruption of critical infrastructure, violations of human rights, or damage to property, communities, or the environment. This formulation closely tracks the definition adopted by the OECD, which aims to provide an interoperable baseline for regulators and policymakers across jurisdictions.

The term captures a wide spectrum of failures: from a chatbot producing dangerous advice, to an autonomous vehicle crashing, to an algorithmic system making discriminatory decisions in employment or credit. AI incidents are not limited to deliberate misuse; they equally encompass unintentional failures arising from system design, data quality, deployment context, or inadequate human oversight.

A related but distinct concept is an AI hazard, which the OECD defines as an event or circumstance that could plausibly lead to an AI incident, i.e., a near-miss or precursor condition that has not yet caused actual harm. The distinction between actual harm (an AI incident) and potential harm (an AI hazard) is foundational to effective risk management and responsible deployment practices.

In modern computational and legal systems, an Artificial Intelligence (AI) incident represents a critical paradigm shift in how technology-induced harms are identified, evaluated, and legally governed ¹ . Traditionally, standard software failures have been deterministic—manifesting as loud, compile-time errors, execution exceptions, or system crashes that immediately trace back to a specific, broken line of code.In contrast, artificial intelligence systems rely on complex mathematical models that represent the transition dynamics of physical or digital environments. This probabilistic, data-driven architecture allows the system to execute tasks with varying degrees of autonomy and adaptiveness after deployment because of this self-learning, adaptive behaviour, an AI system does not fail in a conventional, binary manner. Instead, it undergoes "silent" and "fluent" degradation, producing plausible but incorrect predictions, discriminatory profiling, or hazardous physical trajectories while the underlying hardware, server connections, and system infrastructure continue to operate normally and log healthy metrics.

To capture and address these unique failures, the concept of an AI Incident has emerged. It signifies a realised event or series of events where the operational execution, output, or malfunction of an AI system functions as a direct, "but-for" cause of tangible or intangible harm to human health, safety, civil liberties, property, or environmental systems.

The concept is vital because it establishes a causal baseline for accountability in an era where decision-making is increasingly delegated to automated agents. It bridges the gap between high-level ethical principles and concrete legal liability, moving the governance of technology from abstract "responsible AI" guidelines to enforceable, audit-ready technical and operational safety standards.

Importance of AI Incidents

AI incidents matter because they constitute direct, real-world evidence of the harms AI systems can and do cause. According to reporting drawing on the Stanford AI Index, documented AI safety incidents rose from 149 in 2023 to 233 in 2024, representing an increase of approximately 56.4% in a single year. These are not theoretical risks; they reflect actual financial losses, legal consequences, physical safety failures, and, in the most serious cases, loss of life.

Tracking and reporting AI incidents serves several governance objectives: it enables regulators to identify high-risk systems and emerging threat patterns in real time; it builds accountability for developers and deployers; and it facilitates collective learning across organisations and jurisdictions. Empirical monitoring — for example through the OECD AI Incidents and Hazards Monitor (AIM) — already shows a steep increase in reported incidents and hazards over time, strengthening the case for proactive policy intervention.

History and Evolution

AI incident tracking as a formal discipline has emerged incrementally alongside the deployment of algorithmic systems in consequential settings such as healthcare, finance, transportation, and public administration. It draws from mature incident-reporting traditions in other domains — notably aviation (e.g., the FAA’s voluntary reporting system), healthcare (sentinel event reporting), and cybersecurity (the Common Vulnerabilities and Exposures, CVE, programme) — where structured disclosure has long been central to safety improvement.

The AI Incident Database (AIID), launched in 2021, is among the earliest dedicated repositories of recorded AI harms. It collects incident reports from news media, research papers, and direct submissions, and has grown substantially, with approximately half of its 800+ entries added since 2022, mirroring the rapid expansion of AI deployment. Parallel initiatives include the AI, Algorithmic, and Automation Incidents and Controversies (AIAAIC) repository, which adopts an “outside‑in” perspective focused on ethical and societal impacts, and the OECD AI Incidents and Hazards Monitor (AIM), a global project that tracks incidents and hazards in near real time based on media reporting.

Between roughly 2010 and the early 2020s, AI-related incidents reported in such repositories increased by an order of magnitude, prompting institutions including the OECD and the Center for Security and Emerging Technology (CSET) to develop standardised definitions, taxonomies, and reporting frameworks so that lessons can be systematically shared across organisations and borders.

OFFICIAL DEFINITION OF AI INCIDENTS

AI Incidents in Indian Legislation and Policy

Digital Personal Data Protection Act, 2023

India’s Digital Personal Data Protection Act, 2023 (DPDP Act) provides the country’s first comprehensive statutory framework for personal data protection and directly governs AI systems that rely on large-scale personal data processing. It mandates lawful and purpose-limited processing, informed consent, data minimisation, and safeguards against misuse — all principles that apply squarely to AI-related harms.

While the DPDP Act does not define “AI incident” as a standalone legal category, its provisions on personal data breaches, fiduciary duties, and the powers of the Data Protection Board of India (DPBI) are directly relevant. The DPBI has authority to inquire into data breaches caused or facilitated by AI systems and to impose significant financial penalties on data fiduciaries that fail to comply with the Act.

Digital Personal Data Protection Rules, 2025, notified by the Ministry of Electronics and Information Technology (MeitY), operationalise the Act by specifying breach reporting obligations, cross‑border data transfer procedures, and the functioning of the DPBI — mechanisms that apply to AI‑driven data harms, including those arising from automated decision‑making systems. With the Rules’ publication, the DPDP Act is scheduled to become fully applicable to entities and government departments from May 2027, creating a binding enforcement environment for AI‑related data incidents.

India AI Governance Guidelines, 2025

The India AI Governance Guidelines 2025, published by MeitY, establish a “whole‑of‑government” framework for AI oversight built around horizontal principles and sectoral regulation rather than a single omnibus AI statute. While the Guidelines primarily address risk classification, accountability, and institutional architecture rather than incident reporting as a discrete obligation, they are directly relevant to how AI‑related harms are identified, mitigated, and managed in practice.

The Guidelines situate AI incident governance within the existing legal architecture — including the Information Technology Act 2000, the Bharatiya Nyaya Sanhita 2023, the DPDP Act 2023, the Consumer Protection Act 2019, and the Telecommunications Act 2023 — all of which contain provisions that can be invoked in response to AI-related harms such as deepfakes, unfair trade practices, or cybersecurity breaches. The Telecommunications Act in particular includes provisions on cybersecurity, critical infrastructure, and incident reporting that extend to AI systems used in telecommunications networks and services.

The Guidelines also introduce an AI Safety Institute (AISI) as a dedicated technical arm responsible for safety testing, risk evaluation, audits of high‑risk AI systems, and research on safety methods — functions that feed directly into incident prevention and post‑incident analysis.

AI Incidents in Official Government Reports

NITI Aayog

NITI Aayog has significantly shaped India’s AI policy trajectory beginning with its 2018 “National Strategy for Artificial Intelligence (#AIforAll),” which framed AI as an instrument for social empowerment and inclusive growth while acknowledging associated risks. Subsequent NITI Aayog work engaged more directly with high‑risk AI systems, impact assessments, and the need for guardrails, thereby laying the conceptual groundwork for later, more formal incident governance mechanisms.

The 2025 India AI Governance Guidelines can be read as the maturation of this trajectory: a shift from NITI Aayog’s strategic vision toward a principle‑based, techno‑legal governance framework anchored in MeitY and supported by cross‑sector regulators. This represents a decisive step toward comprehensive AI accountability in India, even in the absence of a dedicated AI Act.

Indian Judicial Context

In the draft Regulations for Use of Artificial Intelligence (AI) in Courts, 2026, prepared under the aegis of the Artificial Intelligence Committee of the Supreme Court of India and released for public consultation in June 2026, Regulation 3(e) proposes the following definition: ‘AI Incident’ means any event in which the use, operation, failure, erroneous output or malfunction of an AI System directly or indirectly results in, or creates a substantial risk of harm to any person, or infringement of any right, or disruption of any Court process, or a breach of data security or data privacy.

FUNCTIONAL ASPECTS OF AI INCIDENTS

Core Functional Criteria for Identifying AI Incident

CSET’s AI Harm Taxonomy, developed for use in the AI Incident Database, proposes four necessary elements for classifying an event as an AI incident:

• An entity that experiences harm — such as an individual, group, community, organisation, or system.
• A harmful event or issue — a concrete negative outcome such as physical injury, financial loss, privacy violation, or rights infringement.
• An implicated AI system — a specific AI system whose behaviour is causally connected to the harm.
• A direct link to the behaviour of the AI system — the AI’s operation must be a substantive cause of the harm, not merely coincidental.

This four‑part test is important for distinguishing genuine AI incidents from unrelated failures or general operational issues in which AI plays only a peripheral role. The taxonomy further distinguishes between tangible harms (e.g., physical injury, property damage, financial loss) and intangible harms (e.g., human rights violations, privacy breaches, reputational damage), and between harms that have already materialised and those that may occur in the future (hazards).

India's Approach to AI Incident Governance

The DPDP Act, 2023 and the "Significant Data Fiduciary"

Under the DPDP Act, entities notified as Significant Data Fiduciaries (SDFs) face heightened obligations, including Data Protection Impact Assessments (DPIAs), periodic independent audits, appointment of a Data Protection Officer, and additional transparency duties. Crucially, an organisation does not become an SDF merely because it processes large volumes of personal data or deploys AI systems. Under Section 10 of the DPDP Act, the Central Government must notify the Data Fiduciary (or a class of Data Fiduciaries) as significant after considering the statutory factors (such as volume and sensitivity of data, risk to rights of data principals, impact on sovereignty/security/public order, and use of new technologies).

In policy analysis, this SDF designation can function as a risk proxy for AI‑incident potential: where the government determines that the scale, sensitivity, or strategic importance of data processing (including through automated or AI‑driven systems) warrants stricter safeguards, the resulting SDF obligations are more rigorous.This approach parallels (but is not legally equivalent to) risk‑tiering in other AI governance frameworks, such as the EU AI Act’s concept of “high‑risk” systems; the comparison is analytical, not an assertion of identical legal mechanisms or triggers.

The Consumer Protection Act 2019 provides a complementary layer, empowering the Central Consumer Protection Authority to sanction unfair trade practices, misleading advertisements, and defective services, which can encompass AI‑mediated harms such as deceptive AI‑generated advertising or unfair automated rankings.

The India AI Governance Guidelines (2025)

Rather than creating a single centralised AI incident regulator, the India AI Governance Guidelines adopt a federated, sector‑specific model. Sectoral regulators are expected to manage domain‑specific incident risks, while the AI Governance Group (AIGG) and a Technology and Policy Expert Committee (TPEC) provide cross‑sector coordination, guidance, and standard‑setting.

Several regulators have already issued AI‑relevant guidance. For example, frameworks emerging from the Reserve Bank of India and financial sector bodies emphasise explainability in credit and risk decisions, bias audits, and the need for human‑in‑the‑loop review for adverse decisions — safeguards that directly bear on preventing particular categories of AI incidents in finance and payments.

TYPES OF AI INCIDENTS

Taxonomic Classifications

Global failure repositories, such as the AI Incident Database (AIID), utilize a three-tiered system to index automated failures:

• AI Incident (Realized Harm): An event where an AI system is implicated as a "but-for" cause of actual harm to people, property, or environments.
• AI Issue (Latent Risk): A documented vulnerability, model drift, or training dataset representational gap that demonstrates a risk of harm but has yet to cause a verified real-world event.
• AI Incident Variant (Recurring Systemic Fault): An event that shares identical causative factors, produces similar harms, and involves the same model design as an existing AI incident. These are indexed under a shared namespace to identify recurring structural failure modes without artificially inflating raw incident statistics.

Sector-based types

AI incidents occur across sectors wherever AI systems are deployed in consequential roles. Based on incident taxonomies and repositories such as AIID and AIM, heavily affected sectors include:

• Information and communication — disinformation campaigns, content moderation failures, algorithmic amplification of harmful content, deepfakes, and synthetic media.
• Transportation — autonomous vehicle crashes, navigation system failures, perception errors in self‑driving or driver‑assistance systems.
• Law enforcement and public administration — facial recognition misidentification, wrongful arrests, algorithmic profiling, and unfair allocation of public services.
• Health and social work — diagnostic AI errors, harmful chatbot health advice, incorrect or unfair insurance claim denials.
• Retail and finance — discriminatory credit scoring, AI‑facilitated fraud, algorithmic market manipulation, and exploitative personalised pricing.

Function-based types

A systematic review of AI misuse and harm identifies nine primary functional domains of AI incidents:

1. Adversarial Threats — attacks that manipulate AI model outputs through crafted inputs (e.g., adversarial examples that fool image classifiers)
2. Privacy Violations — unauthorised data access, training data leakage, mass surveillance
3. Disinformation, Deception, and Propaganda — AI-generated fake news, deepfake audio and video, synthetic personas and influence operations
4. Bias and Discrimination — algorithmic perpetuation of racial, gender, socioeconomic, or other biases in high-stakes decisions
5. System Safety and Reliability Failures — malfunctions in safety-critical autonomous systems (vehicles, medical devices, infrastructure)
6. Operational Misuses — AI systems taking unwanted autonomous actions or failing to align with their intended goals
7. Violence and Extremism — AI enabling or facilitating violent content, radicalisation, or weapons development
8. Economic Harm — AI-facilitated financial fraud, market manipulation, exploitative automation
9. Child Harm — AI-generated child sexual abuse material, predatory chatbot interactions targeting minors

Legal-trigger-based types

From a regulatory perspective, AI incidents can also be classified by the legal trigger they activate:

• Serious incidents under the EU AI Act: Article 3(49) defines a serious incident as one that directly or indirectly results in death, serious impairment of health, significant disruption of critical infrastructure, or serious violation of fundamental rights.
• Data breach incidents under the DPDP Act: AI‑related processing that results in unauthorised access, disclosure, or destruction of personal data triggers breach reporting obligations to the DPBI and may attract penalties.
• Near‑misses / hazards under the OECD framework: events that could plausibly lead to an AI incident but have not yet caused harm, which the OECD encourages jurisdictions to track alongside actual incidents to enable proactive risk management.

NOTABLE AI INCIDENTS

The real-world manifestation of these incident types is illustrated by documented case studies across safety-critical and socio-economic domains:

Autonomous Systems and Physical Safety: The Uber ATG Tempe Crash

On March 18, 2018, in Tempe, Arizona, an autonomous test vehicle operated by Uber Technologies, Inc., struck and killed pedestrian Elaine Herzberg as she pushed a bicycle across an unlit road.

The National Transportation Safety Board (NTSB) investigation revealed that while the vehicle's sensors registered Herzberg $5.6$ seconds prior to impact, the perception system failed to correctly classify her. Because she was crossing mid-block without a crosswalk, the classification software toggled erratically between labeling her as an unknown object, a vehicle, and a bicycle.

Each time the classification model switched its label, the system reset the target's tracking history, failing to calculate her expected trajectory.

Furthermore, Uber engineers had programmed a $1$ second suppression of emergency braking maneuvers to prevent erratic vehicle reactions during public trials, relying entirely on a distracted in-vehicle operator who failed to intervene in time. The NTSB cited Uber's inadequate safety culture and Arizona’s insufficient regulatory oversight as major contributing factors.

Systemic Actuarial Bias in Criminal Justice: State v. Loomis

In the 2016 Wisconsin Supreme Court case State v. Loomis (881 N.W. 2d 749), criminal defendant Eric Loomis challenged the court's reliance on a proprietary risk-assessment tool, COMPAS, during sentencing. The algorithm used static criminal records and dynamic interview answers to generate scores predicting Loomis's risk of recidivism. Relying on these high-risk COMPAS scores, the sentencing judge ruled out probation and sentenced Loomis to six years of state imprisonment.

Loomis appealed, asserting that the trade-secret, proprietary nature of COMPAS hid its calculations and methodology from the court, the defense, and the defendant, violating his Fourteenth Amendment due process rights.

The Wisconsin Supreme Court denied Loomis’s motion for resentencing, holding that a circuit court’s consideration of a COMPAS risk assessment at sentencing does not violate due process, provided the court observes the limitations and cautions set out in the judgment and does not rely on the score as the determinative sentencing factor.

The court required that presentence investigation reports containing COMPAS scores must include written advisements drawing attention to:

1. the proprietary nature of COMPAS (which prevents disclosure of how the risk scores are calculated);
2. the tool’s limitations for individualised sentencing (scores are based on group data and cannot identify specific high‑risk individuals);
3. the absence of a cross‑validation study for a Wisconsin population;
4. evidence that studies have raised questions about whether COMPAS scores disproportionately classify minority offenders as having a higher risk of recidivism; and
5. the fact that COMPAS was developed primarily to assist the Department of Corrections in post‑sentencing determinations, not as a sentencing tool.

The judgment also stressed that the risk score may not be used to determine whether an offender is incarcerated or the severity of the sentence, and that judges must explain the other independent factors supporting the sentence imposed.

Algorithmic Discrimination in Employment: Amazon's Automated Hiring Tool

Between 2014 and 2017, Amazon attempted to build an internal, machine learning-based hiring tool to automate the screening and ranking of job applicants' resumes.

By 2015, developers discovered that the automated system was systematically discriminating against female candidates applying for software engineering and other technical positions. Because the algorithm was trained on historical resumes submitted to the company over the preceding decade (reflecting a male-dominated workforce), the system learned that male candidates were the preferred standard.

Although the system did not explicitly use a gender variable, it identified and utilized text-based proxies to penalize female applicants. The algorithm downgraded resumes containing the word "women's" (e.g., "women's debate captain") and lowered the scores of graduates from specific all-women's colleges, while favoring masculine-coded verbs like "executed" and "captured". Amazon abandoned and officially dismantled the project in early 2017 due to Title VII disparate impact exposure.

Facial Recognition Failures and Civil Liberties: Detroit Police Department Misidentifications

On January 9, 2020, Robert Williams, a Black man, was wrongfully arrested in his driveway in front of his wife and young children based on an erroneous facial recognition match. Following a 2018 shoplifting event at a Shinola retail store in Detroit, detectives obtained a blurry, low-quality surveillance image and sent it to the Michigan State Police to run a facial recognition search.

The software, developed by Rank One Computing, returned an incorrect match pointing to Williams' expired driver's license photo. Relying on this match as an investigative lead, the detective prepared a physical photo lineup presented to an off-site loss-prevention contractor who was not present during the theft.

Following the contractor's identification, the detective secured an arrest warrant by omitting the low quality of the probe image and the automated origin of the match from the warrant application.

Represented by the ACLU of Michigan, Williams filed a federal civil rights lawsuit under 42 U.S.C. § 1983. The litigation culminated in a historic settlement on June 28, 2024, representing the nation's strongest constraints on law enforcement use of FRT. Under the agreement, the DPD is prohibited from making arrests based solely on FRT results, must corroborate all matches with independent physical evidence, must conduct photo lineups double-blind, and must implement mandatory training on algorithmic bias.

During the same period, the DPD wrongfully arrested two other Black individuals under similar circumstances:

• Michael Oliver (2019): Wrongfully arrested for cellphone theft based on an FRT match, despite having prominent, visible tattoos on his arms that were completely absent from the actual perpetrator in the surveillance video.
• Porcha Woodruff (2023): Wrongfully arrested and detained for 11 hours on charges of carjacking and robbery based on an outdated FRT match, despite being eight months pregnant at the time of her arrest, whereas the actual suspect was not.

Predictive Healthcare Failures: Optum and Epic Systems

• The Optum Cost-Proxy Bias (2019): Ziad Obermeyer et al. analyzed a commercial risk-prediction algorithm developed by Optum, which was utilized by major healthcare organizations to coordinate care for over 70 million patients. The model was designed to generate "health risk scores" to identify sicker, high-risk patients for enrollment in intensive, specialized high-risk care management programs. The audit revealed that the algorithm systematically underestimated the health needs of sicker Black patients, prioritizing healthier white patients for enrollment. The root cause of this systemic bias was the selection of the target variable. To estimate clinical severity, the developers programmed the algorithm to predict future healthcare costs. However, systemic socio-economic inequalities, unequal healthcare access, and institutional biases meant that far less money was historically spent on the treatment of sicker Black patients compared to white patients with identical clinical diagnoses. At any given health risk score, Black patients were significantly sicker than white patients, but the algorithm failed to prioritize them because it misconstrued historical spending disparities as a reflection of healthier clinical status.

• The Epic Sepsis Model Validation Failure (2021): Developed by Epic Systems Corporation, the Epic Sepsis Model (ESM) was an inpatient predictive analytics tool deployed across hundreds of hospitals to analyze electronic health records in real-time and alert clinicians to the early onset of sepsis. While Epic Systems marketed the ESM with a claimed Area Under the Receiver Operating Characteristic (AUROC) curve of 0.76 to 0.83, an independent clinical audit of 27,307 hospitalizations by University of Michigan researchers measured an actual AUROC of only 0.63. The audit revealed that the model missed 67% of patients who went on to develop sepsis, while sending out alerts for nearly 20% of all hospitalized patients, creating a positive predictive value (PPV) of only 12% and inundating medical staff with severe alarm fatigue. The discrepancy in performance arose because the model was trained on billing codes (ICD-10) rather than real-time clinical definitions, and suffered from label leakage. The model’s training data included clinical indicators—such as the ordering of antibiotics—that occurred after physicians had already recognized and begun treating sepsis. Importantly, a separate validation study conducted at Prisma Health evaluated the ESM across 11,512 inpatient encounters. The Prisma study measured an AUROC of 83.4%, with 86.0% sensitivity, 80.8% specificity, and a PPV of 33.8%. This stark contrast demonstrates a critical systemic trajectory: predictive clinical algorithms exhibit highly variable and uneven performance when deployed across different hospital systems, working worst in environments serving sicker, more complex patient populations.

Financial System Disruption: Knight Capital Group Trading Loss

Most AI‑incident frameworks (including the AI Incident Database) treat an “AI incident” as an alleged harm or near‑harm where an AI system is implicated, typically understood as systems involving machine learning, statistical learning, or adaptive/autonomous decision‑making rather than purely rule‑based automation.

The Knight Capital malfunction arose from:

• a flawed software deployment (inconsistent update across eight servers);
• reactivation of legacy, dormant code (“Power Peg”) that had been repurposed via a feature flag; and
• a deterministic, rule‑based order‑routing algorithm that executed child orders in a loop because the order‑completion reporting logic had been broken.wikipedia+4

Crucially, post‑mortems and analyses describe SMARS/Power Peg as automated trading logic, not a system employing machine learning or adaptive AI. Sources explicitly contrast the 2012 era (execution algorithms and basic statistical arbitrage) with later “statistical learning” or AI‑driven trading, and note that the Knight system had no learning or adaptation and no contextual awareness—hallmarks of narrow AI.

On August 1, 2012, Knight Capital Americas LLC experienced a catastrophic operational failure in its automated high-frequency trading platform. Within a 45-minute window following the opening of the U.S. stock markets, the firm's automated order-routing system, SMARS, erroneously flooded the New York Stock Exchange (NYSE) with millions of unintended executions.

The incident was triggered by a flawed software deployment intended to support a new NYSE Retail Liquidity Program. Knight Capital's operations team deployed the new SMARS code to seven production servers; however, they failed to copy the update to the eighth server. This server continued to run a dormant, decade-old piece of legacy code that utilized a function called "Power Peg". Power Peg was originally designed to track child orders relative to execution limits, but its safety-check mechanism had been removed from the active code base years prior.

When the market opened, SMARS received 212 legitimate retail orders. The seven updated servers processed these correctly. However, the eighth server activated the Power Peg function. Lacking safety constraints, the system began an endless loop, repeatedly sending buy and sell instructions to the market.

Within 45 minutes, Server 8 executed over 4 million transactions across 154 stocks, trading 397 million shares. Knight Capital assumed an unintended net long position of 3.5 billion USD and a net short position of 3.15 billion USD. Liquidating these positions cost the firm over 460 million USD—an amount that exceeded its entire market capitalization and triggered a liquidity crisis, forcing the firm into a distressed acquisition by competitor Getco LLC.

Intellectual Property and Secondary Copyright: Getty Images v. Stability AI

In Getty Images (US) Inc v Stability AI Ltd [2025] EWHC 38 (Ch), visual content licensor Getty Images filed suit in the High Court of England and Wales against generative AI developer Stability AI. Getty asserted that Stability AI had unlawfully scraped millions of copyrighted images and associated metadata from its web repositories without authorization to train its "Stable Diffusion" text-to-image latent diffusion model.

Getty demonstrated that the model’s synthetic outputs frequently reproduced Getty’s proprietary images and occasionally generated distorted versions of Getty’s corporate watermarks, constituting trademark infringement and passing off.

As the litigation progressed, Getty accepted there was no evidence that the training and development of Stable Diffusion took place in the UK, forcing it to abandon its claims of primary copyright infringement and database right infringement under UK law. Instead, the case centered on a novel claim of secondary copyright infringement under sections 22 and 23 of the CDPA 1988. Getty argued that by importing, distributing, and making the trained Stable Diffusion model files available to users in the UK, Stability AI was distributing an "infringing copy" of an article containing Getty's copyrighted works.

In a landmark ruling on November 4, 2025, the High Court rejected Getty's secondary copyright claim. The court endorsed the prevailing technical view of neural networks: a trained AI model does not contain or store copies of the works on which it was trained. Instead, the model contains mathematical weights (parameters) that represent abstract visual concepts. Because the model does not contain recognizable, literal copies of the underlying images, the model itself cannot be legally classified as an "infringing copy" for the purposes of secondary infringement under the CDPA, even if the training process itself constituted primary infringement in another jurisdiction. Getty was subsequent granted permission to appeal on this narrow question of statutory construction.

Generative Output and Defamation: Walters v. OpenAI

The legal liability of generative AI developers for output inaccuracies was tested in Walters v. OpenAI, LLC (No. 23-A-04860-2), filed in the Superior Court of Gwinnett County, Georgia. Mark Walters, a prominent radio host, sued OpenAI for defamation after ChatGPT hallucinated a false narrative accusing him of financial crimes.

The incident occurred when a journalist researching a federal lawsuit prompted ChatGPT to summarize the legal complaint. ChatGPT generated a highly detailed, entirely fabricated summary asserting that Walters was a named defendant who had embezzled and defrauded the Second Amendment Foundation (SAF) of funds while serving as its chief financial officer—a position Walters had never held.

On May 19, 2025, the court granted summary judgment in favor of OpenAI, dismissing the defamation claim on three distinct legal grounds:

1. No Defamatory Meaning as a Matter of Law: Under Georgia law, a defamatory statement must be reasonably understood as describing actual facts or events. Given ChatGPT's prominent on-screen user disclaimers warning that the system could produce inaccurate outputs, a reasonable reader would not interpret the AI's response as a literal assertion of fact.
2. Absence of Negligence or Actual Malice: Because Walters was deemed a public figure, he was required to prove that OpenAI acted with "actual malice"—meaning it published the statement with direct knowledge of its falsity or with reckless disregard for the truth. The court found no evidence of such reckless disregard, noting that OpenAI had implemented reasonable training protocols to mitigate hallucinations and warned its user base of potential errors.
3. No Proven Damages: The hallucinated output was never published to the general public; it was seen only by the single investigating journalist, who immediately recognized the error and did not disseminate it. Walters admitted during depositions that he suffered no actual professional, personal, or financial harm from the interaction, thereby defeating his claim for compensatory or punitive damages.

Appearance in official databases

Indian judicial and regulatory databases

In India, there is currently no dedicated official database that systematically tags or categorises “AI agents” as a distinct legal or operational concept. Judicial databases maintained by the Supreme Court, High Courts, and the e‑Courts project, as well as regulatory repositories kept by sectoral regulators, typically use general terms such as “artificial intelligence,” “AI systems,” or “algorithms,” and focus on issues like data protection, discrimination, automation, or evidence handling. References to AI in consultation papers, regulatory circulars, or government reports are similarly framed at the level of AI technologies rather than agents as action‑taking systems. As a result, while AI‑related disputes or regulatory matters may be recorded, the specific agentic characteristics—such as autonomy, tool use, multi‑step execution, or interaction with other agents—are not captured in a structured way. This makes it difficult to trace how AI agents are implicated in judicial or administrative processes, or to extract agent‑specific patterns from existing data.

AI incident databases and repositories

AI agents are more explicitly visible in international AI incident databases and research repositories, although these are not formal judicial or regulatory databases. The AI Incident Database (AIID), maintained by the Partnership on AI and associated researchers, is one of the earliest and most widely cited repositories dedicated to documenting real‑world harms or near‑harms caused by AI systems. It collects incident reports submitted by users and organisations, which are then indexed and made discoverable as a public record of AI‑related failures. The AIID is often used by researchers, journalists, and policymakers to study patterns of AI harm, but it does not distinguish AI agents as a separate category; instead, agent‑related incidents are embedded within broader AI incident records.

Complementary repositories such as the AIAAIC Repository (AI, Algorithmic, and Automation Incidents and Controversies) compile incidents and controversies involving AI, algorithms, and automation, including cases where agentic or semi‑autonomous systems played a role. The MIT AI Incident Tracker further classifies more than 1,400 incidents from the AIID using causal and domain taxonomies, risk levels, and severity scales, and is explicitly designed to make incident data more analytically useful. Some India‑focused initiatives, such as the AI Incident Database – India, document state‑wise records of deepfakes, voice‑cloning scams, electoral fraud, financial scams, and surveillance risks, including incidents that may involve AI agents or agentic workflows, though again without a dedicated “agent” tag.

Policy proposals for India, such as the AI Incident Reporting Framework for India, envision a federated database for AI incidents supervised by an ombudsman, drawing on models like the AIID and AIAAIC, but these proposals remain at the conceptual stage and have not yet been implemented as official government databases. In the absence of a federally operated AI incident or agent database in India, most available data on AI‑related harms involving agents continue to rely on public sources, media reports, academic repositories, and civil‑society documentation rather than mandatory legal disclosure or validated official records.

Enterprise and technical agent databases

Outside the judicial and regulatory sphere, AI agents increasingly appear in enterprise and technical databases designed for internal governance and operational control. Industry reports such as the 2026 State of AI Agents highlight how large organisations are building agent strategies, modernising enterprise architecture, and developing internal agent inventories and observability systems to track agent deployments, permissions, and behaviours. Platforms such as Oracle AI Database Private Agent Factory and AgentDB illustrate how organisations are creating specialised databases and infrastructure to build, deploy, and govern private AI agents with controlled access to enterprise data, APIs, and workflows.

These enterprise‑level databases are not public, but they are significant because they show how AI agents are being recorded and managed in practice: as identifiable systems with identities, access rights, activity logs, and lifecycle status. Governance commentaries stress that such inventories and observability tools are essential for traceability, accountability, anomaly detection, and cost control in large‑scale agentic deployments. For Indian regulators and courts, these private databases represent a potential source of data that could, in future, be linked to official reporting or audit requirements if agentic AI becomes subject to more formal oversight.

Data creators and collation methods

At present, the main data creators for AI‑agent‑related information are a mix of academic and civil‑society initiatives (such as the AIID, AIAAIC, MIT AI Incident Tracker, and India‑focused incident trackers), international organisations (such as OECD and associated research groups), and private enterprises that maintain internal agent inventories and governance dashboards. These actors typically collate data through a combination of voluntary incident submissions, media monitoring, academic research, correspondence with developers, and internal logging systems. Methods range from structured submission forms and taxonomy‑based classification to automated scraping and manual curation.

In contrast, official Indian apex departments and agencies—such as the Ministry of Electronics and Information Technology, sectoral regulators, and the judiciary—do not yet maintain dedicated, publicly accessible databases that systematically record AI agents or agentic incidents. Where data on AI‑related matters are collected, they are usually embedded in broader datasets on technology use, cyber incidents, or regulatory enforcement, without agent‑specific fields or tags. As agentic AI deployments grow, there will likely be a need for these bodies to develop explicit schemas and reporting mechanisms that distinguish agents from other AI systems, and to adopt methods for collating and presenting judicial and regulatory data that reflect the operational role of agentic systems.

RESEARCH THAT ENGAGES WITH AI INCIDENTS

Research on AI incidents has grown rapidly in the last few years, moving from ad hoc case studies to the development of formal taxonomies, reporting frameworks, and analysis tools. Scholars and policy institutes now treat AI incident data as a critical empirical foundation for AI governance, akin to accident reports in aviation or sentinel events in healthcare.

Telecommunications law and AI incident reporting (India)

Indian scholarship is increasingly focused on the "regulatory gap" between existing telecommunications/cybersecurity mandates and the unique operational risks posed by AI. Current frameworks—such as the Telecommunications Act (2023) and CERT-In directives—are primarily designed for static infrastructure failures or data breaches, often failing to account for "probabilistic" failures like model drift, hallucinations, or biased optimization.

• Key Academic Proposals: Research by Agarwal and Nene (2025) emphasizes that India’s vast telecom market (over 1 billion subscribers) makes it a critical site for systemic AI risk. They advocate for integrating AI incident reporting into existing telecom licensing conditions, arguing that this leverages established enforcement channels rather than waiting for horizontal, economy-wide AI legislation.
• Sector-Specific Nuance: Scholars propose defining a "Telecommunications AI Incident" to capture harm beyond traditional security, such as:
  • Algorithmic Misrouting: AI-driven optimization failing to route emergency calls correctly.
  • Automated Fraud at Scale: AI systems unintentionally facilitating large-scale phishing or spam campaigns.
• Coordination Needs: There is a strong call for a "nodal agency" approach where the Department of Telecommunications (DoT) coordinates with the Data Protection Board of India (DPBI) and the prospective AI Safety Institute, ensuring that telecom-specific technical data informs broader national safety strategies.

CSET’s “Adding Structure to AI Harm” (CSET, 2023) and“An Argument for Hybrid AI Incident Reporting” (2024)

One of the most influential conceptual contributions is CSET’s report “Adding Structure to AI Harm” (2023), which introduces the CSET AI Harm Framework. The framework offers a standardised conceptual structure for defining, tracking, classifying, and understanding harms caused by AI systems once deployed. It identifies key elements required for the identification of AI harm — including the harmed entity, the harmful event, the implicated AI system, and the causal link between system behaviour and harm — and specifies their basic relational structure.

Importantly, the framework is designed to improve comparability between different harm‑monitoring efforts without imposing a single normative theory of harm on all users. It is intentionally modular, so regulators, researchers, and practitioners can adapt it to their own analytical needs while still mapping back to a common core structure. The report includes an applied example, showing how a jurisdiction or organisation can customise categories (e.g., harm types, severity scales) while preserving interoperability with other incident‑tracking efforts.

The Center for Security and Emerging Technology’s 2024 issue brief “An Argument for Hybrid AI Incident Reporting” is one of the foundational policy analyses on AI incident reporting design. It starts from the observation that AI incidents have been increasing in frequency, but there is still no comprehensive, government‑backed system to monitor, document, and learn from them.

The report makes several key contributions:

• Hybrid reporting model: It proposes a hybrid framework that combines:
  • Mandatory reporting by regulated entities for certain categories of serious incidents.
  • Voluntary reporting by organisations and professionals for incidents outside the mandatory scope.
  • Citizen reporting by affected individuals, journalists, civil‑society organisations, and watchdogs.
• Federated architecture: Rather than a single centralised database, CSET recommends a federated approach, where a central policy framework sets minimum standards and taxonomies, while sectoral agencies adapt reporting formats and thresholds to their own domain (e.g., healthcare vs. finance vs. critical infrastructure).
• Independent external recipient: The brief argues that incidents should be reported to an independent external entity (such as a government agency, professional body, or oversight board), not only managed internally by companies, to promote transparency and public trust.
• Standardised classification: It calls for a standardised, authoritative classification system for incident types, harms, and causes, so that data from different sectors and jurisdictions can be compared and aggregated.
• Independent investigation body: The authors recommend creating an independent AI incident investigation agency, modelled on aviation or nuclear incident boards, tasked with analysing significant incidents and issuing evidence‑based safety recommendations.

These reports draw explicitly on and complement the CSET AI Harm Framework, and they are cited as “must‑read” foundational research in CSET’s own retrospective of its AI‑governance work. Together, they form a coherent body of policy scholarship that links conceptual definitions of AI harm to concrete reporting templates and institutional design choices.

“AI Incidents: Key Components for a Mandatory Reporting Regime” (CSET, 2025)

In 2025, CSET followed up with “AI Incidents: Key Components for a Mandatory Reporting Regime,” which operationalises the earlier hybrid model by specifying what exactly should be reported in a mandatory regime. This report focuses on the content of incident reports rather than the institutional design.

It identifies a set of standardised key components that any AI incident report should capture, including at least:

• Type of incident: classification of the incident (e.g., bias/discrimination, safety failure, privacy breach, fraud, disinformation).
• Nature and severity of harm: what harms occurred (or nearly occurred) and how severe they were, potentially using a standard severity scale.
• Technical information: details of the AI system (model type and version, training or fine‑tuning data sources where possible, deployment context, interfaces).
• Affected entities: who was harmed or at risk — individuals, communities, organisations, or critical infrastructure — and in which jurisdictions.
• Context and circumstances: organisational setting, socio‑technical context, and any relevant environmental or operational factors.
• Contributing factors: technical (e.g., design flaws, adversarial attacks) and organisational (e.g., inadequate oversight, poor training, misaligned incentives).
• Post‑incident response: remedial measures taken, notifications made, system changes, and follow‑up monitoring.

The report argues that if such components are widely adopted by governments, regulators, professional bodies, developers, and researchers, they would:

• Facilitate consistent data collection and reduce blind spots in incident records.
• Enable better tracking, monitoring, and research on AI harms and risk patterns.
• Support information sharing and cross‑jurisdiction learning.
• Provide a foundation for agile reporting that can adapt as AI technology evolves.

Importantly, the authors again recommend that states establish an independent investigative agency to complement reporting templates by uncovering hidden causes and systemic issues that initial reports might miss. These components are also proposed as a template for voluntary and citizen reporting systems, not just mandatory regimes.

Standardised schemas and taxonomies for AI incident databases

A growing body of technical research focuses on standardising the data structures used in AI incident databases so that incident information is captured in a consistent, analysable way.

“Standardised schema and taxonomy for AI incident databases in critical digital infrastructure” (Agarwal & Nene, 2025)

Agarwal and Nene’s 2025 arXiv preprint, “Standardised schema and taxonomy for AI incident databases in critical digital infrastructure”, proposes a unified schema and structured taxonomy for recording AI incidents, with a particular focus on critical digital infrastructure. The authors argue that existing databases suffer from heterogeneous fields, missing information, and incompatible classification schemes, which impede cross‑system analysis and trend detection.

Their proposed schema introduces standard fields for: incident description, system characteristics, causes, harms, severity, affected sectors, and remediation, along with optional fields for contextual details. The taxonomy distinguishes different types of harm (e.g., safety, fairness, security, privacy), causes (e.g., design flaw, data issue, adversarial attack), and severity levels, enabling structured queries and comparative analysis.

The work explicitly aims to support critical digital infrastructure — such as telecommunications, energy, and financial networks — where AI incidents may have cascading societal impacts and where systematic, high‑quality incident data are particularly important. It demonstrates how a carefully designed schema can make incident databases more useful both for day‑to‑day operators and for regulators and researchers conducting systemic risk analyses.

• Standardised schema for incident databases: Recent work proposes a standard schema and taxonomy for AI incident databases, addressing current challenges such as inconsistent fields, incomplete technical detail, and difficulties in cross‑database comparison. Such a schema typically includes structured fields on context, causes, harms, and affected groups, allowing richer quantitative and qualitative analysis.
• AI Incident Database and academic analyses: Studies of the AI Incident Database (AIID) show that documentation practices are uneven: some entries are rich in technical and contextual information, while many are sparse or media‑driven, leading to visibility and data‑quality biases. Researchers have highlighted the need for better standards, editor guidance, and integration with formal reporting frameworks.
• MIT AI Incident Tracker and AI risk taxonomy: The MIT AI Risk Repository’s incident tracker classifies more than 1,400 incidents by sector, harm type, cause, severity, and other dimensions, drawing on both AIID and additional sources. This work shows how structured classification can reveal patterns (e.g., surges in generative‑AI‑driven fraud or disinformation) that would be hard to see in unstructured narratives alone.
• Scalable classification with large language models: Experimental work has used large language models to summarise, classify, and rate AI incidents across harm categories and domains, combining the MIT risk taxonomy with CSET’s harm ratings. These projects demonstrate how AI can be used to amplify expert capacity in incident analysis without replacing human judgement, especially when dealing with hundreds or thousands of cases.

Taken together, this research stream aims to turn scattered incident stories into systematic, analysable data, which in turn can inform safety benchmarks, regulatory thresholds, and sector‑specific standards.

Indian standardisation efforts: TEC 57090 (2025)

In India, the Telecommunication Engineering Centre (TEC) has prepared a draft standard “TEC 57090:2025, Standard for the Schema and Taxonomy of an AI Incident Database in Telecommunications and Critical Digital Infrastructure.” This draft mirrors the broader research trend by defining a standardised schema for AI incident databases in telecom and critical digital infrastructure, and by establishing a structured taxonomy for classifying incidents.

The document specifies key required data fields for incident reporting (e.g., system description, incident type, impact, affected services) as well as optional fields to capture richer contextual information. It frames the database as a tool to “systematically capture AI incidents, ensuring thorough documentation for analysis and improvement of AI systems,” explicitly linking incident documentation to continuous safety improvement. TEC 57090 thus constitutes a concrete example of how academic and policy research on AI incident schemas is being translated into sector‑specific technical standards in India

Biases and methodological challenges in incident data

A further line of scholarship examines the epistemic limitations of current AI incident datasets:

• Visibility and media bias: Incidents that are newsworthy, involve celebrities, or occur in high‑income, English‑speaking countries are far more likely to be documented, leading to over‑representation of some harms and under‑representation of others.
• Detection and disclosure bias: Organisations may not detect all incidents, or may have weak incentives to disclose them publicly; many “quiet fixes” never enter public databases.
• Geographic and language gaps: Incidents in the Global South or in non‑English languages often go unrecorded, skewing our understanding of how AI harms are distributed globally.

Researchers argue that these biases mean raw incident counts cannot be interpreted naively as a direct measure of overall risk; instead, policymakers must read them alongside qualitative knowledge, targeted surveys, and context‑specific investigations. This has led to calls for more inclusive reporting channels, multilingual monitoring, and stronger incentives for organisations to disclose incidents beyond minimal legal requirements.

AI INCIDENTS IN INTERNATIONAL INSTRUMENTS

OECD AI Incidents Framework

The Organisation for Economic Co-operation and Development (OECD) has led the most systematic international effort to define and monitor AI incidents. Following over 18 months of deliberations within the OECD.AI Expert Group on AI Incidents and the Working Party on AI Governance (AIGO), the OECD released its foundational definitions of AI incidents and related terms. The core definition is:

"An AI incident is an event, circumstance or series of events where the development, use or malfunction of one or more AI systems directly or indirectly leads to: (a) injury or harm to the health of a person or groups of people; (b) disruption of the management and operation of critical infrastructure; (c) violations of human rights or a breach of obligations under the applicable law intended to protect fundamental, labour and intellectual property rights; (d) harm to property, communities or the environment."

These definitions are explicitly designed to promote international interoperability, allowing different jurisdictions to adopt their own reporting thresholds and scopes while still speaking a common conceptual language. The OECD’s public page on “AI risks and incidents” emphasises that trustworthy AI requires interoperable, risk‑based governance, underpinned by a rigorous understanding of incidents and hazards and the patterns they reveal.

Operationally, the OECD has created the AI Incidents and Hazards Monitor (AIM) as an empirical complement to its definitional work. AIM tracks incidents and hazards reported in the media, applying a structured set of criteria that mirror the OECD definitions and using a standardised template to capture incident characteristics (harm type, sector, geography, severity, etc.). The 2025 report “Towards a Common Reporting Framework for AI Incidents” builds on this infrastructure, proposing a global benchmark for incident reporting fields so that incident data from governments, firms, and civil society can be aggregated and compared across sectors and borders.

European Union: EU AI Act

The EU AI Act (Regulation (EU) 2024/1689) contains one of the most legally binding AI incident reporting regimes currently in force globally. Article 3(49) defines a "serious incident" as an incident or malfunction of an AI system that directly or indirectly results in the death of a person; serious harm to health; significant disruption of critical infrastructure; or serious violation of fundamental rights.

Article 73 of the EU AI Act (Regulation (EU) 2024/1689) establishes detailed reporting obligations for providers of high‑risk AI systems, which will become applicable from 2 August 2026, when the high‑risk rules under Title III of the Act enter into application for systems classified under Annex III. From that date, providers of high‑risk AI systems placed on the EU market will be required to report any serious incident to the relevant national market‑surveillance authorities “immediately” after establishing a causal link or a reasonable likelihood of such a link, and in any event not later than 15 days after becoming aware of the incident. For widespread infringements or critical‑infrastructure incidents, the report must be submitted within two days, while in the event of a death, reporting must occur immediately and not later than 10 days after the provider or deployer becomes aware of the incident. After reporting, providers will be required to conduct investigations into the incident and the high‑risk AI system concerned, including a risk assessment and corrective actions, and to cooperate with competent authorities without altering the system in ways that could compromise the investigation. The European Commission has issued draft guidance and a reporting template to help providers interpret Article 73 and align AI incident reporting with other sectoral regimes, with final versions expected to apply from 2 August 2026 alongside the Article 73 obligations themselves.

Council of Europe: Framework Convention on AI, Human Rights, Democracy and the Rule of Law

The Council of Europe’s Framework Convention on Artificial Intelligence, Human Rights, Democracy and the Rule of Law (opened for signature in 2024) does not define “AI incidents” as a term of art, but it embeds incident‑relevant obligations into a binding human‑rights and rule‑of‑law framework. The Convention adopts a risk‑based approach and requires states to ensure that activities across the AI lifecycle are compatible with human rights, democracy, and the rule of law, with particular emphasis on accountability, transparency, and effective remedies for harms caused by AI systems.

Key provisions relevant to incidents include:

• Requirements for risk management and oversight mechanisms capable of identifying, assessing, and mitigating risks from AI systems, including those that materialise as concrete harms.
• Obligations on documentation, transparency, and access to remedies for individuals whose rights are adversely affected by AI systems.
• The expectation that public authorities—and private actors acting on their behalf—maintain effective procedures to address AI‑related harms and failures, which in practice will require structured identification and treatment of AI incidents.

While the Convention is principles‑based, it provides an international legal backbone for national incident‑reporting and redress regimes, aligning them with human‑rights obligations rather than treating AI incidents purely as technical or consumer‑protection issues.

United States

In the United States, the White House AI Action Plan directs the National Institute of Standards and Technology's (NIST) Center for AI Standards and Innovation to create federal AI incident response frameworks. Analysis identifies three categories for AI incidents requiring distinct response protocols: Operational Safety and Reliability (misaligned systems, unwanted autonomous actions); Security and Integrity (model theft, data breaches, prompt injection); and Deployment Misalignment (chatbots causing self-harm, biased automated decisions).

• The NIST Artificial Intelligence Risk Management Framework (AI RMF 1.0) treats incident management and response as core elements of AI risk governance, recommending that organisations establish mechanisms to monitor deployed AI systems, detect harmful events, and respond through investigation, mitigation, and continuous improvement. While it uses the broader language of “negative impacts” rather than a strictly defined “AI incident,” it explicitly links AI risk management to existing cybersecurity incident‑response practices.
• NIST’s general incident‑response guidance (SP 800‑61 Rev. 3), updated in 2025, provides a structured process for incident handling (preparation, detection and analysis, containment, eradication, recovery, and post‑incident activity) and is now being profiled for AI use through a Cybersecurity Framework Profile for Artificial Intelligence that discusses integrating AI‑specific events and failure modes into incident‑response plans.

The earlier Executive Order on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence (October 2023) directed federal agencies to take 150 distinct actions to improve AI safety and security, drawing in part on the NIST AI Risk Management Framework issued in January 2023.

Other Countries and International Guidance

Singapore

Singapore’s AI governance documents embed incident management and reporting as a core governance function, especially for higher‑risk or generative AI systems.

• The draft Model AI Governance Framework for Generative AI explicitly calls on organisations to implement an incident management system that covers timely detection, notification, remediation, and continuous improvement in response to AI‑related failures. The framework distinguishes vulnerability reporting (incentivising pre‑emptive disclosure of weaknesses) from incident reporting (processes for notifying relevant stakeholders and authorities when harms or significant failures occur), and stresses that reporting should be proportionate to the scale and impact of the incident.
• Singapore’s Model AI Governance Framework for Agentic AI similarly highlights the need for robust monitoring and incident‑response processes for agentic systems that can act with higher degrees of autonomy, including escalation paths and clear accountability when harmful outcomes occur.

Although these frameworks are non‑binding “soft law”, they function as widely cited benchmarks in the Asia‑Pacific region and effectively set expectations that serious AI‑related harms be treated and reported as incidents, with structured internal and, where appropriate, external notification processes.

Peru and emerging Latin‑American approaches

Peru is one of the first Latin‑American states to refer expressly to AI‑related incidents in national legislation.

• Law No. 31814, the Law Promoting the Use of Artificial Intelligence for the Economic and Social Development of the Country, establishes a comprehensive framework for ethical and responsible AI use. It includes provisions on protocols for reporting and addressing AI‑related incidents, signalling a legislative intent to treat AI‑caused harms as a distinct regulatory concern rather than leaving them entirely to general civil or criminal law. The law also stresses accountability for developers and users of AI systems where AI use leads to violations of fundamental rights.

A separate body of Peruvian scholarship on digital extortion using AI notes that, while Law No. 32314 introduces an aggravated circumstance for offences committed through AI, there remain gaps in procedures for swift removal of harmful synthetic content, evidence preservation, and technical standards for analysing AI‑generated media, all of which are relevant to properly investigating and documenting AI incidents. This literature situates Peru’s efforts in comparative perspective with the EU AI Act, the UK’s Ofcom duties, and Australia’s eSafety regime.

De facto global practice

Even where “AI incident” is not a defined legal term, several jurisdictions and guidance documents converge on a common pattern:

• AI‑related harms are increasingly expected to be handled through formal incident‑management processes, often by extending existing cybersecurity or operational incident frameworks to explicitly include AI system failures and misuse.
• Soft‑law frameworks (such as Singapore’s model frameworks) encourage organisations to build internal incident detection, triage, and reporting pipelines for AI systems, with escalation to regulators where harms cross specified thresholds.
• Newer AI‑specific laws, such as those in Peru, are beginning to speak of protocols for AI‑related incidents and accountability for AI‑caused rights violations, foreshadowing more explicit statutory incident‑reporting regimes in Latin America and beyond.

These developments complement the OECD definitions and the EU AI Act by showing how national frameworks outside Europe are starting to internalise the idea that AI failures and harms should be treated as a distinct class of reportable incidents, even where terminology and legal form vary.

Other guidance

Beyond the OECD, EU, and Council of Europe instruments, several other international or quasi‑international documents engage with AI incidents more implicitly:

• The OECD’s “Already Happening — The AI Incident Record” project illustrates how a curated, representative record of real‑world AI harms can be used to communicate emerging failure modes to policymakers and practitioners, built explicitly on OECD’s incident and hazard definitions.
• Regional and international initiatives (for example, EU–OECD cooperation around AIM and the EU AI Act’s implementation) stress interoperability between national incident reporting systems and the OECD’s definitions, to facilitate cross‑border learning and joint responses to transnational AI incidents.

Overall, these instruments collectively signal a shift from viewing AI failures as ad hoc “accidents” to treating AI incidents as a regulated category of safety and rights‑relevant events, subject to common definitions, reporting duties, and remediation obligations across jurisdictions.

WAY AHEAD: ADDRESSING CHALLENGES IN AI INCIDENT REPORTING

The next step for AI incident governance is not just to define incidents, but to build systems that reliably detect, report, classify, and learn from them. At present, the biggest gap is the absence of mandatory, standardised, and interoperable reporting regimes in most jurisdictions, which leaves many harms undocumented and many near-misses invisible.

Establishing mandatory reporting frameworks

A strong AI incident regime should combine mandatory reporting for serious incidents, voluntary reporting for lower-severity incidents and near-misses, and citizen reporting for harms experienced by affected individuals and communities. Mandatory reporting is essential where the harm is severe or systemic, while voluntary channels help capture early warning signs before harms escalate. Citizen reporting is equally important because many victims never come to the attention of regulators or firms.

The framework should also be federated rather than entirely centralised. A central authority can set minimum definitions, thresholds, and formats, while sector-specific regulators can tailor implementation to the risks in their domain. This is especially important in sectors such as finance, telecom, health, transport, and public administration, where AI incidents may look very different but still require a common reporting backbone.

Standardising taxonomies and monitoring systems

Incident reporting becomes useful only when the data are comparable. That requires agreed taxonomies for incident type, harm severity, affected sector, system function, and contributing cause. Without such standardisation, databases become fragmented collections of isolated stories rather than reliable sources of policy evidence.

Future systems should therefore adopt a common reporting schema that includes at least: the AI system involved, the context of deployment, the nature of the harm, the affected entity, the likely cause, and the response taken. Real-time or near-real-time monitoring tools should also be developed so that regulators can identify patterns early, rather than reacting only after major harm has already occurred.

Interoperability and cross-border cooperation

AI systems operate across borders, so AI incidents often do too. A model developed in one country may be deployed in another, trained on data from several jurisdictions, and produce harm in a third. For that reason, incident reporting systems must be interoperable across legal systems, sectors, and languages.

International cooperation is also needed so that countries can share lessons, compare patterns, and coordinate responses to transnational harms. This is particularly important for incidents involving deepfakes, cyber-enabled misuse, election interference, financial fraud, and safety-critical systems, all of which can have effects far beyond a single national border. A shared reporting vocabulary is therefore not a technical luxury; it is a governance necessity.

Balancing transparency, privacy, and innovation

A workable reporting regime must also balance transparency with confidentiality and privacy. Incident reports may contain personal data, trade secrets, security-sensitive information, or details that could be misused if disclosed too broadly. The challenge is to design reporting rules that preserve enough information for accountability and learning without exposing victims or weakening system security.

At the same time, regulation should not discourage innovation. The aim is not to punish every failure, but to create a learning environment in which developers, deployers, regulators, and affected users can improve system safety. In this sense, AI incident reporting should function as a safety infrastructure, not merely as a compliance burden.

Building institutional capacity

Even the best legal framework will fail without institutions capable of using it. Regulators need technical expertise, data-analysis capacity, and procedures for triage and investigation. Independent bodies may also be needed to examine major incidents, issue recommendations, and publish lessons learned in a form that is useful for both policymakers and practitioners.

For India, this means aligning incident governance with the DPDP Act, sectoral regulation, and the emerging AI governance framework, while also strengthening institutional capacity in bodies such as sector regulators, technical standards organisations, and any future AI safety institution. The broader goal should be to turn incident reporting into a cycle of learning, prevention, and accountability rather than a purely reactive compliance exercise.

AI INCIDENTS ARE ALSO KNOWN AS

• AI harms
• AI safety failures
• AI-related adverse events
• AI system failures or malfunctions
• Serious AI incidents (in the EU AI Act context, denoting the most severe category)

REFERENCES

Contributors & review status

Curated / edited bynot yet recorded

Peer reviewpending

Project reviewpending

Expert reviewpending

Page created 2026-08-21  ·  Last updated 2026-08-23