Jump to content

Red Teaming

From Justice Definitions

What is Red Teaming?

Red teaming is an aggressive, proactive testing methodology in which authorized ethical hackers and domain experts replicate real-world adversarial attacks against an organization's systems, defenses, or AI models using predefined rules of engagement. The primary goal of red teaming is to detect vulnerabilities, attack vectors, and failure modes before they are exploited by malevolent actors. It is a comprehensive, goal-oriented methodology that goes beyond detecting software faults or code errors.

It is critical to distinguish between the objectives of the authorized red team and those of the simulated attacker. The red team does not intend to steal data, obtain unauthorized access to secured infrastructure, or spread false information. Rather, it is officially allowed by the organization to act out enemy actions in a safe controlled place following specific rules. The main goal is to find spots so they can be fixed before a real enemy uses them.[1] For example a red team testing a Large Language Model (LLM) used in a court setting would carefully change questions on purpose to get answers that're unfair, bad or not right. This shows problems with how the model's set up and the rules around it. The team does not want to really hurt anyone[2].

As the Indian regulatory framework evolves, particularly through the MeitY India AI Governance Guidelines (November 2025) and the Information Technology Act 2000, including the 2026 synthetic content amendments[3], red teaming is increasingly recognized as a critical operational mechanism for demonstrating intermediary due diligence and meeting data fiduciary obligations. It may provide the empirical evidence required for Data Protection Impact Assessments (DPIAs) under the Digital Personal Data Protection (DPDP) Act 2023, allowing significant data fiduciaries to document how they identify and mitigate risks such as data poisoning, model inversion, and unauthorized personal data leakage[4].

The thing is, red teaming does not mean you are automatically protected by the law. The Information Technology Act of 2000 has a section called Section 79 that says you can be protected. Only if you follow the rules. These rules are in the Information Technology Act. The rules made in 2021 like being careful and not helping people do bad things. You also have to do what the law says when someone complains about something online[5]. Red teaming is one thing that can show you tried to be careful but it is not enough, on its own to say you deserve this protection. Red teaming is one piece of the puzzle it does not mean you get to say you are protected under Section 79 of the Information Technology Act of 2000.

Red teaming is a way to make sure that algorithms are fair. When an artificial intelligence system can produce results that're not fair to people and go against their basic rights, like the right to be treated equally and the right to live freely, red teaming helps find and fix these problems before the system is used. The process of teaming has many steps that are done over and over[6]. There are five parts to this process:

  • Phase 1 :It is about defining what the red teaming exercise is trying to do. The red team and the organization that asked for the exercise decide on the rules what kind of threats they are looking for, which artificial intelligence systems they want to test, like a system that summarizes court decisions and what they want to achieve from the exercise.
  • Phase 2: It is where the team thinks about the things that could happen to the system and how someone might try to attack it. They look at all the ways things could go wrong, like if someone puts data in or tries to trick the system or uses language that the system is not used to. The team checks for things, like data poisoning, model inversion, injection and sociolinguistic bias triggers. This helps the organisation see where it might be vulnerable before anything bad actually happens.
  • Phase 3: It is where the team really tests the system to see how strong it is. They use techniques to try to hack into the system like a hacker would and they also use automated tools to try to find weaknesses. They check to see if the systems interfaces are strong and they try to figure out how the system makes its decisions. They also test the systems defenses to see if they can withstand inputs.
  • Phase 4: It is where the team looks at what they found out in Phase 3 and tries to understand what it means. They make a list of all the weaknesses they found. They rank them by how serious they are. Then they use this information to figure out what they need to do to fix the problems. They consider both how well the system is working and what the law says they have to do. This helps them decide which problems to fix first.
  • Phase 5: It is about fixing problems and making sure everything is okay again. The people who make the software do what was suggested in Phase 4. They fix the security issues. Make sure everything is working together properly. The red team might do some testing to make sure the fixes are working and that no new security problems were created.

The red team has a lot of tools to help them test the software. They use algorithms, like the Fast Gradient Sign Method to test how images are classified. They also use automated fuzzing engines to test the softwares application programming interface. The red team uses frameworks to test Large Language Models. They even use safe environments to test the software without hurting the main system. All these tools help the red team do their job and make sure the software is secure. The red team uses these tools to support their work and make sure the software is safe[7].

OFFICIAL DEFINITIONS OF RED TEAMING

Statutory Definitions

As of July 2026 Indian legislation does not have a definition of AI red teaming. The AI red teaming in the Information Technology Act 2000 the Digital Personal Data Protection Act 2023 or the Information Technology Rules 2021[8]. Indian law does not define what AI red teaming is it does not say what AI red teaming should do. It does not have rules for AI red teaming. The thing is Indian law has not made AI red teaming a law yet so AI red teaming works, within a system that has AI governance policy, cybersecurity guidance and data-protection standards. These do not have the power of a law.

Policy Descriptions in Official Government Documents

The IndiaAI Governance Guidelines, published by the Ministry of Electronics and Information Technology, are the closest official Indian policy description of red teaming.[9] According to the Guidelines, red teaming is a simulation exercise that takes place under real-world situations and involves simulated hostile attempts to compromise business processes in order to offer a full assessment of an information system and organization's security capability. [10]While this concept is rooted in cybersecurity tradition and terminology, it is the first unambiguous Indian government-backed policy description of red teaming.

It must be clearly noted that this is a policy description contained within a non-binding governance document. It neither establishes that the Government of India plans to legislate a statutory definition of red teaming nor creates a legally enforceable requirement to perform red teaming. The India AI Governance Guidelines serve as a voluntary accountability and transparency framework. The Guidelines encourage enterprises to disclose reports on red-teaming results, impact assessments, and risk-mitigation procedures for public review, but there is no required duty to do so. The Indian AI governance approach, as expressed in these Guidelines, is distinguished by voluntary disclosure and transparency procedures rather than strict regulatory mandates.

Legal Provisions Conceptually Relating to Red Teaming

Sectoral Frameworks: RBI FREE-AI Recommendations

The Reserve Bank of India's Framework for Responsible and Ethical Enablement of Artificial Intelligence in the Financial Sector (FREE-AI), released as a committee study, is a key sectoral policy reference.[11] The FREE-AI research suggests appropriate, periodic, and trigger-based AI red teaming for higher-risk financial AI systems as part of a larger AI assurance, audit, incident reporting, and model-risk governance framework. [12]However, until the RBI converts these committee recommendations into binding supervisory directions or circulars under the Banking Regulation Act 1949 or the Reserve Bank of India Act 1934, FREE-AI should be viewed as a major regulatory-policy recommendation rather than a statutory or regulatory mandate.

State-Level and Judicial Policy Requirements

The Telangana AI Roadmap from September 2024 asks the states AI Advisory Council to create review frameworks for AI systems used in government services. [13]These frameworks may involve rules for testing. However the Roadmap does not use the words " teaming." The Tamil Nadu Safe and Ethical AI Policy from 2020 says AI systems used in services must be tested before they are put into use.. The policy does not talk about a red teaming method[14]. The Kerala High Court AI Policy from July 2025 says judicial AI tools must be tested thoroughly for accuracy for bias and, for strength before they are used in court. The way this is written matches the goals of teaming but does not require the practice directly[15].

These three frameworks reflect a developing regional consensus on the value of pre-deployment validation for public-sector AI systems. The trajectory progresses from Tamil Nadu's 2020 demand for comprehensive pre-deployment testing without a defined methodology to Telangana's 2024 directive to build formal assessment frameworks, and finally to the Kerala High Court's 2025 requirement for extensive audits that address bias, accuracy, and robustness. Neither of these mechanisms, however, establishes a formal requirement to engage in red teaming.

International Comparative Definitions

European Union Artificial Intelligence Act 2024

The term " teaming" is not really explained in the European Union Artificial Intelligence Act.. It does have something similar that works in a similar way. This is for intelligence models that could cause big problems.[16] Article 55(1)(a) says that the people who make these models have to check them using method[17]s. These methods are what most people consider the way to do things right now. They have to test these models to see if they can be tricked and to reduce the risks.

This shows the difference between what the European Union and India are doing. The European Union says that people have to do this testing for intelligence models that could cause big problems[18]. India is handling " teaming" in a different way. They have guidelines that're not laws they ask companies to tell them about problems voluntarily and they make recommendations, for different industries. The European Union and India are taking approaches to "red teaming" and artificial intelligence.

OECD AI Principles and UNESCO Recommendation on the Ethics of AI

The OECD Recommendation on Artificial Intelligence (revised 2023) does not give a definition of red teaming. However it backs the practice because of its principles of robustness, security and safety. These principles say that AI systems need to be strong, protected and safe during their life.[19] This includes when they are used normally when they are used in ways that were expected when they are misused and in difficult situations. Also there should be ways to stop, fix or take down systems that might cause harm or act in a way. Red teaming shows this way of thinking by looking for abuse and bad situations before they happen.

The UNESCO Recommendation, on the Ethics of Artificial Intelligence (2021) does not offer a definition of red teaming.[20]It supports the practice by focusing on human rights, dignity, fairness, transparency, human control, responsibility and no discrimination. According to the UNESCO plan red teaming is best explained as a way to check and keep track of AI systems. It helps to see if an AI system creates bias or causes treatment.

United States Federal Frameworks

Executive Order 14110, signed by President Biden on October 30, 2023, provided the most formal definition of AI red teaming in US federal governance[21]. It defined "AI red-teaming" as "a structured testing effort to find flaws and vulnerabilities in an AI system, often in a controlled environment and in collaboration with AI developers." However, Executive Order 14110 was repealed by Executive Order 14179[22], signed by President Trump on January 23, 2025, and titled "Removing Barriers to American Leadership in Artificial Intelligence." Executive Order 14110 must thus be acknowledged as a historical source rather than the present US government structure.

Following the revocation of Executive Order 14110, the National Institute of Standards and Technology's voluntary guidelines serve as the operational US federal references for AI red teaming. The NIST AI Risk Management Framework (AI RMF 1.0), which was published in January 2023, describes red teaming as a technique for assessing AI system resilience and finding vulnerabilities. The NIST Generative AI Profile (NIST AI 600-1)[23], published in July 2024, provides a more practical definition, describing AI red-teaming as a structured testing exercise to identify flaws and vulnerabilities in AI systems—such as inaccurate, harmful, or discriminatory outputs—in a controlled environment and in collaboration with system developers.

The Profile states that red teaming can take place before or after public release, by expert teams, general public participation, combination teams, or human/AI teams, and that testing panels should reflect demographic diversity[24]. These NIST guidelines are still in effect and serve as the current official US federal reference for AI red teaming, despite the fact that they are voluntary standards rather than legally binding mandates.

TYPES OF RED TEAMING

Red teaming is something that can be looked at in different ways. These include the NIST Generative AI Profile, the MITRE ATLAS threat matrix, the OWASP Top 10 for Large Language Model Applications and the EU Artificial Intelligence Act 2024.[25] The EU Artificial Intelligence Act 2024 has been changed to fit the laws, languages and constitution of India. Teaming and artificial intelligence systems are being evaluated using these documents.

Classification by Operator

The identity of the people doing the teaming exercise the position they have, in the organization and what they are allowed to do all affect what the exercise covers, how believable the results are and if the findings are legally important. The red teaming exercise is only as good as the people doing it and what they are trying to find out. The red teaming exercise findings are what matter. These findings are determined by the entity conducting the red teaming exercise.

Operator Category Description Institutional Advantage Limitation Indian Context
Internal Red Teaming The people who make the system are in charge of making sure it is safe and secure. They have control over how the system is built what information it uses, what it is told to do and how it is put into action. Sometimes the people in charge do not see problems because they are too close to the system or they have reasons to not want to find faults. This is important for the people who make intelligence systems and who follow the rules set by the IndiaAI Governance Guidelines[26].
Independent Red Teaming Third party auditors or experts from universities can test the system to see if it works correctly. This helps people trust the system especially if it is used for things that're very important or that affect peoples rights. These tests look at the system from viewpoints like how users see it how people who might try to hurt it see it and how the government sees it. The people who test the system may not be able to see all of the details about how it works, unless they are given permission.
Community Red Teaming It involves people like users and community members. This includes people who speak languages and lawyers and journalists. They help identify problems that expert teams might miss. Community Red Teaming captures what people really experience in their lives. It also looks at knowledge and vulnerabilities that experts might overlook. It needs to be done so that participants are protected. May lack technical depth for sophisticated attack vectors; requires careful ethical safeguards for participants. The National Institute of Standards and Technology or NIST thinks Community Red Teaming is very valuable. [27]This is because it can find problems that experts might not see. In India Community Red Teaming is critical for testing intelligence systems. This is because India has 22 scheduled languages and many different cultures.
Expert Red Teaming It is done by specialists like lawyers and doctors. These experts have a lot of knowledge in their fields. They can find problems that others might miss. Deep domain knowledge enables the identification of subtle, context-specific vulnerabilities. Expert Red Teaming might be limited. The experts might only know about their field. NIST recognizes the importance of Expert Red Teaming. In India expert red teams[28] for artificial intelligence should include judges and lawyers.
Combination Red Teaming This involves both experts and ordinary people working together. It combines knowledge with real-world experience. This type of testing can find both problems and everyday issues. Logistically complex; requires careful coordination and deconfliction of findings. NIST recommends Combination Red Teaming for projects. This is especially true for intelligence systems in India. These systems are used in law and healthcare and finance. They need to be both technically sound and trustworthy.[29]
Human-AI Red Teaming Human-AI Red Teaming uses intelligence to test systems. It can generate different tests and find problems quickly. Enables high-throughput testing across thousands of input variations; efficient for regression testing after model updates. Human-AI Red Teaming might miss some problems that are nuanced or specific to a context. It is useful for testing intelligence translation tools. This is to make sure that the tools are not biased.
Regulatory Red Teaming Regulatory Red Teaming is done by government agencies or regulators. It carries a lot of authority. The findings can inform decisions about enforcement or licensing. May be constrained by regulatory mandate, resource limitations, or political considerations. It is most relevant for intelligence applications in finance and healthcare and law enforcement. This includes agencies, like the Reserve Bank of India and the Election Commission of India.

Classification by Attack Type

The category of risk or vulnerability being tested determines the technical methodology, the applicable legal framework, and the constitutional implications of the red-teaming exercise.

Attack Type What Is Tested Primary Technical Sources Indian Legal / Constitutional Nexus
Safety Red Teaming Whether an artificial intelligence system creates content that is harmful, illegal, violent, hateful, extreme, self-harming, or violates kid safety standards. NIST AI 600-1 classifies harmful content generation as a major generative AI risk[30]. IT Act 2000, Sections 67-67B; Indian Penal Code / Bharatiya Nyaya Sanhita 2023 restrictions on hate speech, obscenity, and provocation.
Security Red Teaming Prompt injection, data poisoning, adversarial examples, unsecure plugin exploitation, tool misuse, model extraction, RAG poisoning, and API abuse. NIST AI RMF 1.0; MITRE ATLAS; OWASP Top 10 LLM Applications[31]. IT Act 2000, Sections 43, 43A, and 66; DPDP Act 2023, Section 8(5)
Privacy Red Teaming AI systems can sometimes give away information in a few different ways like when they do something called membership inference or model inversion or training-data extraction or memorization or what is known as RAG leaking. Shokri and colleagues wrote about this in a paper called 'Membership Inference Attacks Against Machine Learning Models back in 2017[32]. There are some laws that talk about this kind of thing like the DPDP Act 2023 which has sections 8(5)–(6).
Bias and Fairness Red Teaming People face discrimination because of their caste, tribe, religion, gender, handicap language, where they live, their class, surname, location. Whether they are from a rural or urban area. I found this information in the NIST AI 600-1 and the UNESCO Recommendation on AI Ethics from 2021[33]. The Constitution of India in Articles 14 15 17 21 and other laws, like the RPWD Act 2016 and the SC/ST Act 1989 talk about discrimination and how to prevent it.
Hallucination Red Teaming fraudulent citations, created case law, invented statute provisions, inaccurate quotations, incorrect legal propositions, and fraudulent factual statements. The NIST AI 600-1 talks about "confabulation" where we are talking about making up information that's false or wrong and then presenting it like it is true and are very sure about it. Bharatiya Sakshya Adhiniyam 2023 (expert and scientific evidence) and Supreme Court e-Committee AI White Paper (judicial AI reliability requirements)[34].
Synthetic-Content and Deepfake Red Teaming Can artificial intelligence made audio, video or pictures really look like they are from people and can they avoid being detected get rid of labels remove information about where they came from or spread false information. Rules like NIST AI 600-1 and the Content Authenticity Initiative standards. There are also rules like the IT Amendment Rules 2026 that talk about content.[35]
Content Provenance Red Teaming Can things like watermarks, information, about where something came from labels and signals that say something is real survive when people take screenshots compress them post them again crop them translate them or share them on platforms. We have the C2PA Technical Specification and NIST AI 600-1 to help with this. The IT Intermediary Rules 2026 also say that people who make content have to label it and say where it came from.
Rights-Based Red Teaming That Artificial Intelligence systems a threat to important things like equality and privacy. The United Nations Educational, Scientific and Cultural Organization made a recommendation about the ethics of intelligence in 2021. The Organisation for Economic Co-operation and Development also made some principles for intelligence in 2023[36]. The Constitution of India has some articles that're relevant here like articles 14 19 21 and 25. There was also a court case called Justice K.S. Puttaswamy v Union of India in 2017[37].
Emotional AI and Cognitive Biometric Red Teaming Whether systems can infer, classify, or manipulate mental or emotional states using facial expressions, voice, gaze, wearables, XR behaviour, or neural/cognitive signals. Ienca and Andorno, 'Towards New Human Rights in the Age of Neuroscience and Neurotechnology' (2017); UNESCO Recommendation on the Ethics of Neurotechnology (2025). [38] Constitution of India, art 21 (mental privacy, cognitive liberty); EU AI Act 2024, art 5(1)(f) (prohibition on emotion inference in workplace/education). [39]
Agentic AI Red Teaming AI agents interacting with tools, APIs, memory, file access, and transactional capabilities; permissions escalation, excessive agency, tool misuse, unsafe delegation, and data exfiltration. NIST AI 600-1; OWASP Top 10 for LLM Applications (2025 update). DPDP Act 2023, s 8 (enacted; not yet in force); IT Act 2000, ss 43, 66.

Classification by AI System Type

Red teaming methodology varies according to the architecture and deployment context of the AI system under evaluation.

AI System Type Description Primary Red Teaming Focus Indian Deployment Context
LLM Red Teaming Things like the Chatbots, text generators, and other summarization tools. The primary function is prompt injection, jailbreaking, hallucination, and sorting out the bias, toxic output generation, and report to the operators SUPACE (case summarisation); LegRAA (legal research); general-purpose legal chatbots.
RAG Red Teaming Systems that get information and create things from documents, law databases or places that store knowledge. These systems like when they get information people making up references changing what is around the important words and getting the source of the information wrong. Some tools that use intelligence look at the NJDG, Indian Kanoon or court records.
Vision Red Teaming Vision Red have systems that can recognize faces scan documents and watch people and they use machines to read writing. Adversarial image perturbation, demographic bias, misidentification, OCR errors in regional scripts. These systems for things, like watching people with cameras making court documents digital and helping the police recognize faces.
Audio Red Teaming Voice cloning, transcription, speech-to-text, and voice authentication software. Voice spoofing, transcription mistakes in accented or dialectal speech, and incorrect speaker identification. Court transcription tools (TERES) are voice-based accessibility solutions for people with disabilities.
Multimodal Red Teaming Systems that integrate text, image, audio, and video inputs and outputs. Cross-modal adversarial attacks, varying outputs across modalities, and compound bias. Virtual hearing platforms (Digital Courts 2.1) and multimodal evidence analysis tools.
Foundation-Model Red Teaming General-purpose AI models with systemic risk, capable of diverse downstream applications. Broad adversarial testing across all risk categories; systemic-risk evaluation. EU AI Act 2024, art 55(1)(a) requires providers of general-purpose AI models with systemic risk to conduct and document adversarial testing. [19] No equivalent Indian statutory mandate exists as of July 2026.

India-Specific Contextual Classification

Red teaming in India must be contextualized, as AI risks do not stem solely from generic model vulnerabilities. Red teaming in India must be contextualized, as AI risks do not stem solely from generic model vulnerabilities. They stem from India's multilingual legal system, caste and religious hierarchies, welfare reliance on digital infrastructure, AI-assisted judicial administration, financial inclusion gaps, political deepfakes, and the use of AI in public-sector decision-making. They stem from India's multilingual legal system, caste and religious hierarchies, welfare reliance on digital infrastructure, AI-assisted judicial administration, financial inclusion gaps, political deepfakes, and the use of AI in public-sector decision-making.

Red Teaming Type Indian Deployment Context India-Specific Risk Vector Applicable Framework
Linguistic Bias Red Teaming AI tools used for judicial translation, legal research, e-filing scrutiny, transcription and public-service chatbots; examples include SUVAS, LegRAA, Digital Courts 2.1, ASR-SHRUTI and PANINI translation tools. Performance degradation for low-resource languages; inaccurate translation of legal terms; disadvantage to litigants in non-English and non-Hindi proceedings; access-to-justice barriers for regional-language users. The test coverage should extend to all 22 Eighth Schedule languages even where the present tool covers fewer languages. eCourts Phase III; Supreme Court AI White Paper; Article 14 equality; Article 21 access to justice and due process; Eighth Schedule language context.[40]
Caste, Religion and Social Bias Red Teaming AI systems used, or proposed to be used, in bail-support tools, policing analytics, HR screening, credit scoring, insurance underwriting, welfare eligibility, public-service targeting and social-benefit disbursement. Encoding historical caste disadvantage into predictive systems; proxy discrimination through surname, locality, school, occupation, dialect, food habit or religion-coded markers; profiling of minorities; unequal false positives or false negatives across vulnerable groups. Articles 14, 15 and 21 of the Constitution; DPDP Act section 10 for Significant Data Fiduciaries where applicable; IndiaAI Governance Guidelines on fairness, equity, bias mitigation and vulnerable groups.[41]
Hallucination Red Teaming for Judicial and Legal AI Legal research tools, LegRAA, SUPACE, AI-assisted pleadings, judicial summarisation tools, automated drafting tools and case-law retrieval systems. Fake case citations, non-existent judgments, wrong statutory cross-references, invented quotations, false procedural history, misleading ratio decidendi and overreliance by lawyers, judges or quasi-judicial authorities. Supreme Court AI White Paper; eCourts AI pilots; KMG Wires v NFAC, Bombay High Court; Article 21 due process; professional duty to verify AI-generated legal material.[42]
Financial AI Red Teaming Credit scoring, insurance underwriting, UPI fraud detection, KYC automation, customer-risk profiling, collections automation, algorithmic trading and stock-market surveillance systems. Discriminatory lending against rural, tribal, unbanked or informal-sector populations; proxy discrimination; fraud evasion; model drift; explainability failure; flash-crash or market-integrity risk from untested algorithmic trading systems. RBI FREE-AI Framework Recommendation 20; RBI Model Risk Management draft guidance 2026; SEBI algorithmic-trading circulars and market-integrity framework.[43]
Deepfake and Synthetic Content Red Teaming Social-media intermediaries, election campaign platforms, generative-video tools, political communication AI, synthetic voice tools and content-moderation AI. Electoral manipulation; impersonation of public officials; defamation through synthetic video; communal incitement through AI-generated audio; failure of watermarking or metadata; removal of labels; evasion of synthetic-content detection. IT Rules 2021 as amended in 2026; Rule 2(1)(wa), Rule 3(3) and Rule 4(1A); Representation of the People Act 1951; BNS provisions where applicable.[44]
Government and Welfare AI Red Teaming Aadhaar-linked welfare systems, Direct Benefit Transfer eligibility, public distribution systems, grievance redressal, tax administration, policing tools, smart-city surveillance and public employment or education platforms. Exclusion of marginalised beneficiaries; false criminal profiling; inability to appeal automated decisions; language barriers; opaque risk scoring; surveillance overreach; denial of livelihood, welfare or public services. Article 21 life, liberty, dignity and livelihood; Article 14 non-arbitrariness; DPDP Act 2023; Aadhaar Act 2016; IndiaAI risk-classification and AI incidents framework.[45]
Election and Political Communication Red Teaming AI-generated campaign material, political microtargeting systems, party communication tools, bot networks, recommendation systems and election-related deepfake detection. Misinformation, voter manipulation, communal targeting, impersonation of candidates, synthetic speeches, AI-generated hate speech, and removal of provenance labels during cross-platform circulation. IT Rules 2026 synthetic-content obligations; Representation of the People Act 1951; Election Commission/MCC framework; Article 19 and democratic participation values.[46]
Emotional AI and Cognitive Biometric Red Teaming AI proctoring, workplace monitoring, attention tracking, wearables, XR systems, neurotechnology, mental-health tools, affective computing and biometric-emotional classification tools. Inferring stress, anger, attention, deception, vulnerability, political preference or mental-health state; emotional manipulation; chilling effect on mental autonomy; discrimination through cognitive or affective proxies. Article 21 privacy, dignity and mental autonomy; freedom-of-thought scholarship; cognitive biometrics and mental privacy literature.[47]

SHOW UP IN THE OFFICIAL DATABASE

Agency Mandate / Relevance to Red Teaming
Supreme Court e-Committee Oversees technology adoption under the eCourts Project and judicial AI pilots. Official material confirms AI use in translation, prediction/forecasting, automated filing, intelligent scheduling, case-information systems and litigant communication. LegRAA has been developed to aid legal research, document analysis and judicial decision support; Digital Courts 2.1 includes judgment databases, document management, ASR-SHRUTI and PANINI; AI/ML tools have been integrated with e-filing software for defect identification; SUPACE remains in experimental development. Red teaming is relevant here because these systems affect access to justice, procedural fairness, translation accuracy and citation reliability.[48]
MeitY / IndiaAI Mission Issues the IndiaAI Governance Guidelines, supports voluntary accountability, and recognises transparency reports through which firms may publish red-teaming results, impact assessments or risk-mitigation steps. MeitY also anchors the broader IndiaAI ecosystem, including AIKosh, Safe and Trusted AI tools, AISI, AI incidents systems and regulatory coordination.[49]
AI Safety Institute (AISI) The IndiaAI Governance Guidelines identify AISI as the technical body for AI safety research, draft standards, evaluation metrics, testing methods, benchmarks, risk assessment and safety testing. AISI should be treated as the institutional anchor for future Indian red-teaming standards and sectoral benchmarks, but avoid claiming that it already operates a full red-teaming registry unless a later official source confirms this.[50]
Reserve Bank of India (RBI) RBI’s FREE-AI framework recommends AI red teaming and breach preparedness for financial-sector AI. The safer formulation is “recommends” or “proposes,” not “mandates,” unless a binding RBI direction has been issued. RBI’s 2026 draft Model Risk Management guidance also strengthens the financial-AI framework by requiring board-approved model-risk frameworks, model inventories, independent validation, human oversight and enhanced controls for AI/ML models.[51]
Securities and Exchange Board of India (SEBI) SEBI’s algorithmic-trading circulars are relevant for financial AI red teaming because trading algorithms require auditability, exchange/broker controls, market-integrity safeguards and risk-management mechanisms. Red teaming should test automated trading systems for flash-crash risk, manipulation, excessive order-to-trade ratios and failure of kill-switch or risk-control mechanisms.[52]
National Informatics Centre (NIC) NIC is central to judicial AI infrastructure. Official material records that LegRAA was developed by NIC’s AI Division and Centre of Excellence (eCourts), NIC Pune, under the guidance of the eCommittee. NIC’s role makes it relevant for technical testing, secure deployment, eCourts integration and future judicial red-team infrastructure.[53]
Department of Justice / Ministry of Law and Justice Coordinates eCourts Phase III and judicial digital transformation. Since Phase III includes future technological advancement such as AI and blockchain, the Department of Justice is institutionally relevant for ensuring that judicial AI pilots include red-team testing, human oversight, auditability and public accountability.[54]
Data Protection Board of India (DPBI) DPBI is relevant where AI failures involve personal data. The DPDP Act does not expressly require red teaming, but section 8(5) requires reasonable security safeguards to prevent personal data breach. Privacy red teaming may therefore become evidence of due diligence where a Data Fiduciary processes personal data through AI systems.[55]
CERT-In CERT-In is relevant to AI red teaming where the vulnerability is also a cybersecurity incident, such as prompt-injection-enabled data exfiltration, model extraction, API abuse, malware generation, or compromise of AI infrastructure. Red-team findings should feed into incident response and coordinated vulnerability disclosure, but do not claim that CERT-In currently maintains an AI red-team database unless official confirmation exists.[56]

Red Teaming in Key Official Repositories

National Judicial Data Grid (NJDG)

The NJDG should not be considered an AI red-teaming registry. Officially[57], it is a national repository of data on pending and resolved cases in district and taluka courts, allowing for case management and monitoring via searchable case data. As a result, the correct formulation is that NJDG currently does not maintain a red-teaming registry, but it may in the future be linked to an AI Quality Assurance Dashboard for judicial AI tools. Such a dashboard could monitor translation and transcription errors, false e-filing defect flags, hallucinated legal authorities, and user-reported AI failures.

IndiaAI Mission Repository and Transparency Reporting

The IndiaAI Governance Guidelines acknowledge transparency reports as an accountability mechanism through which firms can publish red-teaming results, impact assessments, or risk-mitigation strategies[58]. As a result, IndiaAI should be positioned as a future repository for transparency disclosures, benchmark results, AI safety tools, and risk-mitigation resources, rather than as a fully operational red-teaming repository. IndiaAI plans to host benchmark summaries, testing methods, and public safety reports for high-risk AI systems once AISI testing frameworks are mature.

eCourts Phase III

eCourts Phase III is one of the most significant Indian contexts for red teaming. Official documentation confirms the development or deployment of LegRAA, Digital Courts 2.1, ASR-SHRUTI, PANINI [44], e-filing defect-identification tools, metadata extraction prototypes, and SUPACE. These tools should be scrutinized for hallucinated citations, mistranslation, transcription errors, e-filing defect gaming, metadata extraction errors, factual summarization failure, judicial officers' overreliance, and unequal performance across Indian languages. The current official position is that AI-based solutions remain limited to controlled pilot deployments and that operational frameworks will be governed by relevant High Court rules and policies.

National AI Incidents Database

India currently does not have a fully operational public National AI Incidents Database. However, the IndiaAI Governance Guidelines recommend gathering empirical data on AI-related harms and developing a national database of AI incidents to better understand what harms are caused by AI, how AI contributes to harm, when such harms occur, and what causes them[59]. The action plan also includes "operationalising a national AI incident database with localised reporting and feedback loops," according to the Guidelines. The correct framing is that red-team findings should be integrated into the future AI incidents system, but they are not yet part of a mature official red-team database.

AIKosh

The correct name is AIKosh or AIKosha, not "Kosh AI." IndiaAI defines AIKosh as a centralized platform for high-quality AI artifacts relevant to India, including datasets, models, toolkits, use cases, and a sandbox. The IndiaAI Governance Guidelines also state that AIKosh has onboarded datasets and AI models and provides permission-based access. As a result[60], AIKosh should serve as a suitable future repository for adversarial test suites, Indian-language red-team datasets, synthetic-content detection benchmarks, bias probes, legal hallucination datasets, and privacy-preserving evaluation artifacts. Do not claim that AIKosh currently hosts or does not host red-teaming datasets without first consulting the live platform inventory on the date of publication.

RESEARCH ON RED TEAMING IN THE INDIAN JUDICIARY

Research on red teaming in the Indian judiciary is still emerging. There is no settled Indian judicial red-teaming standard yet. However, the research and policy discourse now clearly points toward four requirements: pre-deployment testing, human verification, auditability, and incident reporting. These requirements arise from Indian state-level AI policies, High Court AI-use policies, the Supreme Court’s draft AI Regulations, international judicial AI guidance, and recent empirical research on LLMs as legal decision tools.

State-Level and Court-Level Requirements

Tamil Nadu’s Safe and Ethical AI Policy 2020 was one of the earliest Indian state instruments to require evaluation of AI systems before public rollout. It does not use the term “red teaming,” but its requirement of pre-public-use evaluation supports the same objective: testing AI systems before they affect public-service delivery or assisted decision-making.[61]

Telangana’s AI policy framework is more ecosystem-oriented. The Telangana AI Advisory Council and AI Research and Collaboration Network are designed to steer AI development, build indigenous datasets, models, applications and benchmarks, and enable safe, responsible and equitable AI in the State. This does not amount to a binding red-teaming mandate, but it creates the institutional basis for future adversarial testing and benchmark-based evaluation of government AI systems.[62]

Kerala’s High Court policy is more directly relevant to the judiciary. It restricts the use of AI tools in the district judiciary and prohibits AI tools from being used to arrive at findings, reliefs, orders or judgments. The policy also requires caution because indiscriminate AI use may create privacy, data-security and public-trust risks, and it requires records of AI-tool use and human verification. This is not a full technical red-teaming framework, but it establishes the judicial governance principle that AI outputs must remain assistive, verified and auditable.[63]

Gujarat High Court’s AI policy also supports a red-teaming logic. It recognises that AI systems may encode or perpetuate bias relating to gender, religion, caste, ethnicity or socio-economic status, requires users to understand AI limitations, and makes human supervision and verification central to AI-assisted work. This gives a more explicit basis for bias testing, AI literacy, data-protection compliance and auditability in judicial administration.[64]

The most developed Indian judicial framework is the Supreme Court’s Draft Regulations for Use of Artificial Intelligence in Courts, 2026. The draft regulations are not final law, but they are the strongest official Indian articulation of judicial AI testing. They provide for human primacy, rule of law, fairness, non-discrimination, auditability, privacy, purpose limitation, proportionality, inclusivity, data integrity and cybersecurity. Most importantly, they expressly include controlled-environment testing, periodic technical/legal/ethical audits, an AI Register and an AI Incident Database.[65]

What Judicial Red Teaming Should Test

Judicial red teaming should test whether AI tools used in courts can produce fake citations, incorrect legal propositions, mistranslations, biased summaries, inaccurate transcriptions, defective filing flags, privacy leakage, procedural errors or unequal performance across languages and litigant groups. This is especially important for tools used in legal research, case summarisation, translation, e-filing scrutiny, transcription, scheduling and case-management.

Hallucination red teaming is central. In legal AI, hallucination does not merely mean factual error; it can mean fake case law, invented statutory provisions, wrong quotations, false procedural history or misleading ratio decidendi. The risk is heightened because lawyers, judges, court staff or litigants may treat AI-generated legal material as authoritative unless independent verification is mandatory.

Persuadability testing is another new research requirement. Suttle and Lillis show that LLMs proposed as legal decision assistants or first-instance decision tools can be influenced by the quality of legal arguments made to them. This matters because an AI decision-support system may privilege parties with better advocacy, stronger AI tools or more sophisticated prompting, thereby affecting access to justice and equality of arms.[66]

Access to Justice and AI-Assisted Litigation

Research on AI and access to justice is relevant because AI tools may increase the ability of self-represented litigants to enter the court system, but may also generate new burdens for courts if filings become more numerous, formulaic or legally unreliable. Shah’s SSRN paper on access to justice in the age of AI should be used for this access-to-justice dimension, not as a direct red-teaming authority. Its relevance is that judicial red teaming should test whether AI-assisted filings improve meaningful access or merely increase low-quality filings that shift verification burdens onto courts.[67]

International Judicial AI Guidance

International guidance supports the same conclusion. UNESCO’s 2025 judicial AI guidelines emphasise that AI should strengthen, not undermine, human-led justice. The National Center for State Courts similarly advises that courts using AI for document summarisation, case research and public engagement must review AI-generated content to prevent errors, bias and misinformation, and should use a human-in-the-loop approach.[68]

The Judicial AI Consortium’s resources, including “AI in Chambers,” are useful as practical guidance for judges and chambers considering AI tools, but they should be treated as professional guidance rather than binding authority. They support the need for judicial literacy, verification and responsible use, but they do not create legal obligations by themselves.

DATA CHALLENGES

Poor data quality, incompleteness, and inconsistency.

Adversarial test datasets for Indian-language AI are extremely underdeveloped. The majority of commercially available red teaming toolkits (PyRIT and Garak) focus on English, making them insufficient for testing AI systems in Hindi, Tamil, Telugu, Bengali, or Marathi contexts. Indian legal corpus quality issues—OCR errors in digitized law reports, inconsistent citation formatting, and a lack of structured metadata—degrade the quality of LLM training data and make hallucination testing more difficult to design.

Fragmented, non-interoperable systems.

Red teaming findings generated by various entities—private firms, CERT-In, the DPBI, and the RBI—are not consolidated in a single registry. The lack of a national AI Incidents Database means that vulnerability intelligence discovered through red teaming at one financial institution cannot inform the testing protocols of a judicial AI provider, even if the underlying AI model is identical.

Privacy, Confidentiality, and Security Restrictions

To be effective, red teaming exercises against AI systems processing personal data (judicial case records, medical records, and financial data) must be conducted with realistic data. Providing such access to red teams raises disclosure concerns under the DPDP Act 2023. Synthetic data generation for red teaming alleviates this, but it has its own fidelity limitations. As of 2026, DPDP-compliant red teaming protocols remained undeveloped.

Opacity and unauditability

The majority of AI systems deployed in India, including commercial LLMs accessed via API—are proprietary black boxes. Red teaming of black-box systems is inevitably limited to behavioral observation rather than internal architectural audit. This opacity impedes comprehensive vulnerability assessment and reduces remediation effectiveness. The PSA White Paper's "compliance by design" mandate cannot be enforced without access to model architecture documentation, which is currently not required under Indian law.

Algorithmic biases and representation gaps

Red teaming for bias necessitates representative adversarial probe datasets that include all 22 Indian scheduled languages, all protected characteristics under Article 15, and regional cultural contexts. Such datasets are not currently available at scale. Existing English-language bias benchmarks are not applicable to multilingual Indian AI contexts and must be completely rebuilt.

Infrastructure and Resource Constraints

Red teaming large foundation models requires a significant amount of computational resources. India's 10,372 crore IndiaAI Mission compute capacity is primarily used for AI training rather than evaluation and testing. Dedicated red teaming compute infrastructure, including GPU clusters and sandboxed testing environments, remains a significant resource gap for the AISI as it goes live.

WAY AHEAD

Standardisation and Harmonisation of Existing Data

AISI should develop an Indian AI Red Teaming Standard (IARTS) by 2027 – a national standard that articulates mandatory red teaming protocols for high-risk AI systems as classified under India’s risk taxonomy. The standard should mandate coverage of multilingual adversarial test suites for all 22 scheduled languages; mandatory probe datasets to test caste and religious bias; and sector-specific attack libraries for judicial, financial, healthcare and electoral AI. The National AI Incidents Database needs to combine red teaming finding reports with a standardized schema that corresponds to the OECD AI Incidents Monitor. AI Kosh should host open access adversarial test suites for consistent and reproducible red teaming across India’s AI ecosystem.

Better Data Collection in the Future

The DPBI should work to develop privacy-by-design red teaming protocols that outline how to perform adversarial testing on AI systems that process personal data without creating new privacy violations. AI-assisted quality controls for red teaming datasets (i.e. AI to generate high-quality adversarial probes at scale) should be added to AI Kosh’s dataset curation pipeline. High Court Judicial Data Labs should be able to do red teaming, which is adversarial testing of AI tools used in that jurisdiction.

Allowing Systemic Analysis

High-risk AI systems under India’s AI governance framework must be subject to mandatory independent public red teaming audits by AISI or accredited third-party firms prior to judicial deployment and annually thereafter. The audits should lead to public reports as per the transparency mandate of MeitY AI Governance Guidelines. Longitudinal equity studies should track red teaming findings across protected characteristics to detect persistent patterns of bias across successive model versions. Annual reporting on red teaming results for government deployed AI systems should be made to a parliamentary oversight mechanism, potentially through a Joint Parliamentary Committee on AI Governance.

Contributors & review status
Curated / edited bynot yet recorded
Peer reviewpending
Project reviewpending
Expert reviewpending
Page created 2026-08-21  ·  Last updated 2026-08-23
  1. ↑ National Institute of Standards and Technology, Artificial Intelligence Risk Management Framework (AI RMF 1.0) (NIST 2023) 42 https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.100-1.pdf accessed 30 July 2026.
  2. ↑ Ministry of Electronics and Information Technology, 'Advisory on Due Diligence by Intermediaries/Platforms under the Information Technology (Intermediary Guidelines and Digital Media Ethics Code) Rules, 2021' (15 March 2024) para 2(c) https://www.meity.gov.in/writereaddata/files/Advisory%20on%20AI%20GenAI.pdf accessed 30 July 2026. https://www.meity.gov.in/static/uploads/2024/02/9f6e99572739a3024c9cdaec53a0a0ef.pdf
  3. ↑ Information Technology Act 2000, s 79; Information Technology (Intermediary Guidelines and Digital Media Ethics Code) Rules 2021, SI 2021/GSR 139(E).
  4. ↑ Digital Personal Data Protection Act 2023, ss 8(5)–(6), 10(2)(a).
  5. ↑ Information Technology Act 2000, s 79(1)–(2); Information Technology (Intermediary Guidelines and Digital Media Ethics Code) Rules 2021, rr 3–4.
  6. ↑ Constitution of India 1950, arts 14, 21; Justice K.S. Puttaswamy v Union of India (2017) 10 SCC 1. https://cdnbbsr.s3waas.gov.in/s380537a945c7aaa788ccfcdf1b99b5d8f/uploads/2024/07/20240716890312078.pdf
  7. ↑ Ian J. Goodfellow, Jonathon Shlens and Christian Szegedy, 'Explaining and Harnessing Adversarial Examples' (2015) arXiv:1412.6572 https://arxiv.org/abs/1412.6572 accessed 30 July 2026.
  8. ↑ Information Technology Act 2000; Digital Personal Data Protection Act 2023; Information Technology (Intermediary Guidelines and Digital Media Ethics Code) Rules 2021, SI 2021/GSR 139(E). The term "red teaming" does not appear in any of these instruments. https://www.indiacode.nic.in/bitstream/123456789/1999/1/A2000-21%20%281%29.pdf?spm=a2ty_o01.29997173.0.0.1a5a55fbhaRZ52&file=A2000-21%20%281%29.pdf
  9. ↑ Ministry of Electronics and Information Technology, 'IndiaAI Governance Guidelines' (Government of India, November 2025). https://static.pib.gov.in/WriteReadData/specificdocs/documents/2025/nov/doc2025115685601.pdf
  10. ↑ ibid (definition of red teaming as a simulation exercise for comprehensive security assessment).
  11. ↑ Digital Personal Data Protection Act 2023. The term "red teaming" does not appear in any section of the Act. https://egazette.gov.in/WriteReadData/2023/247847.pdf?spm=a2ty_o01.29997173.0.0.1a5a55fbhaRZ52&file=247847.pdf
  12. ↑ ibid ss 8(5)–(6).
  13. ↑ Government of Telangana, 'Telangana AI Roadmap' (September 2024). https://negd-media.digitalindiacorporation.in/2024/09/AI-Powered-Telangana-Strategy-Document-and-Implementation-Roadmap.pdf
  14. ↑ Government of Tamil Nadu, 'Tamil Nadu Safe and Ethical AI Policy' (2020).https://it.tn.gov.in/sites/default/files/2021-06/TN_Safe_Ethical_AI_policy_2020.pdf
  15. ↑ Kerala High Court, 'AI Policy for the Judiciary' (July 2025). https://images.assettype.com/theleaflet/2025-07-22/mt4bw6n7/Kerala_HC_AI_Guidelines.pdf
  16. ↑ Regulation (EU) 2024/1689 of the European Parliament and of the Council of 13 June 2024 laying down harmonised rules on artificial intelligence (Artificial Intelligence Act), OJ L 2024/1689. https://legalinstruments.oecd.org/en/instruments/OECD-LEGAL-0449?spm=a2ty_o01.29997173.0.0.1a5a55fbhaRZ52
  17. ↑ ibid art 55(1)(a).
  18. ↑ ibid. The EU AI Act makes adversarial testing legally binding for systemic-risk GPAI models; India does not. https://eur-lex.europa.eu/eli/reg/2024/1689/oj?spm=a2ty_o01.29997173.0.0.1a5a55fbhaRZ52
  19. ↑ OECD, 'Recommendation of the Council on Artificial Intelligence' (adopted 21 May 2019, updated 2023) OECD/LEGAL/0449, Principle 1.5 https://legalinstruments.oecd.org/en/instruments/OECD-LEGAL-0449 accessed 30 July 2026.
  20. ↑ UNESCO, 'Recommendation on the Ethics of Artificial Intelligence' (adopted 23 November 2021, 41st General Conference) https://unesdoc.unesco.org/ark:/48223/pf0000381137 accessed 30 July 2026.
  21. ↑ Executive Order 14110, 'Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence' (30 October 2023) 88 Fed Reg 75191, s 3.2 (revoked).
  22. ↑ Executive Order 14179, 'Removing Barriers to American Leadership in Artificial Intelligence' (23 January 2025) 90 Fed Reg 8763, s 2: "The order [EO 14110] is hereby revoked."
  23. ↑ National Institute of Standards and Technology, 'Artificial Intelligence Risk Management Framework (AI RMF 1.0)' (NIST 2023) 42 https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.100-1.pdf accessed 30 July 2026.
  24. ↑ National Institute of Standards and Technology, 'Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile' (NIST AI 600-1, July 2024) 22 https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf accessed 30 July 2026.
  25. ↑ National Institute of Standards and Technology, 'Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile' (NIST AI 600-1, July 2024) https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf accessed 30 July 2026; MITRE Corporation, 'ATLAS: Adversarial Threat Landscape for Artificial-Intelligence Systems' https://atlas.mitre.org accessed 30 July 2026; OWASP Foundation, 'OWASP Top 10 for Large Language Model Applications' (2025) https://owasp.org/www-project-top-10-for-large-language-model-applications/ accessed 30 July 2026; Regulation (EU) 2024/1689, OJ L 2024/1689.
  26. ↑ Ministry of Electronics and Information Technology, 'IndiaAI Governance Guidelines' (Government of India, November 2025).https://static.pib.gov.in/WriteReadData/specificdocs/documents/2025/nov/doc2025115685601.pdf
  27. ↑ NIST AI 600-1 (n 1) 23: "Red-teaming can be performed… by… general public participants."
  28. ↑ ibid 23: "Red-teaming can be performed… by expert teams."
  29. ↑ National Institute of Standards and Technology, 'Artificial Intelligence Risk Management Framework (AI RMF 1.0)' (NIST 2023) https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.100-1.pdf accessed 30 July 2026; MITRE ATLAS (n 1); OWASP Top 10 for LLM Applications (n 1).
  30. ↑ ibid 12–15 (harmful content generation as a significant generative AI risk).
  31. ↑ National Institute of Standards and Technology, 'Artificial Intelligence Risk Management Framework (AI RMF 1.0)' (NIST 2023) https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.100-1.pdf accessed 30 July 2026; MITRE ATLAS (n 1); OWASP Top 10 for LLM Applications (n 1).
  32. ↑ Reza Shokri and others, 'Membership Inference Attacks Against Machine Learning Models' (2017) IEEE Symposium on Security and Privacy 3.
  33. ↑ UNESCO, 'Recommendation on the Ethics of Artificial Intelligence' (adopted 23 November 2021, 41st General Conference) https://unesdoc.unesco.org/ark:/48223/pf0000381137 accessed 30 July 2026.
  34. ↑ Supreme Court of India, e-Committee, 'White Paper on AI and the Judiciary' (2025).https://cdnbbsr.s3waas.gov.in/s3ec0490f1f4972d133619a60c30f3559e/uploads/2025/11/2025112244.pdf
  35. ↑ Information Technology (Intermediary Guidelines and Digital Media Ethics Code) Amendment Rules 2026 (synthetic content provisions). https://www.meity.gov.in/static/uploads/2026/02/550681ab908f8afb135b0ad42816a1c9.pdf
  36. ↑ OECD, 'Recommendation of the Council on Artificial Intelligence' (adopted 21 May 2019, updated 2023) OECD/LEGAL/0449 https://legalinstruments.oecd.org/en/instruments/OECD-LEGAL-0449 accessed 30 July 2026; UNESCO (n 11).
  37. ↑ Justice K.S. Puttaswamy v Union of India (2017) 10 SCC 1.
  38. ↑ Marcello Ienca and Roberto Andorno, 'Towards New Human Rights in the Age of Neuroscience and Neurotechnology' (2017) 13(1) Life Sciences, Society and Policy 5; UNESCO, 'Recommendation on the Ethics of Neurotechnology' (adopted November 2025, 43rd General Conference) https://www.unesco.org/en/ethics-neurotech/recommendation accessed 30 July 2026.
  39. ↑ Regulation (EU) 2024/1689, OJ L 2024/1689, art 5(1)(f).https://eur-lex.europa.eu/eli/reg/2024/1689/oj/eng
  40. ↑ Ministry of Law and Justice, ‘Use of Artificial Intelligence in Legal Field’ (Press Information Bureau, 11 December 2025) https://www.pib.gov.in/PressReleasePage.aspx?PRID=2202159 accessed 28 June 2026; Centre for Research and Planning, Supreme Court of India, White Paper on Artificial Intelligence and Judiciary (November 2025) 11–12, PIB records LegRAA, Digital Courts 2.1, e-filing defect tools, SUPACE and eCourts Phase III AI allocation; the White Paper states that AI in courts raises accountability, bias, due process, privacy and constitutional concerns.
  41. ↑ Constitution of India 1950, arts 14, 15 and 21; Digital Personal Data Protection Act 2023, s 10; Ministry of Electronics and Information Technology, India AI Governance Guidelines (5 November 2025) 3, 37 https://static.pib.gov.in/WriteReadData/specificdocs/documents/2025/nov/doc2025115685601.pdf accessed 28 June 2026, IndiaAI says the framework is rooted in fairness/equity and aims to prevent algorithmic bias and protect vulnerable groups; AISI is to support bias mitigation, fairness testing and explainability.
  42. ↑ Ministry of Law and Justice, ‘Use of Artificial Intelligence in Legal Field’ (PIB, 11 December 2025); KMG Wires Private Limited v National Faceless Assessment Centre 2025:BHC-OS:19789-DB, Bombay High Court, 6 October 2025; Centre for Research and Planning, Supreme Court of India, White Paper on Artificial Intelligence and Judiciary (November 2025) 65–73.
  43. ↑ Reserve Bank of India, Framework for Responsible and Ethical Enablement of Artificial Intelligence in the Financial Sector (FREE-AI): Committee Report (13 August 2025) Recommendation 20; Reserve Bank of India, Guidance on Regulatory Principles for Model Risk Management, 2026 (Draft, 24 June 2026); Securities and Exchange Board of India, ‘Safer Participation of Retail Investors in Algorithmic Trading’ Circular No SEBI/HO/MIRSD/MIRSD-PoD/P/CIR/2025/0000013 (4 February 2025), https://naavi.org/uploads_wp/2026/draft_ai_guidelines_rbi_24062026.pdf?utm
  44. ↑ Information Technology (Intermediary Guidelines and Digital Media Ethics Code) Rules 2021, https://www.meity.gov.in/static/uploads/2026/02/550681ab908f8afb135b0ad42816a1c9.pdfrr 2(1)(wa), 3(3), 4(1A), as amended by Information Technology (Intermediary Guidelines and Digital Media Ethics Code) Amendment Rules 2026, GSR 120(E), 10 February 2026.
  45. ↑ Digital Personal Data Protection Act 2023, ss 8(5), 8(6), 10; Aadhaar (Targeted Delivery of Financial and Other Subsidies, Benefits and Services) Act 2016; Constitution of India 1950, arts 14 and 21; Ministry of Electronics and Information Technology, India AI Governance Guidelines (5 November 2025) 27, 31, https://static.pib.gov.in/WriteReadData/specificdocs/documents/2025/nov/doc2025115685601.pdf
  46. ↑ Information Technology (Intermediary Guidelines and Digital Media Ethics Code) Rules 2021, rr 2(1)(wa), 3(3), 4(1A), as amended in 2026; Representation of the People Act 1951.
  47. ↑ Alberto Rinaldi, ‘The Inviolable Mind: Freedom of Thought as a Constitutional Limit to Emotional AI’ (The Digital Constitutionalist, 17 October 2025) https://digi-con.org/the-inviolable-mind-freedom-of-thought-as-a-constitutional-limit-to-emotional-ai/ accessed 28 June 2026; Patrick Magee, Marcello Ienca and Nita A Farahany, ‘Beyond Neural Data: Cognitive Biometrics and Mental Privacy’ (2024) 112 Neuron 3017.
  48. ↑ Ministry of Law and Justice, ‘Use of Artificial Intelligence in Legal Field’ (PIB, 11 December 2025) https://www.pib.gov.in/PressReleasePage.aspx?PRID=2202159 accessed 28 June 2026.
  49. ↑ Ministry of Electronics and Information Technology, India AI Governance Guidelines (5 November 2025) 32 https://static.pib.gov.in/WriteReadData/specificdocs/documents/2025/nov/doc2025115685601.pdf accessed 28 June 2026
  50. ↑ Ministry of Electronics and Information Technology, India AI Governance Guidelines (5 November 2025) 37–38 https://static.pib.gov.in/WriteReadData/specificdocs/documents/2025/nov/doc2025115685601.pdf accessed 28 June 2026.
  51. ↑ Reserve Bank of India, Framework for Responsible and Ethical Enablement of Artificial Intelligence in the Financial Sector (FREE-AI): Committee Report (13 August 2025) Recommendation 20; Reserve Bank of India, Guidance on Regulatory Principles for Model Risk Management, 2026 (Draft, 24 June 2026).
  52. ↑ Securities and Exchange Board of India, ‘Safer Participation of Retail Investors in Algorithmic Trading’ Circular No SEBI/HO/MIRSD/MIRSD-PoD/P/CIR/2025/0000013 (4 February 2025) https://www.sebi.gov.in/legal/circulars/feb-2025/safer-participation-of-retail-investors-in-algorithmic-trading_91614.html accessed 28 June 2026.
  53. ↑ Ministry of Law and Justice, ‘Use of Artificial Intelligence in Legal Field’ (PIB, 11 December 2025).https://www.pib.gov.in/PressReleasePage.aspx?PRID=2202159&lang=1&reg=3
  54. ↑ Ministry of Law and Justice, ‘Use of Artificial Intelligence in Legal Field’ (PIB, 11 December 2025).
  55. ↑ Digital Personal Data Protection Act 2023, ss 8(5), 8(6), 10, sch item 1. Use this for DPBI’s role in personal-data AI failures.
  56. ↑ Information Technology Act 2000; Indian Computer Emergency Response Team and Manner of Performing Functions and Duties Rules 2013; Ministry of Electronics and Information Technology, India AI Governance Guidelines (5 November 2025) 35–37.
  57. ↑ e-Committee, Supreme Court of India, ‘National Judicial Data Grid’ https://ecommitteesci.gov.in/service/national-judicial-data-grid/ accessed 28 June 2026.
  58. ↑ Ministry of Electronics and Information Technology, India AI Governance Guidelines (5 November 2025) 32.
  59. ↑ Ministry of Electronics and Information Technology, India AI Governance Guidelines (5 November 2025) 27, 39–40.
  60. ↑ IndiaAI, ‘Now Open: Expression of Interest (EOI) to Contribute Datasets and AI Artefacts to AIKosh’ (7 July 2025) https://indiaai.gov.in/article/now-open-expression-of-interest-eoi-to-contribute-datasets-and-ai-artefacts-to-aikosh accessed 28 June 2026; Ministry of Electronics and Information Technology, India AI Governance Guidelines (5 November 2025) 15.
  61. ↑ Government of Tamil Nadu, Safe and Ethical Artificial Intelligence Policy 2020 14 https://it.tn.gov.in/sites/default/files/2021-06/TN_Safe_Ethical_AI_policy_2020.pdf accessed 29 June 2026, p 14 says the policy aims to establish guidelines for evaluating AI systems before rollout for public use and to build fairness, equity, transparency and trust in AI-assisted decision-making.
  62. ↑ Research and Innovation Circle of Hyderabad, Annual Report 2024 27 https://rich.telangana.gov.in/assets/pdfs/Reports/RICH-Annual-Report-2024.pdf accessed 29 June 2026, p 27 says the Telangana AI Advisory Council will set direction for AI, and the AI Research and Collaboration Network will develop indigenous datasets, models, applications and benchmarks and enable safe, responsible and equitable AI.
  63. ↑ High Court of Kerala, Policy Regarding Use of Artificial Intelligence Tools in District Judiciary HCKL/7490/2025-DI-3-HC KERALA I/140639/2025 (19 July 2025); ‘Kerala HC bars district courts from using AI for legal reasoning, decisions’ Business Standard (20 July 2025) https://www.business-standard.com/india-news/kerala-hc-bars-district-courts-from-using-ai-for-legal-reasoning-decisions-125072000216_1.html accessed 29 June 2026, the Business Standard/PTI report quotes the policy as stating that AI tools shall not be used to arrive at findings, reliefs, orders or judgments, and that courts must maintain detailed audit records of AI-tool use and human verification.
  64. ↑ High Court of Gujarat, Policy on the Use of Artificial Intelligence in the Judicial and Court Administration (2026) 3–4 https://gujarathighcourt.nic.in/hccms/sites/default/files/miscnotifications/Policy%20on%20the%20use%20of%20Artificial%20Intelligence%20in%20the%20Judicial%20and%20Court%20Administration.pdf accessed 29 June 2026, pp 3–4 recognise risks of bias relating to gender, religion, caste, ethnicity and socio-economic status, require users to understand AI limitations, and require human supervision and verification in AI-assisted work.
  65. ↑ Supreme Court of India, Draft Regulations for Use of Artificial Intelligence (AI) in Courts, 2026 (Notice dated 3 June 2026) regs 9, 36–39 https://cdnbbsr.s3waas.gov.in/s3ec0490f1f4972d133619a60c30f3559e/uploads/2026/06/2026060342.pdf accessed 29 June 2026, regulation 9 requires continuous monitoring and periodic technical, legal and ethical audits; regulation 36 provides controlled-environment testing for accuracy, reliability, fairness, explainability, cybersecurity and court-process compatibility; regulation 37 creates an AI Register; regulation 38 requires audits; regulation 39 creates an AI Incident Database.
  66. ↑ Oisin Suttle and David Lillis, ‘Persuadability and LLMs as Legal Decision Tools’ (2026) arXiv:2604.26233 https://arxiv.org/abs/2604.26233 accessed 29 June 2026; Maynooth University, ‘AI “Judges” Can Be Swayed by Better Arguments’ (11 June 2026) https://www.maynoothuniversity.ie/law/news/ai-judges-can-be-swayed-better-arguments-dr-oisin-suttle-conducts-research-highlights-issues accessed 29 June 2026.
  67. ↑ Anand Shah, ‘Access to Justice in the Age of AI: Evidence from U.S. Federal Courts’ (2026) SSRN https://ssrn.com/abstract=6766859 accessed 29 June 2026.
  68. ↑ UNESCO, ‘AI in the Courtroom: UNESCO’s New Guidelines for the Judiciary’ (3 December 2025, updated 8 December 2025) https://www.unesco.org/en/articles/ai-courtroom-unescos-new-guidelines-judiciary accessed 29 June 2026; National Center for State Courts, ‘Guidance for Implementing AI in Courts’ https://www.ncsc.org/resources-courts/guidance-implementing-ai-courts accessed 29 June 2026.
Retrieved from "https://www.lawandjusticewiki.org/w/index.php?title=Red_Teaming&oldid=15676"
Cookies help us deliver our services. By using our services, you agree to our use of cookies.