Algorithmic Bias and Accountability
RAS Mains, Paper II, Unit I, point 7: "Artificial Intelligence (AI) versus conscience in administrative decision making"; closely linked to point 8: "accountability in governance"
- Updated for the Jan 2026 syllabus
- Read time: about 8 minutes
Exam weight (our count from real 2018-2024 questions)
0 direct questions (new topic). Nearest: 2021 Part C Q12 (ethical vs administrative decision) and 2023 Part C Q13 (moral dilemma). Related UPSC question: GS Paper 4, 2024, Q1(a): "The application of Artificial Intelligence as a dependable source of input for administrative rational decision-making is a debatable issue." (number and wording from secondary sources; match against the official paper). This is our count, not an RPSC notice. The full topic frame, the six principles and two hypothetical cases are on page 7.6; this page is a short, fast revision on bias and accountability.
30-second summary
Remember
- Algorithmic bias: a repeated, unfair tilt in AI results, usually coming from incomplete or skewed data and design choices. "Biased data in, biased result out."
- Bias enters by five routes (our grouping): old history in the data, groups missing from the data, wrong proxies, design and threshold choices, and feedback loops with blind trust in the machine.
- Documented cases: the Dutch child-benefit scandal (cabinet resigned 15 January 2021), Amazon's hiring tool (Reuters, 2018), a US health algorithm (Science, October 2019), the Gender Shades study (2018).
- Accountability means every automated decision has a named human owner who can give reasons, hear appeals and correct errors.
- The formula stays the same: "AI proposes, a responsible human decides" (page 7.6).
Part A box (every line 15 words or fewer)
Remember
- Question: What is algorithmic bias?
- Answer: Systematic unfairness in AI output, usually inherited from skewed data or design choices. (13 words)
- Ready line 2: Accountability means a named human owns every automated decision and answers for it. (13 words)
- Ready line 3: A PIN code or surname proxy can carry caste or religion bias without naming it. (15 words)
1. Terms
| Term | Meaning |
|---|---|
| Proxy | A data point that quietly stands in for something sensitive (for example PIN code, surname, past spending) |
| Automation bias | Trusting the machine's answer over one's own judgment |
| Feedback loop | People flagged by the machine are checked more, more faults are found, and the machine appears "right" |
| False positive / false negative | Wrongly flagging an innocent person / missing a guilty one; in welfare, a wrongful denial usually costs far more |
| Explainability | Being able to give the reason in plain words |
2. How bias enters (our five routes)
1. Historical bias: if past decisions were unfair, the machine learns and repeats them at scale.
2. Missing or thin groups: if some groups are scarce in the training data, errors for them are higher.
3. Wrong proxy or target: using "spending" as the goal instead of "health need".
4. Design and threshold choices: where the "suspicious" line is drawn, and who carries the burden of false positives.
5. Feedback loops and automation bias: "human oversight" exists on paper, but the officer signs without thinking.
3. Fairness is contested
In the US COMPAS case, ProPublica (23 May 2016) reported a racial gap in false-positive rates; the maker disputed this. The dispute comes down to two different definitions of "fair". The Wisconsin Supreme Court (July 2016, Loomis) allowed the score with cautions about its limits (page 7.6). Lesson: decide first what "fair" means, and test errors group by group.
4. Documented cases
| Case | What happened | Lesson |
|---|---|---|
| Dutch child-benefit scandal | Between 2005 and 2019, about 26,000 parents were wrongly accused of fraud. The Dutch Data Protection Authority found that (dual) nationality was used as a risk indicator, so applications were automatically risk-flagged; the tax administration was fined 2.75 million euros (December 2021). Parliamentary report "Unprecedented Injustice" (17 December 2020); cabinet resigned 15 January 2021 | A model built on a sensitive attribute plus blind compliance means harm at scale |
| US health algorithm (Obermeyer and others, Science, October 2019) | The study found that using health spending as a proxy for health need made Black patients systematically look less "at risk", even though at the same score they were sicker | A proxy that looks sensible can still carry bias |
| Amazon's hiring tool | Started in 2014; trained on 10 years of applications (mostly from men); applications containing "women's" and graduates of two women's colleges were marked down; bias recognised by 2015; per Reuters (10 October 2018) the team was disbanded by early 2017 | Historical bias moves into the machine; fixes can leave new routes open |
| Gender Shades (2018) | In the facial-analysis systems tested, error rates were up to about 35 percent for darker-skinned women and under 1 percent for lighter-skinned men | Group-wise testing is essential |
Indian "reported" references (Haryana Parivar Pehchan Patra, reports of stopped pensions) are on page 7.6, with the "it is reported" caution.
5. Accountability: who answers to whom
Policy-maker: where AI may be used and where not (human mandatory for adverse and high-risk decisions).
Department head (the user): the decision to deploy, choice of targets, monitoring and the grievance system.
Signing officer: apply own judgment; "the system said so" is no defence (page 7.6: Nuremberg IV, Arendt).
Vendor: transparency, test data and audit access.
Accountability tools (our list): named owner · decision log · reasons in plain words · right to hearing and appeal · independent audit (group-wise error) · error reporting and a stop button · procurement terms giving access to model and data checks.
Legal and policy base (see page 7.6): Maneka Gandhi (1978): procedure must be just, fair and reasonable; Puttaswamy (24 August 2017): privacy and proportionality; DPDP Act 2023 Section 8(3) (completeness, accuracy and consistency of data used in decisions) and Sections 11 to 13 (information, correction, grievance); NITI Aayog responsible-AI document (February 2021); MeitY AI governance guidelines (5 November 2025), which include accountability and "People First" (human oversight). The DPDP Act has no right against solely automated decisions like Article 22 of the European GDPR (page 7.6).
6. The officer's five-question checklist (ours), before adopting a tool
1. What data was it built on? Who is missing?
2. Whom could it wrongly flag, and who bears the burden of false positives?
3. Can I give the citizen the reason in plain words?
4. Where will the citizen appeal, and within how many days is the hearing?
5. How, and how often, will we measure group-wise error?
How to use it in an answer: definition → one route of entry (proxy or history) → one documented case (Netherlands or Obermeyer) → three accountability tools → "AI proposes, the human decides".
Hypothetical case
No separate case on this page; both hypothetical cases on page 7.6 ("the pension risk score" and "signing hundreds of files") are solved there on both bias and accountability.
Thinkers and quotes
| Name | Idea | Usable line |
|---|---|---|
| Hannah Arendt (1963) | Harm from thoughtlessness | "the fearsome, word-and-thought-defying banality of evil" |
| Nuremberg Principle IV (1950) | An order is not a full defence | "provided a moral choice was in fact possible to him" |
| Commissioner Holmes, Robodebt report (7 July 2023) | Automated harm, ignored warnings | "venality, incompetence and cowardice" |
| Mahatma Gandhi | The inner voice | "The 'still small voice' within you must always be the final arbiter when there is a conflict of duty." (Young India, 4 August 1920) |
Diagram: pipeline, leak points and checks
Data
Node (grey): · leak: · check
historical bias, missing groups
data review
Model and target
Arrow. Node (blue): · leak: · check
wrong proxy, threshold choices
group-wise error testing
Score or recommendation
Arrow. Node (blue): · leak: · check
opacity
plain reasons, decision log
Officer's decision
Arrow. Node (orange): · leak: · check
automation bias
human review of adverse decisions, written reasons
Impact on citizen
Arrow. Node (navy): · check
appeal, grievance, independent audit, feedback
AI proposes; a responsible human decides.
Footer strip
No question by this name (2018, 2021, 2023, 2024).
Earlier question on this concept: none found (new syllabus point). Our prediction (not an RPSC notice): Part A "What is algorithmic bias" or "human-in-the-loop"; Part B sources of bias or accountability.
Common mistakes
Remember
- Assuming the machine is neutral, or the opposite: writing a total ban on AI.
- Writing "remove the sensitive attributes and bias ends"; proxies and historical bias remain.
- Blaming only "the company" or only "the machine"; final responsibility also lies with the human who decides.
Self-check (5 questions)
Remember
- Define algorithmic bias.
- Name any three routes by which bias enters.
- What lesson does the Dutch case teach (with year)?
- Name three accountability tools.
- Why is "the system said so" no defence?
Practice Questions
Practice questionabout 50 words
How can an algorithm be biased even if it uses no data on caste, religion or gender?
Model answer
Yes, through proxies: PIN code, surname or past spending can stand in for caste, religion or class. Training on past biased decisions makes the machine repeat the same injustice. A wrong target also harms: in a US health algorithm (2019), treating spending as need left Black patients behind. Hence group-wise testing is essential.
Practice questionabout 100 words
If an AI tool used by a government department wrongs a citizen, who should be accountable? Suggest safeguards.
Model answer
Accountability is layered. The department head answers for deployment, targets and the grievance system; the signing officer for own judgment, since "the system said so" is no defence; the vendor for transparency and audit access. The Dutch child-benefit scandal (cabinet resigned, 2021) showed that blind compliance is costly. Safeguards: a named owner, decision log, plain-language reasons, human review of adverse decisions, right of appeal and independent group-wise audits. In the end the final decision must stay with an accountable human, and the citizen must get a hearing. The just, fair and reasonable procedure demanded in Maneka Gandhi (1978) requires exactly this.
Real past questions
2021NearestPaper II, Part C, Q1210 marks
Explain the factors essential in 'Ethical decision' making. In case of ethical decision being against to administrative decision, how will you harmonise them? Explain with examples.
2023NearestPaper II, Part C, Q1310 marks
What do you mean by moral dilemma? Should personal ethics take precedence over professional ethics in a situation of moral conflict?
How to Write This Answer
Introduction
define bias
Body
- routes, case, layers of accountability
Conclusion
human owner, appeal, audit
Value addition
- Arendt, Nuremberg IV, Maneka Gandhi
One-page Revision
- Bias = unfair tilt; biased data in, biased result out
- five routes: history, missing groups, wrong proxy, threshold choices, feedback and automation bias
- Netherlands (resignation 15.1.2021; fine 2.75 million euros, Dec 2021)
- Obermeyer (Science, Oct 2019): spending vs need
- Amazon (Reuters 10.10.2018)
- Gender Shades 2018
- COMPAS: two definitions of fairness
- accountability: named owner, log, reasons, appeal, audit, stop button
- Maneka Gandhi 1978
- DPDP 8(3)
- NITI Feb 2021
- MeitY 5.11.2025
- "AI proposes, the human decides"
- cases: page 7.6.