Case 9.19: The order written by AI

Difficulty: Medium
Syllabus point: 7: AI versus conscience
Values and laws tested: Ownership of the decision, natural justice (a hearing and a reasoned order), confidentiality, verification; Conduct Rules 13 (unauthorised communication of information) and 3(1); Nolan's "accountability"

Illustrative exampleThe order written by AI

Sanjana is a Sub-Divisional Officer in a district of Rajasthan with 60 revenue appeals pending. There is pressure from above to dispose of them. She has an order drafted with a public AI tool, entering the parties' names and survey numbers as well. One case is about a claimed right of way; the tool's draft rejects the claim and cites "a Supreme Court judgment" and a "section". Sanjana looks for them and finds neither that judgment nor that section with such wording. The draft looks very professional and confident, and it would save hours. Her reader says, "Madam, everybody does this, just sign." One party has not yet been given a full chance to state its case at the hearing.

Stakeholders: both parties, especially the one who did not get a full hearing (the weakest party), Sanjana, the reader and office, the seniors, the department, and the public's trust in the system.
Issue: speed and relief from workload against a correct, just and reasoned decision; the convenience of AI against the decision-maker's conscience and responsibility; also the confidentiality of the parties' information.
Options: A. Sign the draft (speed; but unverified citations, an incomplete hearing, the risk of the order being set aside and of her own answerability). B. Stop using AI altogether (safe; but a heavy workload and the loss of legitimate help). C. Use AI only as an aid for structure or language, check every citation against an official source, not enter the parties' private information in an external tool, complete the hearing and write her own reasons, and drop any citation she is unsure of (it takes time; but the decision is true and her own).
Best: C. The principle of natural justice demands a hearing for both parties and a reasoned order; Nolan's "accountability" says responsibility for a decision lies with the decision-maker, not the tool. In our reading, putting the parties' information in an outside tool may go against Conduct Rule 13.

Lesson

Action: Short term: complete the hearing, check citations against official sources, write the order on her own reasons, tell a senior of the risk from the information already entered in the tool, instruct the reader. Long term: a departmental AI-use instruction (what may not be entered, what help is allowed, verification compulsory), a secure government tool, training, disclosure of use.

Practice Questions

Practice questionabout 100 words

(10 marks, 100 words) What should Sanjana do? Solve with the five-step card.

Model answer

The weakest stakeholder is the party denied a full hearing; the other party, the office and public trust matter too. The issue is speed and convenience against a just, reasoned decision. Three routes: sign the draft, drop AI, or keep AI as an aid and own the decision. The first rests an order on unverified citations; the second adds to the burden. I choose the third, because natural justice demands a hearing and reasons, and Nolan's accountability rests on the decision-maker. I would complete the hearing, check citations in official sources and keep private data out. Long term: an AI-use instruction.

Practice questionabout 50 words

(5 marks, 50 words) Who remains responsible for the decision when an AI drafts it? Answer from the case.

Model answer

The responsibility is always the decision-maker's; the tool is an aid and can be wrong with confidence.

Practice questionabout 50 words

(5 marks, 50 words) Suggest long-term rules for the use of AI in quasi-judicial work.

Model answer

What may not be entered, compulsory verification, a secure government tool, training, disclosure.