Deloitte refunds Australian government after errors found in AI-generated report
What's the story
Deloitte has agreed to partially refund the Australian government after errors were discovered in a report that was partly generated using artificial intelligence (AI) technology. The Big Four firm had been contracted for a seven-month project worth A$440,000 (nearly $290,000) to conduct an assurance review of the country's Targeted Compliance Framework (TCF), an IT system used for welfare and benefits payments.
Report discrepancies
Errors in report flagged by academic
The final report of the project, published in July, was found to have several errors. These included academic references to non-existent individuals and a fabricated quote from a Federal Court judgment. The discrepancies were first flagged by Australian welfare academic Chris Rudge. An updated version of the report was later published on the Department of Employment and Workplace Relations's website, correcting these mistakes.
AI involvement
Report revealed use of generative AI
The updated report revealed that Deloitte's methodology "included the use of a generative artificial intelligence (AI) large language model (Azure OpenAI GPT — 4o) based tool chain licensed by DEWR and hosted on DEWR's Azure tenancy." This detail was not included in the original report published in July. A spokesperson for the Department of Employment and Workplace Relations confirmed to Business Insider that Deloitte has agreed to repay the final installment under its contract.
Uncertain impact
Updates do not affect substantive content, says Deloitte
While Deloitte acknowledged its use of AI in the updated report, it has not confirmed whether this technology was responsible for the errors. The company maintains that the updates made do not affect the substantive content, findings, and recommendations in the report. Senator Deborah O'Neill criticized Deloitte over this incident, saying it highlights an over-reliance on AI and calling for government agencies to verify who is actually doing their work.