KPMG Withdraws “Redefining Excellence in the Age of Agentic AI” Report Over Hallucinations

KPMG withdrew its report titled “Redefining Excellence in the Age of Agentic AI” after researchers identified inaccuracies, including “questionable citations” and case studies that appeared to contain AI-generated hallucinations.

“A new probe into Big Four KPMG’s report on agentic AI found that the majority of its references were flawed, amid the latest news of AI-hallucinated reports published by professional services firms”

City AM

The report, released in October, examined how organisations were allegedly using AI agents to automate and manage complex tasks, and it claimed UBS had integrated AI agents into investment advisory, risk management and compliance systems through a customised platform.

City AM

GPTZero’s review found that only five of the report’s 45 citations correctly matched the sources they referenced, while the remaining citations were described as misleading, partially fabricated or difficult to verify.

KPMG removed the report from its websites and launched an internal review, with a company spokesperson saying, “The report has been removed and we are reviewing the circumstances surrounding its publication.”

The organisations named in the KPMG report challenged its assertions, with UBS reportedly describing the claims as “factually incorrect” and seeking their removal.

Swiss Federal Railways said the description of its AI usage was not accurate, while a spokesperson for Transport for London characterised the claims as misleading.

Engadget

The dispute extended to the report’s NHS Greater Manchester example, which Futurism said was based on a press release about an AI tool for lung cancer that had “nothing to do about an AI agent performing the above hospital tasks.”

GPTZero CEO Edward Tian warned that inaccurate reports from widely trusted organisations can spread quickly after being cited by media outlets and other publications, potentially amplifying misinformation, telling the Financial Times that error-riddled publications can “poison the well of information.”

KPMG said it takes “accuracy and integrity of its published content seriously,” and it expected employees to follow guidelines on responsible AI that include human oversight and source verification.

“Accessibility helpSkip to navigationSkip to main contentSkip to footer > KPMG report contained AI hallucinations on benefits of”

Financial Times

PCMag reported that the now-deleted report was cited in responses generated by OpenAI’s ChatGPT and Google Gemini, and it said the report had 40 citations that appeared to be at least partially AI hallucinations out of the 45 in the entire document.

The same PCMag account described how researchers found that many citations had incorrect titles, authors or dates, and that researchers were unable to match many citations to an existing study due to a lack of information.

Against that backdrop, GPTZero’s Edward Tian warned that the problem can spread beyond the original article, with TechRadar saying “vibes have consequences” as KPMG’s influence raises the risk that its findings are “likely… cited globally.”