California prosecutors used AI for criminal filings, leading to errors
Prosecutors in Nevada County, California used artificial intelligence to write at least one criminal court filing, but the AI made up legal citations and misquoted rulings—mistakes known as "hallucinations."
The flawed document was pulled after the errors came to light. Defense lawyers say similar issues showed up in other cases too.
Legal fallout and new rules on AI use
The District Attorney's office admitted using AI in one filing, but denied it was used in the case of defendant Kyle Kjoller.
Still, defense teams pushed back hard, worried that these errors could hurt defendants' rights or fairness in court.
While their request for sanctions was denied by an appeals court, they've taken the fight to California's Supreme Court with support from legal scholars.
In response, the DA's office has rolled out new policies and mandatory staff training on using AI tools responsibly—though it's still waiting on a final ruling from the state's top court.