Study finds 13,000 phrases that could reveal AI-generated writing
What's the story
As the use of AI-generated text becomes more common, people are looking for ways to identify it. A new study by marketing firm Graphite has revealed that even advanced models have certain telltale habits in their writing. These "tells" are specific words and phrases that are more frequently used in AI-generated content than human-written prose.
Research findings
Graphite's study examined frontier models' writing habits
Graphite's study examined the writing habits of frontier models, identifying their preferred words and phrases.
The researchers found that while some early indicators of AI writing, like em-dashes and "delve," have disappeared, contrast-heavy constructions still remain.
Each model version has its own unique quirks, with a surprising range of tells discovered.
The team identified 13,000 phrases that were at least twice as common in AI content than human content, what they define as a "tell."
Model comparison
Comparing AI and human writing
The study also found that Anthropic's Claude models are getting closer to human word distribution over time, while OpenAI's GPT models are moving further away.
To compare AI and human writing, Graphite used a corpus of 10,000 pre-ChatGPT articles as a control group.
They had different AI models rewrite these articles from summaries to eliminate source bias as much as possible.
This way, they could compare the frequency of certain words and phrases in both human and AI writing.
Model traits
OpenAI's Astra model often hedges claims with specific phrases
The study found Claude Opus 5.5 overuses word "dependable," appearing 23 times more than in human samples. It also prefers phrases like "this matters" and "why X matters."
Meanwhile, OpenAI's Astra model often describes an "another dimension" of a topic and hedges claims with phrases like "may provide" or "can provide."
Its biggest tell is what Graphite calls "corrective framing," where a topic is defined as "not simply X" or offered as an alternative, "rather than relying on X."
Punctuation patterns
Labs have worked to reduce em-dash usage
Interestingly, all frontier labs have acknowledged the issue of overusing em-dashes in their models.
In Graphite's samples, Opus 5.5 used the punctuation mark 99% less than its predecessor. Astra now uses it 88% less than human samples, while Gemini 3.1 Pro has almost completely removed it from its writing.
Despite these individual changes, Graphite says the overall number of tells remains mostly unchanged across different model versions.
Ongoing challenge
Do labs really need to worry about this?
Despite their efforts to create human-like writing styles, AI labs are still struggling with the persistence of tells.
Anthropic claimed that its Opus 5.5 model "communicates more naturally than prior models," while OpenAI made similar claims for its GPT-6 versions of Sol and Luna.
However, Graphite's chief AI officer Greg Druck is skeptical about how much these labs can do to completely eliminate telltale constructions or phrases due to the complexity of their models with billions of parameters.