Last year I wrote about how to recognize an email written by an AI language model. That post covered 12 telltale signs — generic language, excessive formality, suspiciously perfect grammar. It was useful for spotting default-settings AI output in your inbox.
But emails are just one surface. AI-generated text is everywhere now — blog posts, documentation, marketing copy, product descriptions, news articles, LinkedIn posts. And the patterns that give it away go deeper than “I hope this message finds you well.”
So I built something more comprehensive: an open-source guide that catalogs the patterns systematically, organized for both people who write with AI and people who review what AI helps produce.
The guide: Guide to Identifying and Improving AI-Assisted Content (GitHub, MIT license, open source)
What the guide covers
The Guide to Identifying and Improving AI-Assisted Content is a framework organized into five categories:
- Language and Tone Patterns — importance inflation (“stands as a testament to…”), promotional language (“rich cultural heritage”), editorial commentary (“it’s important to note that…”), the formulaic Rule of Three
- Style and Structural Indicators — excessive em dashes, bulleted lists with bolded lead-ins, mechanical formatting consistency
- Technical and Formatting Tells — placeholder text left in published content, chatbot artifacts (“If you have any questions, feel free to ask!”), broken or fabricated links
- Citation and Sourcing Issues — hallucinated citations (real-sounding sources that don’t exist), vague attribution (“experts say,” “studies have shown”)
- Context-Specific Indicators — industry-specific slop patterns, absence of personal detail, superficial depth without genuine expertise
Version 2.0 has 27 patterns across these categories. Each includes concrete examples and most have before/after comparisons showing what the pattern looks like and how to fix it.
The part that surprised me
When I started cataloging these patterns, I expected to produce a detection checklist — something for reviewers. What emerged was something more interesting: the same framework works in both directions.
If you’re a content creator, understanding what makes text read as AI slop lets you systematically edit it out of your own drafts. Not to hide that you used AI — but because these patterns genuinely make writing worse. “Stands as a testament to” is bad writing whether a human or an LLM produced it.
If you’re a reviewer or editor, the same catalog gives you a systematic vocabulary for what you’re seeing instead of just a vague sense that “this reads like AI.”
The guide frames this as a GAN dynamic — the same mechanism behind Generative Adversarial Networks in machine learning. Generators and discriminators improve each other through competition. As reviewers get better at spotting lazy AI output, creators are forced to produce higher-quality work. As creators get better, reviewers need sharper tools. The floor rises for everyone.
The end state is not some arms race where AI learns to evade detection. It’s a world where the minimum standard for published content goes up. That’s a good outcome regardless of how the text was produced.
What does not work
The guide also catalogs detection approaches that are unreliable, which is just as important as knowing what works:
- Perfect grammar does not indicate AI. Skilled writers and professional editors produce flawless text all the time.
- “Bland” prose does not indicate AI. Corporate communications have always sounded robotic. (Ask anyone who’s read internal memos.)
- Common phrases do not indicate AI. Real humans write “rich cultural heritage” and “moreover” without any machine assistance.
- Vocabulary blacklists — flagging words like “delve” or “tapestry” — produce false positives constantly and are trivially evaded.
- Perplexity scoring alone cannot reliably distinguish AI from human text, especially for edited or domain-specific content.
Single indicators prove nothing. Detection requires recognizing clusters of patterns across a piece — which is why the guide includes a four-tier confidence framework (high, medium, low, unreliable) to help prioritize which signals matter most.
A before and after
Here is an example from the guide. AI output first:
The conference was a resounding success, bringing together industry leaders, innovators, and thought leaders for three days of engaging discussions. Attendees praised the event for its comprehensive programming, networking opportunities, and inspiring keynotes. The event stands as a testament to the organization’s commitment to fostering collaboration and driving innovation in the field.
Count the patterns: importance inflation (“resounding success,” “stands as a testament”), Rule of Three (twice — “leaders, innovators, and thought leaders” plus “programming, networking, and keynotes”), promotional language, zero specifics.
Now the human revision:
About 400 people showed up, which surprised the organizers who’d planned for 250. The keynote on supply chain automation ran 20 minutes long because the Q&A wouldn’t stop. I overheard two CTOs in the hallway comparing notes on the same vendor pitch — turns out neither was buying. The real value was in the unscheduled conversations: I came away with three potential partnerships and one job lead I hadn’t expected.
Specific numbers, a personal observation, an overheard detail, a concrete outcome. No one would mistake this for AI output. The difference isn’t talent — it’s specificity.
How to use it with any LLM
The guide is a markdown file. You can paste it into any LLM — Claude, ChatGPT, Gemini, Llama, whatever you use — and ask it to review content against the framework. No proprietary software, no API keys, no subscription.
Here are prompts that work:
Self-editing check after an AI-assisted draft:
Review this article against the AI Content Quality Guide. Flag every instance where the text exhibits patterns from the guide — importance inflation, Rule of Three, promotional language, vague attribution, formulaic transitions, or any other cataloged pattern. For each flag, quote the specific text and suggest a concrete revision. Be thorough — I want to catch everything before publishing.
Editorial review for a team:
You are a senior editor. Using the AI Content Quality Guide as your rubric, evaluate this submission. Rate the overall AI-pattern density (low/medium/high), identify the top 5 most concerning patterns with specific examples, and provide a recommendation: publish as-is, revise and resubmit, or reject. Explain your reasoning.
Quick confidence assessment:
Scan this text using the Detection Confidence Framework from the AI Content Quality Guide. List any high-confidence indicators first, then medium, then low. How many medium-confidence indicators cluster together? What is your overall assessment of whether this text was likely unedited AI output?
Content improvement (not just detection):
I wrote this with AI assistance. Using the AI Content Quality Guide’s revision process, help me eliminate AI patterns while preserving my intended meaning. For each change, explain which pattern you’re addressing. Focus especially on adding specificity, personal voice, and genuine expertise where the text currently sounds generic.
How it compares to AI detectors
Tools like GPTZero, Originality.ai, and Copyleaks approach the problem differently. They use statistical models trained on AI-generated text to produce a probability score. They have their place, but they also have well-documented limitations: false positives on non-native English speakers, inconsistent results across runs, and an inherent lag as new models are released.
This guide takes a different approach. Instead of a black-box score, it gives you a transparent catalog of specific patterns with explanations of why each one matters. You can understand every assessment it produces because the reasoning is visible. It works with any LLM as the analysis engine — not just one vendor’s API. It’s open-source under the MIT license, so you can adapt it to your organization’s specific needs, add patterns relevant to your industry, or integrate it into your editorial workflow however you see fit.
The approaches are complementary, not competing. Use a statistical detector for initial screening if you want. Use this guide for the deeper, more actionable analysis: understanding exactly what needs to change and why.
Get the guide
The full guide is on GitHub: Guide to Identifying and Improving AI-Assisted Content. MIT license. Use it, fork it, adapt it.
It’s part of the ai-knowledge-ragbot repository — a collection of open-source runbooks and frameworks for working effectively with AI tools.
Rajiv Pant is President of Flatiron Software and Snapshot AI, where he leads organizational growth and AI innovation. He is former Chief Product & Technology Officer at The Wall Street Journal, The New York Times, and Hearst Magazines. Earlier in his career, he headed technology for Conde Nast’s brands including Reddit. He writes about AI, engineering leadership, and the evolving relationship between human expertise and machine capabilities. Connect with him on LinkedIn or read more at rajiv.com.