Model collapse used to be a research term. Now it shows up in AI-risk conversations, usually with the same framing: models trained on the output of other models degrade, the open web is filling with synthetic text, and the companies training frontier models have a data-quality problem on their hands. That framing invites a conclusion I keep hearing from technology leaders: this one is not our problem. We do not train foundation models. We buy them. If a vendor’s training pipeline fills with synthetic sludge, we can switch vendors.
Half of that reasoning holds. You do not control what the labs train on, and no policy you write will clean the open web. The other half fails quietly: the same feedback shape runs inside organizations, one layer down, in systems you do control. Your knowledge base. Your retrieval pipelines. At that layer the defense is not waiting for the labs. It is practice.
One distinction first, because the rest of this argument depends on it and the loose version of it is wrong. What happens in your knowledge base is not model collapse. Model collapse is a specific claim about weights: train a model on the output of earlier models, repeat, and the distribution degrades. Nothing trains in your knowledge base. What degrades there is provenance and grounding, as summaries of summaries drift from what was actually decided and retrieval starts returning the drift. The mechanism is different and the name should be too. What the two share is the structure that causes both: a system consuming its own output as though it were fresh evidence, with no human check at the point of re-entry. That shape is the thing worth defending against, and it is portable across layers even though the failure it produces is not the same failure.
The practice I have in mind is synthesis engineering, the discipline I have been writing about for building production systems with AI while keeping architectural control and human understanding intact. I did not frame it as an answer to model collapse. It turns out to be one anyway.
What the research found
The term comes from a line of research that culminated in a 2024 Nature paper: train generative models on data produced by earlier generations of generative models, recursively, and they degrade in a characteristic way. The tails of the distribution go first. Rare knowledge and unusual cases fade while outputs concentrate around the most probable, most generic center. Each generation compounds the drift of the one before. The researchers’ most practical finding points at the remedy: collapse is held off by continued access to fresh human-generated data.
Last spring I wrote about a related idea: pre-AI content as the new low-background steel. Steel made before atmospheric nuclear testing is prized for radiation-sensitive instruments because everything produced afterward carries traces of the test era, and text written before late 2022 may hold the same position for AI training. That piece was about the ecosystem: what publishers and archives should do about provenance at the scale of the open web. This piece is about the version of the problem you can actually fix, inside your own walls.
The same loop, one layer down
You do not need a training pipeline to run a synthetic-data feedback loop. Three shapes to look for.
AI-drafted documents summarizing AI-drafted documents. A model writes the first draft of a design doc from a meeting transcript. Another model summarizes that doc into a wiki page. A third answers questions from the wiki. Nobody in the chain checked the first draft against reality, and by the third hop nobody can.
Retrieval corpora that ingest whatever exists. RAG gets sold as the fix for hallucination: the model answers from your documents instead of from its memory. That only helps if what the documents say is itself grounded. Index unreviewed AI output and the loop closes inside your retrieval layer, where its results now arrive with citations attached.
Knowledge bases that cite themselves. AI-generated FAQ pages built on AI-generated summaries, while the human sources they once derived from go stale underneath. Each regeneration cycle moves the text further from the person who knew the answer.
Inside an organization, the failure does not look like the gibberish of the lab experiments. It is quieter. Documentation that reads plausibly and says nothing specific to you. Answers that drift toward what is true of companies in general instead of what is true of yours. A useful test for whether the drift has started: pick a page from your internal knowledge base and ask whether it could have been written about any company in your industry. If it could, the mean is winning.
What you control
You control what your systems treat as ground truth. You decide which documents the retrieval index trusts, and what a page has to pass before it becomes the version people and machines rely on. You decide whether an AI draft can become a source of record without a human ever having owned it. These are architecture decisions, and they are yours.
The synthesis engineering framework rests on four pillars: human architectural authority, systematic quality standards, active system understanding, and iterative context building. I framed them for software teams. Read them as principles for governing knowledge and they describe an anti-collapse program.
Context that pulls back
A language model’s prior pulls every answer toward the statistically common center: the average architecture, the average policy, the average company. That pull is what collapse strengthens. The counterweight is context: durable, human-authored records of your actual decisions and constraints, kept where your AI systems read them.
This is iterative context building practiced at the level of the organization rather than the coding session. Project records that say what was decided and why. Architecture notes. A knowledge layer designed for AI to query rather than for humans to file. The more of your reality exists in verified written form, the less room the model has to substitute its average for your specifics. Teams that maintain this notice a second effect: the context compounds. Every corrected record makes the next generated draft start closer to true.
Gates that catch the drift
Collapse amplifies two failures: blandness and confident fabrication. Both are the failures human review exists to catch, which is why the second and third pillars carry so much of the defense.
Systematic quality standards means AI-generated text meets the same bar as human-written text before it ships, whether it is code or documentation. In practice that is a promotion gate: everything a model produces is a draft until a person has read it, corrected it, and taken ownership of it. Only then does it enter the reference layer that other people, and other models, will build on. Label provenance while you are at it: which pages are human-verified and which are AI drafts awaiting review, with the retrieval index respecting the difference by default.
Active system understanding supplies the test for whether the gate is real. For code the test is whether an engineer could debug the system at 2 a.m. For knowledge, the question becomes: when this page turns out to be wrong, can someone trace the claim to its source and fix it there? A document that cannot be traced cannot be repaired, and untraceable text is what the loop mass-produces.
Judgment does not delegate
The first pillar, human architectural authority, generalizes past software: humans decide what is true and what the organization will treat as knowledge. Models draft, summarize, translate, and refactor at a speed no team can match, and none of that output becomes canon on the model’s authority. Someone owns every fact your organization relies on, or nobody does. The part of the work that was always judgment is the part that does not delegate.
Make new steel
In the steel analogy, the clean material is finite. The supply is whatever was forged before the first nuclear test, and instrument-makers salvage it from shipwrecks and old bridges. In the earlier piece I listed the places where the analogy breaks down, and one of them matters most here: unlike steel, humans have not stopped producing the clean material. People in your organization write, decide, correct, and verify every working day.
That break in the analogy is the strategy. An organization practicing this discipline does more than preserve its pre-2022 archive as a vault of uncontaminated text. Every document a person read and corrected this week is new low-background material. Every verified record added to the knowledge base grows the clean corpus. The archive compounds instead of depleting.
The remedy the Nature researchers identified, fresh human data, is scarce at web scale because nobody owns the supply. Inside your organization, you own it. Verification gates are the mill that produces it, and human judgment is the quality mark stamped on each batch. Durable context is the warehouse where it accumulates.
The constraint on how much AI can do for your organization is shifting from the capability of the models to the quality of the material you ground them in. The labs will keep fighting over what remains of the clean open web. You get to run a cleaner operation entirely: keep judgment human, keep the gates honest, and keep making new steel.