AI Content Risks

AI tools have made content production faster and cheaper. Used well, they can help with research, drafting, and editing. Used recklessly (as a pipeline for publishing at scale without genuine editorial input), they introduce risks that tend to compound over time: algorithmic demotion, loss of trust signals, and reduced visibility in the AI-generated answers they were often deployed to target.

What does Google’s spam policy say?

Google’s position is frequently misquoted. The policy is not that AI-generated content is prohibited. It is that content produced primarily to manipulate search rankings, regardless of how it was created, violates the spam policy.

The relevant category is “scaled content abuse”: generating large volumes of content, with or without AI, where the primary purpose is ranking rather than serving readers. A site publishing 300 AI-generated articles on loosely related topics, with thin coverage and no meaningful editorial review, falls under this policy whether or not the content is technically accurate.

The distinction that matters is intent and quality, not the tool. AI-assisted content, where a subject matter expert drafts with AI help, reviews the output, and publishes under their name, sits in a different category entirely from an automated pipeline publishing content at scale.

Hallucination and factual accuracy

AI language models generate text that is confident, well-formatted, and sometimes wrong. This is the hallucination problem: the model produces a plausible-sounding claim that is factually incorrect, and the error is often invisible to anyone without domain knowledge.

For general topics, a hallucinated sentence might be embarrassing. For specialist content - SEO, health, finance, legal - it can be a material trust liability. A page stating an incorrect algorithm date, a misattributed statistic, or a tool feature that no longer exists damages credibility in ways that are hard to recover from, particularly on a site making authority claims.

The risk is not that AI writes poorly. It is that AI writes poorly in a way that passes a surface-level read. Catching hallucinations requires someone who knows enough to know what to check, which is the domain expertise that a purely automated content pipeline typically removes from the process.

Outdated training data

AI models have knowledge cutoffs. They write about the world as it was when their training data was collected, not as it is now. For stable topics this is a minor issue. For anything that changes regularly - Google features, algorithm behaviour, tool interfaces, crawler support - the gap between training data and current reality can be significant.

SEO content is particularly exposed. AI Overviews roll out and change behaviour month by month. Google Search Console adds and removes reports. Third-party tools update their interfaces. An AI-drafted article about any of these topics, published without review by someone tracking current developments, may be factually incorrect from day one and progressively more so as time passes.

The fix is a review layer with genuine current knowledge. Proofreading is insufficient; the reviewer must understand the domain well enough to catch errors the writer didn’t see.

What happens when AI content becomes AI’s source?

Hallucination is usually described as one model inventing one fact. The compounding problem is what happens next, when that output becomes a source other models retrieve.

This is not hypothetical, and it is already well advanced on both sides of the loop.

On the supply side, Ahrefs sampled 900,000 newly published pages in April 2025 and found 74.2% contained at least some AI-generated content. The detail matters more than the headline: only 2.5% were written entirely by AI, while 25.8% were purely human and the remaining majority were a mixture.¹ Most “AI content” on the web is a human-machine blend, not pure machine output, and claims that three-quarters of the web is AI-written misread the finding.

On the retrieval side, Graphite estimated that 42.7% of the references ChatGPT cited in June 2026 were themselves AI-generated, up from 38.9% five months earlier.² Read that figure with one caveat the study itself depends on: the classification was made with an AI-content detector, and detectors are the weakest instrument in this field, prone to flagging human writing as machine-written. The direction of travel is credible and the precise share is not. On any reading, a substantial and growing slice of what the model treats as evidence is machine-written.

Keep two very different things apart when reading any claim about how much content is AI-written. A third-party detector makes a statistical guess from style, which is why its output is unreliable at the level of an individual page. Provider-side marking is not a guess at all: the system that generated the text inserts a signal deliberately. That second category is arriving under regulatory pressure rather than voluntarily. Article 50 of the EU AI Act, which applies from 2 August 2026, requires providers of generative systems to ensure output is “marked in a machine-readable format and detectable as artificially generated or manipulated”,³ and Anthropic has published plans to watermark all text Claude generates, worldwide rather than only in the EU.⁴ Google’s SynthID and C2PA Content Credentials are the same category of signal arriving by a different route.

Two limits stop this rescuing the statistics above. Detection tooling for these marks is mostly unpublished, so nobody can act on them yet. And the marks answer a narrower question than people assume: Anthropic states that a detected mark means content “may have been processed by Claude”, which covers proofreading and translation of human writing, while unmarked content may still be machine-written because marks do not survive heavy editing or format conversion.⁴ Provider marking will make provenance more knowable than a detector ever could. It will not produce a clean split between human and machine text, because most content is neither.

Under simulation, that tips into convergence: AI answers collapsed onto the same homogenised result in 79.6% of runs (1,216 of 1,528 simulations across 1,019 questions and three model providers) once AI-generated references entered the retrieval pool.² The researchers are careful about what this does and does not show, and so should we be: they state they “do not definitively prove that AI search collapse is already happening”. It is a demonstrated mechanism, not a measured state of the live web. The consequences for citation strategy are covered in Study: AI search collapses onto identical answers when models cite AI content. The same loop also propagates errors, which is the subject here.

An AI-drafted article publishes an unverified claim. The article gets indexed. The next model retrieves it alongside others repeating the same claim, all tracing back to the same unchecked origin, and reports it as settled fact rather than a contested assertion, because from the inside it looks like consensus.

The underlying problem is that volume of agreement reads as corroboration when it is not. Ten sources saying the same thing are worth no more than one if all ten copied that one, and a search summary across them cannot tell the difference.

A worked example. In 2025, a research paper described “360Brew”, a large language model for ranking, written by authors from LinkedIn. The paper was later withdrawn, with arXiv removing every version because the submitter did not have the right to agree to the licence.⁵ LinkedIn’s VP of Engineering then addressed the claim directly. Asked whether 360Brew was used in ranking, he answered: “Short answer: No!”, explaining that the model had been tested with a small group of members, judged not the right fit, and the test shut down.⁶

None of that stopped it. Marketing articles continued to announce 360Brew as LinkedIn’s new algorithm, AI tools retrieving from those articles repeated it as fact, and practitioners built ranking advice on top of it.⁷ The primary source had been retracted and the platform had publicly denied the claim, yet the derivative layer registered neither, because nothing in that layer checks.

Two failure modes follow from this, and both are checkable.

Retractions do not propagate. When a paper is withdrawn or a company corrects the record, the correction lands on the original source. The articles, summaries and model outputs already built on it are not revisited, so the claim continues to circulate in the derivative layer after its basis has gone.

Models fabricate citations. Asked to source a claim, a language model will produce a plausible URL that has never existed, assembled from the pattern of real URLs on that domain. A fabricated citation is indistinguishable from a genuine one until it is opened, which is why opening it is the only check that catches it.

How do you defend against it?

Three habits, all mechanical rather than attitudinal, because general scepticism is easy to profess and hard to apply.

Check what the model is sourcing, not only what it is saying. A confident answer with no source, or with a source nobody has opened, is not evidence.

Open every URL before citing it. This catches fabricated references, which no other check will.

Treat heavy repetition of a specific claim as a reason for more scrutiny, not less. Widespread agreement about a named internal detail, an algorithm, a model, a system, that appears only in marketing content and never in a primary source, is a signal to trace the claim back to its origin and confirm the origin still stands.

The practical consequence for anyone using AI in an SEO workflow is that these tools are not only capable of being wrong. They distribute other people’s errors efficiently, in confident and well-formatted prose that carries no indication of whether it has been checked.

E-E-A-T erosion

Experience, expertise, authoritativeness, and trustworthiness are assessed at the page level and the site level. Content produced at scale without named authors, without first-hand experience, and without genuine expertise behind it is structurally weak on every E-E-A-T dimension.

A site that publishes 500 articles with no bylines has made a choice to trade long-term authority for short-term output. Google’s quality systems are designed to identify this pattern. More practically, AI retrieval systems - the same surfaces that GEO aims to influence - are trained on quality signals that correlate closely with E-E-A-T. Content from named, credible authors on sites with demonstrated topical depth is more likely to be cited than anonymous content from a site with broad but shallow coverage.

The core irony

The sites most commonly targeted by AI content strategies - AI Overviews, Perplexity, ChatGPT Search - retrieve and cite based on: factual accuracy, clear attribution, verifiable expertise, and well-sourced claims. These are the signals that reckless AI content production systematically fails to provide.

A site publishing AI-generated content at volume, without meaningful human review, is producing exactly the kind of content that AI retrieval systems are trained to pass over. The strategy sold as a way to win AI search visibility tends to reduce it.

This is not a coincidence. AI retrieval systems learn from the same quality signals as traditional search. The content that earns citations is the content that would have ranked well anyway.

What does responsible AI-assisted content look like?

The question is not whether to use AI tools. It is what the human layer looks like.

Content produced with AI assistance and published responsibly typically involves: a subject matter expert directing or reviewing the output, fact-checking against current sources rather than relying on the model’s training data, a named author with genuine credentials, and editorial judgment about what to publish and what to cut.

The practical test: would a knowledgeable person in the relevant field put their name to this and stand behind it? If not, the content is not ready to publish, regardless of how it was produced.

AI tools work well for first drafts, research summaries, structural suggestions, and editing passes. They work poorly as a replacement for the expertise and judgment that determines whether a piece of content is worth publishing.

The same split applies to the SEO work around the content. Using ChatGPT for SEO is reliable for organising, summarising and drafting; it is unreliable for anything that requires live data, because the model will produce confident keyword volumes and rankings it has no way to know.

Frequently asked questions

Is AI-generated content against Google’s guidelines?
Not inherently. Google’s policy targets content produced primarily to manipulate rankings, regardless of the tool used. High-quality AI-assisted content, reviewed and published by a named expert, is not in conflict with any Google guideline. Mass-produced, low-quality AI content published at scale is.

How can I use AI tools without risking penalties?
Treat AI as a drafting and research aid, not a publishing pipeline. Have a subject matter expert review every piece before publication. Fact-check against current sources. Publish under a named author with verifiable credentials. Maintain topical depth rather than breadth: cover a defined area well rather than everything superficially.

Does AI-generated content affect E-E-A-T?
Only if it lacks the signals E-E-A-T depends on: named authorship, demonstrated expertise, first-hand experience, and factual accuracy. Those signals can be present in AI-assisted content if a genuine expert is involved in producing and reviewing it. They are typically absent from fully automated content pipelines.

Footnotes

1. What percentage of new content is AI-generated? — Ahrefs ↩

2. AI search collapse: AI responses collapse when AI retrieves its own generations — Graphite ↩ ↩²

3. Article 50: Transparency Obligations for Providers and Deployers of Certain AI Systems — EU Artificial Intelligence Act. Article 50 sits in Chapter IV, which applies from 2 August 2026. ↩

4. How Claude marks AI-generated content — Anthropic ↩ ↩²

5. 360Brew: a decoder-only foundation model for personalized ranking and recommendation (withdrawn) — arXiv ↩

6. Is 360Brew used as part of your ranking system? Short answer: No — Tim Jurka, VP Engineering, LinkedIn ↩

7. LLMs and combatting misinformation about LinkedIn’s algorithm — Phil Szomszor ↩