Generative AI makes content and makes a mess. What to do? Use a tried-and-tested content production system.
I think most people who read about AI know that there’s a phenomenon called “vibe coding”, which is when people prompt AI tools to generate code rather than writing it by hand.
Now, there’s a new twist on that: vibe citing – when AI-hallucinated citations end up in academic papers (or, as in South Africa’s case, in government policy documents).
The term “vibe citing” was coined by an engineer at GPTZero, a company which makes AI detection software. In a May 2026 article, GPTZero says that after it began investigating the question of vibe citations, it found them in a government publication, reports by major business consulting firms and in papers at artificial intelligence conferences. The conclusion: “What we’ve found suggests that the vibe citing epidemic is already endemic, even among the major players.”
It’s worth noting that the company has skin in the game – one of their products is a Hallucination Check tool. But there’ve been enough reports around the globe to suggest that they ain’t wrong. French lawyer and data scientist Damien Charlotin has a database that tracks legal decisions in cases where generative AI produced hallucinated content – typically fake citations, but also other types of AI-generated arguments. At the time of writing, he’d found 1,624 instances.
Why citations (aka listing sources or referencing) are important
The University of Leeds puts it succinctly: “Referencing is an important part of academic work. It puts your work in context, demonstrates the breadth and depth of your research, and acknowledges other people’s work. You should reference whenever you use someone else’s idea.”
What holds for academic work holds for reports of all kinds. The key phrase in that Leeds definition? Demonstrating your research – if a consultancy white paper or government policy or news report has references or links to sources that are made up, what does that mean about the research that went into the document? Was the text all researched and generated by an AI tool? If so, does the person or group claiming to be the author have the right to make that claim? And so on.
In the case of the South African’s government’s AI policy (now withdrawn) and those fabricated references, the question that arises is this: does the policy represent the work of the government we elected to do this kind of thing, or does it represent the thoughts of a disembodied, non-human artificial intelligence?
But what to do beyond hand-wringing?
I’m with journalist and AI trainer Lloyd Coutts on this: none of this is a surprise (read his thoughts here: Yes, AI hallucinates, can we please take it into account and move on?).
It’s well-known that GenAI makes things up; it’s also known that it has its uses. The question is: what do we do about things like vibe citing? The answer lies in governance and systems.
Coutts has good guidance on the question of AI governance in institutions, suggesting that:
AI use inside organisations is already moving fast… Managers are approving tools without always knowing where data goes, how outputs are checked, or who carries responsibility when something fails. Organisations can start now by setting the rules. Name the owners. Classify the data. Check the outputs. Keep the evidence. The tools may be new but the management problem is not. When AI output becomes organisational output, the buck still stops with people.
How to manage content flow and deal with hallucinations?
I’d like to suggest that there’s a time-honoured way of checking the veracity of content, which could be used to help prevent AI-generated messes: the production process used by news organisations. I’ve written about that at some length here. I’ve tried to outline the process as simply as possible below, with some thoughts on wider applicability (there’s a more detailed version to download, too). Some things to bear in mind: table:
- In the news production process, the question of who did what to a piece of content as it moves from person to person or department to department is usually handled by a content management system (CMS).
- Statuses are attached to a piece of content as it moves through the system, and changes made are digitally recorded.
- In organisations in which there is no CMS, a system of files and folders could be used, along with indications in the file name as to who touched the draft. (This is called “version control” and is vitally important). Two links that have some insight on that: How do publishing systems handle content version control? – Smartium and The Ultimate Guide to Document Version Control
Content production – the nitty gritty
Stage 1 – Writing: Reporter writes article (interviews/research/sourced links). That equates to a human writing content, or a human prompting an AI tool to produce content with cited sources. The document needs to be classified at this point: important or lightweight? The relative importance will determine how much editing and checking is needed..
Stage 2 – First edit: In news production, the newsdesk edits for grammar/clarity, checks facts with reporter, sends back for rewriting if unsatisfactory. Outside of news production, that maps to a different human from the original writer editing content, manually checking all AI- or human-provided links, and, if needed, sending back to the AI tool (with rewrite prompts) or the human writer.
Stage 3 – Second edit: In news production, a department of sub-editors check for inconsistencies, grammar, and known factual problems; links are rechecked if time allows. In other contexts, this could be an outside editor/proofreader (or an AI tool with careful, detailed prompting) doing the same. Human review of any links the AI may have added or changed is done, again.
Stage 4 – Final eye/publication: In news content flows, a senior staffer gives a final once-over; accountability for errors sits with the editor. That equates to a human staff member making the final call to publish. Accountability must sit with one named person or department – never the AI tool – and should be built into KPIs.
Download the detailed version here
Real-world suggestions
If all this seems time-consuming, think again. In my days in production at an afternoon newspaper, an article could be written by 7am, go through all the steps and be sent for printing in the newspaper by 9.15am. The key to that is skilled and experienced content workers (known, actually, as journalists), who all clearly understand their roles and responsibilities.
Of course, these people only have one job. In cases where an organisation publishes content alongside other functions, people will be working on content alongside many other deliverables. And that will make them even more likely to make mistakes: all the more reason to have a clear step-by-step process for the production of content.
To give an example of how this works in a small team: I work in a three-person comms team for a global company. For any given piece of content, one of us writes it, the other edits it and the third person takes a final look before publication. We manage version control by adding our initials and version numbers (V1, V2, Final) to the file name as it moves between the three of us.
How does this work for solo content creators?
If you are a writer or researcher without access to colleagues or organisational resources, there are still some things you can do to ensure quality.
1. Be clear in your working processes about what you will and won’t use generative AI for. If you only use it for research, your responsibility is to check the sources the AI tool gives you. If you also use it for writing, you need to check all the facts hidden in the AI output alongside checking the cited sources.
2. Time is your best friend. Whatever you have written will always be easier to self-edit if you leave it alone for a day or two; mistakes and oddities will leap out in ways that you didn’t see the first time round. If you don’t have time, try looking at your writing in a different way – change the font, look at it in a different format (for instance, if you wrote it in Word, copy and paste it to Google Docs).
The one non-negotiable
However you use generative AI to create content, if it gives you a source and/or a link you are going to have to check it. Always.
Some quick tips on how to do that:
1. If you already have a link, click on it. Compare the page you land on to the text of the reference in your source text. Do they match? If so, you are good to go. If not, back to the drawing board. This applies when a source is simply part of a list, perhaps at the end of a text. If the document says, for instance, that the United Nations says such-and-such and gives a link, you need to check the link AND check that the UN has been correctly quoted.
2. If you don’t have a link, you will have to use an internet search engine to find the source. You can paste in the text of the entire source (or the title of the alleged document), and see what you get. Click on the link you get in the search engine results and see what happens. I’d suggest using a search engine in which you have the option to turn off AI overviews. You want links, not more AI synthesis. I use Brave, and then go to Google if needed.
(If you want to stick with Google but want to take some control over how much it uses AI, there’s a good guide here: How to Tell Google’s AI Features to Leave You Alone, Once and for All – CNET)
Does all that link checking seem like a lot of work? That’s because it is, but there’s no way round it. Vibe citing is with us, everywhere, and human intervention is the only way to deal with it.
Main picture: A screenshot from 2007, Kylu, Wikimedia Commons, CC BY-SA 3.0
Previous Sensible Guide articles
What journalists do (part two): the checking of the facts – In a sea of confusing information, journalism is one way of sifting truth from nonsense. How do journalists do that?
Tips for editors: Academic referencing 101 – One of the things a proofreader or copy editor might be asked to do is to check academic references. Here’s a guide to getting that done…
Research skills: Five ways to assess and use reliable sources -With our wondrous phones in our hands, we can all find any information we want. Here’s how to apply research skills to that information.
How to get away from Google search – Searching for a better search engine? Here’s my rough-and-ready guide; no affiliate links, no reviews, no spam…
If any of this is relevant to your work…
You might find one of these useful: