Check an AI answer for invented facts

Audit this answer for things that may not be true. Do not improve it.

The answer:
[PASTE IT]

The question it was answering: [PASTE]
What I plan to do with it: [E.G. PUT IT IN A REPORT, MAKE A DECISION, POST IT]
What I already know about the subject: [SO YOU KNOW WHERE MY OWN JUDGEMENT IS THIN]

Produce five sections.

1. SPECIFIC CLAIMS — every statement of fact: numbers, dates, names, citations, quotes, versions, prices, legal or medical assertions. List them individually. Do not summarise them into themes.

2. HOW EACH ONE COULD FAIL — for each claim: is it the kind of thing that is stable, the kind that goes out of date, or the kind that models routinely fabricate? Citations, statistics with decimal places, quotations, and case names are the highest-risk categories. Say which of these you can and cannot verify from your own knowledge, and be explicit when you cannot.

3. PLAUSIBLE BUT UNSUPPORTED — the part I most need. Statements written with the confidence of fact that are actually inference, generalisation, or filler. These are the ones that survive a read-through because nothing about them looks wrong.

4. WHAT TO VERIFY, RANKED — ordered by what it would cost me to be wrong given what I said I am using it for, not by how uncertain each item is. For each, where to check.

5. WHAT IS PROBABLY FINE — so I do not verify all of it. Say why.

You are not a fact-checker with access to sources. Where you are uncertain, say uncertain. Do not replace one confident claim with another.

Everyone now knows models make things up, and almost nobody checks, because the parts that are invented look exactly like the parts that are not. Fabrication does not arrive with hedging attached — it arrives as a citation with a plausible author and year, a statistic with a decimal place, a quotation that sounds like the person. The fluency is uniform, which means reading carefully does not help.

What does help is splitting the answer into individual claims and sorting them by failure mode, because the risk is not evenly spread. A definition of a well-known concept is nearly always right; a specific figure with a source attached is where the invention lives. Section three then catches the other category, the one people miss entirely: sentences that are not false so much as unsupported — an inference stated as a finding, a generalisation stated as a rule.

Section four is what makes it usable. Verifying everything is not going to happen, so the ranking is by consequence rather than by uncertainty: what you are about to do with the answer decides which three things are worth ten minutes.

1. Say what you are using it for

   The ranking is built on consequence. A claim that would be embarrassing in a published report and harmless in a private note is the same claim with a different priority.

2. Check every citation by opening it

   Fabricated references are the single most common failure, and they are the most convincing: real-sounding author, plausible journal, right decade. A title that cannot be found usually does not exist.

3. Use a different model, or a search, for the verification

   Asking the same model whether it was right invites it to agree with itself. This prompt is for producing the list of what to check, not for doing the checking.

4. Read section three even when the facts hold up

   An answer can contain no false statements and still be mostly unsupported inference. That is the failure mode that survives fact-checking untouched.

• The auditor has the same blind spots as the authorA model checking a model shares its training data and its gaps. It will miss fabrications in areas where it is itself weak, and it cannot open a link. This produces a list of what to check, not a verdict.
• Anything recentEvents, prices, versions, and personnel after the training cutoff cannot be assessed at all. Treat every claim of that type as unverified regardless of what section five says.
• Medical, legal, and financial claimsThe consequences of a plausible wrong answer here are not proportionate to the effort of checking. Take these to a qualified person rather than to a second model.

• ResearchRagnarökCompare options and see where they differCompares the options on the things that would actually change your decision, and says separately which differences do not apply to you.
• WritingApolloRemove AI tells from a draftRewrites text that reads as machine-written, then lists the specific phrases and structures that gave it away so you stop producing them.
• LearningApolloLearn a topic and get quizzed on itExplains something at three levels of detail, then questions you on it and reports which specific part you did not actually have.
• CodingRagnarökDebug a stack traceTurns an error and its stack trace into an ordered list of what could cause it, each with the specific check that would rule it in or out.