How to Fact-Check an AI Answer with Primary Sources
Fact-checking an AI answer means separating its individual claims and verifying them against original, authoritative evidence rather than trusting fluent wording or a list of links. You’ll need the answer you want to check, access to a web browser or relevant source databases, and enough time to inspect the evidence behind important claims.
What You’ll Need
- The complete AI answer, including its citations, links, dates, and any relevant prompt or context.
- A web browser and access to primary sources, such as laws, research papers, official datasets, court decisions, company filings, or original statements.
- A notes document or table for recording claims, sources, publication dates, and findings.
- For specialized or paywalled material, access to a library, official archive, or the full document through another legitimate source.
Step-by-Step
- Define what you are checking. Save the answer as written and note the question that produced it. An answer can change depending on wording, date, location, or assumptions. Decide whether you need to verify every factual statement or only the claims that affect a decision. For high-stakes uses, check every material claim.
- Break the answer into checkable claims. Split long sentences into individual statements. Mark claims about numbers, dates, quotations, causation, rules, and named people or organizations. Separate facts from interpretations and recommendations. For example, “The policy began in 2022 and reduced costs by 15%” contains two claims: a start date and a measured effect. Each needs evidence suited to it.
- Prioritize claims by importance. Check claims that could mislead or cause harm first, followed by precise statistics, quotations, and claims central to the answer. Do not let a large number of low-impact details distract from a decisive unsupported statement. If a claim is an opinion or forecast, identify it as such instead of treating it as a settled fact.
- Classify each citation before relying on it. Open every supplied link and identify what it actually is: an original source, a news report, a commentary page, a search result, or something else. A citation is not proof merely because it looks formal. Confirm that the page exists, is accessible, and discusses the specific claim. Search the exact title or quoted phrase separately if the link is broken or suspicious.
- Find the primary source. A primary source is direct evidence produced by the people or organization involved, or a record created at the time of the event. Examples include the text of a statute, a government dataset, an original study, a court opinion, a transcript, or a company’s filed report. Search for the document title, author, organization, date, and distinctive wording. Follow citations in a secondary article back to the underlying record when possible.
- Check source identity and context. Verify that the document is hosted by its issuing organization or a reliable archive, and that the author or publisher is identifiable. Read beyond a search snippet, abstract, headline, or isolated quotation. Inspect relevant definitions, footnotes, methods, limitations, and surrounding paragraphs. A primary source can still be incomplete, mistaken, outdated, or selective; its status does not make every interpretation of it correct.
- Match the evidence to the exact claim. Confirm that the source supports the same date range, population, location, unit, and meaning as the answer. For a statistic, check how it was measured and whether it is a count, percentage, estimate, or rate. For a quotation, compare the wording with the original and preserve qualifications. For a rule, use the applicable official text and check its effective date and jurisdiction.
- Check dates and changes over time. Record when the source was published and when the underlying information applies. A page’s latest update may not be the date of the data it describes. For current facts, look for newer official releases or amendments. For historical questions, use records from the relevant period and distinguish what was known then from later findings.
- Seek independent confirmation when appropriate. Look for another primary record or a separate dataset that measures the same thing. Two websites repeating one press release are not two independent confirmations. When sources differ, compare their methods, definitions, and time periods, then report the discrepancy rather than choosing whichever result fits the answer.
- Record a verdict and cite clearly. Mark each claim as supported, partly supported, unsupported, contradicted, or not verifiable from available evidence. Keep the source title, author or issuing body, date, URL, and the exact passage, table, or page that supports your conclusion. Link directly to the relevant document or section where possible, and explain any caveats. Do not present a citation as support if it only discusses the same general topic.
- Revise the answer without overstating certainty. Correct inaccurate details, remove claims with no adequate evidence, and qualify results that depend on assumptions or incomplete data. Preserve a distinction between what the source directly says and what you infer from it. If you cannot verify a claim, say so plainly and explain what evidence would be needed.
Common Mistakes
- Trusting a citation by appearance. A polished link or plausible title can lead to a source that does not support the statement. Open and inspect it.
- Stopping at a search snippet or abstract. Snippets omit context, and abstracts may not include methods or limitations. Read the relevant full text or official record.
- Treating “primary” as synonymous with “correct.” Original documents are closest to the evidence, but may contain errors, conflicts of interest, or limited scope. Assess their reliability and context.
- Confusing correlation with causation. A dataset showing that two things changed together does not by itself establish that one caused the other. Look for a suitable study design and cautious conclusions.
- Ignoring dates, units, or jurisdiction. A correct figure or rule can be wrong for a different year, place, population, or measurement.
- Using one source for every claim. A source may support one sentence but not the next. Match each citation to the specific statement it is intended to verify.
Tips
- Use a simple table with columns for claim, best source, supporting passage, date, and verdict.
- Search exact quotations in quotation marks, then compare any match with the original document.
- For research, inspect the paper’s methods, data, and limitations, not just its abstract or press coverage.
- For official statistics, check the dataset’s definitions, release date, and revision notes.
- Keep citations specific: link to the document or section, and give enough bibliographic detail for readers to find it if the URL changes.
- Set a stopping point. If a key claim cannot be verified after reasonable searches, label it unverified rather than treating repeated online copies as confirmation.
Troubleshooting
The cited link is broken. Search the exact document title, author, or a distinctive sentence. Check the issuing organization’s website or an established public archive. Record that the original link failed and cite the replacement source you actually inspected.
The full source is paywalled. Look for an author-posted manuscript, an institutional repository, a library copy, or an official abstract and supporting data. Do not claim to have verified details that are only visible in inaccessible text. State the limitation.
Primary sources disagree. Compare dates, definitions, methods, and scope. Check whether one source supersedes another or whether they measure different things. If the conflict remains, describe both findings and avoid a definitive conclusion.
You cannot find the claimed source or quotation. Search key wording and names, including likely variations, and inspect the original publication or record. If it remains unavailable, mark the claim unsupported or unverified—not automatically false—and do not repeat it as established fact.
The answer is too broad to check in one pass. Divide it into smaller claims, prioritize the most consequential ones, and state which parts you checked. Ask for a narrower answer or the exact sources behind a particular assertion if that would make the review manageable.
Frequently Asked Questions
What counts as a primary source?
A source that provides direct evidence, such as an original dataset, study, official record, transcript, or firsthand document. What counts depends on the claim being checked.
Are official sources always reliable?
No. They can be authoritative for their own records or rules, but can still contain errors, omit context, or use definitions that do not fit your question.
Can a secondary source help?
Yes. It can point you to relevant records and explain context. For important claims, follow its references and check the original evidence yourself.
How many sources should I check?
There is no fixed number. Use the best source for each claim, and seek independent confirmation for consequential or disputed facts when possible.
What if a claim cannot be verified?
Label it unverified, note what you searched, and avoid presenting it as fact. Lack of confirmation alone does not prove it is false.
Do citations prove an AI answer is accurate?
No. Citations must be genuine, relevant, current, and accurately represented. Check the underlying evidence and the answer’s interpretation of it.
Related
Method: the standard formula for this calculation, rounded as each result states. See how the calculators work or report an error.