Verify AI-Assisted Reports: Data and Citations
Verifying an AI-assisted report means checking that its figures can be reproduced from reliable data and that each citation supports the claim it accompanies. Before sharing or relying on a report, you will need the report itself, access to its cited sources and underlying data where available, and a way to recalculate key results.
What You’ll Need
- The report, including charts, tables, footnotes and references.
- Original datasets or source documents, not just summaries of them.
- A spreadsheet or other suitable tool for repeating calculations.
- Access to source links, publication details and any relevant data definitions.
- A record of your checks, including unresolved gaps and corrections.
Step-by-Step
- Define what must be verified. Identify the report’s purpose, intended audience and the decisions it may inform. Mark every important numerical claim, chart, table and factual statement that needs evidence. Prioritize claims that could change a conclusion, affect a decision or be difficult to reverse. Do not assume a polished chart or confident wording is proof.
- Make an inventory of claims and sources. Create a review table with columns for the claim, location in the report, stated source, calculation or method, your finding and status. Separate claims that can be checked directly from source documents from calculations that require underlying data. Record missing citations rather than silently treating them as verified.
- Trace each citation to the original. Open the linked document or locate it using its author, title, publisher and date. Confirm that the source exists and that the citation identifies the correct work, version and page, table or section when applicable. Prefer original research, official datasets or primary records for factual claims; use a secondary summary only when the primary material is unavailable or the claim concerns the summary itself.
- Check whether the source actually supports the claim. Read the relevant passage, inspect the table or review the dataset documentation. Compare the report’s wording with the source’s population, time period, location, definitions and level of certainty. A source about correlation does not establish causation, and a finding about one group or period may not support a broad generalization. Note qualifications the report leaves out.
- Get the exact data and definitions. Find the dataset version and the fields used. Check units, date ranges, inclusion and exclusion rules, missing values, duplicate records and whether figures are preliminary or revised. If the report provides only a chart or rounded summary, mark that limitation: you may be able to check the displayed values, but not independently reproduce the full calculation.
- Recalculate key figures independently. Write down the formula and inputs before using a spreadsheet or script. Reproduce totals, percentages, averages, rates, changes over time and sample sizes that drive the report’s conclusions. For example, confirm whether a percentage uses the stated denominator and whether a change is reported as a percentage change or a percentage-point difference. Keep full precision during calculations and round only for display.
- Audit charts and tables. Check that labels, units, totals and category names agree with the source data. Verify that axes start and end where shown, time periods are consistent, and the plotted values match the underlying numbers. Look for truncated axes or uneven intervals that may exaggerate a difference. Reconcile subtotals with totals, allowing only for clearly explained rounding.
- Test the method and assumptions. Confirm that the calculation fits the question. Check the denominator, comparison group, time window, weighting, filters and treatment of missing data. Ask whether another reasonable choice would materially change the result. If the report does not explain a consequential assumption, request clarification or report the result as unverified rather than guessing.
- Review the conclusion against the evidence. Compare each headline claim with the checked figures and sources. Ensure the conclusion does not claim more than the data show, especially about cause, forecasts or differences between groups. Separate facts, estimates and interpretations. Label uncertainty and limitations where a reader will see them.
- Document findings and resolve discrepancies. Record the source, version, formula, inputs, recalculated result and any difference from the report. Classify each item as verified, verified with a caveat, incorrect or unable to verify. Ask the report’s author to explain discrepancies; then correct the report or disclose what remains unresolved. Repeat the checks on the final edited version, since revisions can change numbers or citations.
Common Mistakes
- Checking only that a link opens: A real source may still be irrelevant, outdated or inconsistent with the report’s claim.
- Using a search snippet as evidence: Snippets omit context and can display text from another part of a page. Read the source itself.
- Recalculating from rounded numbers: Rounded inputs can produce a different result. Seek unrounded data and record when only displayed values are available.
- Mixing up percentage change and percentage points: Moving from 20% to 25% is a rise of 5 percentage points, or a 25% relative increase.
- Ignoring changed definitions or dates: A revised dataset or different population may explain a mismatch, but it must be documented.
- Treating a plausible result as a verified result: A number can look reasonable while using the wrong denominator, filter or source.
Tips
- Start with the figures and citations most important to the report’s conclusion; check all remaining material before publication if it is presented as factual.
- Save dated copies of sources and datasets, along with the report version reviewed. Note retrieval dates for web material that may change.
- Use a second reviewer for high-impact claims or complex calculations, and ask them to repeat the check without relying on your result.
- Keep calculations transparent: use labeled cells or readable code, show units, and avoid hiding assumptions in formulas.
- Use consistent status labels and distinguish “not checked” from “checked but not independently reproducible.”
Troubleshooting
The citation is broken or the source is unavailable: Search by the exact title, author and publication date, or use the publisher’s archive or an institutional repository. Do not substitute a different source without confirming it supports the same claim. If you cannot retrieve it, mark the claim unable to verify.
Your calculation does not match: Compare the source version, units, date range, denominator, filters, missing-value handling and rounding in that order. Check for a transcription error and reproduce the formula with a small hand-worked example. If the report omits essential inputs or methods, request them and disclose the limitation.
The source supports a narrower statement: Revise the report to match the source’s scope, population and level of certainty, or find stronger evidence. Do not broaden a finding simply because the report’s wording sounds reasonable.
Different reputable sources disagree: Compare their definitions, collection methods, dates and revisions. Explain the difference and choose the source most appropriate to the report’s question; do not average conflicting values without a defensible method.
Frequently Asked Questions
Do I need to check every number?
Check every material figure and all figures presented as evidence. Prioritize high-impact calculations, but do not describe unchecked claims as verified.
Can I verify a calculation without the original dataset?
You can check arithmetic based on published inputs, but you cannot fully reproduce a result that depends on unavailable records, filters or methods. State exactly what you could confirm.
What counts as a reliable citation?
It is retrievable, accurately identified, relevant to the claim and sufficiently authoritative for the question. Reliability also depends on source quality, date and methodology.
How much rounding is acceptable?
Keep full precision through intermediate steps, then round consistently to a sensible number of places. Explain rounding when it affects totals or comparisons.
What should I do if a source is later corrected?
Check which version the report used, update the calculation and citation, and note any change to the findings. Preserve the earlier version in your review record.
When is a report ready to share?
When important claims are supported, key calculations are reproducible or their limits are disclosed, discrepancies are resolved or clearly flagged, and the final version has been rechecked.
Related
Method: the standard formula for this calculation, rounded as each result states. See how the calculators work or report an error.