The short answer: treat every number, date, law and quote from a chatbot as a lead, not a fact. Spend five minutes: ask for the source, open the source yourself, confirm the figure matches, check the date, and look for a second source. If any step fails, do not act on the answer.
This is educational content, not financial, legal or medical advice.
Why chatbots get facts wrong with a straight face
AI assistants generate text that sounds right. That is not the same as looking something up. They can invent a source, mix up two similar rules, or quote a figure that was true two years ago. The wording stays calm and confident either way, so the tone never warns you. The fix is a short routine you run every time the answer will affect money, health, legal choices or anything you will publish.
The 5-minute routine
Step 1: Ask for the source, and the exact wording
Follow up with: “Which official page or document says this? Quote the sentence.” If the tool turns up nothing specific, or gives a vague answer like “according to experts”, you already have your answer: unverified.
Step 2: Open the source yourself
Do not trust a link just because it appears. Click it and confirm that the page exists, is run by who the tool says it is, and contains the claim. Made-up links and real links that say something different are both common failure modes. Prefer primary sources: government sites ending in .gov, the company’s own pricing or help page, the original study.
Step 3: Match the number
Find the figure on the page and compare it digit by digit with what the chatbot told you. Percentages, dollar thresholds and deadlines are where small changes hide. If you are doing math, redo the calculation in a spreadsheet. Chatbots are better at explaining arithmetic than at doing long chains of it.
Step 4: Check the date and the place
Look for “last updated” on the page. Tax limits, interest rates, subscription prices and consumer rules change often. Also check the jurisdiction: a rule that applies in the UK may not apply in the US, and a state rule may differ from a federal one.
Step 5: Find one independent confirmation
Search the claim in your own words and see whether a second reliable source agrees. Not a copy of the first, an independent one. For high-stakes questions, such as taxes, loans or medication, the second source should be a professional: an accountant, a pharmacist, a licensed advisor.
A worked example: a tax question
Suppose you ask a chatbot how much self-employment tax you owe on $800 of side income, and it answers “about $113.” Here is what the routine looks like:
- Ask for the source: it points to the IRS rule that self-employment tax is 15.3% on 92.35% of net earnings.
- Open the IRS page on self-employment tax and confirm the rate.
- Match the number: $800 x 0.9235 x 0.153 = $113.04. Matches.
- Check the date: rates and wage caps update yearly, so confirm it is current for your tax year.
- Confirm with a second source or a tax pro if the amount is large.
That took under five minutes and caught nothing wrong, which is what you want. When it does catch something, it usually saves far more than it costs. For the full picture on this topic, read why side income is taxable without a 1099.
Red flags that a claim needs extra checking
- A precise statistic with no source (“73.4% of people…”).
- A quote attributed to a named person with no link to where they said it.
- A law or deadline described without a year or jurisdiction.
- A price for a software plan. Prices change often, so check the pricing page.
- A source that, once you open it, is a different article from what was described.
Make the tool work for you
- Use tools that browse the web and show clickable citations for factual questions, then still open the citations.
- Ask: “What would make this answer wrong?” The reply often reveals the weak assumption.
- Ask it to separate “what you know” from “what you are guessing”.
- Never paste private data to get a faster answer. See our guide to AI chat privacy settings.
Who this is for, and who can skip it
For: anyone using AI for money decisions, work documents or anything that gets published. You can skip the full routine for low-stakes uses such as brainstorming names or rewording an email, where a mistake is cheap and obvious. This check is also the first question of our AI tool scorecard.
Our take
The habit matters more than the tool. Whichever assistant you use, the five minutes you spend verifying are the difference between a useful draft and an expensive mistake. Make “show me the source” your default follow-up and open every link yourself.
This is educational content, not financial, legal or medical advice.