Why AI Hallucinates and How to Fact-Check Any AI Answer

By Ravindra Bagale, trainer · LinkedIn · Last updated: 19 October 2026

Generative AI course · Chapter 13 · Estimated reading time: 14 minutes

1. Suppose we asked for 3 good books

Suppose we are Kiran, an MBA student in Bengaluru. Our professor asked for 3 books on retail in India. We ask an AI:

Suggest 3 books on the growth of retail in India, with author and year.

A typical answer (yours will be similar, not the same):

1. "The Kirana Revolution: How Small Shops Built Indian Retail" – Meera Krishnan (2016)
2. "Retail Bharat" – S. Raghunathan (2019)
3. "From Mandi to Mall" – Anil Deshpande (2012)

The titles sound real. The authors sound real. The years look right. But when we search each title on Google Books, an online bookstore and the library catalogue, we may find nothing. The AI produced titles that look like book titles, because that is what the next-text prediction does well.

(We made up the 3 titles above on purpose to show what this looks like. Do not search for them.)

That is a hallucination:

A hallucination is an AI answer that sounds confident and correct, but is false or made up.

2. Why does AI hallucinate? 5 simple reasons

1. It predicts, it does not look up. An LLM writes the most likely next words. "Likely" and "true" are usually the same, but not always.
2. It never says "I don't know" on its own. Training rewards a helpful, complete answer. A confident guess often looks more helpful than "I am not sure".
3. Rare facts are weak. Famous facts (the capital of India) appear thousands of times in training text. A small book, a local rule or a person's exact date of birth may appear once or never.
4. Knowledge cutoff. Anything after the training date is unknown to the model alone, so it may fill the gap with an old or invented answer.
5. Pressure from the prompt. If you ask "give 10 sources", it will try to give 10, even if it knows only 4.

Google's own Gemini help page says it plainly: Gemini can hallucinate and present inaccurate information as factual. Every AI company has a similar warning under the chat box.

3. A real case: an AI-made judgment in the Bombay High Court

This is not a classroom problem. In January 2026, the Bombay High Court imposed costs of ₹50,000 on a party in a flat dispute in Mumbai. The party's written submissions relied on a case called "Jyoti w/o Dinesh Tulsiani vs Elegant Associates". The judge and the court's law clerks searched for this judgment and could not find it anywhere. The judge noted signs that the submissions were prepared with an AI tool such as ChatGPT.

Justice Milind Sathaye wrote that using AI for research is welcome, but the person using it has a great responsibility to cross-verify that the references are relevant, genuine and really exist. He also warned that an advocate doing this could face stricter action, including a referral to the Bar Council. (Source: LiveLaw report of 16 January 2026 on the judgment of 7 January 2026 in Deepak Shivkumar Bahry vs Heart & Soul Entertainment Ltd.)

The lesson for every job: the person who signs is responsible, not the AI. That is true for a court filing, a client email, a college assignment or a bank form.

4. Worked example 1: catch an invented book

Let us test the AI on purpose with a book that does not exist.

Steps:

1. Open any AI tool. Turn web search off if there is a button.
2. Paste:

Summarise the book "The Silent Monsoon Ledger" by Arvind Kale in 5 bullet points.

(We invented this title and author for this test.)

Expected output, two kinds (yours will be similar, not the same):

A careful answer:

I could not find a book called "The Silent Monsoon Ledger" by Arvind Kale. It may be a
very new or small book, or the title may be slightly different. Can you share where you
heard about it?

A hallucinated answer:

"The Silent Monsoon Ledger" by Arvind Kale is a novel set in a coastal Konkan village...
- An old accountant discovers secrets in a family ledger
- ...

What to learn: newer models refuse more often, but not always. Try the same prompt in 2 tools. If one invents a plot, you have seen a hallucination live.

How to check any book quickly:

1. Search the exact title in quotes on Google: "The Silent Monsoon Ledger".
2. Search it on an online bookstore and on Google Books.
3. Look for an ISBN (the book's ID number). A real published book usually has one.

If you cannot find it in 3 places, do not use it.

5. Worked example 2: catch a wrong date

Dates and numbers are where hallucinations hurt most, because they look so precise.

Steps:

1. Ask:

When did GST start in India? Give the exact date and one line on what it replaced.

2. Compare the answer with an official source.

A wrong answer you may see (example):

GST was launched in India on 1 April 2017, replacing VAT, service tax and excise duty.

The check: GST in India started on 1 July 2017 (the launch was at a special midnight session of Parliament on 30 June to 1 July 2017). The official GST portal and CBIC pages give this date. The answer above is wrong by 3 months, even though everything else in it sounds right. "1 April" is the start of the financial year in India, so it is a very "likely" date, which is exactly why a model might predict it.

Follow-up prompt when you catch a mistake:

That date is wrong. The official GST portal says 1 July 2017. Please correct your answer, and
tell me which other facts in your answer I should double-check.

Most tools will correct it. But remember: the AI agreed because you checked. Never let it be the only source for a date in a document you send.

6. Worked example 3: check the sources with Gemini and Copilot links

Tools with web search show sources: small links or numbers next to the sentences. Sources help a lot, but a link is not proof until you open it.

Steps:

1. In Copilot, ask (Copilot's search answers show citations next to the text):

What is the minimum balance for a basic savings bank deposit account (BSBD) in India? Give sources.

2. In Gemini, ask the same question. When Gemini has sources, a Sources button appears below the answer (or links appear inside it). Some versions also have a Double-check response option in the menu under the answer; use it if you see it.
3. For each tool, open at least 2 of the links.
4. For each link, check 3 things:
   1. Is it a trusted site? For banking: rbi.org.in or the bank's own site, not a random blog.
   2. Does the page actually say this? Use Ctrl + F (or Find in page on the phone) to search for the key word, like "minimum balance".
   3. Is it current? Look for a date on the page.

Expected output (yours will be similar, not the same):

A BSBD account has no minimum balance requirement. [1] [2]
Account holders get basic facilities like an ATM card and a minimum number of free withdrawals each month. [1]

What you may find when you open the links:

• Link [1] is an RBI page or a bank's BSBD page that says there is no minimum balance requirement. Good, the claim is supported.
• Link [2] is a general blog page about savings accounts that does not mention BSBD at all. This link does not support the claim. The AI attached it anyway.

This is common: the sentence is right, but one source is weak, or the sentence adds a detail that no source says. (For the record, RBI's rules say a BSBD account has no minimum balance and at least 4 free withdrawals a month; the AI's "minimum number" is vague, so check the exact number on the RBI page.) Trust only what you found in the source yourself.

7. The 5-step fact-check (2 minutes)

Use this on any AI answer before you send, submit or sign it:

1. Mark the facts. Underline every name, number, date, price, rule, quote and source in the answer.
2. Ask: does it matter? If a wrong fact would embarrass you, cost money or break a rule, it must be checked. A wrong adjective in a birthday wish does not.
3. Find the official source. Government site, company site, the original document, the book itself. Not another AI, not a forwarded message.
4. Match exactly. Open the source and find the same fact, with Ctrl + F. Same number, same date, same name.
5. Fix or remove. If you cannot confirm a fact, change it or delete it. "I could not verify" is better than a confident mistake.

8. Prompts that reduce hallucinations

You cannot switch hallucinations off, but these lines in your prompt help a lot:

A ready "careful answer" prompt (copy and save):

Answer my question below. Rules:
1. If you are not sure about a fact, say "Not sure" next to it.
2. Do not invent names, books, case laws, links or numbers.
3. At the end, list every fact I should verify, with the best official website to check it.
Question: [your question]

9. Where hallucinations are most dangerous

10. Compare the 4 tools

No tool is free of hallucinations. Tools with search hallucinate less on fresh facts, but they can still misread a page or attach a weak source.

11. Beginner mistakes and quick fixes

12. Try at home

Recap

Take the last AI answer you used for work or study, and run the 5-step fact-check on it now. You will be surprised how often one small fact needs fixing.

Previous chapter: Tokens and Context Window Explained: Why AI Forgets Long Chats

Next chapter: AI with Web Search, Deep Research and AI Mode: When the Answer Is Fresh

Ravindra Bagale, trainer: linkedin.com/in/ravindra-bagale. The people, book titles and author names in the examples are made up for learning; they are not real books. The Bombay High Court case is real and is reported by LiveLaw (16 January 2026). The GST start date and the BSBD rules (RBI directions in force from 1 April 2026) were checked on 19 October 2026. AI answers change every time, so your output will be similar, not the same.