What Is RAG? Build a Chatbot from Your Own PDF (Step by Step)

By Ravindra Bagale, trainer · LinkedIn · Last updated: 23 October 2026

Generative AI course · Chapter 38 · Estimated reading time: 15 minutes

1. Suppose HR answers the same 20 questions every week

Suppose Deepa works in HR at a small software company. Every week, people ask her the same things: How many leave days do I get? Can I work from home? What is my notice period? The answers are all in a 2-page policy PDF, but nobody reads it.

She tries a general chatbot. It answers "You can usually carry forward 10 days", which is not her company's rule (her policy says 6). Wrong answers about leave and notice periods cause real problems.

What she needs is a bot that:

1. finds the right paragraph in her PDF,
2. answers only from that paragraph,
3. says which page it came from, and
4. says "I could not find this" when the PDF does not cover the question.

That is RAG.

2. What RAG means

Retrieval-augmented generation has two halves:

• Retrieve: search your documents and pick the few pieces most related to the question. We do this with embeddings, the meaning-numbers from the previous chapter.
• Generate: send those pieces to a model with the question, and ask it to answer only from them, with a citation.

The 5 steps in plain words:

1. Read the PDF into text.
2. Chunk: cut the text into small pieces (chunks), such as one section or paragraph each.
3. Embed every chunk once and keep the numbers.
4. Retrieve: embed the question and pick the 2 or 3 closest chunks.
5. Answer: send the chunks and the question to the model with clear rules.

Install what we need (steps 1 to 4 run free on a laptop):

pip install pypdf fastembed numpy

3. Worked example 1: an HR policy PDF bot

We made a sample 2-page policy PDF for a made-up company, "Bluehill Softworks", with 6 numbered sections: working hours, leave, sick leave, work from home, notice period and travel expenses.

Steps 1 to 4 in code:

# RAG step 1-3: read the PDF, cut it into chunks, find the chunks that answer a question.
import re
import numpy as np
from pypdf import PdfReader
from fastembed import TextEmbedding

# 1. Read the PDF, page by page
reader = PdfReader("hr_policy.pdf")
chunks = []
for page_no, page in enumerate(reader.pages, start=1):
    text = page.extract_text()
    # 2. Cut into chunks: one numbered section = one chunk
    for part in re.split(r"\n(?=\d+\. )", text):
        part = " ".join(part.split())
        if re.match(r"\d+\. ", part):
            chunks.append({"page": page_no, "text": part})
print("Chunks:", len(chunks))

# 3. Embed the chunks once, then embed each question and pick the closest chunks
model = TextEmbedding("BAAI/bge-small-en-v1.5")
vecs = np.array(list(model.embed([c["text"] for c in chunks])))
vecs = vecs / np.linalg.norm(vecs, axis=1, keepdims=True)

def retrieve(question, top=2):
    q = np.array(list(model.embed([question])))[0]
    q = q / np.linalg.norm(q)
    scores = vecs @ q
    return [(float(scores[k]), chunks[k]) for k in np.argsort(-scores)[:top]]

question = "How many leave days can I carry forward?"
print("Question:", question)
for score, c in retrieve(question):
    print(f"  {score:.2f}  page {c['page']}: {c['text'][:70]}...")

Real output (we ran this on 23 October 2026):

Chunks: 6
Question: How many leave days can I carry forward?
  0.77  page 1: 2. Leave Every employee gets 18 days of paid leave in a calendar year....
  0.71  page 2: 4. Work from home Employees may work from home up to 2 days a week wit...

The right section (Leave, page 1) is on top. We also tried two more questions:

Question: Do I need a doctor's note for 3 days of sick leave?
  0.76  page 1: 3. Sick leave Employees get 8 days of sick leave a year. For...
  0.64  page 1: 2. Leave Every employee gets 18 days of paid leave in a cale...
Question: Is there a gym allowance?
  0.56  page 2: 6. Travel expenses Travel bills must be submitted within 15 ...
  0.55  page 1: 2. Leave Every employee gets 18 days of paid leave in a cale...

Look at the last one. The policy says nothing about a gym, and the best score is only 0.56. Retrieval still returns something, because it always returns the closest chunks. That is why the next step's rules matter.

Step 5: the prompt for the model. The program builds this from the retrieved chunks:

Answer the question using ONLY the policy text below. Cite the page like (page 1).
If the answer is not in the text, say "I could not find this in the policy."

Policy text:
[page 1] 2. Leave Every employee gets 18 days of paid leave in a calendar year. Up to 6 unused days can be carried forward to the next year. Leave must be applied for in the HR portal at least 3 working days in advance, except in an emergency.
[page 2] 4. Work from home Employees may work from home up to 2 days a week with their manager's approval. Work from home is not allowed during the first 3 months after joining.

Question: How many leave days can I carry forward?

Send it with the API call from the chapter on your first API call (for example, Gemini's generate_content with your key in .env).

Expected output (yours will be similar, not the same):

You can carry forward up to 6 unused leave days to the next year (page 1).

For the gym question, the expected answer is "I could not find this in the policy." If the model invents an allowance instead, make the rule stronger or add a score floor (worked example 2).

4. Worked example 2: a course-notes bot

Now a bot for your own study notes, a plain text file with 5 short paragraphs on Python basics (lists, dictionaries, for loops, f-strings, and try and except).

First try: chunks of 40 words with a 10-word overlap. This is a common default. With our notes, the chunk that best matched "How do I stop my program crashing on bad input?" started in the middle of the f-string paragraph and ran into the try-and-except paragraph. Fixed-size chunks cut ideas in half.

Better: one paragraph per chunk, plus a score floor: if the best score is too low, the bot says "not in your notes" without calling the model at all.

# Course-notes bot, retrieval part: one paragraph per chunk, then retrieve with a score floor.
import numpy as np
from fastembed import TextEmbedding

text = open("notes.txt", encoding="utf-8").read()
chunks = [p.strip() for p in text.split("\n\n") if p.strip()]   # one paragraph = one chunk
print("Words:", len(text.split()), "| Chunks:", len(chunks))

model = TextEmbedding("BAAI/bge-small-en-v1.5")
vecs = np.array(list(model.embed(chunks)))
vecs = vecs / np.linalg.norm(vecs, axis=1, keepdims=True)
FLOOR = 0.65      # below this, we say "not in the notes" instead of asking the model

for question in ["How do I stop my program crashing on bad input?", "What is a Python decorator?"]:
    q = np.array(list(model.embed([question])))[0]; q = q / np.linalg.norm(q)
    scores = vecs @ q; k = int(np.argmax(scores))
    print(f"\nQuestion: {question}\nBest chunk {k + 1}, score {scores[k]:.2f}")
    if scores[k] < FLOOR:
        print("Answer: This is not in your notes yet.")
    else:
        print("Send to model:", chunks[k][:90] + "...")

Real output (23 October 2026):

Words: 122 | Chunks: 5

Question: How do I stop my program crashing on bad input?
Best chunk 5, score 0.68
Send to model: try and except catch an error so the program does not crash. Put the risky line under try ...

Question: What is a Python decorator?
Best chunk 1, score 0.62
Answer: This is not in your notes yet.

Now the chunk is exactly the try-and-except paragraph, and the decorator question (not in the notes) is stopped before it reaches the model.

Be honest about the floor: 0.68 and 0.62 are close. A floor of 0.65 works for these notes and this model, not for every file. Test 10 questions you know are in the notes and 5 you know are not, then choose the floor.

5. Worked example 3: a cited-answer check

Even with good chunks, a model can copy a number wrongly. A small check catches this: every number in the answer must appear on the page it cites.

# Cited-answer check: do the numbers in the answer really appear on the page it cites?
import re
from pypdf import PdfReader
pages = {i: " ".join(p.extract_text().split()) for i, p in enumerate(PdfReader("hr_policy.pdf").pages, start=1)}

def check(answer):
    cited = [int(n) for n in re.findall(r"\(page (\d+)\)", answer)]
    if not cited:
        return "NO CITATION: do not trust this answer"
    numbers = re.findall(r"\b\d+\b", re.sub(r"\(page \d+\)", "", answer))
    text = " ".join(pages.get(n, "") for n in cited)
    missing = [n for n in numbers if not re.search(rf"\b{n}\b", text)]
    return "OK: every number is on the cited page" if not missing else f"CHECK: {missing} not found on page {cited}"

answers = [
    "You can carry forward up to 6 unused leave days to the next year (page 1).",
    "You can carry forward up to 10 unused leave days (page 1).",
    "Yes, you get a gym allowance of 500 rupees a month.",
]
for a in answers:
    print(check(a), "<-", a)

Real output (23 October 2026):

OK: every number is on the cited page <- You can carry forward up to 6 unused leave days to the next year (page 1).
CHECK: ['10'] not found on page [1] <- You can carry forward up to 10 unused leave days (page 1).
NO CITATION: do not trust this answer <- Yes, you get a gym allowance of 500 rupees a month.

The check caught both bad answers: a wrong number with a real citation, and an invented allowance with no citation. It is simple (it only checks numbers), but numbers are where the costliest mistakes hide: days, amounts, dates.

6. RAG vs fine-tuning

People often ask: "Should I train the model on my PDF instead?" That is called fine-tuning.

For "answer from our documents", start with RAG.

7. Ready-made RAG tools

You do not always need to build it yourself:

• Chat tools with file upload: ChatGPT projects, Gemini and Claude projects can answer from files you upload. Good for personal use; follow the privacy rules from the chapter on safe use.
• Gemini API File Search: Google offers a managed File Search tool in the Gemini API that stores, chunks and retrieves your files for you.
• Build your own (this chapter): you control every step, can print what was retrieved, and can add checks like the cited-answer check.

For a few thousand chunks or more, store the embeddings in a vector database instead of in memory.

Privacy rule for every option: an HR policy is usually fine to use, but employee records, salaries and personal data are not. Ask your company before you upload internal documents to any AI service.

8. Worked example 4: three stacks for the same HR PDF bot

Same goal: answer questions from one HR policy PDF with citations. Three stacks (health-check: EXPANDOCT23-RAG-NOTEBOOKLM-CHROMA-BEDROCK). Pick the stack that matches your skill — do not jump to AWS on day one.

Stack A — no-code (NotebookLM):

1. Upload the HR PDF to a NotebookLM notebook.
2. Ask: "How many casual leaves? Cite the section."
3. Share only with people allowed to see HR policy. NotebookLM is not a public chatbot host.

Stack B — local Python + Chroma:

# Learning sketch — install versions change; check current docs
# pip install chromadb pypdf
# 1) Split PDF text into chunks
# 2) Store embeddings in Chroma
# 3) On each question: retrieve top chunks, send to an LLM with citations
# Never commit API keys; use environment variables.

Run only on sample public policies while learning. Expected output: answers that quote chunk text you can open in the PDF.

Stack C — AWS Bedrock Knowledge Base (when you already use AWS):

1. Put the PDF in an S3 bucket your account owns.
2. Create a Bedrock Knowledge Base pointing at that bucket (console wizard).
3. Test in the console. Watch billing: delete test indexes when done so leftover vector stores do not surprise you.
4. Do not paste production employee data into a tutorial account.

Expected output (yours will be similar, not the same): the same leave-policy question answered in all three stacks, with a citation you can verify. Costs and UI labels change — read AWS and Google docs in the month you build.

9. Beginner mistakes and quick fixes

10. Try at home

Recap

Build the HR bot on a made-up policy this week and try to make it give a wrong answer. Every way you break it shows you exactly which step to improve.

Previous chapter: Embeddings Explained Simply: How AI Finds Similar Meaning

Next chapter: Vector Databases for Beginners: Chroma, pgvector and OpenSearch

Ravindra Bagale, trainer: linkedin.com/in/ravindra-bagale. Deepa, Bluehill Softworks, its policy and the study notes on this page are made up for learning. Gemini API File Search was checked on Google's Gemini API pages on 23 October 2026. The Python output was produced on 19 October 2026 with pypdf, fastembed and the BAAI/bge-small-en-v1.5 model. AI output changes every time, so your output will be similar, not the same.