Multimodal AI: Ask Questions About Photos, PDFs and Voice Notes

By Ravindra Bagale, trainer · LinkedIn · Last updated: 19 October 2026

Generative AI course · Chapter 15 · Estimated reading time: 13 minutes

1. Suppose the meeting notes are on paper

Suppose we are Meena, a store manager at a supermarket in Thane. At the morning briefing our area manager writes 6 points on a whiteboard and we copy them on paper in a hurry. Later we must send these points to 12 staff members on the team WhatsApp group.

Typing them again takes 10 minutes and our handwriting is not easy to read. Instead we take a photo of the page and ask an AI:

This is a photo of my handwritten meeting notes. Type them out exactly, as a numbered list.
If you cannot read a word, write [unclear] instead of guessing.

Expected output (yours will be similar, not the same):

1. Diwali stock display ready by Thursday
2. Billing counter 3 – new scanner from Monday
3. Staff leave: no leave from 25 Oct to 5 Nov
4. Check expiry dates in dairy section daily
5. Customer feedback box near exit – [unclear] every Saturday
6. Target: sweets section sales up 20%

In 20 seconds we have clean text, and the AI has marked the one word it could not read. We look at the paper, see the word is "empty", fix it, and send. That is multimodal AI: the model looked at a picture and gave us text.

2. What "multimodal" means

A multimodal AI model can take more than one kind of input, such as text, images, PDFs, audio and video, and answer about all of them in one chat.

Earlier AI tools needed separate programs: one to read text from images (OCR), one to turn speech into text, and one to write. Now one model does all of it, and it can combine them. For example: "Look at this photo of the bill and this PDF of the price list. Are the prices on the bill correct?"

What each tool accepts (checked October 2026)

The Gemini numbers come from Google's "Upload & analyze files" help page: up to 10 files in one prompt, each file up to 100 MB (videos up to 2 GB), total audio up to 10 minutes on the free plan and up to 3 hours on Google AI Pro or Ultra. Limits change, so check the help page if an upload fails.

3. Worked example 1: a handwritten note

You saw the result in section 1. Here are the steps, and how to get the best reading.

Steps:

1. Put the paper on a flat table near a window. Avoid shadows from your phone.
2. Hold the phone straight above the page, not at an angle.
3. Take the photo and crop away the table.
4. In the AI app, tap + (or the attach icon) and choose the photo.
5. Paste the prompt from section 1.

Then improve the result with follow-ups:

Now rewrite these points as a short WhatsApp message to my staff. Simple English, under 80 words, start with "Good morning team".

Expected output (yours will be similar, not the same):

Good morning team,
1. Diwali stock display must be ready by Thursday.
2. Counter 3 gets a new scanner from Monday.
3. No leave from 25 Oct to 5 Nov.
4. Check dairy expiry dates every day.
5. Empty the feedback box near the exit every Saturday.
6. Our target: sweets sales up 20%.
Thank you!

Check it: compare every number and date with the paper. Handwritten "25" and "26", or "5" and "6", are easy to confuse.

4. Worked example 2: explain a graph

Graphs in reports and news are easy to misread. AI can explain them in simple words, but it can also misread the numbers on the axes. So we ask it to show what it read.

Suppose our manager shares a screenshot of a bar chart: monthly sales of the sweets section from April to September.

Prompt:

This is a chart from our sales report.
1. First, write the value you read for each bar, in a table: Month | Value.
2. Then explain the trend in 3 simple lines.
3. Point out one thing that looks unusual.
Do not guess values you cannot read; write "unclear".

Expected output (yours will be similar, not the same):

| Month     | Value (₹ lakh) |
|-----------|----------------|
| April     | 4.2            |
| May       | 3.8            |
| June      | 3.5            |
| July      | 3.9            |
| August    | 5.6            |
| September | 4.1            |

Trend: sales fell from April to June, then rose. August was the highest month.
Unusual: August is much higher than the months around it, maybe because of
Raksha Bandhan sweets. Check if there was a festival offer in August.

Check it:

1. Compare 2 or 3 values in the table with the chart. If the chart has labels on the bars, they must match exactly.
2. If the AI read a value wrong, tell it: August is 5.2, not 5.6. Please redo the explanation.
3. Treat the "why" (Raksha Bandhan) as an idea to check, not a fact. The AI only sees the picture, not the reason.

Tip: if you have the original numbers in Excel, give the AI the numbers instead of a picture. Reading numbers from a picture is always less exact than reading the numbers themselves.

5. Worked example 3: summarise a voice note

Voice notes are fast to send and slow to listen to. A 6-minute voice note from a supplier can hide one important date.

Gemini accepts audio files. On the free plan, the total audio in one prompt can be up to 10 minutes.

Steps (Android, WhatsApp to Gemini):

1. In WhatsApp, long-press the voice note.
2. Tap Share (or the forward/share icon) and choose Gemini, or save the file to your phone and then attach it in Gemini with + → Files.
3. Type this prompt with the file:

This is a voice note from our supplier. Please:
1. Give a 3-line summary.
2. List every date, amount and quantity mentioned, with the exact words used.
3. List what I need to do, as action points.
If any part is unclear, say so.

Expected output (yours will be similar, not the same):

Summary: The supplier says the Diwali sweets boxes will arrive in 2 lots.
The first lot is ready; the second lot is delayed because of packaging.
He wants the payment for the first lot before dispatch.

Dates, amounts and quantities:
- "First lot of 200 boxes on Wednesday"
- "Second lot of 150 boxes by next Monday, maybe Tuesday"
- "Payment of 46,000 for the first lot before dispatch"

Action points:
1. Confirm the first lot delivery time for Wednesday.
2. Arrange payment of ₹46,000 for the first lot.
3. Ask for a firm date for the second lot.

Check it: play the voice note at the part with the money and the dates. Speech-to-text can mix up "16" and "60", or "40" and "14", especially on a noisy phone line. Asking for "the exact words used" makes this check fast.

If your phone cannot share the voice note to Gemini, play it on your phone's speaker and use the AI app's voice input, or use your phone's built-in transcribe feature, then paste the text into any AI tool with the same prompt.

Privacy: a voice note can contain other people's names, numbers and private matters. Send only work voice notes you are allowed to share, and never voice notes with OTPs, bank details or personal family matters of others.

6. Ask questions about a PDF with pictures

Many PDFs are not just text: they have tables, charts and scanned pages. Multimodal models can read these too, but scanned pages and small tables are where mistakes happen.

A good PDF prompt:

From the attached PDF only:
1. What does the table on page [N] show? Give the column names.
2. Copy the row for [item] exactly.
3. If the page is a scan and you cannot read part of it, say which part.

Use what you learned in the context window chapter: ask for exact quotes with page numbers, and split very long PDFs.

7. Medical reports and X-rays: be very careful

People search "ChatGPT X-ray reader" and upload scans and blood reports. Here is what is safe and what is not.

Safe uses:

• "Explain what HbA1c means in simple words."
• "What questions should I ask my doctor about this report?"
• "Explain the words in this report that I do not understand." (with your name and ID hidden)

Not safe:

• Asking the AI to diagnose you from an X-ray, a scan or a report.
• Changing a medicine or a dose because of an AI answer.
• Uploading reports with your name, hospital ID, Aadhaar or phone number visible.

An AI can describe an image confidently and still be wrong, and an X-ray needs a trained doctor who also knows your history. Google's Gemini help page says not to rely on Gemini's responses as medical, legal or financial advice; other companies give similar warnings. Use AI to prepare for the doctor's visit, not to replace it.

8. How to check what the AI saw or heard

Multimodal mistakes have a simple pattern: the AI misreads first, then reasons correctly on the wrong reading. So always check the reading step.

9. Compare the 4 tools

In the tool comparison chapter, all 4 tools read clear printed photos well. Results drop with blurry photos and messy handwriting in every tool.

10. Beginner mistakes and quick fixes

11. Try at home

Recap

Take a photo of one handwritten page from your desk today and turn it into clean text with the [unclear] rule. You will never type from paper again.

Previous chapter: AI with Web Search, Deep Research and AI Mode: When the Answer Is Fresh

Next chapter: How to Make a Resume Using ChatGPT (Free Prompts and Template)

Ravindra Bagale, trainer: linkedin.com/in/ravindra-bagale. The people, store, notes, sales figures and supplier on this page are made up for learning. Upload limits were checked on Google's Gemini Apps help page on 19 October 2026 and can change. AI answers change every time, so your output will be similar, not the same.