If your AI agent says something isn’t in a document when it clearly is, check the PDF text extraction before you blame the model. Extraction often drops the spaces and punctuation between list items, so a word like "Lakeview" reaches the agent buried inside one long string it can’t match.
We ran into this on a client’s document agent built on Azure AI Foundry. A user asked whether Lakeview was a departure city on one of their certificates (we have changed the city names in this post). The agent said no. The certificate PDF lists Lakeview, in plain sight.
This post shows what the agent actually read, why PDFs lose their spaces, how big the problem turned out to be, and the five steps we used to fix it.
Quick Answer PDF text extraction can drop the spaces between words in lists, tables and columns, so a fact that’s on the page reaches your AI agent glued inside one long string. Search and the model then miss it and answer "not found". Inspect the extracted text, repair it, re-index, and test every document type.
The extracted text explained the wrong answer. This is what the agent was given for the list of departure cities:
...MapletonLakeviewOakridgeRiverton...
A person reads that without thinking. The model saw one long word, didn’t find "Lakeview" as a city in it, and answered "it’s not in the list". It wasn’t a hallucination. The agent answered faithfully from text that was broken before it ever arrived.
The QServices Framework Clean extraction = Extract → Inspect → Repair → Re-index → Test
The order matters: you can’t repair what you haven’t inspected, and a repair does nothing until the index is rebuilt and tested.
A PDF isn’t a text file. It stores characters with positions on a page, so a "space" is often just a gap between two pieces of text rather than a space character. Extractors have to guess where one word ends and the next begins.
Most of the time they guess right. They guess wrong most often in:
The agent never sees the page. In Azure AI Foundry’s file search tool, a file you add is parsed, split into chunks (800 tokens with a 400-token overlap by default) and embedded, and up to 20 chunks go to the model for each answer. Whatever the parser produced is the only version of your document the model ever reads.
Missing spaces look cosmetic. They aren’t, for two reasons.
Search can’t find the word. A question about "Lakeview" is looking for that word. A chunk containing "MapletonLakeviewOakridgeRiverton" doesn’t contain it as a word, so any search that matches words ranks the right chunk lower or misses it.
The model can’t reliably split the string. Even when the right chunk is retrieved, the model has to break the run into city names on its own. Ours didn’t. Then it did the worst possible thing: it gave a confident "no" instead of "I’m not sure".
Key Insight A wrong "no" is harder to spot than a vague answer, because it sounds like the agent checked.
When we checked all 198 certificate types the agent covers, 7 had glued text, and most of it came from a single certificate type. The other 191 extracted cleanly.
That’s why this kind of fault survives testing. Spot-check ten documents and you’ll probably see clean text and correct answers. The failures sit in a few document types, and they only show up when someone asks about the exact list that broke.
These are the faults we look for first. The fix for each one has a cost, so the last column says what it takes.
| Fault | What the extracted text looks like | What the agent does | Fix | What the fix costs |
|---|---|---|---|---|
| List items glued together | MapletonLakeviewOakridge |
Says an item isn’t there when it is | Split against the known values, then re-index | You need the list of valid values |
| Table read straight across | Column headers followed by every value in one line | Pairs a value with the wrong row | Layout-aware extraction that keeps tables as tables | A second extraction service to run and pay for |
| Words split at line ends | recon- ciliation |
Misses the term in search | Re-join hyphenated breaks before chunking | Can wrongly join real hyphenated words |
| Headers and footers inside the text | Page titles and page numbers mid-sentence | Quotes boilerplate as if it were content | Strip repeated header and footer lines | Rules per document template |
| Two columns read as one | Sentences from both columns interleaved | Gives answers that mix two topics | Extract with reading order, or split columns first | Slower extraction on long files |
Standards & References
- Microsoft Learn: Foundry file search tool: how agent files are parsed, chunked and retrieved
- Microsoft Learn: RAG with Document Intelligence: Markdown layout output for chunking
1. Measure it across every document type. Scan the extracted text for long runs of letters with capitals in the middle, such as 25 or more letters with no space. Make the check cheap enough to run on the whole collection, not a sample. That tells you how many documents are affected and which ones.
2. Repair the text before indexing. The simple fix is to put a separator wherever a lowercase letter runs straight into a capital. Be careful with that, because real names break it: McAllen, DeKalb and LaGrange would all get split. Where you know the vocabulary, as we did with a list of cities, split against that list instead. It’s slower to set up and it doesn’t damage real names.
3. Re-index. Cleaning the source file doesn’t help if the old chunks are still in the vector store. Rebuild the index from the repaired text so search and the model both see the fixed version.
4. Use layout-aware extraction where structure matters. Azure AI Document Intelligence’s layout model can return a document as Markdown, with paragraphs, headings and tables kept apart. Microsoft recommends using that output to chunk on paragraph boundaries and give tables their own chunks. For documents that are mostly lists and tables, that structure is the difference between a usable index and a guessing game.
5. Keep regression questions per document type. For every document type, write a few questions whose answer sits in a list or a table, including at least one where the right answer is "no". Run them after every re-index. That’s how the next glued list gets caught by a test instead of by a customer.
We test the extracted text the same way we test the answers, because the agent can only be as right as what it reads.
Implementation Checklist
- Extracted text reviewed for every document type, not a sample
- Scan for words over 25 letters run on the whole collection
- Repairs use a list of valid values where one exists
- Index rebuilt after every repair
- At least one list question and one table question per document type
- At least one test where the right answer is "no"
- Regression questions run after every re-index
If an agent is giving answers your team can’t trust, start with the documents, not the model. Pilot the five steps on the one document type that fails most, and measure the share of list and table questions answered correctly before and after. Book a process assessment and we’ll show you which of your document types break in extraction and what fixing them takes.
Written by Rohit Dabra
Co-Founder and CTO, QServices IT Solutions Pvt Ltd
Rohit Dabra is the Co-Founder and Chief Technology Officer at QServices, a software development company focused on building practical digital solutions for businesses. At QServices, Rohit works closely with startups and growing businesses to design and develop web platforms, mobile applications, and scalable cloud systems. He is particularly interested in automation and artificial intelligence, building systems that automate routine tasks for teams and organizations.
Usually because the text it was given doesn’t match what’s on the page. If PDF text extraction dropped the spaces between words, the term the user asked about may sit inside one long string that search and the model don’t recognize. Check the extracted text for the page in question before changing prompts or models.
Don’t assume it does. File search parses and chunks what it gets from the file. Look at the extracted text for your hardest documents, especially lists and tables, and repair or re-extract before you rely on the answers.
Run a scan for abnormally long words, for words with capitals in the middle, and for repeated header and footer lines across every document type. Then ask a few questions per type whose answers sit in lists and tables, including ones where the right answer is no.
For documents with tables, lists or several columns, yes, it usually helps. A layout-aware extractor that outputs Markdown keeps that structure, so you can chunk by paragraph and keep each table together instead of slicing through it.
When PDF text extraction drops the spaces between words, AI agents can’t find facts that are right there. How we found it in Azure AI Foundry and fixed it.
Pint AE explained for UAE finance teams with one sample invoice: see which fields come from your ERP, which go missing, and how to test before go-live.
Days sales outstanding explained for CFOs: the formula, worked examples, named industry benchmarks, and why seasonal sales distort DSO. See what moves it.
Dynamics GP end of life is closer than it feels. See the real Microsoft dates, your three options, and what each one means for AR and open invoices.
Follow one payment through the cash application process, step by step, and see the exact moment it stalls so your AR team can clear unapplied cash faster.
UAE e invoicing requirements for Phase 2 businesses under AED 50M: the dates, the ERP data to fix first, and how to pick an accredited provider.
When PDF text extraction drops the spaces between words, AI agents can’t find facts that are right there. How we found it in Azure AI Foundry and fixed it.
Pint AE explained for UAE finance teams with one sample invoice: see which fields come from your ERP, which go missing, and how to test before go-live.
Days sales outstanding explained for CFOs: the formula, worked examples, named industry benchmarks, and why seasonal sales distort DSO. See what moves it.
Dynamics GP end of life is closer than it feels. See the real Microsoft dates, your three options, and what each one means for AR and open invoices.
Follow one payment through the cash application process, step by step, and see the exact moment it stalls so your AR team can clear unapplied cash faster.
UAE e invoicing requirements for Phase 2 businesses under AED 50M: the dates, the ERP data to fix first, and how to pick an accredited provider.
Earning Global Recognition: A Testament to Quality Work and Client Satisfaction. Our Business Thrives on Customer Partnership
Founder and CEO
Chief Sales Officer
Join our live webinar on Sept 24 and see five
ways to automate meetings, approvals, and
document search with tools already in your
tenant.
