Why raw OCR text isn't the same thing as structured invoice data

Any expense-management app, bookkeeping tool, or accounts-payable workflow that already “does OCR” on receipts and invoices has usually solved only the first half of the problem. Optical character recognition turns the pixels in an image into text. It does not turn that text into the vendor name, the line items, and the total that a system of record actually needs — that’s a separate step, and it’s the one that decides whether the tool saves anyone real time.

What generic OCR actually returns

A general-purpose OCR engine returns recognized characters, roughly in the order and position they appeared on the page. What it doesn’t return is labels: nothing marks which line is the vendor name, which line is a line-item description versus a subtotal, or which number is tax versus the grand total. That labeling is exactly the part that varies most from one receipt or invoice to the next — a grocery receipt, a services invoice, and a restaurant bill don’t share a layout. Code that tries to guess field boundaries from raw OCR text by position or keyword-matching tends to work until the next vendor’s format shows up, then breaks again.

The fields a system of record actually needs

What actually gets typed into an accounting system or an expense report is a small, fixed set of fields: vendor name, date, a line-item array (description, quantity, unit price, line total), subtotal, tax, grand total, and currency. That’s the shape the Invoice/Receipt OCR API returns directly from an image or a PDF (sent as base64) — no intermediate wall of text to re-parse, and no per-vendor layout rules to write and maintain.

Structured output still isn’t a guarantee of accuracy

Turning an image into typed fields doesn’t make extraction perfect — it can’t. Skewed photos, faded thermal receipts, handwriting, and unusual invoice layouts can all still produce partially wrong or incomplete output. That’s why every response carries a confidence score (0–1): the extraction provider’s own estimate of how reliable that particular result is. A low-confidence response, or any field returned as null where a value was expected, should be treated as needing a human to check it, not as ground truth.

Where the numbers can be checked against each other, the API checks them. If the line items don’t add up to the subtotal (or the total), or the subtotal plus tax doesn’t equal the total, within a small rounding tolerance, the response carries a totals_mismatch warning — often the sign of a page or a block of items that was never read, which otherwise looks exactly like a complete, confident extraction. And a number or date the model reports but cannot trace to what is printed on the document comes back as null with an unreadable_number warning, rather than as a plausible guess. No warning means none of these checks found a problem, not that every value was read correctly. A high confidence next to a warning is exactly the case the warning exists for: read both.

A real gotcha if you’re integrating this yourself: how you send the image matters

This is worth knowing before you build against any hosted OCR/vision API, not just this one: sending a link to an image (file_url) is not always as reliable as sending the image bytes directly (base64_image). In live testing, a request against a legitimate, publicly-hosted test image failed to download on the extraction provider’s side — most likely a hotlink or bot-protection mismatch on that specific host, not something either side directly controls. That’s a real, reproducible finding, not a hypothetical edge case. The API deliberately does not fetch a customer-supplied URL itself on its own infrastructure (fetching an arbitrary caller-supplied URL from your own server is a known security anti-pattern), so file_url reliability depends on the target host cooperating with the extraction provider’s own fetcher. If you control the image bytes — you scanned it, a user uploaded it to your own storage — sending base64_image sidesteps that dependency entirely.

If you’re building an accounts-payable, expense, or bookkeeping tool and want structured fields instead of a block of recognized text, see the Invoice/Receipt OCR API for the request/response shape and error cases. As with any OCR or vision extraction, always check warnings and confidence before writing a result into a system of record — this API does not guarantee accuracy. Extraction runs on an AI model, so each document is sent to our model provider, Anthropic; Aervik Labs does not store it.