Receipt OCR Benchmark with LLMs

Extracting knowledge from receipts is crucial for companies since tens of millions of workers are submitting their work associated bills by way of receipts.

With the most recent developments in generative AI and large language models, knowledge extraction accuracy has reached roughly human ranges.

Benchmark outcomes

We used Claude 3.5 Sonnet to measure the receipt knowledge extraction accuracy of LLMs:

Dataset

We divided our dataset into two components:

• Top quality: Scanned, excessive decision receipts. These photos are aligned properly, with excessive distinction.

• Low high quality: Photographed, low high quality receipts. These photos will not be aligned correctly, with no pre-processing to make distinction larger.

Our purpose is to cowl real-life instances as a lot as attainable.

We requested for a JSON output to make analysis simpler. Our immediate is: Please output the textual content on the PDFs in a correct JSON format.

Methodology

Outcomes had been evaluated at key-value pair stage:

• If a discipline contains the proper label and worth, it’s marked as appropriate.

• If there are any character variations vs the bottom reality within the label or the worth, that row is marked as false.

Extraction accuracy: Variety of appropriately extracted key-value pairs divided by the overall variety of key-value pairs.

Subsequent steps

We are going to add extra LLMs (ChatGPT and so forth.) to this benchmark to look at their capability to knowledge extraction higher.

What’s receipt OCR?

Receipt OCR (Optical Character Recognition) is a know-how that extracts knowledge from scanned and digital receipts utilizing synthetic intelligence and machine studying algorithms. Receipt OCR parses the info, converts it to a structured format and captures particulars within the receipt, like date, objects, and costs.

To extend the accuracy of the OCR, the pictures needs to be:

• In larger decision

• Aligned properly

• Freed from printing errors

Try to be conscious of:

A lot of the receipt OCR instruments fail in matching the proper merchandise with appropriate value when there’s a observe concerning the merchandise within the subsequent line with no pricing listed. In that case, it’s common for instruments to learn the following merchandise’s value because the observe’s value. To see clearly, let’s have a look at the instance:

In such instances, the output of OCR might match “SpcyDlx +PJ” with the value 0.40, which isn’t appropriate. It’s attainable particularly within the instances the place picture decision and high quality is low, and the picture isn’t aligned straight.

We observed that within the case of low decision or printing errors (ink doesn’t cowl the letter utterly and so forth.), instruments are having hassle in absolutely figuring out comparable letters and numbers. Like “8” and “9” or “5” and “6”. Additionally having hassle in figuring out “/” and “1” is a typical case, particularly in dates.

• Receipt quantity

• Date

• Vendor identify

• Subtotal quantity

• Tax quantity

• Complete quantity

• Bought objects

• Receipt scanning: Scanning the receipt with excessive decision. OCR receipt scanning helps getting extra prime quality photos than taking images of the receipts.

• Receipt processing: To extend distinction and readability of the enter picture, processing receipts could also be wanted.

• Receipt parsing: Parsing the receipt picture is crucial to research and seize knowledge, it breaks down knowledge into extra organized parts.

• Utilizing structured knowledge: Structured knowledge can be utilized to automate knowledge entry in current techniques like accounting software program. Related knowledge can be utilized in so many instances like following the transaction date in monetary information and expense administration. By routinely extract knowledge from receipts through the use of LLMs or receipt OCR apis can scale back errors and handbook entry and will increase total effectivity with excessive accuracy.

FAQ

Exterior Hyperlinks