IDP Activities

Version: 1.18.0  ·  ID: zoan-packages-idp  ·  Author: Zoan Software

The IDP (Intelligent Document Processing) package reads documents whose layout changes from one issuer to the next: equipment records, invoices, certificates. The idea is to describe what each value means rather than where it sits, so one configuration handles every layout of the same kind of document.

How this package works

• No session: each activity takes the document path and works on it.
• The route picks itself: IDP Load Document checks whether the PDF has a text layer. If it does, it reads it (free and exact); if not, it scans it with Azure Document Intelligence. Both routes return the same structure, so nothing downstream changes.
• Coordinates are unified: every word comes back in pixels at the dpi you ask for, with a top-left origin, whichever route the document took.

Why the coordinates matter

Each word carries its box (x, y, width, height). That is what lets you go back to the document from an extracted value: highlight where a serial number came from, show it to a person to validate, or audit an extraction months later.

Use the same dpi in PDF To Images and the boxes land exactly on top of the rendered page, with no conversion at all. That is the foundation any visual review is drawn on.

Activities

The first two are the normal path. The ones below are still there for when you need to get in between.

Activity	Type	Output	What it does
IDP Classify	idp-classify	object	Reads a document and tells which kind it is. Returns the document already read
IDP Extract	idp-extract	object	Extracts an extraction project’s fields, routes to the format profile and locates every value on the page
IDP Load Document	idp-load-document	object	Loads a document and returns its text and its words with positions
IDP Build Classifier	idp-build-classifier	object	Builds the schema that identifies which kind of document it is
IDP Build Schema	idp-build-schema	list	Splits a taxonomy into the schemas that fit in one call
IDP Ground	idp-ground	object	Locates every extracted value and flags what needs review

The pipeline

Two nodes:

IDP Classify  →  is it the kind you want?  →  IDP Extract

IDP Classify returns the document already read along with the verdict, and IDP Extract reuses it: the file is read once. With a scan that is not a minor optimisation — reading it again is paying Azure twice.

They are separate on purpose. What to do with a document that does not belong is your decision, not an activity’s: one that quietly decides not to do its job hides the control flow, and it also rules out the one thing you actually need when the queue brings a mix — classify once and route to the right project.

One call, not a pipeline

IDP Extract extracts all fields in a single model call (no strict grammar), grounds and persists the result — all inside, without your playbook seeing it. It used to split the schema into batches; not anymore. The granular activities (IDP Build Schema, IDP Build Classifier) are still registered for anyone assembling the pipeline by hand.

Two guarantees, and they are not the same

Worth keeping apart in your head, because they protect against different things:

• Classifying answers “is this document the kind I think it is?”. Without that step, an invoice processed as an equipment record can slip a wrong value through with high confidence.
• Grounding answers “is this value written in the document?”. It catches the model making a value up, not the document being the wrong one.

Neither covers the other. See IDP Build Classifier for the exact case that slips through.

Scanned documents

The scan route uses Azure Document Intelligence (the prebuilt-layout model), which reads handwriting — which matters because in many documents the part nobody ever digitised (a maintenance history, a signature, a margin note) is handwritten. It needs no training and no labelling: it works on any document from day one.

Can’t use the cloud?

If the client refuses to let documents leave their network, compose the on-prem route in the playbook: PDF To Images to rasterise and the OCR package (Tesseract) to read. Assume Tesseract will not read the handwriting.

The full playbook

Enqueue (once per batch):

List Files   path = = env("HOSPITAL_DOCS_DIR")   pattern = "*.pdf"   recursive   → files
Foreach   items = = files   itemVariable = path
    Queue Add Item   queueName = "device-records"   reference = = path
                     specificContent = { "path": = path }

reference = the path. It is the queue’s idempotency key and the key the document will carry in Zoan Cloud, so re-running enqueues nothing twice and you can go from the item to its document.

Process:

Queue Process   queueName = "device-records"   itemVariable = item   onError = continue

  Set Variable   name = content   value = = item.SpecificContent

  IDP Classify   path = = content["path"]
                 projects = ["device-record"]
                 credential = = credential("anthropic-key")
                 dpi = 200
                 azureCredential = = credential("azure-di-key")
                 → doc

  If   condition = = doc["recognized"] == false
    Queue Set Transaction Status   item = = item   status = "successful"
                                   output = { "discarded": true, "reason": = doc["reason"] }
    Continue

  IDP Extract   document = = doc                    ← already read: not read again
                project = "device-record"
                credential = = credential("anthropic-key")
                azureCredential = = credential("azure-di-key")
                → res

  PDF To Images     path = = content["path"]   dpi = 200   outputDir = = tmpFolder   → pages
  Documents Upload  path = = content["path"]   reference = = item.Reference
                    pages = = pages   dpi = 200   extraction = = res
                    documentType = = res["documentType"]   → uploaded

  Queue Set Transaction Status   item = = item   status = "successful"
                                 output = { "documentId": = uploaded["id"],
                                            "needsReview": = res["needsReview"],
                                            "values": = res["values"] }

The project goes by name, and carries everything

project = "device-record" is the name of an extraction project of the environment, edited in the portal. It carries the canonical schema, the format profiles, the model, the reading mode and the OCR endpoint — so the person who knows which fields matter (a biomedical engineer, not whoever writes playbooks) changes them without opening the Designer or redeploying, and the playbook stays free of model/mode/endpoint.

For CI and the CLI, which have no Zoan Cloud session, project also accepts an inline project-resolve JSON.

The dpi must be the same in both places

PDF To Images renders at the same dpi you gave IDP Classify. That is what makes the boxes land exactly on the image in the validation station, with no conversion.

With mismatched dpi nothing fails: the boxes just land in the wrong place, and nobody tells you.

Uploading is the playbook’s call, not the IDP’s

The IDP uploads nothing — it knows how to read documents, not how to talk to Zoan Cloud. Uploading is a separate node you choose to call.

That is what keeps the on-prem route alive: a client who won’t let documents leave their network simply doesn’t install the Documents package, and the rest works the same.

Telemetry: close the item yourself

Queue Process closes each item on its own — successful if the body finishes, failed if it throws — but it closes it with an empty output: it throws the result away. With the auto-close, your telemetry is “processed 400, 12 failed” and nothing else.

That is why the example calls Queue Set Transaction Status by hand: if the body already set the status, the auto-close steps aside. Each document then leaves a queryable trail — what was extracted, at what confidence, why it went to review, and why it was discarded if it was.

That gives you:

Where	What you get
GET /queues/:id/metrics	Total, counts by status, success rate, throughput per hour, average and p95 times.
GET /queues/:id/items	Each document with its reference, status, and the output you attached.
The *-review queue	How many, and which, need a person.

Zoan Cloud’s review status is not IDP validation

It is an intake gate: with accept_automatically = false, items are born in review and approving them releases them to the bot. It answers “should this be processed?”, not “are these fields right?”. IDP validation is the second queue.

Next steps

• IDP Load Document — load the document.
• IDP Build Classifier — know what kind it is before extracting.
• IDP Build Schema — describe what to extract, with one taxonomy per document type.
• IDP Ground — locate what was extracted and decide what goes to review.
• OCR — on-prem route for clients without cloud.