A document-processing demonstration can look convincing while failing the first operational test. One clean invoice is read correctly, the fields appear in a table and the summary sounds plausible. Then a scan arrives with two pages missing, a handwritten correction and a total expressed in a different currency. The workflow still needs to know what is evidence, what is uncertain and whether it is safe to act.
“Document AI” and “generative AI” are not competing products to select in isolation. A document process may need file checks, optical character recognition, layout parsing, field extraction, interpretation, deterministic validation, human review and integration. The right architecture assigns each task to the lowest-uncertainty component that can do it.
Decompose the document job before choosing a model
Start with the output and its consequences. Extracting a reference for search has a different error cost from extracting the bank details used for a payment. Summarising a policy for an expert has a different control need from deciding whether a claim is complete.
The Government Digital Service: Artificial Intelligence Playbook for the UK Government advises teams to define the goal, use the right tool and understand limitations. Applied to documents, that means separating at least five capabilities:
1. Acquisition and file control
Verify the source, file type, completeness, malware status and version before interpreting content. Detect blank pages, password protection, corrupt files and duplicates. If a document can be replaced or amended, keep its identity and history.
2. OCR and layout parsing
Optical character recognition turns an image into text. Layout-aware parsing preserves relationships such as headings, tables, columns and key-value pairs. Neither step proves that the source is genuine or that an extracted value is correct.
3. Schema-led extraction
For stable document families, specialised extraction can map known fields and tables into a defined structure. It is easier to validate when the expected output is explicit: invoice number, date, currency, line value, tax and total.
4. Interpretation
A generative or multimodal model can compare clauses, classify variable correspondence, summarise context or propose an answer where language and layout vary. Its flexibility introduces uncertainty: it may omit a qualification, join unrelated passages or produce a claim that is not supported by the document.
5. Workflow and action
Routing, validation, approval, exception handling and writing to the system of record produce the operational outcome. A capable model inside an uncontrolled workflow remains an unreliable process.
Use a hybrid path for consequential work
A controlled architecture keeps source evidence and irreversible action apart. A common path is:
- Register: capture source, version, time, sender and file integrity.
- Read: run OCR and layout parsing while retaining page coordinates and the original file.
- Extract: map known fields into a typed schema and preserve confidence or validation status.
- Interpret selectively: use a generative model for bounded classification, comparison or summarisation.
- Validate: apply deterministic rules to critical formats, calculations, reference data and cross-field consistency.
- Escalate: send uncertain or consequential cases to a reviewer with the relevant source passage visible.
- Commit: write only approved values or actions to the system of record and retain an event trail.
This pattern is not mandatory. Stable, high-volume forms may need no generative component. Highly variable expert material may need interpretation but no automated write-back. The architecture should become simpler when the task allows it.
Treat documents as data with duties attached
Documents often contain more information than the target task requires: names, addresses, health details, signatures, financial data, internal comments and data about third parties. Sending the full file to every component because it is convenient conflicts with purposeful design.
The Information Commissioner’s Office: data minimisation principle requires personal data to be adequate, relevant and limited to what is necessary for the purpose. The ICO’s more specific Information Commissioner’s Office: security and data minimisation in AI recommends mapping the machine-learning processes in which personal data is used and considering minimisation from design and procurement.
For each component, record what content it receives, why, where it is processed, how long it remains, whether it is used to improve a service and which organisation decides the purpose and means. Redaction, field selection and page selection can reduce exposure before a generative model is called.
When several organisations provide OCR, model hosting, storage or workflow services, controllership and processor responsibilities may differ by operation. The Information Commissioner’s Office: accountability and governance implications of AI stresses the need to document those roles and use a DPIA where the processing is likely to create high risk for people.
For EU operations, the European Data Protection Board: Opinion 28/2024 on personal data in AI models is relevant to questions about anonymity, legitimate interest and the effect of unlawfully processed personal data. It does not answer every deployment question; it is a reason to establish the data route rather than assume a model is detached from its inputs.
Build the evaluation set before the interface
Three hand-picked documents establish that a path runs. They do not show whether it works. Construct a versioned evaluation set from the real variation in the workflow: good files, poor scans, rotated pages, unusual tables, changed templates, missing pages, handwriting, multilingual content, contradictory values and documents that should be rejected.
The NIST: AI Risk Management Framework places measurement inside continuous risk management. Its NIST: Generative AI Profile adds risks and suggested actions for generative systems. Translate that discipline into measures at the level where the process can fail:
- File acceptance: proportion of valid documents accepted and invalid or incomplete documents rejected.
- Field quality: performance for each critical field, including false acceptance and false rejection.
- Grounding: whether every interpretive output is supported by the cited page and passage.
- Calculation: whether totals and relationships pass independent deterministic checks.
- Exception quality: whether uncertain cases are routed with enough evidence for efficient review.
- Workflow outcome: whether approved information reaches the right system without duplication or silent loss.
- Change resilience: performance after a template, prompt, model or upstream system changes.
Do not rely on one average accuracy number. A system can perform well overall and still fail on the field that controls payment, eligibility or filing. Set thresholds by consequence and make important failure classes visible.
Keep source, inference and decision separate
Data-protection accuracy and model statistical accuracy are related but distinct. The Information Commissioner’s Office: accuracy and statistical accuracy in AI distinguishes factual accuracy of personal data from how often a model predicts the expected answer. Document systems should label extracted facts, model inferences and human decisions accordingly.
Preserve provenance for each material output: document ID and version, page, region or passage, extraction method, model and prompt version, validation result and reviewer decision. A summary without source references may help someone browse; it should not quietly become an authoritative record.
If two documents conflict, the model should not invent a reconciliation. The workflow needs a source hierarchy or an exception. If a policy has expired, retrieval should exclude it or warn the user. If a value is inferred rather than stated, label it as an inference.
Design review around consequence
Human review is useful when the reviewer has the source, the extracted or generated result, the reason for escalation and the authority to change the outcome. Sending every case to a person can recreate the original workload and encourage automatic approval. Sending only low-confidence cases can miss confidently wrong output.
Combine confidence with risk signals: critical field, unusual format, rule conflict, new counterparty, missing evidence or high-consequence action. Sample apparently successful cases as well as exceptions. Reviewers’ corrections should feed an error taxonomy and evaluation set, not disappear into an inbox.
Plan for change and failure
Document systems drift even when the model does not. Suppliers revise templates; organisations update policy wording; scanners change; users discover workarounds. Maintain a change register and rerun the relevant evaluation before releasing a new model, prompt, extractor, rule or document version.
ISO: ISO/IEC 42001 AI management systems provides a useful organisational frame for objectives, responsibilities, risk, change and continual improvement. The standard cannot certify that a particular extraction is correct; it can help make the process for owning and reviewing the system explicit.
The right answer is to stop automated processing when critical outputs cannot be traced to the source, representative test material cannot be obtained lawfully, exceptions cannot be reviewed competently, or an error would trigger an action that cannot be contained. In those cases, improve the input process, narrow the task or retain manual handling.
Use generative AI where bounded interpretation adds value. Use schema-led or deterministic components where consistency can be specified. Put both inside a workflow that exposes uncertainty and prevents an attractive output from becoming an unsupported action.
Further reading: Information Commissioner’s Office: guidance on AI and data protection; European Data Protection Board: Opinion 28/2024; NIST: AI Risk Management Framework; ISO: ISO/IEC 42001 AI management systems.
