Consider an RFQ for a packaged pump system. The buyer sends a data sheet, a 180-page project specification, two arrangement drawings, a motor schedule, a commercial workbook and a link to customer standards. The pump duty sits in the data sheet. The hazardous-area requirement sits in the specification. A drawing note changes the coating system. The workbook asks for a price option that applies only to one site.
A generic document assistant may produce a plausible summary. The bid team needs something stricter: the current source set, each requirement with its location and context, conflicts between documents, missing references, and a review queue for the engineers who own the decisions. That is the standard an industrial OEM should use when evaluating AI document review.
Define the review record before choosing a model
Start with the record an engineer would accept. Each extracted requirement needs a stable ID, the customer document and revision, the page, section, table cell, drawing zone or model entity, the exact source text or feature, the system’s proposed interpretation, applicability, review state and owner. Add the downstream work it can affect: configuration, cost, supplier scope, schedule, inspection, compliance or proposal wording.
The NASA Systems Engineering Handbook uses similar requirement metadata: a unique ID, rationale, traceability, owner, verification method, verification lead and verification level. An RFQ review does not need a space-program process, but it does need enough structure to trace a customer statement into a bid decision.
| Review field | What the engineer needs to see | Why it matters |
|---|---|---|
| Source identity | Document number, title, revision, issue date and customer transmittal | Prevents a correct answer from the wrong issue |
| Location | Page and section, table row and column, drawing zone or model entity | Lets the reviewer check the evidence quickly |
| Requirement | The exact text, symbol, value, unit and stated condition | Preserves qualifiers that a summary can drop |
| Interpretation | The proposed internal meaning and affected product or scope | Exposes where the system has reasoned beyond extraction |
| Decision state | Proposed, accepted, rejected, superseded or waiting for clarification | Stops draft output from becoming an approved answer |
| Effect | Configuration, cost, supplier, schedule, quality, compliance or proposal work to review | Carries the decision into the quote |
Control the source set first
Inventory the package before extracting requirements. Identify duplicates, revisions, referenced documents, password-protected files and attachments that never arrived. Record which transmittal supplied each file. A model cannot compensate for a missing standard or decide that “final_v7.pdf” supersedes a numbered customer revision.
IEC 82045-1 defines document-management metadata and information models for documents associated with an object across their life cycle. ASME Y14.35 covers identification and recording of revisions to engineering product definition data. Those controls belong at the start of AI review. Document identity and revision state are inputs to the review.
Create one active source set for the bid and preserve every superseded set. When the buyer sends a replacement file, compare it with the baseline behind the active quote and reopen the affected work. The separate drawing and specification revision-control guide covers that change process in detail.
Parse each file according to its structure
Engineering packages mix document types that carry meaning in different ways. A specification uses sections, definitions, tables, exceptions and references. A spreadsheet uses headers, formulas, merged cells and tabs. A drawing uses views, zones, leaders, symbols, dimensions and notes. A 3D model can contain product and manufacturing information that never appears as ordinary text.
Flattening every file into a stream of words breaks those relationships. A value may lose its unit, a table entry may lose its column header and a note may lose the feature its leader points to. OCR also needs explicit checks for decimal marks, diameter symbols, superscripts, subscripts and rotated text.
Document layout remains a measurable source of error. IBM Research built DocLayNet because models trained on scientific papers did not generalize well to diverse business documents. Its baseline layout models remained about 10% below human annotator agreement. Engineering drawings add another layer of spatial and symbolic meaning, so test them as their own document class.
| Source type | Structure to preserve | Checks before review |
|---|---|---|
| Specification PDF | Sections, definitions, tables, footnotes, exceptions and references | Page count, headings, table structure and missing referenced sections |
| Scanned document | Page image, text region, rotation and reading order | OCR confidence, units, symbols and a visible source crop |
| Spreadsheet | Sheet, row, column, merged header, formula and named range | Hidden sheets, filtered rows, units and formula results |
| 2D drawing | Title block, revision block, view, zone, callout, leader, note and dimension | Document identity, scale, symbols, tolerances and referenced details |
| CAD or STEP model | Assembly structure, geometry, PMI, attributes and validation properties | File integrity, coordinate system, units and product structure |
| Email or clarification | Thread, sender, date, attachments and the question it answers | Authority, applicable scope and relationship to the governed document |
For STEP files, NIST’s STEP File Analyzer and Viewer reports entity and attribute data, semantic and graphical PMI, validation properties and format errors. ASME Y14.41 covers preparation and revision of digital product definition data sets. A review pipeline should use that structured product data where it exists.
Test retrieval, comprehension and compliance separately
The Autodesk DesignQA benchmark divides engineering document understanding into rule extraction, rule comprehension and rule compliance. That split maps well to industrial RFQ review. First find the governing requirement. Then understand what it means in context. Finally decide whether a proposed configuration, drawing or response satisfies it.
DesignQA combines textual requirements, CAD images and engineering drawings. The researchers reported that the multimodal models they tested struggled to retrieve relevant rules reliably, recognize technical components in CAD images and analyze engineering drawings. A vendor demo that answers five prepared questions does not establish production accuracy across those jobs.
| Review stage | Question | Example test |
|---|---|---|
| Retrieval | Did the system find every applicable source? | Find all coating requirements across the specification, drawing notes and customer standard |
| Comprehension | Did it preserve conditions, units, exceptions and scope? | Explain whether the C5-M requirement applies to internal surfaces, external surfaces or both |
| Compliance | Does the proposed product or response meet the accepted requirement? | Compare the offered coating system and inspection plan with the customer requirement |
| Decision | Who has authority to accept the result or state a deviation? | Route the proposed answer to coatings engineering and the commercial owner |
Require evidence for every answer
Show the source beside the proposed result. For prose, include the document, revision, page, section and exact passage. For a table, show the row and column headers with the cell. For a drawing, show the zone and a crop that includes the note, symbol, dimension or leader in context. For a model, show the entity or feature and the relevant PMI.
The source has to prove the statement. A link to page 86 does not support a claim if the cited value came from page 112. A cropped table cell does not support a result if its unit sits in a header outside the crop. Make the reviewer’s correction part of the record: accepted as written, corrected, rejected, superseded or sent for clarification.
When no source supports the answer, return “not found in the active package” and name the documents searched. That result can become a customer question. An invented answer can become a bad configuration, an omitted cost or a false compliance statement.
Find conflicts across the package
Many expensive review failures involve two valid-looking statements. The data sheet says 60 Hz while the motor schedule says 50 Hz. A general paint specification calls for one system while a drawing note names another. A customer standard sets a default tolerance, but the part drawing gives a tighter value. Finding one answer is not enough. The system has to find the competing answer and preserve any rule of precedence.
Group requirements by the product, interface or obligation they govern. Normalize units for comparison while keeping the customer’s original value. Then flag contradictions, overlapping ranges, missing referenced documents and requirements whose applicability is unclear. The engineer decides which source governs or sends a technical clarification to the customer.
| Conflict check | Example | Review action |
|---|---|---|
| Value conflict | 460 V in the data sheet; 480 V in the motor schedule | Confirm the required supply and affected motor selection |
| Scope conflict | Customer standard applies plant-wide; drawing note excludes stainless parts | Confirm which surfaces or items the exception covers |
| Revision conflict | Assembly drawing points to Rev B; received component drawing is Rev C | Resolve the product baseline before costing |
| Unit conflict | Flow stated in US gpm in one file and m³/h in another | Convert, compare and retain both source values |
| Missing reference | Specification invokes a test procedure that was not supplied | Request the procedure or state an approved bid assumption |
| Precedence unknown | Proposal deviation conflicts with a later customer email | Ask the person with contractual authority to decide |
Route review by consequence and confidence
Do not send every extracted field to a senior engineer. Route objective, high-confidence fields to a quick verification queue. Send safety, performance, fit, compliance and acceptance requirements to the qualified owner even when extraction confidence is high. Low confidence also needs review when the field can change cost, feasibility or delivery.
The routing rule should use both consequence and confidence. Confidence answers how sure the system is about what it read. Consequence answers what happens if the result is wrong. A clear but conflicting hazardous-area requirement still needs engineering judgment. A low-confidence shipping contact can wait.
| Consequence | Confidence | Treatment |
|---|---|---|
| High | Any | Named qualified reviewer must accept or correct the result |
| Medium | High | Batch review with source evidence and exception flags |
| Medium | Low | Individual review before dependent work starts |
| Low | High | Auto-fill as proposed data and audit a sample |
| Low | Low | Leave blank or send to an administrative review queue |
Build the test set from your own documents
Use completed RFQs from the product families you plan to automate. Include clean digital PDFs, scans, wide tables, rotated notes, similar part numbers, customer-specific terms, revised drawings, model-based definition and files with no answer. Record the answer your engineers accept and the source that proves it. Keep a separate test set that the implementation team does not tune against.
Weight the cases by business consequence. Missing one of 200 repeated document references matters less than missing the one test requirement that changes acceptance. Keep false positives in the test too. A system that extracts every number on a page as a requirement will create more review work than it removes.
NIST’s work on testing and evaluating industrial AI makes the same basic point: useful results depend on a test and evaluation method that reflects the industrial system and its risks. Measure the complete review workflow, including the engineer’s corrections and downstream updates.
| Measure | How to calculate it | Failure it exposes |
|---|---|---|
| Source-set accuracy | Correct active documents and revisions ÷ documents required for the case | Review against a missing or superseded file |
| Requirement recall | Applicable requirements found ÷ applicable requirements in the checked case | A cost, performance or compliance obligation disappears |
| Field accuracy | Correct values, units, conditions and scope ÷ extracted fields reviewed | The words are present but their meaning changed |
| Citation accuracy | Claims supported by the exact cited region ÷ claims reviewed | Plausible answers with decorative citations |
| Conflict recall | Known cross-document conflicts found ÷ conflicts in the checked case | The system selects one answer and hides the disagreement |
| Abstention quality | Unsupported or uncertain questions correctly withheld ÷ questions that should be withheld | The model guesses when the package cannot answer |
| Reviewer correction time | Minutes spent checking and correcting each accepted requirement | Automation moves work into a slower review queue |
| Downstream completion | Approved changes reflected in affected quote work ÷ changes that required an update | The review is correct but the estimate or proposal stays stale |
Worked example: review a pump-package RFQ
The system first inventories the request: data sheet DS-104 Rev 2, project specification PS-880 Rev 0, arrangement drawing GA-22 Rev C, motor schedule MS-7 Rev 4, commercial workbook Q-17 and two referenced customer standards. One standard is missing, so the review opens a document request before claiming full compliance.
It extracts the duty point and fluid properties from the data sheet, hazardous-area classification from the project specification, connection orientation from the drawing and voltage from the motor schedule. Each result includes the exact source location. It then flags two conflicts: the data sheet names 60 Hz while the motor schedule names 50 Hz, and the drawing invokes a newer coating specification than the package includes.
Application engineering owns the frequency question because it changes pump speed and motor selection. Coatings engineering reviews the drawing note and asks for the missing specification. Estimating can continue on the unaffected baseplate and instrumentation work. Once the customer answers, the accepted requirements update the configuration, supplier RFQs, cost build, inspection plan and proposal compliance matrix. The system has reduced search and coordination work without taking the engineering decision.
How Bourne handles engineering document review
Bourne assembles the active RFQ package, identifies document revisions and parses specifications, tables, drawings, spreadsheets and product data with their structure intact. It prepares a requirement record with the source beside the proposed interpretation, groups conflicts across files and routes each decision to the engineer who owns it.
After review, Bourne carries the accepted requirement into configuration, costing, supplier work, compliance and the proposal. Rejected and superseded results stay in the history. Missing or conflicting information becomes a controlled customer question with an owner and due date. Engineers make the decision. Bourne handles the search, evidence, routing and follow-through.
Arda Bulut
Arda Bulut is the co-founder and CTO of Bourne and HockeyStack. He leads engineering at Bourne, building the platform people use to create AI products, agents and automations.