The Response Shape
The response contains these top-level fields:Markdown
Themarkdown field is a single string containing the document in reading order, following CommonMark 0.31.2 with a few extensions. Use it directly for RAG, search indexing, or anything that takes Markdown as input. Every range elsewhere in the response points into this string, using Unicode code point offsets as declared by metadata.range_units.
Math formulas are transcribed as LaTeX, wrapped in $...$ for inline math or $$...$$ for display math, matching the original layout. Superscript and subscript text that is not part of a math formula is wrapped in <sup> and <sub> tags.
Figures
A figure is wrapped in an HTML-style element whosetype attribute is the classification label: <figure type="TYPE">...</figure>. Inside the element, the figure’s visible text is transcribed, and generated descriptions of the visual content appear in <description>...</description> blocks. A CHART figure also transcribes the chart’s data as an HTML <table>.
The figure type labels are CHART, FLOWCHART, DIAGRAM, ILLUSTRATION, and PHOTOGRAPH, with FIGURE as the fallback.
Example
A bar chart could render like this:
Attestations
An attestation is a block that has a signature, stamp, or seal. It renders as a header line of one or more bracketed type labels, followed by the block’s transcribed content. The type labels areSIGNED, E-SIGNED, STAMPED, and SEALED. The type labels stack when a block has more than one.
Within the content, readable text is transcribed as prose, and every visual mark is replaced by a short bracketed description of what it is. The model generates these descriptions, so treat their wording as variable rather than a fixed set of labels.
Two of these descriptions are fixed literals you can build logic around:
[ILLEGIBLE_SIGNATURE]: a signature that can’t be read.[ILLEGIBLE_TEXT]: any other unreadable text.
Structure: Pages and Blocks
Thestructure field is a document node whose children are pages, and each page’s children are the blocks on that page. Every node below the root document carries its location inline in a grounding object, so hierarchy and spatial data live in one tree.
Common Block Fields
Every block in the structure tree shares the same base shape:Page Fields
page nodes carry their own grounding plus parse status fields:
Block Types
recognizes the following block types:Tables and Cells
Atable block has its own id and grounding, and a children array of table_cell blocks in row-major order. Every cell uses the table_cell type, including cells in header rows. Each cell carries its grid position:
Cell positions are 0-indexed even though page numbers are 1-indexed.
A table is the one block type that has children but no
atomic_grounding; its cells are leaf blocks whose atomic_grounding is always an empty array. A cell’s range points to its own content in the Markdown, so an empty cell has a zero-length range.
Grounding: Where Things Are
Every node below the rootdocument carries a grounding object. The same shape appears on pages, blocks, and each atomic_grounding entry, so any grounding object is self-contained: you can lift it out of the tree and it still locates its content.
Working with Boxes
Boxes are resolution-independent. To convert a box to pixels, multiply itsx values by the width and its y values by the height of whatever rendering of the page you draw on. The same box lines up at any resolution because it scales with the dimensions you supply.
For images with EXIF orientation metadata (such as phone photos), the API applies the orientation before parsing, so boxes align with the upright image as a viewer sees it. Use the upright dimensions when converting.
Python
Atomic Grounding
Leaf blocks (every type excepttable) also carry atomic_grounding: an array of grounding objects, each with the same {page, range, box} shape as the block’s own grounding, one entry per finer-grained segment.
- For
textandmarginalia, there is one entry per visual line, so you can highlight or extract content at the line level. Line ranges are ordered, non-overlapping sub-ranges of the block’s range; consecutive lines are separated by the newline that joins them in the Markdown. - For
figure,logo,card,scan_code, andattestation, there is one entry per transcribed segment the model can localize, or a single entry covering the block’s full range and box when no finer segmentation exists. - For
table_cell, the array is always empty; a cell has no finer granularity than itself. - Blocks suppressed with
blocks.<type>.markdown=falsealso return an empty array. - The field never appears on
document,page, ortablenodes.
options.atomic_grounding to false.
Example: Multi-Line Paragraph
A paragraph that wraps across two visual lines produces onetext block with two atomic_grounding entries. The block-level box covers both lines; each atomic entry covers one line:
Look Up Blocks by ID
To look up blocks byid instead of walking the tree, flatten it into a map once:
Python
Slice the Markdown with Python
To get a block’s text, index themarkdown string with its range offsets. Python string indexing is code-point based, so the offsets map directly onto the string:
Python
options.inline_markdown to true; each node then carries its own markdown field.
Slice the Markdown with JavaScript
In JavaScript, slice a code-point array so the offsets stay aligned. JavaScript strings index by UTF-16 code units, so building the array withArray.from() first keeps ranges correct even when the Markdown contains characters outside the Basic Multilingual Plane, such as emoji or some CJK characters. Build the array once and reuse it across ranges:
JavaScript
Metadata
Themetadata field describes the job.
A complete
metadata block could look like this: