Parse a Directory of Documents

Parse every supported file in a folder by submitting each one as a Parse Job on the standard tier, then waiting for the results. standard jobs consume half the credits of a synchronous Parse request, with a slower turnaround, so they suit bulk work where no one waits on each document. See Credit Consumption. For each document, the script saves two files to an output folder, named after the file name and its extension so that invoice.pdf and invoice.png don’t overwrite each other:

<name>_<extension>.md, such as invoice_pdf.md: the document as Markdown, ready to send to Extract.
<name>_<extension>_parse_output.json, such as invoice_pdf_parse_output.json: the full parse response, including the structure tree with the page number and bounding box of every block. Keep it if you plan to ground extracted fields later.

standard jobs count against your plan’s hourly page limit. See Rate Limits.

These examples require the Python or TypeScript client library. Before running a script, set your API key and install the library and any required dependencies.

Scripts

from pathlib import Path
from landingai_ade import LandingAIADE
from landingai_ade.lib.v2_errors import JobWaitTimeoutError

# Initialize client (uses the API key from the VISION_AGENT_API_KEY environment variable)
client = LandingAIADE()

# Replace "data/" with your folder path
data_folder = Path("data/")
output_folder = Path("output/")
output_folder.mkdir(exist_ok=True)

# File types the Parse API accepts (adjust as needed for your use case)
supported_types = {
    ".pdf", ".png", ".jpg", ".jpeg", ".gif", ".bmp", ".tif", ".tiff", ".webp",
    ".docx", ".odt", ".rtf", ".pptx", ".xlsx", ".csv",
}

# Submit one Parse Job per file on the standard tier
jobs = {}
for filepath in sorted(data_folder.iterdir()):
    # Skip folders and file types the Parse API doesn't accept
    if not filepath.is_file() or filepath.suffix.lower() not in supported_types:
        continue
    job = client.v2.parse_jobs.create(
        document=filepath,
        model="dpt-3-pro-latest",
        service_tier="standard",
    )
    jobs[filepath] = job.job_id
    print(f"Submitted {filepath.name}: {job.job_id}")

# Wait for each job, then save its Markdown and the full parse response
for filepath, job_id in jobs.items():
    try:
        # Wait up to an hour for this job (timeout is in seconds)
        job = client.v2.parse_jobs.wait(job_id, timeout=3600)
    except JobWaitTimeoutError:
        print(f"Still running: {filepath.name} ({job_id}). Check it later with client.v2.parse_jobs.get().")
        continue

    if job.status != "completed":
        print(f"Failed: {filepath.name}: {job.error.message if job.error else job.status}")
        continue

    result = job.result
    block_count = sum(len(page.children) for page in result.structure.children)
    print(f"Parsed {filepath.name}: {block_count} blocks")

    # Keep the extension in the output name, so invoice.pdf and invoice.png don't overwrite each other
    output_name = f"{filepath.stem}_{filepath.suffix[1:]}"
    (output_folder / f"{output_name}.md").write_text(result.markdown, encoding="utf-8")
    (output_folder / f"{output_name}_parse_output.json").write_text(result.to_json(), encoding="utf-8")

import LandingAIADE, { JobWaitTimeoutError } from "landingai-ade";
import fs from "fs";
import path from "path";

// Initialize client (uses the API key from the VISION_AGENT_API_KEY environment variable)
const client = new LandingAIADE();

async function main(): Promise<void> {
  // Replace with your folder path containing documents to parse
  const dataFolder = "data/";
  const outputFolder = "output/";
  fs.mkdirSync(outputFolder, { recursive: true });

  // File types the Parse API accepts (adjust as needed for your use case)
  const supportedTypes = new Set([
    ".pdf", ".png", ".jpg", ".jpeg", ".gif", ".bmp", ".tif", ".tiff", ".webp",
    ".docx", ".odt", ".rtf", ".pptx", ".xlsx", ".csv",
  ]);

  // Submit one Parse Job per file on the standard tier
  const jobs: { filename: string; jobId: string }[] = [];
  for (const filename of fs.readdirSync(dataFolder).sort()) {
    // Skip folders and file types the Parse API doesn't accept
    if (!fs.statSync(path.join(dataFolder, filename)).isFile()) continue;
    if (!supportedTypes.has(path.extname(filename).toLowerCase())) continue;
    const job = await client.v2.parseJobs.create({
      document: fs.createReadStream(path.join(dataFolder, filename)),
      model: "dpt-3-pro-latest",
      service_tier: "standard",
    });
    jobs.push({ filename, jobId: job.job_id });
    console.log(`Submitted ${filename}: ${job.job_id}`);
  }

  // Wait for each job, then save its Markdown and the full parse response
  for (const { filename, jobId } of jobs) {
    let job;
    try {
      // Wait up to an hour for this job (timeout is in milliseconds)
      job = await client.v2.parseJobs.wait(jobId, { timeout: 3_600_000 });
    } catch (error) {
      if (error instanceof JobWaitTimeoutError) {
        console.log(`Still running: ${filename} (${jobId}). Check it later with client.v2.parseJobs.get().`);
        continue;
      }
      throw error;
    }

    if (job.status !== "completed") {
      console.log(`Failed: ${filename}: ${job.error?.message ?? job.status}`);
      continue;
    }

    const result = job.result as LandingAIADE.V2ParseResponse;
    const pages = result.structure?.children ?? [];
    const blockCount = pages.reduce((sum, page) => sum + (page.children?.length ?? 0), 0);
    console.log(`Parsed ${filename}: ${blockCount} blocks`);

    // Keep the extension in the output name, so invoice.pdf and invoice.png don't overwrite each other
    const extension = path.extname(filename);
    const outputName = `${path.basename(filename, extension)}_${extension.slice(1)}`;
    fs.writeFileSync(path.join(outputFolder, `${outputName}.md`), result.markdown ?? "", "utf-8");
    fs.writeFileSync(path.join(outputFolder, `${outputName}_parse_output.json`), JSON.stringify(result, null, 2), "utf-8");
  }
}

main();

Collect Jobs That Are Still Running

The wait timeout only stops the script from waiting. The job keeps running on the server, so collect a job the script reports as still running with client.v2.parse_jobs.get() (Python) or client.v2.parseJobs.get() (TypeScript) and its job_id. See Monitor a Parse Job.

When Someone Is Waiting on the Results

Set service_tier to priority in the create call. priority jobs consume credits at the full rate and draw from the same per-minute limit as synchronous requests. See Choose a Service Tier.

Overview

Parse Only

Parse & Extract

Parse a Directory of Documents

Scripts

Collect Jobs That Are Still Running

When Someone Is Waiting on the Results

​Scripts

​Collect Jobs That Are Still Running

​When Someone Is Waiting on the Results

Scripts

Collect Jobs That Are Still Running

When Someone Is Waiting on the Results