pdfplumber for LLMs, RAG and agents

import statistics

import pdfplumber

def table_markdown(rows):
    rows = [[(c or "").replace("\n", " ").replace("|", "\\|") for c in row] for row in rows]
    out = ["| " + " | ".join(rows[0]) + " |", "|" + "---|" * len(rows[0])]
    out += ["| " + " | ".join(r) + " |" for r in rows[1:]]
    return "\n".join(out)

def page_markdown(page, body_size):
    page = page.crop((0, 50, page.width, page.height - 40))   # drop running header and footer
    tables = page.find_tables()
    boxes = [t.bbox for t in tables]

    def outside(obj):  # keep chars that are not inside any table
        return not any(x0 <= obj["x0"] and obj["x1"] <= x1 and top <= obj["top"] and obj["bottom"] <= bottom
                       for x0, top, x1, bottom in boxes)

    blocks = []          # (top, bottom, markdown, is_body_text)
    for line in page.filter(outside).extract_text_lines():
        size = max(c["size"] for c in line["chars"])
        level = 1 if size > body_size * 1.8 else 2 if size > body_size * 1.15 else 0
        text = ("#" * level + " " if level else "") + line["text"]
        prev = blocks[-1] if blocks else None
        if prev and level == 0 and prev[3] and line["top"] - prev[1] < size * 0.6:
            blocks[-1] = (prev[0], line["bottom"], prev[2] + " " + line["text"], True)   # same paragraph
        else:
            blocks.append((line["top"], line["bottom"], text, level == 0))
    for t in tables:
        blocks.append((t.bbox[1], t.bbox[3], table_markdown(t.extract()), False))
    return "\n\n".join(b[2] for b in sorted(blocks, key=lambda b: b[0]))

with pdfplumber.open("samples/report.pdf") as pdf:
    body = statistics.median(c["size"] for p in pdf.pages for c in p.chars)
    md = "\n\n".join(page_markdown(p, body) for p in pdf.pages)

print(md.count("\n| Site "), "table rows,", md.count("\n## "), "section headings,", len(md), "characters\n")
print(md[:620], "…")

50 table rows, 2 section headings, 1717 characters

# Annual inspection report

## 1. Summary

Forty-eight sites were inspected between March and September 2026. Most sites passed; sites marked Follow-up need a second visit before 31 December 2026.

Overall status: 11 sites need follow-up (see table below).

Methodology: pdfplumber.dev/cookbook/

## 2. Results by site

| Site | Result | Reading (ppb) |
|---|---|---|
| Site 01 | Pass | 3.9 |
| Site 02 | Pass | 11.9 |
| Site 03 | Pass | 7.2 |
| Site 04 | Pass | 7.3 |
| Site 05 | Pass | 14.4 |
| Site 06 | Pass | 3.1 |
| Site 07 | Pass | 2.9 |
| Site 08 | Pass | 10.8 |
| Site 09 | Follow-up | 9.8 |
| Site 10 | Pass |  …

Does pdfplumber use AI?

No. It reads what the PDF stores (characters with their positions and fonts, lines, rectangles) and groups them with fixed rules and tolerances. That has three consequences for AI pipelines:

• Deterministic. The same file gives the same text every run, so a chunk’s ID or embedding cache stays valid until the document changes.
• Positions for citations. Every word has a page number and a box, so an answer can link back to the exact spot (search, highlight matches).
• No reading of pixels. Scanned pages have no characters to read. Run OCR first, or use a model-based parser for those files.

Rules have limits that models do not. On our two-column test article, pdfplumber’s default text kept the reading order with a similarity of only 27.7%, because it reads straight across both columns; docling scored 99.6% (benchmark). With column-aware code pdfplumber gets it right (multi-column layouts), but you have to write that code.

Why tables need special handling

Plain extract_text() output loses a table’s columns: the row becomes Site 17 Follow-up 4.0, and a value with a space in it (Follow-up visit) makes the columns ambiguous. A pipe table keeps every cell separate and puts the header with the data, so a model does not have to guess where one column ends:

| Site | Result | Reading (ppb) |
|---|---|---|
| Site 17 | Follow-up | 4.0 |

That is why the converter above extracts tables separately and filters their characters out of the body text. The same filter() technique works with any layout: find the regions you will handle specially, extract them, and exclude them from the rest.

Long tables need one more step for retrieval: when you split a table across chunks, repeat its header row in each chunk, so that a chunk retrieved on its own still says what its columns mean. Tables that continue across pages can be stitched first (multi-page tables).

LangChain: PDFPlumberLoader

LangChain’s built-in loader for pdfplumber is PDFPlumberLoader, from the langchain-community package:

from langchain_community.document_loaders import PDFPlumberLoader

docs = PDFPlumberLoader("samples/report.pdf").load()

print(len(docs), "documents, one per page")
meta = docs[1].metadata
print({k: meta[k] for k in ("source", "page", "total_pages")})
print(docs[1].page_content[:160])

2 documents, one per page
{'source': 'samples/report.pdf', 'page': 1, 'total_pages': 2}
Hollow Pine Supply Co. · Annual inspection report 2026
Site Result Reading (ppb)
Site 15 Pass 4.4
Site 16 Pass 8.4
Site 17 Follow-up 4.0
Site 18 Pass 13.7
Site 

Three things to know, the first two visible in that output:

• One document per page, with the page’s extract_text() as content. The table comes through flattened, as shown above.
• page is zero-based. The second page has page: 1. Add one before showing a page number to a person.
• The package is being retired. LangChain announced on 22 May 2026 that langchain-community is being sunset and archived its repository on 19 June 2026 (announcement). Version 0.4.2 still installs and works; importing it prints a DeprecationWarning pointing to the announcement. The announcement names no replacement package for PDF loaders, and we found none on PyPI. (A package called langchain-pdf exists, but it generates PDFs from text; it is unrelated.)

A loader of your own is short, depends only on langchain-core, and can do better than the old one: tables as separate Markdown chunks, text without the table rows, and page numbers as people count them:

import pdfplumber
from langchain_core.documents import Document

def md_table(rows):
    rows = [[(c or "").replace("\n", " ") for c in r] for r in rows]
    lines = ["| " + " | ".join(rows[0]) + " |", "|" + "---|" * len(rows[0])]
    return "\n".join(lines + ["| " + " | ".join(r) + " |" for r in rows[1:]])

docs = []
with pdfplumber.open("samples/report.pdf") as pdf:
    for page in pdf.pages:
        tables = page.find_tables()
        for t in tables:                                   # each table: one chunk, as Markdown
            docs.append(Document(md_table(t.extract()),
                                 metadata={"page": page.page_number, "kind": "table"}))
        text = page.filter(lambda o: not any(                # text outside the tables
            t.bbox[0] <= o.get("x0", 0) and o.get("x1", 0) <= t.bbox[2] and
            t.bbox[1] <= o.get("top", 0) and o.get("bottom", 0) <= t.bbox[3] for t in tables
        )).extract_text()
        docs.append(Document(text, metadata={"page": page.page_number, "kind": "text"}))
        page.close()

for d in docs:
    print(d.metadata, len(d.page_content), "chars:", d.page_content[:60].replace("\n", " / "))

{'page': 1, 'kind': 'table'} 415 chars: | Site | Result | Reading (ppb) | / |---|---|---| / | Site 01 | 
{'page': 1, 'kind': 'text'} 396 chars: Hollow Pine Supply Co. · Annual inspection report 2026 / Annua
{'page': 2, 'kind': 'table'} 945 chars: | Site | Result | Reading (ppb) | / |---|---|---| / | Site 15 | 
{'page': 2, 'kind': 'text'} 56 chars: Hollow Pine Supply Co. · Annual inspection report 2026 / 2

Swap extract_text() for the Markdown converter above if you also want headings. The resulting Document objects work with any LangChain text splitter, vector store or retriever; split text chunks freely, but pass table chunks through whole or with their header repeated.

Chunking for retrieval

What we would do for a typical business document:

1. Remove running headers and footers by cropping each page to its body (crop). The report’s Hollow Pine Supply Co. · Annual inspection report 2026 line otherwise appears in every chunk and pulls unrelated chunks together in search.
2. Split at headings, detected from font size (headings by font size), then by length inside long sections.
3. Keep each table whole, or split it by rows with the header repeated.
4. Store the page number and a bounding box with each chunk, so answers can cite and highlight their source.
5. Check a few chunks by eye. Wrong reading order and missing spaces are easy to see in text and hard to see in retrieval scores (missing spaces).

Agents and tools

An agent with a Python tool can call pdfplumber directly. The functions on this site map well onto tool calls: “find the page containing X” (search), “read the region next to label Y” (value next to a label), “list the tables on page N” (tables). Narrow tools like these give a model positions and structure it can reason about, which a single dump of a 200-page document does not.

Treat PDFs as untrusted input in agent pipelines. Text inside a document can contain instructions aimed at the model, and malformed files can make any parser slow; run extraction with time and memory limits (performance).

When a model-based parser is the better tool

Use an AI document parser (docling, marker and similar) when the files are scanned, when layouts vary from document to document and you cannot write rules for each, or when reading order across complex layouts matters more than speed. They are much slower: docling took 66 s for the 15 pages pdfplumber read in 1.6 s in our test. The AI parsers comparison has the details. Many pipelines use both: pdfplumber for born-digital PDFs with known layouts, a model for the rest.