Troubleshooting
Damaged and malformed PDFs
Damaged PDFs fail in two ways: loudly, with an exception, or quietly, with a page that comes out empty. The quiet failures are the dangerous ones. We broke a three-page bank statement in seven realistic ways and recorded exactly what pdfplumber and its Ghostscript repair do with each.
Short answer
| Message | Usual cause | First thing to try |
|---|---|---|
PdfminerException: No /Root object! - Is this really a PDF? |
Not a PDF (HTML, empty file), or the cross-reference table is missing | Check the first bytes are %PDF-; then repair=True |
PdfminerException: Unexpected EOF |
The file is cut off: interrupted download or upload | Get the complete file; repair=True may save some pages |
| No error, but a page has no text | A damaged page stream, or a scanned page | Count page.chars on every page and flag zeros |
RecursionError |
Circular or extremely deep object references | Upgrade pdfplumber (0.11.6 handles more of these), then repair=True |
pdfplumber wraps pdfminer.six’s exceptions in PdfminerException; the original is e.args[0].
Seven broken files, tested
Each file below is a copy of the three-page bank statement, damaged in one way that happens in real life. Download them to test your own pipeline:
| File | Damage | Happens when |
|---|---|---|
| empty.pdf | Zero bytes | A failed download or upload |
| html-page.pdf | An HTML page with a .pdf name | A login wall or error page saved by a scraper |
| truncated-33.pdf, truncated-90.pdf | Cut off at a third and at 90% | Interrupted transfers, size limits |
| bad-startxref.pdf | The pointer to the cross-reference table is wrong | Files edited by tools that do not update it |
| no-xref.pdf | Cross-reference table missing | Damaged or hand-assembled files |
| bad-page-stream.pdf | Page 2’s compressed content is garbage | Storage or transfer corruption inside one object |
What pdfplumber makes of each, with no options:
from pathlib import Path
import pdfplumber
from pdfplumber.utils.exceptions import PdfminerException
for path in sorted(Path("samples/broken").glob("*.pdf")):
try:
with pdfplumber.open(path) as pdf:
counts = [len(page.chars) for page in pdf.pages]
print(f"{path.name:20} opened, chars per page {counts}")
except PdfminerException as e:
inner = e.args[0]
print(f"{path.name:20} {type(inner).__name__}: {inner}")bad-page-stream.pdf opened, chars per page [1181, 0, 801]
bad-startxref.pdf opened, chars per page [1181, 1140, 801]
empty.pdf PDFSyntaxError: No /Root object! - Is this really a PDF?
html-page.pdf PDFSyntaxError: No /Root object! - Is this really a PDF?
no-xref.pdf PDFSyntaxError: No /Root object! - Is this really a PDF?
truncated-33.pdf PSEOF: Unexpected EOF
truncated-90.pdf PSEOF: Unexpected EOFThree things stand out:
- pdfminer.six recovers some damage on its own. A wrong
startxrefpointer is common, and the parser falls back to scanning the file for objects: all three pages came out complete. - The messages are not specific. An empty file, an HTML page and a missing cross-reference table all give No /Root object!. Look at the file’s first bytes to tell them apart.
- One damaged page fails silently.
bad-page-stream.pdfopened without an exception or a warning, and page 2 simply has no characters. A pipeline that only catches exceptions would store an empty page and move on.
Read damaged files safely
For batch jobs, check what you can before parsing, catch errors per page rather than per file, and treat a page without characters as a problem to report:
from pathlib import Path
import pdfplumber
def read_pdf(path):
"""Text per page, plus a list of problems, without crashing on damaged files."""
data = Path(path).read_bytes()
if not data.lstrip()[:5] == b"%PDF-":
return None, [f"not a PDF (starts with {data[:15]!r})"]
if b"%%EOF" not in data[-1024:]:
problems = ["no %%EOF at the end: probably truncated"]
else:
problems = []
try:
pdf = pdfplumber.open(path)
except Exception as e: # PdfminerException and friends
return None, problems + [f"cannot open: {e.args[0]!r}"]
texts = []
with pdf:
for page in pdf.pages:
try:
texts.append(page.extract_text())
if not page.chars:
problems.append(f"page {page.page_number}: no characters")
except Exception as e:
texts.append(None)
problems.append(f"page {page.page_number}: {type(e).__name__}")
page.close()
return texts, problems
for name in ["bank-statement.pdf", "broken/bad-page-stream.pdf", "broken/truncated-90.pdf", "broken/html-page.pdf"]:
texts, problems = read_pdf(Path("samples") / name)
print(f"{name:28} pages read: {0 if texts is None else len(texts)} problems: {problems or 'none'}")bank-statement.pdf pages read: 3 problems: none
broken/bad-page-stream.pdf pages read: 3 problems: ['page 2: no characters']
broken/truncated-90.pdf pages read: 0 problems: ['no %%EOF at the end: probably truncated', "cannot open: PSEOF('Unexpected EOF')"]
broken/html-page.pdf pages read: 0 problems: ["not a PDF (starts with b'<!DOCTYPE html>')"]The intact statement passes; the damaged page is caught; the truncated file and the HTML page are identified before or at opening, with a reason you can log. A page without characters can also be a scanned page rather than damage: the check catches both, and either needs a look.
Run untrusted files with limits as well. A malformed PDF can make any parser loop or allocate memory without end, so extract in a separate process with a timeout (performance shows the process pattern) and a memory limit.
repair=True: rewriting with Ghostscript
pdfplumber.open(path, repair=True) first rewrites the file with Ghostscript, then opens the rewritten copy. Ghostscript is a separate program: install it from your package manager or ghostscript.com, or pass its location with gs_path=. On the files that pdfplumber could not open:
from pathlib import Path
import pdfplumber
for name in ["no-xref.pdf", "truncated-90.pdf", "truncated-33.pdf", "empty.pdf", "html-page.pdf"]:
path = Path("samples/broken") / name
try:
with pdfplumber.open(path, repair=True) as pdf: # rewrites the file with Ghostscript first
counts = [len(page.chars) for page in pdf.pages]
print(f"{name:18} repaired: chars per page {counts}")
except Exception as e:
print(f"{name:18} repair failed: {str(e).splitlines()[0][:60]}")no-xref.pdf repaired: chars per page [1181, 1140, 801]
truncated-90.pdf repaired: chars per page [1181, 1140, 0]
truncated-33.pdf repaired: chars per page [0, 0, 0]
empty.pdf repaired: chars per page [0]
html-page.pdf repair failed: GPL Ghostscript 10.02.1 (2023-11-01)| File | After repair |
|---|---|
| no-xref.pdf | Fully recovered: every character of all three pages |
| truncated-90.pdf | Pages 1 and 2 recovered; page 3 is blank (its content was in the missing tenth) |
| truncated-33.pdf | Three pages, all blank. No error |
| empty.pdf | One blank page. No error |
| html-page.pdf | Fails; the error message is Ghostscript’s banner, not a description |
Repair is worth trying, but its successes and failures look alike. Always count characters per page after repairing, and compare the page count with what you expected. A repaired file that is all blank pages is a failed repair.
To keep the repaired file, for example to repair once and parse many times, use pdfplumber.repair():
import pdfplumber
pdfplumber.repair("samples/broken/no-xref.pdf", outfile="repaired.pdf") # repair once, keep the file
with pdfplumber.open("repaired.pdf") as pdf: # then open it as often as needed
print(len(pdf.pages), "pages,", sum(len(p.chars) for p in pdf.pages), "characters")3 pages, 3122 characterssetting= chooses Ghostscript’s quality preset ("default", "prepress", "printer", "ebook", "screen"); the default changed from prepress to default in 0.11.3. Lower presets downsample images, which does not affect text.
When the file is not the problem
seek of closed file: the file was closed before you used a page. Keep work inside thewithblock (details).- Password errors are
PdfminerExceptionwrappingPDFPasswordIncorrect(passwords). (cid:…)in the text: the file is fine; the font has no usable character mapping (cid errors).