{"id":1310,"date":"2026-11-06T12:14:09","date_gmt":"2026-11-06T20:14:09","guid":{"rendered":"https:\/\/salemdata.net\/johnpress\/?p=1310"},"modified":"2026-11-06T12:14:09","modified_gmt":"2026-11-06T20:14:09","slug":"local-ai-pdf-digestion","status":"publish","type":"post","link":"https:\/\/salemdata.net\/johnpress\/?p=1310","title":{"rendered":"Local AI: PDF Digestion"},"content":{"rendered":"
<h1>Introduction<\/h1>\n<p>My local AI, the <a href="https:\/\/github.com\/Niko1221\/Strata">Strata<\/a> project model <strong>qwen3.8-flash-next-iq3_s,<\/strong> does not accept PDFs for upload.<\/p>\n<img class=" wp-image-1320" src="https:\/\/salemdata.net\/johnpress\/wp-content\/uploads\/2026\/11\/20261106_095014_Fri.png" alt="" width="346" height="218" \/> Strata Chat Interface\n<p>The interface <em>does<\/em> accept binary images.\u00a0 So if I want my chat to consider the contents of a PDF, I have to extract its text and images and then upload the text and images to my local chat.\u00a0 This study compares two tools that can be used for what I call “digestion” of PDF files: 1) <strong>pymupdf<\/strong> and 2) <strong>docling<\/strong><\/p>\n<h1>Methodology<\/h1>\n<p>Since special libraries and AI models are needed, I created separate Python environments so we know what additional packages are needed and what space they consume.\u00a0 When I installed docling, I was surprised — it looks like another AI model is staged for the docling process.<\/p>\n<h2>Libraries<\/h2>\n<p>Installing either library will consume appreciable disk space.\u00a0 Notably, Docling could contribute as much as 9 GB.<\/p>\n<h2>Setting Up the Python Environments<\/h2>\n<h3>pymupdf<\/h3>\n<blockquote>\n<pre>jlpoole@ryzdesk ~ $ python3 -m venv ~\/venvs\/pymu\r\njlpoole@ryzdesk ~ $ du -s ~\/venvs\/pymu\r\n13380 \/home\/jlpoole\/venvs\/pymu\r\njlpoole@ryzdesk ~ $ du -sh ~\/venvs\/pymu\r\n14M \/home\/jlpoole\/venvs\/pymu\r\njlpoole@ryzdesk ~ $\r\n<\/pre>\n<\/blockquote>\n<h3>docling<\/h3>\n<blockquote>\n<pre>jlpoole@ryzdesk ~ $ python3 -m venv ~\/venvs\/docling\r\njlpoole@ryzdesk ~ $ du -sh ~\/venvs\/docling\/\r\n14M \/home\/jlpoole\/venvs\/docling\/\r\njlpoole@ryzdesk ~ $<\/pre>\n<\/blockquote>\n<h3>Updated disk-space accounting<\/h3>\n<table>\n<thead>\n<tr>\n<th>Component<\/th>\n<th>PyMuPDF<\/th>\n<th>Docling<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>Initial virtual environment<\/td>\n<td>14 MiB<\/td>\n<td>14 MiB<\/td>\n<\/tr>\n<tr>\n<td>Installed virtual environment<\/td>\n<td>294 MiB<\/td>\n<td>6.2 GiB<\/td>\n<\/tr>\n<tr>\n<td>Increase over baseline<\/td>\n<td>~280 MiB<\/td>\n<td>~6.19 GiB<\/td>\n<\/tr>\n<tr>\n<td>Hugging Face model weights<\/td>\n<td>\u2014<\/td>\n<td>506 MiB<\/td>\n<\/tr>\n<tr>\n<td>Recently modified pip cache<\/td>\n<td>Not yet apportioned<\/td>\n<td>3,169.61 MiB (shared)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>If we provisionally attribute all 3,169.61 MiB of recently modified pip cache files to Docling, the combined footprint would be approximately 9.8 GiB, including the environment growth, model weights, and cached downloads.\u00a0\u00a0That figure is an upper-bound-style estimate, not a verified total: some cached packages may belong to PyMuPDF or other pip activity.<\/p>\n<h2>Test PDFs<\/h2>\n<p>I’m using these two PDFs since they contain lots of details in formats other than text that could be relevant to an AI’s analysis.<\/p>\n<ul>\n<li>Sensirion specifications for its transducer <a title="https:\/\/www.digikey.com\/en\/products\/detail\/sensirion-ag\/SDP810-125PA\/6605488?s=N4IgTCBcDaIMoBEAKAOAjABgLRrAViQEMQBdAXyA" href="https:\/\/www.digikey.com\/en\/products\/detail\/sensirion-ag\/SDP810-125PA\/6605488?s=N4IgTCBcDaIMoBEAKAOAjABgLRrAViQEMQBdAXyA" data-from-md="">SDP810-125PA<\/a><\/li>\n<li>“Implementation, Characterization, and Evaluation of an Inexpensive<br \/>\nLow-Power Low-Noise Infrasound Sensor Based on a Micromachined<br \/>\nDifferential Pressure Transducer and a Mechanical Filter” (2012)<\/span>1<\/sup><\/a><\/span><\/li>\n<\/ul>\n<h2>Results<\/h2>\n<h3>Commands<\/h3>\n<p>The command for the pymupdf is this script:<\/p>\n<blockquote>\n<pre>import fitz # PyMuPDF\r\nimport os\r\n\r\ndoc = fitz.open("ABC.pdf")\r\nos.makedirs("output_images", exist_ok=True)\r\n\r\n# 1. Extract all text\r\nwith open("extracted_text.txt", "w", encoding="utf-8") as f:\r\nfor page_num in range(len(doc)):\r\npage = doc[page_num]\r\nf.write(f"--- Page {page_num + 1} ---\\n")\r\nf.write(page.get_text())\r\n\r\n# 2. Extract all embedded images\r\nimage_count = 0\r\nfor page_num in range(len(doc)):\r\npage = doc[page_num]\r\nfor img_index, img in enumerate(page.get_images(full=True)):\r\nxref = img[0]\r\nbase_image = doc.extract_image(xref)\r\nimage_bytes = base_image["image"]\r\nimage_ext = base_image["ext"]\r\nimage_filename = f"output_images\/page_{page_num + 1}_img_{image_index}.{image_ext}"\r\nwith open(image_filename, "wb") as img_file:\r\nimg_file.write(image_bytes)\r\nimage_count += 1\r\n\r\nprint(f"Extracted text to extracted_text.txt and {image_count} images to output_images\/")\r\njlpoole@ryzdesk ~\/work\/HDC\/pdftrial\/pymupdf $ cat >install.sh<\/pre>\n<\/blockquote>\n<p>The command for the docling is simply:<\/p>\n<blockquote>\n<pre>docling ABC.pdf --to md --image-export-mode referenced --output output_dir\/<\/pre>\n<\/blockquote>\n<h3>Extractions Results<\/h3>\n<h4>text extractions<\/h4>\n<h5>pymupdf<\/h5>\n<p>Johnson Study – pymupdf (<a href="https:\/\/salemdata.net\/dev\/wp\/pdf_digesters\/pymupdf_johnson_extracted_text.txt">text<\/a>)<\/p>\n<p>Sensirion specification – pymupdf (<a href="https:\/\/salemdata.net\/dev\/wp\/pdf_digesters\/pymupdf_sensirion_extracted_text.txt">text<\/a>)<\/p>\n<h5>docling<\/h5>\n<p>Johnson Study – docling (<a href="https:\/\/salemdata.net\/dev\/wp\/pdf_digesters\/docling_Johnson.md">markdown<\/a>)<\/p>\n<p>Sensirion specification – docling (<a href="https:\/\/salemdata.net\/dev\/wp\/pdf_digesters\/docling_sensirion.md">markdown<\/a>)<\/p>\n<h4>proof sheets of the image extractions<\/h4>\n<p><<\/p>\n<table>\n<tbody>\n<tr>\n<td><\/td>\n<td><strong>Johnson PDF<\/strong><\/td>\n<td><strong>Sensirion PDF<\/strong><\/td>\n<\/tr>\n<tr>\n<td><strong>pymupdf<\/strong><\/td>\n<td>\n<p><img class=" wp-image-1314" src="https:\/\/salemdata.net\/johnpress\/wp-content\/uploads\/2026\/11\/johnson_output_images_proof_sheet_20261106_082149.png" alt="" width="345" height="180" \/> pymupdf Johnson<\/td>\n<td>\n<p><img class=" wp-image-1315" src="https:\/\/salemdata.net\/johnpress\/wp-content\/uploads\/2026\/11\/sensirion_artifacts_proof_sheet_20261106_082432.png" alt="" width="237" height="270" \/> pymupdf Sensirion<\/td>\n<\/tr>\n<tr>\n<td><strong>docling<\/strong><\/td>\n<td>\n<p><img class=" wp-image-1316" src="https:\/\/salemdata.net\/johnpress\/wp-content\/uploads\/2026\/11\/Johnson__artifacts_proof_sheet_20261106_082344.png" alt="" width="342" height="249" \/> docling Johnson<\/td>\n<td>\n<p><img class=" wp-image-1313" src="https:\/\/salemdata.net\/johnpress\/wp-content\/uploads\/2026\/11\/sensirion_output_images_proof_sheet_20261106_082240.png" alt="" width="328" height="644" \/> docling Sensirion<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<h3>Local AI Chats Analyses Base On A Particular Extraction<\/h3>\n<p>pymupdf-based chat\u00a0 <a href="https:\/\/salemdata.net\/dev\/wp\/pdf_digesters\/pymufpdf_strata-chat-2026-11-06-08-49.md">pymufpdf_strata-chat-2026-11-06-08-49.md<\/a><\/p>\n<p>docling-based chat <a href="https:\/\/salemdata.net\/dev\/wp\/pdf_digesters\/docling_strata-chat-2026-11-06-09-04.md">docling_strata-chat-2026-11-06-09-04.md<\/a><\/p>\n<h3>Analysis Of Both Chats<\/h3>\n<p>Local AI Analysis of its pymupdf and docling responses <a href="https:\/\/salemdata.net\/dev\/wp\/pdf_digesters\/Analysis_of_both_chats_strata-chat-2026-11-06-17-21.md">Analysis_of_both_chats_strata-chat-2026-11-06-17-21.md<\/a><\/p>\n<h1>Conclusion<\/h1>\n<p>Definitely go with the <strong>docling<\/strong>.\u00a0 For technical PDFs that contain tables and graphs, you want the benefit of that information being digested for AI.<\/p>\n<\/div>

Introduction<\/h1>\n

My local AI, the Strata<\/a> project model qwen3.8-flash-next-iq3_s,<\/strong> does not accept PDFs for upload.<\/p>\n

\"\"
Strata Chat Interface<\/figcaption><\/figure>\n

The interface does<\/em> accept binary images.\u00a0 So if I want my chat to consider the contents of a PDF, I have to extract its text and images and then upload the text and images to my local chat.\u00a0 This study compares two tools that can be used for what I call “digestion” of PDF files: 1) pymupdf<\/strong> and 2) docling<\/strong><\/p>\n

Methodology<\/h1>\n

Since special libraries and AI models are needed, I created separate Python environments so we know what additional packages are needed and what space they consume.\u00a0 When I installed docling, I was surprised — it looks like another AI model is staged for the docling process.<\/p>\n

Libraries<\/h2>\n

Installing either library will consume appreciable disk space.\u00a0 Notably, Docling could contribute as much as 9 GB.<\/p>\n

Setting Up the Python Environments<\/h2>\n

pymupdf<\/h3>\n
\n
jlpoole@ryzdesk ~ $ python3 -m venv ~\/venvs\/pymu\r\njlpoole@ryzdesk ~ $ du -s ~\/venvs\/pymu\r\n13380 \/home\/jlpoole\/venvs\/pymu\r\njlpoole@ryzdesk ~ $ du -sh ~\/venvs\/pymu\r\n14M \/home\/jlpoole\/venvs\/pymu\r\njlpoole@ryzdesk ~ $\r\n<\/pre>\n<\/blockquote>\n
docling<\/h3>\n
\n
jlpoole@ryzdesk ~ $ python3 -m venv ~\/venvs\/docling\r\njlpoole@ryzdesk ~ $ du -sh ~\/venvs\/docling\/\r\n14M     \/home\/jlpoole\/venvs\/docling\/\r\njlpoole@ryzdesk ~ $<\/pre>\n<\/blockquote>\n
Updated disk-space accounting<\/h3>\n\n\n\n\n\n\n\n\n\n
Component<\/th>\nPyMuPDF<\/th>\nDocling<\/th>\n<\/tr>\n<\/thead>\n
Initial virtual environment<\/td>\n14 MiB<\/td>\n14 MiB<\/td>\n<\/tr>\n
Installed virtual environment<\/td>\n294 MiB<\/td>\n6.2 GiB<\/td>\n<\/tr>\n
Increase over baseline<\/td>\n~280 MiB<\/td>\n~6.19 GiB<\/td>\n<\/tr>\n
Hugging Face model weights<\/td>\n\u2014<\/td>\n506 MiB<\/td>\n<\/tr>\n
Recently modified pip cache<\/td>\nNot yet apportioned<\/td>\n3,169.61 MiB (shared)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n
If we provisionally attribute all 3,169.61 MiB of recently modified pip cache files to Docling, the combined footprint would be approximately 9.8 GiB, including the environment growth, model weights, and cached downloads.\u00a0\u00a0That figure is an upper-bound-style estimate, not a verified total: some cached packages may belong to PyMuPDF or other pip activity.<\/p>\n
Test PDFs<\/h2>\n
I’m using these two PDFs since they contain lots of details in formats other than text that could be relevant to an AI’s analysis.<\/p>\n