Close Menu
  • AI
  • Content Creation
  • Tech
  • Robotics
AI-trends.todayAI-trends.today
  • AI
  • Content Creation
  • Tech
  • Robotics
Trending
  • Collectively AI Discloses Immediate Leak After Coding Agent Uncovered Credential – Unite.AI
  • Dragos Finds Immediate Leaks and Stolen Keys in AI Provide Chains
  • Lean and Mathlib Communities Certify Claude-Assisted Riemann Proof – Unite.AI
  • AWS Routes HyperPod Inference Throughout Areas From One Endpoint – Unite.AI
  • DailyAI Briefing: A Lean-checked Riemann proof, plus 4 extra
  • Anthropic Shares Drop 8.5% in Second Session Throughout AI Selloff – Unite.AI
  • OpenAI Ships GPT-Stay-1.1 for Voice Brokers, Cuts Idle Value to $0.02/Min
  • Copywriting: The Full Information | Copyblogger
AI-trends.todayAI-trends.today
Home»Tech»Build an End-to End Document Intelligence pipeline with DeepDoctection

Build an End-to End Document Intelligence pipeline with DeepDoctection

Tech By Gavin Wallace23/08/20268 Mins Read
Facebook Twitter LinkedIn Email
LifelongAgentBench: A Benchmark for Evaluating Continuous Learning in LLM-Based Agents
LifelongAgentBench: A Benchmark for Evaluating Continuous Learning in LLM-Based Agents
Share
Facebook Twitter LinkedIn Email

This tutorial shows how to implement an intelligence pipeline for documents using deepDoctection 1.2.x It combines the layout recognition, OCR and reading-order reconstruction with annotation linking. The analyzer is configured with DocLayNet based layout detection and Table Transformer structure identification, followed by DocTR OCR. We then examine the Page objects that result to better understand the way deepDoctection presents text, figures and tables as well as relationships, provenance and reading order. In addition, we extend the framework to include custom object types as well as our PipelineComponent that extracts monetary and dates entities and classifies documents based on their tabular properties. Finaly, we manually build a pipeline with ServiceFactory. We explore filtering, service rollback and serializing processed pages. Finally, document annotations are transformed into JSONL chunks for use in downstream RAG systems and retrieval.

Install!pip -q "deepdoctection" "transformers>=5.2.0" "timm" "python-doctr" "pdfplumber" "networkx" "lxml"
Import os
os.environ["DD_USE_TORCH"]  = "True"
os.environ["DPI"]           = "200"
os.environ["LOG_LEVEL"]     = "INFO"
os.environ["ENABLE_DYNAMIC_OBJECT_TYPES"] = "False"
Textwrap, import json and re.
Import Path from pathlib
From Collections Import Counter
Numpy can be imported as np
Import matplotlib.pyplot into plt
Import HTML from IPython.display
import deepdoctection as dd
print("deepdoctection:", dd.__version__)
import transformers.integrations.peft as _hf_peft
if _hf_peft.is_peft_available():
   _hf_peft.is_peft_available = lambda: False
   print("patched: PEFT adapter lookup disabled for from_pretrained")
!mkdir -p /content/docs /content/imgs
!wget -q -O /content/docs/paper.pdf 
 

Click to access 2312.13560.pdf

!wget -q -O /content/imgs/finance.png https://raw.githubusercontent.com/deepdoctection/notebooks/main/sample/finance/1bcac3899c9cb1c0b0f650b1431d3d52_7.png The path of PDF is:"/content/docs/paper.pdf") PNG = Path"/content/imgs/finance.png") The path is OUT."/content/out"); OUT.mkdir(exist_ok=True) Def show(img w=16), If image is None: Return plt.figure(figsize=(w, w * 1.3)); plt.axis("off"); plt.imshow(img); plt.show() def analyze_any(pipe, path, **kw): """ Display correctly a directory or PDF file, as well as a single picture. DoctectionPipe can stream a directory or a PDF from disk, but a *single* image has no reader — path= only supplies the file name / provenance, and Pixels must be sent in bytes=. You get the following: When passing the path of a single picture, the bytes in that image are passed. It must be done """ Path = path If path.is_dir(): kw.setdefault("file_type", [".jpg", ".png", ".jpeg", ".tif"]) return pipe.analyze(path=path, **kw) if path.suffix.lower() == ".pdf": return pipe.analyze(path=path, **kw) if path.suffix.lower() (".png", ".jpg", ".jpeg", ".tif"): return pipe.analyze(path=path, bytes=path.read_bytes(), **kw) raise ValueError(f"unsupported input: {path}")

Install the deepDoctection dependency, set up its runtime, and then apply a compatibility fix for Transformers or PEFT. The sample image and PDF files are downloaded and we prepare the output directory. Also, we define functions that help visualize images, analyze PDFs and image files, and perform a consistent analysis of directories and PDFs.

dd.print_model_infos(add_description=False, add_config=False, add_categories=False)
profile = dd.ModelCatalog.get_profile("Aryn/deformable-detr-DocLayNet/model.safetensors")
print("nlayout model categories:", profile.categories)
print("is registered:", dd.ModelCatalog.is_registered("Aryn/deformable-detr-DocLayNet/model.safetensors"))
config_overwrite = [
   "USE_ROTATOR=False",
   "USE_LAYOUT=True",
   "USE_LAYOUT_NMS=True",
   "USE_TABLE_SEGMENTATION=True",
   "USE_TABLE_REFINEMENT=False",
   "USE_PDF_MINER=False",
   "USE_OCR=True",
   "USE_LAYOUT_LINK=True",
   "LAYOUT.WEIGHTS=Aryn/deformable-detr-DocLayNet/model.safetensors",
   "ITEM.WEIGHTS=deepdoctection/tatr_tab_struct_v2/model.safetensors",
   "ITEM.FILTER=['table']",
   "OCR.USE_DOCTR=True",
   "OCR.USE_TESSERACT=False",
   "OCR.USE_TEXTRACT=False",
   "OCR.WEIGHTS.DOCTR_WORD=doctr/db_resnet50/db_resnet50-ac60cadc.pt",
   "OCR.WEIGHTS.DOCTR_RECOGNITION=doctr/crnn_vgg16_bn/crnn_vgg16_bn-0417f351.pt",
   "SEGMENTATION.THRESHOLD_ROWS=0.4",
   "SEGMENTATION.THRESHOLD_COLS=0.4",
   "SEGMENTATION.FULL_TABLE_TILING=True",
   "WORD_MATCHING.RULE=ioa",
   "WORD_MATCHING.THRESHOLD=0.3",
   "WORD_MATCHING.MAX_PARENT_ONLY=True",
   "TEXT_ORDERING.INCLUDE_RESIDUAL_TEXT_CONTAINER=True",
   "TEXT_ORDERING.PARAGRAPH_BREAK=0.035",
   "TEXT_ORDERING.BROKEN_LINE_TOLERANCE=0.003",
   "LAYOUT_LINK.PARENTAL_CATEGORIES=['figure','table']",
   "LAYOUT_LINK.CHILD_CATEGORIES=['caption']",
]
analyzer = dd.get_dd_analyzer(config_overwrite=config_overwrite)
print("n--- pipeline ---")
for sid, name in analyzer.get_pipeline_info().items():
   print(f"{sid}  {name}")
print("n--- what this pipeline produces ---")
print(analyzer.get_meta_annotation())

The model registry of deepDoctection is checked to confirm the supported categories and layout models. Configuring the analyzer explicitly to include layout detection, DocTR OCR (word matching), reading-order reconstruction and layout linking. Then we initialize the analysis and examine its pipeline components as well as the annotation types it produces.

df = analyze_any(analyzer, PDF, session_id="tutorial01", max_datapoints=3)
df.reset_state()
Pages = list (df)
print(f"nparsed {len(pages)} pages")
The page is equal to pages[0]
show(page.viz(show_figures=True, show_residual_layouts=True, show_table_structure=True))
print("== narrative text ==")
print(textwrap.fill(page.text[:900], 110))
print("n== layout blocks in reading order ==")
For doc_id img_id pno ann_id order cat txt txt in the page.chunks[:12]:
   print(f"[{order:>3}] {str(cat): caption ids:", [i[:8] for i in linked])
If page.words
 Page.words = w[0]
   print("nword:", w.characters, "| service:", w.service_id,
         "| model:", w.model_id, "| bbox:", [round(x) for x in w.bbox])
tbl_pages = [p for p in pages if p.tables]
If tbl_pages
 t = Tbl_pages[0].tables[0]
   print(f"table {t.number_of_rows}x{t.number_of_columns}, "
 The f"max_row_span={t.max_row_span}, max_col_span={t.max_col_span}")
   display(HTML(t.html))
 Row in the t.csv[:5]:
       print([c[:22] "c" in row"
 For c, use t.cells[:5]:
       print(f"  r{c.row_number} c{c.column_number} "
 F"(span {c.row_span}x{c.column_span}) {c.text[:40]!r}")
else:
   print("no table on these pages — the finance.png sample below has one")

The configured analyzer is run on the PDF sample and the pages are generated by the lazy data flow. Inspecting narrative text, chunks of reading, categories for annotations, relationships between figure captions and words, bounding boxes, and word provenance. We can also examine tables detected through HTML, CSV or individual cell representations.

@dd.object_types_registry.register("CustomKey")
class CustomKey(dd.ObjectTypes):
   """Custom summary keys — must be registered to be serialisable."""
   MONEY_MENTIONS = "money_mentions"
   DATE_MENTIONS  = "date_mentions"
   DOC_FLAVOUR    = "doc_flavour"
@dd.object_types_registry.register("FlavourLabel")
class FlavourLabel(dd.ObjectTypes):
 TABULAR = "tabular"
 Narrative = "narrative"
 MIXED = "mixed"
MONEY = (r.compile)"(?:[$€£]s?d[d,.]*|d[d,.]*s?(?:USD|EUR|GBP|million|bn))")
DATE = Recompile (r"b(?:d{1,2}[/-]d{1,2}[/-]d{2,4}|d{4}-d{2}-d{2}|"
 The r"(?:Jan|Feb|Mar|Apr|May|Jun|Jul|Aug|Sep|Oct|Nov|Dec)w*s+d{1,2},?s+d{4})b")
class EntityAndFlavourService(dd.PipelineComponent):
   def __init__(self, name="entity_flavour", tabular_ratio=0.25):
       self.tabular_ratio = tabular_ratio
 You can also check out our supersized().__init__(name)
   def serve(self, dp: dd.Image) -> None:
       page = dd.Page.from_image(dp, text_container=dd.LayoutLabel.WORD)
       text = page.text_no_line_break
       money = sorted(set(MONEY.findall(text)))
       dates = sorted(set(DATE.findall(text)))
       tables = page.tables
       table_area = sum((b[2] The - b[0]) * (b[3] The - b[1](t.bbox in the tables for t)
       ratio = table_area / float(page.width * page.height or 1)
       flavor = (FlavourLabel.TABULAR if ratio > self.tabular_ratio
                  else FlavourLabel.NARRATIVE if not tables
 Mixes (FlavourLabel)
       self.dp_manager.set_summary_annotation(
           summary_key=CustomKey.MONEY_MENTIONS, summary_name=CustomKey.MONEY_MENTIONS,
           summary_value=money)
       self.dp_manager.set_summary_annotation(
           summary_key=CustomKey.DATE_MENTIONS, summary_name=CustomKey.DATE_MENTIONS,
           summary_value=dates)
       self.dp_manager.set_summary_annotation(
           summary_key=CustomKey.DOC_FLAVOUR, summary_name=flavour,
           summary_score=round(ratio, 4))
   def clone(self):
       return self.__class__(self.name, self.tabular_ratio)
   def get_meta_annotation(self) -> dd.MetaAnnotation:
       return dd.MetaAnnotation(
           image_annotations=(),
           sub_categories={},
           relationships={},
           summaries=(CustomKey.MONEY_MENTIONS, CustomKey.DATE_MENTIONS, CustomKey.DOC_FLAVOUR),
       )
for k in (CustomKey.MONEY_MENTIONS, CustomKey.DATE_MENTIONS, CustomKey.DOC_FLAVOUR):
   dd.Page.add_attribute_name(k)

Registering custom object types is used to classify document flavors, date and monetary references. Implementing a deepDoctection component, we analyze page text and coverage of tables to produce these page-level summary. We expose these custom summary field as Page attributes, so we can use them directly in processed documents.

from deepdoctection.analyzer import cfg, ServiceFactory
cfg.freeze(False)
cfg.USE_TABLE_SEGMENTATION = True
cfg.freeze(True)
Components = []
layout_detector = ServiceFactory.build_layout_detector(cfg, mode="LAYOUT")
components.append(ServiceFactory.build_layout_service(cfg, detector=layout_detector, mode="LAYOUT"))
components.append(ServiceFactory.build_layout_nms_service(cfg))
item_detector = ServiceFactory.build_layout_detector(cfg, mode="ITEM")
components.append(ServiceFactory.build_sub_image_service(cfg, detector=item_detector, mode="ITEM"))
components.append(ServiceFactory.build_table_segmentation_service(cfg, detector=item_detector))
word_detector = ServiceFactory.build_doctr_word_detector(cfg)
components.append(ServiceFactory.build_doctr_word_detector_service(word_detector))
components.append(ServiceFactory.build_text_extraction_service(cfg, ServiceFactory.build_ocr_detector(cfg)))
components.append(ServiceFactory.build_word_matching_service(cfg))
components.append(ServiceFactory.build_text_order_service(cfg))
components.append(EntityAndFlavourService())
custom_pipe = dd.DoctectionPipe(pipeline_component_list=components)
print("ncustom pipeline:", list(custom_pipe.get_pipeline_info().values()))
df2 = analyze_any(custom_pipe, PNG)
df2.reset_state()
fin_page = next(iter(df2))
print("flavour  :", fin_page.doc_flavour)
print("money    :", fin_page.money_mentions[:10])
print("dates    :", fin_page.date_mentions[:10])
show(fin_page.viz(show_table_structure=True), w=13)
def skip_if_no_table(dp: dd.Image) -> bool:
 You can return to your original language by clicking here. "table" not in {dp.get_annotation: a is not listed in the a.category_name.()}
components[-1].set_inbound_filter(skip_if_no_table)
det_sid = next(sid for sid, n in analyzer.get_pipeline_info().items()
 If (n.startswith)"image_doctr"))
det_comp = analyzer.get_pipeline_component(service_id=det_sid)
df_undo = det_comp.undo(dd.DataFromList([p.base_image for p in pages]))
df_undo.reset_state()
undone = list(df_undo)
print("annotations before/after undo:",
     len(pages[0].get_annotation()),
     len(dd.Page.from_image(undone[0]).get_annotation()))

Manually, we assemble the deepDoctection Pipeline with ServiceFactory. We combine layout analysis and table processing along with OCR, ordering of text, and our own custom component. This custom pipeline is executed on the image of a financial document and we inspect detected flavors, monetary value, dates and table structures. Also, we apply an inbound filters and show how to undo annotations created by selected DocTR services.

For i and p, in the enumerate (pages):
   p.save(image_to_json=False, path=OUT / f"page_{i}.json")
restored = dd.Page.from_file(str(OUT / "page_0.json"))
print("round-trip:", len(restored.get_annotation()), "of",
     len(pages[0].get_annotation()), "annotations restored")
Record = []
For p, see pages
 If you want to put doc_id and img_id in chunks, please use the following:
 If txt.strip and txt.strip():
           records.append({"document_id": doc_id, "page": pno, "order": order,
                           "category": str(cat), "annotation_id": ann_id, "text": txt})
 For p.tables, t.
       records.append({"document_id": p.document_id, "page": p.page_number,
                       "order": -1, "category": "table_html",
                       "annotation_id": t.annotation_id, "text": t.html})
(OUT / "chunks.jsonl").write_text("n"Joining records using.join (json.dumps.r for the rth record)
print(f"n{len(records)} chunks -> {OUT/'chunks.jsonl'}")
print(json.dumps(records[0], indent=2)[:400])

Serializing each page processed to JSON preserves its annotations and structural information without embedding original image data. Reloading a page saved in the database and comparing annotation counts allows us to confirm that structural data survived serialization. Finally, we convert narrative chunks, table HTML, and other HTML records into JSONL files that can be used directly by RAG and retrieval pipelines, as well as downstream document processing.

As a conclusion, we gained a better understanding of how deepDoctection orchestrates a variety of document-analysis services. In addition to running an analyzer, we also examined model registrations and controlled individual services. We also extracted tables, created custom metadata summary, and composed our own pipeline stages. We examined the effect of service filters and undos on annotations. This gave us a finer level of control over document processing workflows. Finally, serializing the structure of the processed documents was done. RAG chunks were generated, providing a base for AI document applications such as document search, knowledge extract, retrieval and augmented generation.


Click here to find out more FULL CODES here. Also, feel free to follow us on Twitter Join our Facebook group! 150k+ML SubReddit Subscribe now our Newsletter. Wait! What? now you can join us on telegram as well.

Want to promote your GitHub repo, Hugging Face page, Product release or Webinar?? Connect with us


Sana Hassan is a dual-degree IIT Madras student and consulting intern with Marktechpost. She loves to apply technology and AI in order to solve real-world problems. He has a passion for solving real-world problems and brings an innovative perspective at the intersection between AI and practical solutions.

intel intelligence
Share. Facebook Twitter LinkedIn Email
Avatar
Gavin Wallace

Related Posts

OpenAI Ships GPT-Stay-1.1 for Voice Brokers, Cuts Idle Value to $0.02/Min

21/11/2026

Meet Pondering Machines Inkling 2 Mini: 26B Looped MoE Runs on a 32GB Mac

21/11/2026

Baseten Releases Bison-1 and Groundhog-1, Topping Choice Mannequin Index

21/11/2026

RadixArk Ships DaisyRose-27B, a 27B Agentic Coding Mannequin Skilled With Reinforcement Studying Solely

20/11/2026
Top News

DailyAI Briefing: Nvidia board probes Huang, plus 4 extra

How to install a Chatbot in Your Own Computer

The AI Image Generator Startup Exposed Database Leaked a Huge Trunk of Nude Pictures

AI Is Making a Mess of Nurses’ Schedules. They Say It’s a Security Situation

AI’s Subsequent Trick? Talking Dolphin

Load More
AI-Trends.Today

Your daily source of AI news and trends. Stay up to date with everything AI and automation!

X (Twitter) Instagram
Top Insights

Anthropic Visitor Essay Urges AI to Return Ranges of Scientific Fashions – Unite.AI

08/11/2026

This is a good example of how too much thought can lead to LLMs breaking: Inverse scaling in test-time computation

30/07/2025
Latest News

Collectively AI Discloses Immediate Leak After Coding Agent Uncovered Credential – Unite.AI

22/11/2026

Dragos Finds Immediate Leaks and Stolen Keys in AI Provide Chains

22/11/2026
X (Twitter) Instagram
  • Privacy Policy
  • Contact Us
  • Terms and Conditions
© 2026 AI-Trends.Today

Type above and press Enter to search. Press Esc to cancel.