feat(perfs): add VLM-based picture/chart extraction to RAG eval pipeline

[ceberam]

Summary

This PR refactors the agentic RAG evaluation pipeline (perfs/agentic_rag_eval.py) to streamline the conversion step, drop a redundant heading-fix step, and introduce new VLM-powered enrichments during PDF conversion. Output directories are now namespaced by dataset name to make multi-dataset runs self-contained and easier to manage.

What changed

Pipeline restructuring (4 steps → 3 steps)

The previous four-step pipeline included a dedicated Step 2 that used DoclingEditingAgent to correct flat heading hierarchies in converted documents. This step has been removed — heading hierarchy detection is now handled natively by Docling's conversion pipeline via HeadingHierarchyOptions(enabled=True). The steps are renumbered accordingly:

Before	After
Step 1 – Convert PDFs	Step 1 – Convert PDFs (richer, with hierarchy)
Step 2 – Fix heading levels (agent)	Step 2 – Enrich with summaries
Step 3 – Enrich with summaries	Step 3 – Evaluate RAG
Step 4 – Evaluate RAG	(removed)

Richer, configurable PDF conversion (Step 1)

The converter is no longer a bare DocumentConverter(). A dedicated _create_document_converter() factory now builds a fully configured converter with:

• Native heading hierarchy detection via HeadingHierarchyOptions.
• Platform-aware OCR selection: Nemotron OCR on Linux with the correct language (English or French, inferred from the dataset directory name); auto-selection on macOS/other platforms.
• CUDA acceleration: when a GPU is available, the threaded pipeline is used with increased batch sizes for OCR, layout, and table detection.
• Optional picture description (--picture-description / picture_description: true in config): captions every image using ibm-granite/granite-vision-4.1-4b.
• Optional chart extraction (--chart-extraction / chart_extraction: true in config): produces a natural-language summary for each detected chart using the Granite Vision v4 model in chart2summary mode.

Conversion errors are now handled gracefully: soft failures (PARTIAL_SUCCESS) are logged as warnings instead of raising, and a per-run summary counter (succeeded / partial / failed) is printed at the end of the step.

Dual output format in Step 1

Each converted document is now saved in two forms:

1. A .dclx archive that bundles the full DoclingDocument together with page images — useful for visual inspection and downstream VLM tasks.
2. A lean .json file with page images stripped out — the format consumed by Steps 2 and 3.

Dataset-namespaced output directories

All step output directories are now scoped under a subdirectory named after the dataset:

output/
  step1_converted/<dataset_name>/
  step2_enriched/<dataset_name>/
  step3_evaluation/<dataset_name>/

This makes it safe to run the pipeline on multiple datasets pointing at the same output root without outputs colliding.

Dependency additions

The evaluation extras group gains:

• ocrmac (macOS only) for OCR engine selection on Apple Silicon.
• nemotron-ocr (Linux x86-64, Python 3.12 only) for GPU-accelerated OCR.
• A pinned transformers range as a temporary workaround for a Docling compatibility issue.

[github-actions]

✅ DCO Check Passed

Thanks @ceberam, all your commits are properly signed off. 🎉

[mergify]

Merge Protections

🟢 Merge protection satisfied — ready to merge.

Show 1 satisfied protection

🟢 Enforce conventional commit

Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/

• title ~= ^(fix|feat|docs|style|refactor|perf|test|build|ci|chore|revert)(?:\(.+\))?(!)?:

[PeterStaar-IBM]

lgtm!

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[PeterStaar-IBM]

lgtm!