Skip to content

Feat/v1 overhaul - #14

Merged
Khubaib8281 merged 5 commits into
mainfrom
feat/v1-overhaul
Oct 1, 2026
Merged

Khubaib8281 merged 5 commits into
mainfrom
feat/v1-overhaul

Conversation

@M-O-J-U

@M-O-J-U M-O-J-U commented Oct 1, 2026

Copy link
Copy Markdown
Collaborator

What does this PR do?

Rebuilds docling-pk into a release-ready v1.0.0: layout-aware extraction for CNIC (front + back), Matric, Intermediate and Degree/Transcript, with per-field confidence, status and reason, plus tests, CI, docs and a PyPI release workflow.

Results

  • Real photos (16 images, private, aggregates only): 34.9% → 94.4% field accuracy; 99.2% of fields marked ok are correct; 30/30 marks rows exact.
  • Synthetic fixtures (committed): 96.2% accuracy, no wrong values returned.
  • Details: docs/benchmark.md, benchmarks/results/real_samples_summary.json

Key changes

  • Fixes v0.1 blockers: extractor.py SyntaxError, missing from docling_pk import extract, unpackaged CLI entry point, orientation picking Urdu noise, no-op deskew.
  • Default OCR engine is RapidOCR (measured more accurate on 4/5 document types; ~370 MB install, no PyTorch). EasyOCR is the optional [urdu] extra for the Urdu address and second-engine verification. Rationale in docs/design_doc.md §5.1.
  • New: degree/transcript parser, auto document type, PDF input, batch CLI, marks-table validation.

How was it tested?

256 tests (214 unit + 42 end-to-end OCR), 93% coverage, run against the built wheel on Python 3.9, 3.10, 3.11, 3.12 on Windows and Linux. ruff, mypy and actionlint clean.

Checklist

  • Tests added; all pass
  • Docstrings on public functions
  • README / CHANGELOG updated
  • No real documents or personal data in code, fixtures or history of this branch

…ields

Library (src/ layout):
- extract(): paths, bytes, arrays, PIL, PDF, CNIC front+back lists; auto
  document type; DocumentResult with per-field status and reason
- OCR backends: RapidOCR (default, CPU, no PyTorch) and EasyOCR (optional
  'urdu' extra, also used as an independent verifier)
- Orientation scoring robust to Urdu-script noise and to engines that rotate
  tall crops; skew from text-line angles; micro-text filtering with an
  adaptive ink-only re-read; quality warnings instead of blur rejection
- CNIC: label geometry, chronological dates, CNIC/gender parity, digit-only
  re-reads, back side, experimental Urdu address with lexicon correction
- Matric/Intermediate: column-aligned marks table validated against TOTAL,
  subject sums and marks-in-words; board detection
- Degree/transcript parser (new)
- CLI: extract, batch, info, --version

Fixes from v0.1: extractor.py SyntaxError, missing top-level extract(),
unpackaged CLI entry point, orientation picking Urdu noise, no-op deskew,
'Name' returned as father's name, non-ASCII Windows paths.
- 214 unit tests on constructed OCR layouts (no models; 92% coverage)
- 42 end-to-end tests with real OCR on synthetic fixtures, including the
  invariant that no field is returned 'ok' with a wrong value
- benchmarks/synthetic/generate.py: fabricated, clearly stamped specimens
  (no emblems, photos or seals); 23 committed fixtures covering clean,
  rotated, skewed, photographed, low-res, blurry and occluded cases
- benchmarks/run_benchmark.py: accuracy, coverage, precision, ok-precision,
  CER, marks rows, latency
- Results: synthetic report, CPU-only run, and aggregate-only real-sample
  summary (no personal data)
- README: install options, usage, supported fields, accuracy, limitations, privacy
- docs/design_doc.md (v1) and docs/benchmark.md
- CHANGELOG, CONTRIBUTING, PR template with benchmark and privacy checks
- CI: ruff, format, mypy; tests on Python 3.9-3.12 (+Windows, macOS) with a
  90% coverage gate; end-to-end OCR job; build, twine check and clean-venv
  install smoke test
- Release: PyPI trusted publishing on version tags
…tale hints

- Depend on opencv-python (the distribution RapidOCR requires) instead of
  opencv-python-headless, so only one cv2 is installed
- CI: install libgl1/libglib2.0-0 on Linux runners; README note for slim images
- requirements.txt points at pyproject.toml (was the stale v0.1 stack)
- RapidOCR import hint no longer references a removed extra

Verified: 256 tests pass against the built wheel on Python 3.9, 3.10, 3.11
and 3.12 on both Windows and Linux (93% coverage); extras resolve on all
four; actionlint clean.
On PR #14 two Linux test jobs stalled for over 15 minutes inside
apt-get update (package mirror), while identical jobs passed. The libraries
are preinstalled on GitHub runners, so the step now only calls apt when
libGL/libgthread are missing, retries mirror fetches, and is capped at 5
minutes.
@M-O-J-U

M-O-J-U commented Oct 1, 2026

Copy link
Copy Markdown
Collaborator Author
01_accuracy_before_after 05_after_cli_cnic_clean 06_after_cli_cnic_rotated image-1790872288343

@Khubaib8281
Khubaib8281 merged commit 2e51f7a into main Oct 1, 2026
9 checks passed
@Khubaib8281
Khubaib8281 deleted the feat/v1-overhaul branch October 1, 2026 16:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants