[kevin9327]

Issue resolved by this Pull Request:
Resolves #4887

Problem

docling convert exits 0 even when documents fail. export_documents logs Document … failed to convert. and counts the failure, but the count never reaches the command, so a run where every input failed still looks successful to a shell script or CI job:

$ printf 'not a pdf' > bad.pdf
$ docling bad.pdf --output out
… WARNING docling.document_converter: Input document …/bad.pdf is not valid.
… WARNING docling.cli.main: Document …/bad.pdf failed to convert.
$ echo $?
0

convert-remote has the same gap, even though its help lists 1 runtime or connection failure (service unreachable, conversion error).

Change

• export_documents returns an ExportSummary(success, partial_success, failure) instead of nothing.
• PARTIAL_SUCCESS documents are exported like SUCCESS ones. A warning names the document, followed by the same [Failure Detail] lines as for failures. They do not count as failures. SKIPPED documents count as neither. The empty-Markdown check still turns a document into a failure.
• New --fail-on {never,failure,partial} option on convert and convert-remote (FailOn is defined once in docling/cli/main.py). Both commands exit 1 when summary.should_fail(fail_on) is true: failure (default) on any failed document, partial also on partially converted ones, never always exits 0. The check runs after all documents have been processed and exported. --abort-on-error is unchanged and only controls when processing stops.
• "Exit codes" section in the convert help, convert-remote exit codes mention --fail-on. Added the --fail-on rows to docs/reference/cli.md and a row to the usage skill's CLI reference.

Verification

• tests/test_cli.py::test_cli_convert_exits_nonzero_when_a_document_fails: converts a file with a .pdf name that is not a PDF and expects exit code 1. Fails on main (assert 0 == 1), passes with this change.
• tests/test_cli_remote.py::test_remote_exit_code_follows_fail_on (fake service client): FAILURE exits 1 by default and 0 with --fail-on never. PARTIAL_SUCCESS exits 0 by default with its Markdown written, and exits 1 with --fail-on partial. SKIPPED exits 0 even with --fail-on partial.
• tests/test_cli_remote.py: 24 passed. ruff format --check and ruff check (v0.16.10) are clean.

AI-assisted: I used an AI coding assistant to help write this change. I reproduced the bug and reviewed and ran the tests myself.

Checklist:

• Documentation has been updated, if necessary.
• Examples have been added, if necessary. (n/a)
• Tests have been added, if necessary.

[github-actions]

✅ DCO Check Passed

Thanks @kevin9327, all your commits are properly signed off. 🎉

[mergify]

Merge Protections

🟢 Merge protection satisfied — ready to merge.

Show 1 satisfied protection

🟢 Enforce conventional commit

Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/

• title ~= ^(fix|feat|docs|style|refactor|perf|test|build|ci|chore|revert)(?:\(.+\))?(!)?:

[github-actions]

AI triage (advisory)

Checked commit 75bd1c23f7. The analysis read the diff only. It did not run code from this pull request.

The PR fixes docling convert and convert-remote to exit with code 1 when documents fail. It adds a --fail-on option (never, failure, partial), an ExportSummary return value from export_documents, and tests for both CLI commands.

CI safety: no concern found. A maintainer must still approve CI.

1 related pull request(s) in the same code

PR	State	Signal	Assessment
#4897	open	changes the same code	related: PR 4897 modifies the same three files to add --trust-remote-code and the Nemotron v3.5 VLM preset; it does not touch the exit-code or --fail-on logic, so both PRs can coexist.

_{The ai:* labels are advisory. They do not approve CI and they do not add tests:full. A new push runs this check again.}

[github-actions]

AI first review (advisory): no blocker found

This review covers the full diff at 440dd49126. It read the code and did not run it.

The PR makes export_documents return its pre-existing failure_count and raises typer.Exit(1) in both convert and convert-remote when that count is non-zero. The logic is correct: --abort-on-error raises before the return, so the paths don't interact. The tests exercise both commands with a real failed-document scenario. No blockers found; verify that PARTIAL_SUCCESS and SKIPPED statuses intentionally cause exit 1, since doc_failed = conv_res.status != ConversionStatus.SUCCESS (line 584) counts those too — but this is pre-existing behavior that the PR merely exposes.

_{A maintainer makes the merge decision. This review never approves the pull request.}

[codecov]

Codecov Report

❌ Patch coverage is 76.19048% with 10 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
docling/cli/main.py	81.57%	4 Missing and 3 partials ⚠️
docling/cli/remote.py	25.00%	3 Missing ⚠️

📢 Thoughts on this report? Let us know!

[dolfim-ibm]

Thanks for picking this up, a non-zero exit on failure is the right direction. One blocker on how partial success is treated, plus a proposal to make the failure threshold configurable.

Blocker: PARTIAL_SUCCESS must not fail the run by default

doc_failed = conv_res.status != ConversionStatus.SUCCESS (main.py:583) counts PARTIAL_SUCCESS (and SKIPPED) as a failure, so with this PR a 300-page PDF where one page times out (--document-timeout) or one page hits a model error now makes docling convert exit 1. Today those runs exit 0, so this turns a lot of currently-green batch jobs and CI pipelines red.

It is also inconsistent with the converter: --abort-on-error passes raises_on_error, and DocumentConverter.convert_all deliberately does not raise on partial success (document_converter.py:773):

if raises_on_error and conv_res.status not in {
    ConversionStatus.SUCCESS,
    ConversionStatus.PARTIAL_SUCCESS,
}:

So the strictest CLI mode (--abort-on-error) treats partial success as OK, while the default mode would exit 1 on it.

Note that the CLI also does not export PARTIAL_SUCCESS documents today. That part is pre-existing (shared with main, unchanged since the CLI was added), but it matters here: exiting 0 on partial success only makes sense if the partial document is actually written. Please fix both together:

• export PARTIAL_SUCCESS documents like SUCCESS ones, and log a warning with the conv_res.errors details (the same [Failure Detail] lines used for failures);
• do not count them as failures by default.

FAILURE keeps exiting 1, as this PR does. The existing "empty markdown export" check (from #3429) should stay a failure.

Proposal: --fail-on to choose how strict the exit code is

Different users want different thresholds (a RAG ingestion job is fine with a missing page, a compliance archive is not), so make it an option instead of hardcoding it. Concretely:

class FailOn(str, Enum):
    """Which conversion outcome makes the CLI exit with a non-zero code."""

    NEVER = "never"        # always exit 0 once the run completes (old behaviour)
    FAILURE = "failure"    # exit 1 if any document has status FAILURE (default)
    PARTIAL = "partial"    # also exit 1 if any document is PARTIAL_SUCCESS

fail_on: Annotated[
    FailOn,
    typer.Option(
        "--fail-on",
        help=(
            "Which outcome makes docling exit with code 1: 'failure' (default) "
            "if any document fails, 'partial' also if any document is only "
            "partially converted, 'never' to always exit 0."
        ),
    ),
] = FailOn.FAILURE,

• Add the same option to convert-remote (define FailOn once in docling/cli/main.py, next to ChunkerType, and import it in remote.py).
• --abort-on-error stays independent: it controls when processing stops; --fail-on controls the exit code at the end. --abort-on-error --fail-on never still raises on the first failure, as today.
• SKIPPED documents should count as neither failure nor partial.

To keep the decision in one place, have export_documents return the counts as a small structured value instead of a bare int (the same split docs/examples/batch_convert.py already uses with its success, partial_success, failure tuple):

@dataclass(frozen=True)
class ExportSummary:
    success: int
    partial_success: int
    failure: int

    def should_fail(self, fail_on: FailOn) -> bool:
        if fail_on == FailOn.NEVER:
            return False
        if fail_on == FailOn.PARTIAL:
            return self.failure > 0 or self.partial_success > 0
        return self.failure > 0

and in both commands:

if summary.should_fail(fail_on):
    raise typer.Exit(1)

Docs and tests

• Add an "Exit codes" section to the convert help (mirroring the one in convert-remote) and extend the convert-remote one to mention --fail-on. Also update docs/reference/cli.md and the usage skill in docling/.agents/skills/docling/, since scripts will start depending on this.

• Tests worth having, on top of the two already added:

  • a PARTIAL_SUCCESS result exits 0 by default and its output file is written;
  • the same result exits 1 with --fail-on partial;
  • a FAILURE result exits 0 with --fail-on never.

  The fake service client in tests/test_cli_remote.py makes the partial cases easy to set up without a real model run.

[kevin9327]

@dolfim-ibm thanks for the detailed review. All of it is in 75bd1c2 (rebased on current main):

• Partial success: PARTIAL_SUCCESS documents are now exported like SUCCESS ones. A warning names the document, followed by the same [Failure Detail] lines used for failures. They no longer count as failures. SKIPPED documents count as neither. The empty-Markdown check from fix: fail on empty markdown export #3429 still marks the document as a failure.
• --fail-on: FailOn (never / failure / partial, default failure) is defined next to ChunkerType in docling/cli/main.py, and remote.py imports it. export_documents returns a frozen ExportSummary(success, partial_success, failure) with should_fail(fail_on), and both commands use if summary.should_fail(fail_on): raise typer.Exit(1). --abort-on-error is unchanged.
• Docs: there is a new "Exit codes" epilog on convert, and the convert-remote exit codes now mention --fail-on. I added the --fail-on rows to docs/reference/cli.md (they match scripts/render_cli_reference.py output) and a row to docling/.agents/skills/docling/references/cli.md.
• Tests: test_remote_exit_code_follows_fail_on uses the fake service client and covers: partial exits 0 by default and writes report.md; partial exits 1 with --fail-on partial; failure exits 0 with --fail-on never; failure exits 1 by default; skipped exits 0 even with --fail-on partial.

[dolfim-ibm]

lgtm