[TINGyu123644]

What this fixes

Both reports point at the same two helpers in api/core/rag/extractor/markdown_extractor.py:

• [Bug]: Markdown image removal keeps inline images and deletes text between wiki images #45183 — with remove_images=True, a standard ![alt](url) was left in the text, while ![[one.png]] useful text ![[two.png]] lost the words between the images. remove_images only knew the wiki form ![[...]] and matched it with a greedy (.*).
• [Bug]: Markdown hyperlink removal corrupts images when image removal is disabled #45184 — with remove_hyperlinks=True and image removal off, ![diagram](diagram.png) became !diagram: the link pattern also matched the [alt](url) half of the image.

How

remove_images now matches both forms with non-greedy character classes, so only the image markup is dropped and the surrounding text stays:

content = re.sub(r"!\[\[[^\]]*\]\]", "", content)
return re.sub(r"!\[[^\]]*\]\([^)]*\)", "", content)

remove_hyperlinks skips anything preceded by !, so an image is never half-unwrapped:

return re.sub(r"(?<!!)\[([^\]]*)\]\([^)]*\)", r"\1", content)

The replacement for a link is still its text, as before.

Tests

Four cases added to api/tests/unit_tests/core/rag/extractor/test_markdown_extractor.py, covering both issues at helper level and through extract() with a real temporary file.

# on main
4 failed, 1 passed  (the four new cases fail)

# with this change
17 passed

Only that unit module was run locally (pytest, API deps installed via uv); the rest of the suite is left to CI.

The change and the tests were drafted with an AI assistant and reviewed by me before opening this PR.

[GYJ99]

Thanks for consolidating #45183 and #45184 and adding regressions for both image syntaxes. I found one valid Markdown URL shape that the new inline-image regex does not consume completely: destinations containing balanced parentheses.

With the exact new expression re.sub(r"!\\[[^\\]]*\\]\\([^)]*\\)", "", text), an isolated Python check gives:

input:  before ![plot](figures/plot(v2).png) after
actual: before .png) after

The first ) in (v2) terminates the regex before the actual closing parenthesis, so remove_images=True leaves a broken filename fragment in the indexed content. Another example, ![chart](https://example.com/charts/(a)/render.png), leaves /render.png). Both forms are legitimate Markdown destinations.

Could you add a regression for a parenthesized image destination with meaningful surrounding prose and use balanced-destination handling (or a Markdown-aware parser) rather than [^)]*? The new image-removal behavior should not leave URL debris. I reproduced only the regex behavior with Python, not the full Dify extractor tests.

Also, #45185 and #45186 overlap these same helper functions; coordination before merging would avoid competing fixes. AI-assisted code review with ChatGPT.

[TINGyu123644]

Replying to @GYJ99's review (balanced parentheses in image destinations):

Thanks for the careful review - you found a real gap. The inline-image matcher stopped at the first ) in the destination, so ![plot](figures/plot(v2).png) left a .png) fragment behind.

Fixed in 5006de6 with balanced-destination handling instead of [^)]*: a destination now matches a sequence of plain characters, backslash escapes, or one level of balanced parentheses ((?:[^()\\]|\\.|\([^()]*\))*). The same handling is applied to remove_hyperlinks, which had the identical debris issue on [plot](figures/plot(v2).png).

Regressions added (all fail on the previous commit, pass now - verified locally with the shimmed extractor test suite, 17 -> 20 passed):

• ![plot](figures/plot(v2).png) and ![chart](https://example.com/charts/(a)/render.png) are fully removed
• an escaped destination ![esc](a\(b\).png) is fully removed
• an extract()-level case with surrounding prose (Quarterly revenue rose. ![plot](figures/plot(v2).png) Costs fell.) leaves no debris
• [plot](figures/plot(v2).png) unwraps to plot under remove_hyperlinks

On coordination: #45185 and #45186 (Zhi-Xiang-Guo, opened earlier the same day) do overlap these helpers - this PR intentionally consolidates both halves (#45183 + #45184) plus the balanced-destination handling. Happy to close this one if maintainers prefer landing #45185/#45186; just say the word.