[sharadvc]

Problem

DeepSeek's DSML protocol injects tool-call routing markers into the content stream before the actual tool_calls delta arrives. These markers (e.g. <‖DSML‖tool_calls>, <‖DSML‖invoke ...>) leak into the UI when tool calls fail or aren't parsed in time.

Fix

Adds _strip_dsml_markers() — a fast regex filter applied to every content chunk yielded by _parse_openai_sse().

• Zero overhead on non-DeepSeek streams: the guard short-circuits on chunks that don't contain 'DSML'
• Catches all known marker variants: full-width ‖, Unicode ｜, ASCII pipe |, and double-escaped forms
• Applied at both the streaming yield point (real-time UI) and the final block assembly (fallback)

Testing

• Verified with synthetic SSE chunks containing <‖DSML‖tool_calls>, <‖DSML‖invoke name="ShellCommand" arguments="{...}"> — all stripped cleanly
• Non-DeepSeek streams unaffected (guard short-circuits)

Closes #804

[sharadvc]

Friendly bump — DeepSeek DSML strip is ready for review when you have a moment. Happy to tweak if needed.

[ZhulongNT]

I tested the current PR implementation against DSML strings observed in real model_responses. Two gaps remain:

1. Double full-width-pipe markers are only partially removed

_strip_dsml_markers('<｜｜DSML｜｜parameter name="x">abc</｜｜DSML｜｜parameter>')
# current result: '<abc</'
# expected:       'abc'

The ｜DSML｜ regex starts matching at the second pipe of ｜｜DSML｜｜, leaving the outer < / </ behind.

2. Markers split across SSE chunks are not removed

chunks = ['hello<‖DS', 'ML‖tool_calls>world']
''.join(_strip_dsml_markers(x) for x in chunks)
# current result: 'hello<‖DSML‖tool_calls>world'

Because each chunk is filtered independently and the fast guard requires "DSML" in text, a marker split at any point inside DSML bypasses the filter. This is especially relevant for streaming, and the PR description explicitly mentions chunk-split regression tests.

Suggested direction: maintain a short carry-over buffer/state across SSE chunks, and match the double-pipe variants before their single-pipe substrings. Please also add regression tests for opening/closing parameter tags and every split position within a marker.

[lsdefine]

Thanks for the PR and @ZhulongNT for the careful review. We've decided not to merge this. DSML leaking into content means the tool call was already not parsed, either on the serving side or because of malformed model output. Stripping the markers on the client doesn't recover the call. It only hides the failure, and it rewrites the history sent back to the model, which stops it from correcting itself. We'd rather keep the model output as-is and fix this at the serving/parser layer. Closing, thanks again!