Repository navigation
Conversation
This was referenced Oct 16, 2026
Merged
Collaborator
Author
|
Superseded by #2198, which carries the rebased version of this change plus the matching GLM-5.2 fix. Closing. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
GLM-5.3-Flash 的训练后缀
<|endoftext|>只在最终 assistant 消息后追加;assistant→system、assistant→assistant 边界不再插入 EOS,与官方chat_template.jinja一致。本 PR 是 #2181 按维度拆分系列的 1/7(失败报告条目 M02/M03)。原 #2181 混在一起 reviewer 难以拆分审查,现按飞书《glm5.3 flash 训推 chat template 一致性》失败报告的维度拆成独立小 PR。
Why
官方模板从不在对话中间输出 EOS(
glm52_chat共享逻辑把"下一条不是 user/tool 就补 EOS"带到了 glm53)。修复前assistant_then_system/consecutive_assistants样本在边界多插一个 EOS(原始报告各差 26/27、21/22 个 token),模型会在分布外位置学到停止信号。Changes
glm53_chat.pyrender 主循环:next_role is None才追加训练 EOSglm53_tokenize_fn_slowspeed对称处理:①前缀渲染末尾的训练 EOS 在前一条是 assistant 时删除(保持 token 级前缀对齐);②assistant→system/assistant 边界截掉监督区间末尾的 EOS(边界 token 本身不计入 loss)TestGlm53OfficialTemplateContract,golden 用例assistant_then_system/consecutive_assistants逐字符对齐官方 jinja(fast/slow 一致)+ 中间边界无 EOS、边界 token 无 loss 断言Test plan
GLM_5_3_FLASH_PATH=<四层 checkpoint> pytest tests/datasets/test_glm53_chat.py:11 passedtransformers.models.glm5_next无法导入,属环境问题,与改动无关)系列 PR(#2181 拆分,全部 base=main,按依赖序)
7 个 PR 全部合入后与 #2181 的 618d3c4 逐字节等价;#2181 将在系列建齐后关闭并指向本系列。层叠 PR 的 Files changed 会包含父 PR 提交直至父合并,审阅增量可用 compare 视图(链接在系列建齐后补充)。