Skip to content

Add experimental native Wan 1.3B and LTX 2B small-video ART recipes - #11

Draft
vanzll wants to merge 14 commits into
mainfrom
codex/small-video-8gpu-20261104
Draft

vanzll wants to merge 14 commits into
mainfrom
codex/small-video-8gpu-20261104

Conversation

@vanzll

@vanzll vanzll commented Nov 4, 2026 •

Copy link
Copy Markdown
Owner

Adds native Wan2.1-T2V-1.3B and LTX-Video 2B 0.9.6 distilled backends, with separate LoRA/full and Original/Flash recipes. LoRA is the documented default. Existing Wan I2V14B and other backend defaults/checkpoint interfaces are preserved.

Draft: native replay science confirmation and all eight training acceptances are pending. The implementation retains native Wan UniPC history and LTX checkpoint stochastic sampling, actual random draws and conditioned decode. The proposed training contract freezes rollout history and evaluates native conditional mean-MSE/a² as stable velocity residuals, preserving canonical sigma weighting and Q2 reference masking. This equivalence remains unapproved; Wan native CUDA AMP currently fails the strict affine replay audit, so CPU algebra alone does not establish its validity. It is not a native-kernel KL claim. Recipes currently fail closed before training until the scientific contract is explicitly confirmed.

The rollout protocol is P16×K8, independently standardized raw VQ/MQ over complete K8, then their equal mean once; TA is monitoring only. Frozen covariance supplies disjoint K4/K4 windows and exactly two optimizer steps per rollout. Each acceptance cell has a cumulative ten-step/five-rollout cap across checkpoint4 and resumed six steps, with fixed eight-prompt paired base/post evaluation. No 50/200-rollout stage is authorized. Complete policy/Adam/RNG/data/timestep-selection checkpoints, durable per-rank actual-step journals, named final exports and generic held-out evaluation are implemented.

Evidence so far

Check Result
New CPU functionality Prior consolidated261 passed; later scoped Wan25, reward/UUID71 and namespace/source57 passed (overlap, do not sum); actual cold native imports and explicit checkpoint constructor checks passed
Affected existing functionality Prior affected261 passed; prior consolidated baseline522 unique cases, plus later diagnostic cases
Native LTX LoRA pretrained B2 parity 115/115 comparisons passed, maximum absolute error0; four matched full-profile videos retained
Native LTX full pretrained B2 parity 99/99 passed, maximum absolute error0; explicitly eval/no-grad BF16 views retaining FP32 masters, not a distributed/backward acceptance
Native Wan pretrained parity Full teacher parity remains failed. First-step source0061b74: four same-B1 replays exact; native prePEFT B4 differs from B1 (max0.105957), while nativeB4/postPEFT/adapterB4 and zero-LoRA-disable are exact. Difference precedes adapter; no specific kernel cause claimed and no tolerance relaxed
Cold installation New isolated model and canonical reward environments installed; pip/import/environment checks passed; real existing-video scorer smoke passed on the same physical visible device; eight-GPU reward/backward acceptance remains pending
Canonical reward CUDA smoke Two native full-profile LTX videos scored finite raw VQ/MQ/TA; VQ/MQ means1.248046875 and1.8984375; scoring2.86s/load18.31s; scorer peak allocated5.262GB/reserved6.325GB. This is no reward-growth or training claim
Wan native CUDA solver-only diagnostic Exact same-velocity replay all50; AMP einsumBF16 fails affine3e-5 at46/47/48, maxrelative6.7012e-5. AMP-off is diagnostic only (not a profile change)
Pinned assets All51 required files,52.44GB, passed model/source/data checksums and preflight
Independent code review One review completed; identified LoRA device placement and custom asset binding issues corrected

LTX uses1216×704,153frames/30fps/eight stochastic native steps,CFG1/STG0; Wan uses832×480,81frames/16fps/50native UniPC steps,CFG6/shift8. Actual transformer batches exceed1. Earlier failed attempts and their evidence remain retained; tolerances, native samplers, reward and scientific dimensions were not relaxed to obtain these results.

Model Mode Original actual steps Flash actual steps
Wan1.3B LoRA 0/10 pending 0/10 pending
Wan1.3B full 0/10 pending 0/10 pending
LTX2B distilled LoRA 0/10 pending 0/10 pending
LTX2B distilled full 0/10 pending 0/10 pending

The allocation reports “H800” names but143771MiB and PCI device2335, identified asH200 in NVIDIA’s driver table. This is not validation on eight H80080GB cards. Native parity above uses one device with a real batch2, and is not presented as an eight-GPU training result. Existing image science runs/services/supervisors/leases are protected; resource handoff only admits verified idle whole nodes through their original generation-fenced controller.

Reproduction and remaining acceptance

Install and pin assets with the setup guide; follow the eight recipe commands and full protocol, LTX official parity, Wan official parity and checkpoint/resume/export. Commands use operator-supplied paths and fresh isolated environments. Dependencies and model/code/reward/data revisions are fixed; native Wan adds optional FlashAttention2.7.4.post1 after Torch2.6cu124, without changing other backend dependencies.

Sanitized source/report hashes, actual numerical results, hardware limits and the complete0/10 matrix are in the checked-in diagnostic evidence. Passing reward GPU source is1e0a31d37bb4659892d82956045838378295241a; final head explicitly supplies the same11352 checkpoint observed in that run, with native constructor/forwarding checks on CPU.

Remaining evidence before readiness: explicit replay scientific decision, Wan native parity, real eight-GPU rollout→reward→backward→optimizer for all eight cells, complete checkpoint/fresh load/eval/resume within each ten-step budget, actual rank-loaded native communication libraries/data paths, per-phase throughput and per-rank/scorer memory, W&B URLs, and paired raw VQ/MQ changes with uncertainty. Flat or negative short-run outcomes will be reported; ten steps cannot establish long-run stability, generalization or absence of reward exploitation. No weights or large videos/logs are included in Git.

Base: b51e6ec.
Exact current head: 0f7ac15.
LTX passing GPU source: c015d8b; later changes add diagnostic/preflight paths, explicit namespace provenance and the fixed native checkpoint11352; unchanged LTX generator arithmetic.

Targets main for local coordinator review. Do not merge automatically.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant