[feat] Add HunyuanVideo 1.5 embedding preprocessing pipeline - #1663

Open

klhhhhh wants to merge 11 commits into

hao-ai-lab:mainfrom

klhhhhh:hunyuan-embed-v2

Commits on Aug 3, 2026

[feat]add second text encoder byt5 for hunyuan model

klhhhhh
authored and
SolitaryThinker
committed
6307bca
[feat]change utils function for 2 encoder output for hunyuan

klhhhhh
authored and
SolitaryThinker
committed
40520e1
[feat] add hunyuan1.5 preprocess overfit

klhhhhh
authored and
SolitaryThinker
committed
fcadd62
run pre-commit

klhhhhh
authored and
SolitaryThinker
committed
d15f1f8
run pre-commit

klhhhhh
authored and
SolitaryThinker
committed
05e0ca0
run pre-commit

klhhhhh
authored and
SolitaryThinker
committed
ab00790
[test]add unitest for preprocessing

klhhhhh
authored and
SolitaryThinker
committed
e06b1bb
[bugfix]: keep the existing preprocess writers working with the widened schema

William Lin
authored and
SolitaryThinker
committed
5aa15db
[misc]: let the overfit preprocess script's paths be overridden

William Lin
authored and
SolitaryThinker
committed
b7f7f66

Commits on Aug 4, 2026

[test] add parquet round-trip coverage for dual text embeddings
klhhhhh
committed
9ca732e
[test]adjust import sequence
klhhhhh
committed
d8d9e0d