Repository navigation
fix(onnx): keep no left-context frames when chunk_size[0] is 0 in streaming Paraformer - #3790
Conversation
…eaming Paraformer overlap_feats[:, -0:, :] returns every frame instead of none, so with chunk_size=[0, 10, 5] (the streaming example in README and docs) the final chunk re-decodes the previous chunk's features and repeats the last characters. Compute the tail start index explicitly; the same applies to the non-final cache when chunk_size[0] + chunk_size[2] is 0.
45c7863 to
c81b11d
Compare
Integrate ychy7001's #3792 inference change and regressions with the #3790 cache checks. The combined tree passed 55 focused CPU tests and 20 real synthetic FP32 L20Z boundary cases, plus exact-head CI and package-byte acceptance. Not pretrained ASR accuracy or a package publication. Signed-off-by: LauraGPT <zhifu.gzf@alibaba-inc.com>
|
Merged normally with the original contributor commit preserved. The zero-context failures were reproduced on main; the candidate passed 50 focused CPU tests and eight actual L20Z CUDA wrapper-boundary cases. Author-head and merged-main Python 3.11/3.12 CI, package source bytes and the three regression logs were verified. #3808 subsequently integrates #3792 while retaining these tests; its combined tree passes 55 focused tests and 20 CUDA boundary cases, plus both new-head/final-main CI matrices. The GPU graphs have explicit fixed synthetic parameters (FP32, batch 1); this does not reproduce or independently confirm the reported AISHELL CER, pretrained recognition quality, or a full repository suite. Existing released packages remain unchanged. |
Summary
Fixes #3789.
add_overlap_chunkstored the left context withoverlap_feats[:, -n:, :]. Withn == 0this returns every frame, so withchunk_size=[0, 10, 5](the streaming example in README and docs) the final chunk re-decodes the previous chunk's features and repeats the last characters, e.g. 火爆 → 火爆爆爆.The start index is now computed explicitly in a small
_tail_frameshelper, used for both the final and non-final cache updates. Behaviour for non-zero context sizes is unchanged.Type of change
Validation
python -m compileall funasr examples testsDocs or links checked
Runtime/deployment command tested
python tests/test_funasr_onnx_paraformer_online_overlap.pyagainst the installed package: 3 passed. Against the unfixed package, the two zero-context tests fail and the non-zero test passes.The test is added to
test-funasr-onnx-package.yml(path filters and a step after the installed-wrapper check), so CI runs it against the built wheel.AISHELL-1 test (7176 utterances),
iic/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-onlinev2.0.5,chunk_size=[0, 10, 5], onnxruntime 1.20.1 CPU, dither 0:PyTorch AutoModel with the same settings: 14.81%.
chunk_size=[5, 10, 5], outputs for the sampled utterances are identical before and after (the fix does not change slicing for non-zero sizes).User impact
Removes the extra tail decoding caused by retaining all overlap frames when the requested context size is zero. This affects users running the streaming ONNX model with the documented
chunk_size=[0, 10, 5]. Other trailing repeats that also appear in the PyTorch reference are out of scope.Notes for reviewers
Only
paraformer_online_bin.pyis affected; the C++ runtime does not use this code. Running black on the file also reformats unrelated pre-existing lines, so those are left untouched.