Skip to content

[ONNX] Add SimplifiedLayerNormalization converter for Relax frontend - #20752

Open
Asthenia0412 wants to merge 1 commit into
apache:mainfrom
Asthenia0412:add-simplifiedlayernorm-onnx
Open

Asthenia0412 wants to merge 1 commit into
apache:mainfrom
Asthenia0412:add-simplifiedlayernorm-onnx

Conversation

@Asthenia0412

Copy link
Copy Markdown

Background

ONNX SimplifiedLayerNormalization is an RMSNorm variant used by models like SmolLM2. TVM's ONNX frontend currently lacks this op converter, blocking end-to-end import of such models.

Problem

Missing converter causes ONNX import to fail with Unsupported operator: SimplifiedLayerNormalization.

Solution

  • Add convert_onnx_simplifiedlayernormalization in onnx_frontend.py
  • Implement RMSNorm semantics: y = x / sqrt(mean(x²) + eps) * weight [+ bias]
  • Support all ONNX attributes: axis, epsilon (default 1e-5), stash_type
  • Handle optional bias input (input CXX API Todo #3)

Testing

  • Unit tests in tests/python/relax/frontend/onnx/
  • Compare against numpy RMSNorm reference
  • Test with/without bias, multiple axis values

Fixes #20625

@Asthenia0412
Asthenia0412 force-pushed the add-simplifiedlayernorm-onnx branch from a3bf78d to 5f386a2 Compare October 17, 2026 01:24
- Implement convert_onnx_simplifiedlayernormalization in onnx_frontend.py
- Map ONNX SimplifiedLayerNorm (RMSNorm variant) to Relax IR
- Support epsilon, axis attributes and optional bias input
- Add unit tests covering basic and edge cases

Fixes apache#20625
@Asthenia0412
Asthenia0412 force-pushed the add-simplifiedlayernorm-onnx branch from 5f386a2 to 90aaa72 Compare October 18, 2026 04:20

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug][ONNX] SimplifiedLayerNormalization and SkipSimplifiedLayerNormalization ops unsupported

1 participant