Skip to content

Commit ab0eba9

Browse files
authored
[ONNX][Autocast] Adds nodes_to_exclude regex support to the QDQ-aware convert_to_f16() API (#2241)
### What does this PR do? Type of change: New feature Adds `nodes_to_exclude` regex support to the QDQ-aware `convert_to_f16()` API, matching the node-name exclusion semantics already supported by `convert_to_mixed_precision()`. This allows callers to keep selected numerically sensitive subgraphs in FP32 while converting the rest of a quantized ONNX graph to FP16 or BF16. Existing `op_block_list` and `tensor_block_dict` behavior remains unchanged. The regression test reuses the existing conversion fixture and verifies that: - op_block_list continues to preserve matching operations in FP32. - nodes_to_exclude preserves regex-matching nodes in FP32. - Non-matching computation is converted to FP16. - The resulting ONNX model passes full validation. ### Usage ```python import onnx from modelopt.onnx.autocast import convert_to_f16 model = onnx.load("model.onnx", load_external_data=True) converted_model = convert_to_f16( model, low_precision_type="fp16", # Preserve Q/DQ operations using the existing op-type policy. op_block_list=["QuantizeLinear", "DequantizeLinear"], # Keep the numerically sensitive RMSNorm calculation in FP32. nodes_to_exclude=[ r"^/rms/(Pow|ReduceMean|Add|Sqrt|Div)$", ], ) onnx.save(converted_model, "model_fp16.onnx") ### Testing ```bash pytest tests/unit/onnx/autocast/test_precisionconverter.py ``` Result: 185 tests passed. Added focused coverage for combining operation-type and node-name exclusions. The test also includes a non-excluded FP16 conversion control. ### Before your PR is "*Ready for review*" Make sure you read and follow [Contributor guidelines](https://github.com/NVIDIA/Model-Optimizer/blob/main/CONTRIBUTING.md) and your commits are signed (`git commit -s -S`). Make sure you read and follow the [Security Best Practices](https://github.com/NVIDIA/Model-Optimizer/blob/main/SECURITY.md#security-coding-practices-for-contributors) (e.g. avoiding hardcoded `trust_remote_code=True`, `torch.load(..., weights_only=False)`, `pickle`, etc.). - Is this change backward compatible?: ✅ - If you copied code from any other sources or added a new PIP dependency, did you follow guidance in `CONTRIBUTING.md`: N/A — no copied code or new dependency. - Did you write any new necessary tests?: ✅ - Did you update [Changelog](https://github.com/NVIDIA/Model-Optimizer/blob/main/CHANGELOG.rst)?: ✅ - Did you get Claude approval on this PR?: ❌ — pending /claude review. ### Additional Information This addresses QDQ-aware mixed-precision conversion of numerically sensitive named subgraphs without requiring callers to expand an entire operation type into op_block_list. No new runtime or PIP dependencies are introduced. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added Muse Glimmer AutoQuantize, Alpamayo QAD, and streaming Kimi-K3 conversion. * Added layerwise checkpoint export and ONNX Q/DQ node-name exclusion support. * Added temporary quantization contexts and improved CUDA capability handling. * **Bug Fixes** * Fixed NVFP4 calibration and export issues. * Ensured excluded nodes and blocked operators remain in FP32 during FP16 conversion. * Improved strict ONNX model validation. * **Documentation** * Documented new quantization contexts and CUDA capability behavior. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Signed-off-by: Jai Prajapati <jprajapati@nvidia.com>
1 parent 029c67f commit ab0eba9

3 files changed

Lines changed: 35 additions & 3 deletions

File tree

CHANGELOG.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,7 @@ Changelog
2323

2424
*Misc*
2525

26+
- Add ``nodes_to_exclude`` regex support to the Q/DQ-aware ONNX ``convert_to_f16`` API, matching ``convert_to_mixed_precision`` node-name exclusion semantics while composing with the existing operation and tensor block lists.
2627
- Add ``modelopt.torch.utils.mlflow.MlflowRunLogger`` for recording a script run on an MLflow tracking server: the invocation, the ModelOpt version, the run log (captured by teeing ``stdout``/``stderr``) and any caller-supplied artifacts, with configuration as searchable params. ``mlflow`` is an optional dependency, imported only when tracking is enabled.
2728
- Add ``--mlflow <tracking-uri>`` to ``examples/hf_ptq/hf_ptq.py`` (MLflow's own ``MLFLOW_TRACKING_URI`` is honoured too). A tracked run records the invocation, the resolved recipe (``$import``\ s expanded), the run log and the quantization summaries, with every command-line argument as a searchable param; failed runs are recorded with their traceback. The experiment defaults to ``$USER/hf_ptq/<checkpoint basename>-<recipe name or --qformat>`` and can be overridden with ``--mlflow_experiment`` / ``--mlflow_run_name``.
2829
- Add ``--mlflow <tracking-uri>`` to ``examples/vllm_serve/vllm_serve_fakequant.py`` (MLflow's own ``MLFLOW_TRACKING_URI`` is honoured too), so a fake-quant serve records what it quantized and an evaluation of that endpoint can be traced back to a recipe. A tracked run uploads the launcher command, the resolved ``RECIPE_PATH`` (or the merged ``QUANT_CFG``/``KV_QUANT_CFG`` when presets are used), the worker log and the quantizer summary; the experiment defaults to ``$USER/vllm_serve_fakequant/<model basename>-<recipe name or quantization config>`` and can be overridden with ``--mlflow-experiment`` / ``--mlflow-run-name``.

modelopt/onnx/autocast/convert.py

Lines changed: 15 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -32,7 +32,11 @@
3232
import modelopt.onnx.utils as onnx_utils
3333
from modelopt.onnx.autocast.graphsanitizer import GraphSanitizer
3434
from modelopt.onnx.autocast.logging_config import logger
35-
from modelopt.onnx.autocast.nodeclassifier import NodeClassifier, NodeRuleBase
35+
from modelopt.onnx.autocast.nodeclassifier import (
36+
DisabledNodeNameRegexRule,
37+
NodeClassifier,
38+
NodeRuleBase,
39+
)
3640
from modelopt.onnx.autocast.precisionconverter import PrecisionConverter
3741
from modelopt.onnx.autocast.referencerunner import ReferenceRunner
3842
from modelopt.onnx.utils import get_min_opset_for_precisions, get_qdq_precisions
@@ -221,6 +225,7 @@ def convert_to_f16(
221225
trt_plugins: list[str] | None = [],
222226
use_standalone_type_inference: bool = False,
223227
opset: int | None = None,
228+
nodes_to_exclude: list[str] | None = None,
224229
) -> onnx.ModelProto:
225230
"""Convert model to mixed precision, using PrecisionConverter.
226231
@@ -240,6 +245,7 @@ def convert_to_f16(
240245
(22 for bf16, 19 for fp16) and Q/DQ node requirements. The opset may be automatically
241246
increased if Q/DQ nodes in the model require a higher version (e.g., FP8 requires 19,
242247
INT4 requires 21, NVFP4 requires 23).
248+
nodes_to_exclude: List of regex patterns to match node names that should remain in FP32.
243249
"""
244250
assert low_precision_type in ["fp16", "bf16"], "low_precision_type must be either fp16 or bf16"
245251
original_network_io_metadata = _capture_network_io_metadata(model, keep_io_types)
@@ -303,9 +309,15 @@ def convert_to_f16(
303309
use_standalone_type_inference=use_standalone_type_inference,
304310
original_network_io_metadata=original_network_io_metadata,
305311
)
306-
high_precision_nodes = [node.name for node in model.graph.node if node.op_type in op_block_list]
312+
node_name_rule = DisabledNodeNameRegexRule(nodes_to_exclude or [])
313+
high_precision_nodes = [
314+
node.name
315+
for node in model.graph.node
316+
if node.op_type in op_block_list or node_name_rule.check(node)
317+
]
318+
high_precision_node_set = set(high_precision_nodes)
307319
low_precision_nodes = [
308-
node.name for node in model.graph.node if node.op_type not in op_block_list
320+
node.name for node in model.graph.node if node.name not in high_precision_node_set
309321
]
310322
model_mod = precision_converter.convert(high_precision_nodes, low_precision_nodes)
311323
return model_mod

tests/unit/onnx/autocast/test_precisionconverter.py

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2220,6 +2220,25 @@ def test_convert_to_f16_restores_public_io_metadata_from_entry_boundary():
22202220
onnx.checker.check_model(converted, full_check=True)
22212221

22222222

2223+
def test_convert_to_f16_combines_op_and_node_exclusions(simple_model):
2224+
model, *_ = simple_model
2225+
converted = convert_to_f16(
2226+
model,
2227+
keep_io_types=False,
2228+
op_block_list=["MatMul"],
2229+
nodes_to_exclude=[r"^add$"],
2230+
)
2231+
2232+
value_types = {
2233+
value.name: value.type.tensor_type.elem_type
2234+
for value in (*converted.graph.output, *converted.graph.value_info)
2235+
}
2236+
assert value_types["gemm_output"] == TensorProto.FLOAT
2237+
assert value_types["add_output"] == TensorProto.FLOAT
2238+
assert value_types["Y"] == TensorProto.FLOAT16
2239+
onnx.checker.check_model(converted, full_check=True)
2240+
2241+
22232242
def test_convert_to_f16_refreshes_gathernd_pre_cast_declaration(monkeypatch):
22242243
def discover_test_plugins_without_trt(self):
22252244
self.custom_ops = {

0 commit comments

Comments
 (0)