Repository navigation
[NV] Muse-Spark-Open NVFP4 GB300 SGLang AgentX configs / [NV] Muse-Spark-Open NVFP4 GB300 SGLang AgentX 配置 - #4454
hshrivastava-droid wants to merge 1 commit into
Conversation
…0261019) Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Signed-off-by: hshrivastava-droid <hshrivastava@nvidia.com>
|
Thanks for the contribution!
中文感谢你的贡献！
|
hshrivastava-droid
added
the
full-sweep-enabled
hshrivastava-droid
changed the title
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
| # The client container has no /model mount; it loads the tokenizer by Hub id. | ||
| HF_HUB_OFFLINE: '0' |
There was a problem hiding this comment.
🔴 All four new multi-node runs may fail at benchmark start because the AIPerf client must fetch the tokenizer from a gated Hub repo. The client env sets HF_HUB_OFFLINE: '0' and no TOKENIZER, so the tokenizer is pulled by the served model id. The sibling Muse-Spark recipes say that repo is unreadable with the job token, and they use TOKENIZER: /model. Fix: make the client tokenizer source match what works for this model on the same cluster, for example TOKENIZER: /model as in agg-variants.yaml:92. If /model really is missing in this mode, point it at a readable tokenizer path instead.
Why this was flagged
The trigger is the job token lacking read access to the gated repo meta-models/Muse-Spark-Open-816B-A42B-NVFP4. Two comments in the repo say it does: agentic.yaml:65 says "The gated Hub repo is unreadable with the job token". In the new recipe, base.benchmark.env (agg-tp8-direct-variants.yaml:79-80) sets HF_HUB_OFFLINE: '0' and no TOKENIZER, so AIPerf loads the tokenizer by Hub id. The existing multi-node recipe in the same directory and on the same cluster:gb300-nv runner (agg-variants.yaml:92) uses TOKENIZER: /model, and this is the only recipe in the repo that sets HF_HUB_OFFLINE: '0'. If the Hub download is denied, every row of musespark-fp4-gb300-sglang-agentic-dflash-multinode (conc 5/10/18/32) fails before any benchmark data is collected, so the full sweep goes red. The new comment says the client has no /model mount, but the sibling recipe gets the same srt multi-node client and uses /model, so that comment does not rule this failure out.
Verification: Triggering condition: the job's HF token cannot read the gated repo meta-models/Muse-Spark-Open-816B-A42B-NVFP4, as gb300-fp4-mtp/agentic.yaml:61 says. agg-tp8-direct-variants.yaml:78-82 sets HF_HUB_OFFLINE: '0' and no TOKENIZER. infx/bench/agentic/replay.py:77 then falls back to values["MODEL"], the gated Hub id. agg-variants.yaml:92 sets TOKENIZER: /model.
|
View unofficial run (performance): https://inferencex.semianalysis.com/inference?unofficialRun=40039644745 View unofficial run (accuracy): https://inferencex.semianalysis.com/evaluation?unofficialRun=40039644745 |
Summary
Add two Muse-Spark-Open NVFP4 GB300 SGLang AgentX configs on
lmsysorg/sglang:nightly-dev-20261019-3c560089(pinned by digest):musespark-fp4-gb300-sglang-agentic-dflash-multinode: aggregated TP8/EP8 over two nodes, concurrency 5, 10, 18 and 32.musespark-fp4-gb300-sglang-agentic-dflash-nightly1019: aggregated TP4/EP4 on one node, concurrency 28 and 52.AI model disclosure
claude-sonnet-5-5)PR checklist and change type
Type of Change
Checklist
python -m infx.matrix.changed --base origin/mainlists affected config keys, I have appended one new entry covering them to the physical end ofinferencex-e2e/perf-changelog.yaml; if it lists none, I have added none. I have not edited historical entriesinferencex-e2e/perf-changelog.yamlentry, it carries exactly one primary sweep label (a maintainer applies it on fork PRs):full-sweep-fail-fast(recommended),full-sweep-enabled, ornon-canary-full-sweep-enabled. Optional modifiersall-evals,evals-only, andagentx-fastrequire a primary label; the last two block reuse while applied.OWNER/MEMBER/COLLABORATOR) has commented/use <run_id>(or the legacy/reuse-sweep-run) on this PR. Do this only once there is a final full sweep that is all green with evals passing, since after this comment the primary sweep label will no longer automatically kick off new sweeps. Remove and re-add the primary sweep label to force a new sweep.中文
新增两个 Muse-Spark-Open NVFP4 GB300 SGLang AgentX 配置，镜像为
lmsysorg/sglang:nightly-dev-20261019-3c560089（按摘要固定）：多节点 TP8/EP8（并发 5、10、18、32）与单节点 TP4/EP4（并发 28、52）。AI 模型使用说明：Claude Sonnet 5.5（
claude-sonnet-5-5），负责准备配置、变更日志和 PR 文本。变更类型：配置变更。🤖 Generated with Claude Code