Skip to content

perf(migrate): reuse syntax trees between unchanged rules - #2987

Open
fengmk2 wants to merge 1 commit into
mainfrom
perf/migrate-reuse-syntax-trees
Open

fengmk2 wants to merge 1 commit into
mainfrom
perf/migrate-reuse-syntax-trees

Conversation

@fengmk2

@fengmk2 fengmk2 commented Oct 11, 2026 •

Copy link
Copy Markdown
Member

Import migration parses each source file again for every rewrite rule. Most rules leave the file unchanged, so large projects repeat the same parsing work many times.

Reuse the syntax tree while the text and rule language stay unchanged. Parse again after each edit or language change. Rules still run in order, and later rules see all earlier edits.

Compare main at d74d7975b with this PR at f4993bc49. The table shows median elapsed times from six alternating pairs on the same Linux runner. Both variants use the release profile, fat LTO, one codegen unit, and four Rayon threads.

Project Input Main This PR Time change Speedup
Dify Original files 77.976s 5.821s 🟢 ↓ 92.5% 13.40×
Dify Already migrated files 72.615s 5.369s 🟢 ↓ 92.6% 13.52×
Vinext Original files 38.232s 5.635s 🟢 ↓ 85.3% 6.79×
Vinext Already migrated files 38.130s 5.357s 🟢 ↓ 85.9% 7.12×

Each measurement starts a fresh process. The driver reads files before the timer starts to make filesystem cache conditions comparable. The timed work calls rewrite_imports_in_directory, which includes file discovery, rule setup, reads, matching, and writes. It excludes compilation, dependency installation, and other migration phases. These percentages apply only to import rewriting.

A separate diagnostic pass counts parser calls without affecting the timed binaries. First-pass counts are:

Project Main parses PR parses Parse reduction Matcher attempts in each variant
Dify 180,848 4,747 🟢 ↓ 97.4% 180,848
Vinext 99,152 2,075 🟢 ↓ 97.9% 99,152

The Vitest group has 128 rules. A file with no edits needs one parse for that group instead of 128. All matcher attempts and replacement counts match between variants. Output hashes, changed-file lists, and warnings also match in every pair.

Benchmark run and raw results · Benchmark source · Individual samples and diagnostics

@fengmk2 fengmk2 added the test: e2e Auto run e2e tests label Oct 11, 2026
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Oct 11, 2026 •

Copy link
Copy Markdown

🚀 Deploying Preview to Cloudflare 🚀

Preview URL: https://perf-migrate-reuse-syntax-trees-viteplus-dev.voidzero-docs.workers.dev (commit f4993bc)

This URL reflects your latest Preview deployment

Preview Deployments by commit

Status Deployment URL Commit Updated (UTC) See this deployment's details
  • Build: Success ✅
  • Deployment: Success ✅

View logs ↗
https://b9618037-viteplus-dev.voidzero-docs.workers.dev f4993bc 2026-10-13T17:05:49.375Z Visit the dashboard ↗
  • Build: Success ✅
  • Deployment: Success ✅

View logs ↗
https://f6213d13-viteplus-dev.voidzero-docs.workers.dev 989e9c4 2026-10-11T02:14:05.275Z Visit the dashboard ↗

@github-actions

github-actions Bot commented Oct 11, 2026 •

Copy link
Copy Markdown
Contributor

CLI artifact sizes (f4993bc)

Final release artifacts built by the canonical build-upstream and build-windows-cli actions.
The dist rows use the Linux build. The core total excludes .node files to match the release artifact.

Artifact Format Base PR Change
packages/cli/dist Directory total 2.21 MiB 2.21 MiB 0 B (0.00%)
packages/core/dist Directory total 4.08 MiB 4.08 MiB 0 B (0.00%)
Combined package dist Directory total 6.29 MiB 6.29 MiB 0 B (0.00%)
vp (Linux x64) Binary 11.42 MiB 11.42 MiB 0 B (0.00%)
vp (Linux x64) gzip -9 4.93 MiB 4.93 MiB 0 B (0.00%)
NAPI (Linux x64) Binary 33.69 MiB 33.69 MiB 0 B (0.00%)
NAPI (Linux x64) gzip -9 13.41 MiB 13.41 MiB +20 B (+0.00%)
vp (macOS ARM64) Binary 8.50 MiB 8.50 MiB 0 B (0.00%)
vp (macOS ARM64) gzip -9 4.31 MiB 4.31 MiB 0 B (0.00%)
NAPI (macOS ARM64) Binary 40.96 MiB 40.96 MiB 0 B (0.00%)
NAPI (macOS ARM64) gzip -9 17.64 MiB 17.64 MiB -21 B (-0.00%)
vp (Windows x64) Binary 9.29 MiB 9.29 MiB 0 B (0.00%)
vp (Windows x64) gzip -9 4.05 MiB 4.05 MiB -1 B (-0.00%)
NAPI (Windows x64) Binary 29.53 MiB 29.53 MiB 0 B (0.00%)
NAPI (Windows x64) gzip -9 12.03 MiB 12.03 MiB +233 B (+0.00%)
Trampoline (Windows x64) Binary 13.50 KiB 13.50 KiB 0 B (0.00%)
Trampoline (Windows x64) gzip -9 7.09 KiB 7.08 KiB -3 B (-0.04%)
Installer (Windows x64) Binary 4.63 MiB 4.63 MiB 0 B (0.00%)
Installer (Windows x64) gzip -9 2.17 MiB 2.17 MiB -1 B (-0.00%)

@fengmk2
fengmk2 force-pushed the perf/migrate-reuse-syntax-trees branch from 989e9c4 to f4993bc Compare October 13, 2026 17:04
@fengmk2 fengmk2 self-assigned this Oct 14, 2026
@fengmk2
fengmk2 marked this pull request as ready for review October 14, 2026 03:42

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

test: e2e Auto run e2e tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants