Skip to content

ADFA-6228: Stop building the per-module classpath trie - #2074

Open
itsaky-adfa wants to merge 2 commits into
perf/ADFA-6228-class-lookupfrom
perf/ADFA-6228-drop-classtrie
Open

itsaky-adfa wants to merge 2 commits into
perf/ADFA-6228-class-lookupfrom
perf/ADFA-6228-drop-classtrie

Conversation

@itsaky-adfa

@itsaky-adfa itsaky-adfa commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor

ADFA-6228

4 of 4, stacked on #2073. After #2073 nothing reads ModuleProject.compileClasspathClasses, but every module still fills it on every open. This deletes the field and its fill, in its own commit, so a single revert restores the trie if a reader turns out to be missing.

Measured on a moto g34 (512 MB heap), cold open, stage + probes vs this stack:

before after
ClassTrie$Node 472,065 4,430 (boot classpath only)
String 58.3 MB 34.5 MB
Classpath indexing phase 16.6 s, 2,870 MB allocated, 10 blocking GCs 9.8 s, 1,035 MB allocated, 2 blocking GCs

A second commit fixes the "Indexing classes (n/m)" total, which climbed for minutes during the first library pass. Each JAR's stored fingerprint was read one at a time, under the same index lock the pass's own jobs take for every 500-row batch, and the JAR was added to the total only after its read returned. On the moto g34, submitting 404 JARs took 152 s. The pass now reads every fingerprint in one query before it starts any job. The new test in JarIndexingServiceProgressTest fails without the fix.

A cold open of this project still runs out of memory on that device, in both builds, later on during Kotlin source indexing. That is a separate problem and is not addressed here.

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This repository is configured for manual code reviews. Comment @claude review for a one-time review, or @claude review always to subscribe this PR to a review on every future push.

Tip: disable this comment in your organization's Code Review settings.

@itsaky-adfa
itsaky-adfa added this pull request to stack #2077 September 25, 2026 09:16
@coderabbitai

coderabbitai Bot commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: e546c302-e166-4482-bb06-789279d95aa9

📥 Commits

Reviewing files that changed from the base of the PR and between ac5f599 and ff81c6a.


📒 Files selected for processing (6)
  • lsp/indexing/src/main/kotlin/org/appdevforall/codeonthego/indexing/InMemoryIndex.kt
  • lsp/indexing/src/main/kotlin/org/appdevforall/codeonthego/indexing/SQLiteIndex.kt
  • lsp/indexing/src/main/kotlin/org/appdevforall/codeonthego/indexing/api/Index.kt
  • lsp/jvm-symbol-index/src/main/kotlin/org/appdevforall/codeonthego/indexing/jvm/JarIndexingService.kt
  • lsp/jvm-symbol-index/src/main/kotlin/org/appdevforall/codeonthego/indexing/jvm/JvmSymbolIndex.kt
  • lsp/jvm-symbol-index/src/test/kotlin/org/appdevforall/codeonthego/indexing/jvm/JarIndexingServiceProgressTest.kt

Included review availability: This review used your included allowance. Your plan provides up to 2 included reviews per hour; 1 remain after this review.



📝 Summary
  • Removed ModuleProject.compileClasspathClasses and the code that built a per-module compile-classpath trie.
  • Updated completion tests and differential-test documentation to reflect the removal.
  • The author reports these cold-open changes on a moto g34 with a 512 MB heap: classpath indexing decreased from 16.6 to 9.8 seconds, allocation from 2,870 MB to 1,035 MB, and blocking garbage collections from 10 to 2. The reported ClassTrie$Node count decreased from 472,065 to 4,430; the remaining nodes cover the boot classpath.
  • Risk: Removing the public compileClasspathClasses property may affect consumers outside this repository.
  • The author reports that cold open still runs out of memory during Kotlin source indexing on the tested device. This change does not address that issue.
  • Test results were not provided.
📝 Summary

Walkthrough

ModuleProject no longer exposes or builds compileClasspathClasses. The index API now returns source fingerprints, and JAR reindexing uses a snapshot of those fingerprints. Related tests and differential-test documentation are updated.

Changes

Classpath Indexing

Layer / File(s) Summary
Remove classpath class indexing
subprojects/projects/src/main/java/com/itsaky/androidide/projects/api/ModuleProject.kt, lsp/java/src/test/java/com/itsaky/androidide/lsp/java/providers/completion/ImportCompletionProviderTest.kt, lsp/jvm-symbol-index/src/test/kotlin/org/appdevforall/codeonthego/indexing/jvm/*
ModuleProject removes compileClasspathClasses and stops enumerating JAR classes into that index. Compile classpath filtering, canonical-file caching, and Android boot-classpath updates remain. The completion test and differential-test documentation reflect the changed indexing setup.

Source Fingerprint Snapshots

Layer / File(s) Summary
Expose source fingerprint maps
lsp/indexing/src/main/kotlin/org/appdevforall/codeonthego/indexing/api/Index.kt, lsp/indexing/src/main/kotlin/org/appdevforall/codeonthego/indexing/{InMemoryIndex,SQLiteIndex}.kt, lsp/jvm-symbol-index/src/main/kotlin/org/appdevforall/codeonthego/indexing/jvm/JvmSymbolIndex.kt
The index implementations and JvmSymbolIndex add sourceFingerprints(), which returns source IDs mapped to fingerprints.
Use fingerprint snapshots during JAR reindexing
lsp/jvm-symbol-index/src/main/kotlin/org/appdevforall/codeonthego/indexing/jvm/JarIndexingService.kt, lsp/jvm-symbol-index/src/test/kotlin/org/appdevforall/codeonthego/indexing/jvm/JarIndexingServiceProgressTest.kt
JarIndexingService accepts an index factory and reads source fingerprints once before submitting JAR jobs. A gated-index test checks progress reporting while fingerprint reads are blocked.

Priority: ➖ Normal

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Refactor

Suggested reviewers: jatezzz


Merge Risk: ⚪ Minimal · up to ff81c

The classpath cleanup and fingerprint change appear mergeable after normal checks. No actionable regression was identified.

Security Architecture Review

Security architecture risk: 🔵 Low · up to ff81c

The inspected refresh flow retains its source-selection, job-validation and failure-recovery controls. No introduced security issue was identified. Risk remains low because public API compatibility and integrations outside the repository were not verified.

Retained concerns
No architecture-level concerns identified.

Security review details

Security Blast Radius

  • observed — The bulk API enumerates source IDs and fingerprints across one backing index, including inactive cached sources; it is not restricted to the active workspace set. The inspected refresh consumer uses that map only to compare JARs selected for the current workspace.

Trust Boundaries and Controls

  • inferred — The inspected bulk-read path does not alter symbol-query visibility or introduce project-controlled index authority. Active-source filtering remains on symbol and package queries; fingerprint access already delegated directly to the backing index. The new SQLite query interpolates only the existing sanitized table identifier, not source IDs. Untrusted external access to the new API was not established.

Resilience and Maintainability Implications

  • inferred — The inspected transition logic retains failure containment: submitted jobs still validate stored fingerprints, fold identical work, and cancel and join superseded work before replacement. SQLite records a successful fingerprint only with final-pass completion; partial failure leaves it absent for retry. Refresh closes its progress pass on failure or cancellation. No introduced cleanup or recovery regression was established by the source comparison.

Pre-merge checks | Passed 4 | Failed 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 31.82% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 22 functions across 10 files. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Description check Passed The description clearly explains removal of ModuleProject.compileClasspathClasses, the fingerprint-read optimization, measured performance improvements, and the remaining out-of-memory issue.
Title check Passed The title clearly and concisely summarizes the main change: stopping construction of the per-module classpath trie.

  • Fix all pre-merge checks with AI
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Commit to this branch
  • Create a new PR

🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR

  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

A rabbit checks the jars at dawn
And gathers fingerprints one by one
The trie grows quiet, classes pass
While source maps shine like dewdrops on grass
Three jars wait, then hop along
The index hums a tidy song

Comment @coderabbitai help to get the list of available commands.

@itsaky-adfa
itsaky-adfa force-pushed the perf/ADFA-6228-drop-classtrie branch from 5f67f3e to 780908a Compare October 7, 2026 13:50
@itsaky-adfa
itsaky-adfa force-pushed the perf/ADFA-6228-drop-classtrie branch from 780908a to 05d2ea8 Compare October 8, 2026 13:39
@itsaky-adfa
itsaky-adfa force-pushed the perf/ADFA-6228-drop-classtrie branch 2 times, most recently from 765c699 to 5b98760 Compare October 9, 2026 12:24
Class lookups now come entirely from the JVM symbol index; reverting this
commit alone restores the trie and its fill.
@itsaky-adfa
itsaky-adfa force-pushed the perf/ADFA-6228-drop-classtrie branch from 5b98760 to ac5f599 Compare October 12, 2026 13:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants