Skip to content

Port the JRE module trim, Jackson/snappy/commons-math3 jars and the jar closure test (#19) - #28

Merged
tae898 merged 2 commits into
mainfrom
port/jre-trim-jar-closure
Oct 24, 2026
Merged

tae898 merged 2 commits into
mainfrom
port/jre-trim-jar-closure

Conversation

@tae898

@tae898 tae898 commented Oct 24, 2026

Copy link
Copy Markdown
Collaborator

Part of #19: the group "JRE module trim, Jackson/snappy/commons-math3 handling, jar closure and payload tests". It comes after #11, #12 and #16, which are merged. The Studio jar exclusion is #20 and is not touched here.

Changes

  1. JRE module trim (Trim the bundled JRE: drop java.desktop, jdk.jfr, java.compiler and strip native libraries (-13 MB per wheel) humemai/arcadedb-embedded-python#452, aebf4373b5). jdeps names java.desktop, jdk.jfr and java.compiler because of optional users (JTS awt, Jackson and snakeyaml java.beans, an unused JFR reporter, agrona's code generator), so the jlink runtime carried modules that no code path of the wheel reaches. New scripts/jre_module_exclusions.txt lists the modules to keep out. scripts/Dockerfile.build and scripts/build-native.sh remove them from the jdeps answer before jlink, stop the build when jdeps finds nothing (the java.se fallback would bring the modules back), and fail the build when one is still in java --list-modules afterwards (a module that another module requires comes back that way). The Linux build also strips the runtime's .so files.
  2. Jackson, snappy-java and commons-math3 ship again (Ship Jackson, snappy-java and commons-math3; polyglot engine off by default; jar closure test humemai/arcadedb-embedded-python#409, bc93f97a67). scripts/jar_exclusions.txt excluded them as "embedded-unused", checked with the embedded suite only. Server mode reaches them: swagger-core builds GET /api/v1/openapi.json with Jackson, the Prometheus remote read and write routes and snappy-compressed bodies need snappy-java, and jvector's ScoreTracker$TwoPhaseTracker needs commons-math3. The file now says so beside the exclusion list.
  3. Jar closure test (#409 and #452). tests/test_jar_closure.py reads the class files of the shipped jars and fails on a reference into a package that no shipped jar or JDK module provides, unless an allow-list entry gives a reason. The second half does the same against the packages of the running runtime for the removed modules, checks the runtime lacks them, and runs Jackson and the YAML mapper on it.
  4. Server payload tests (#409). tests/test_server_packaging.py gains test_openapi_json_is_served and test_snappy_compressed_bodies_are_decoded.
  5. README package-contents numbers, from the wheel measured below.

Measured

Both wheels: linux/amd64, Python 3.12, built with scripts/build.sh from the same engine jars (extracted once from the arcadedata/arcadedb:26.11.1-SNAPSHOT image), baseline is origin/main f042fd3.

before (origin/main) after (this PR)
wheel, compressed 73,973,014 bytes 67,547,260 bytes
wheel, uncompressed 103,089,346 bytes 90,229,775 bytes
bundled JRE, uncompressed 66,204,657 bytes 46,097,654 bytes
jars 65, 36,600,156 bytes 72, 43,850,849 bytes

The JRE shrinks by 20,107,003 bytes uncompressed and the jars grow by 7 files, so the wheel is 6,425,754 bytes smaller overall. For comparison, the source commits recorded "80.8 MB to 67.7 MB" (#452) and "about 6.7 MB" for the three jars (#409), on their own tree.

Tests

  • Smoke on both wheels: import arcadedb_embedded, create a database, SQL count, Cypher MATCH, an LSM_VECTOR index and vectorNeighbors: OK.
  • Against the after wheel: test_jar_closure.py, test_server_packaging.py, test_build_scripts.py, test_jar_provenance.py and test_wheel_size.py pass. After removing the polyglot test (below), test_jar_closure.py alone gives 8 passed.
  • The same tests against the before wheel give 8 failed, 26 passed: the new tests catch the old exclusions and the old runtime.

Not included

🤖 Generated with Claude Code

tae898 and others added 2 commits October 24, 2026 10:34
…bedded-python

Ported from humemai/arcadedb-embedded-python#409 (bc93f97a67) and #452 (aebf4373b5):

- scripts/jre_module_exclusions.txt lists the JDK modules the bundled JRE must not contain (java.desktop, jdk.jfr, java.compiler). Dockerfile.build and build-native.sh remove them from the jdeps answer before jlink, stop when jdeps finds nothing (the java.se fallback would bring them back), and fail when one is still in `java --list-modules`. The Linux build also strips the runtime's .so files.
- scripts/jar_exclusions.txt no longer excludes jackson, snappy-java and commons-math3: server mode (openapi.json, snappy bodies) and jvector refer to them.
- tests/test_jar_closure.py reads the class files of the shipped jars and fails on a reference into a package no shipped jar or JDK module provides, unless listed with a reason.
- tests/test_server_packaging.py gains the openapi.json and snappy body tests.

Refs #19

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…the built wheel

The polyglot test checks the -Darcadedb.polyglotEngineEnabled=false default in jvm.py, which comes with the runtime-fixes port. The README numbers are from the 26.11.1.dev0 linux/amd64 wheel built from this tree.

Refs #19

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@tae898

tae898 commented Oct 24, 2026

Copy link
Copy Markdown
Collaborator Author

No review needed from you for this one, @robfrank: it is a build trim (JRE modules, jars; wheel 73.97 MB to 67.55 MB) and CI is green. I will merge it after 24 hours unless you object here. Sorry for the earlier ping.

@robfrank robfrank left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@tae898
tae898 merged commit ef1d9c3 into main Oct 24, 2026
46 checks passed
tae898 added a commit that referenced this pull request Oct 24, 2026
Conflicts in README.md, both in sections this branch deliberately removes:
main's #28 updated the size figures in the Features bullets and in the Package
Contents list, and this branch drops every size figure and the Features list.
Kept this branch's text in both places. Also brought the README in line with what
main added: the Documentation list names the new Which package for which mode and
Concurrency pages, and the examples count is 27 (example 29 landed).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants