Skip to content

Python runtime fixes in src: serialized start_jvm(), heap default, log directory, XML limits, read-only open, async docs - #27

Merged
tae898 merged 4 commits into
mainfrom
port/runtime-fixes
Oct 24, 2026
Merged

tae898 merged 4 commits into
mainfrom
port/runtime-fixes

Conversation

@tae898

@tae898 tae898 commented Oct 24, 2026 •

Copy link
Copy Markdown
Collaborator

Part of #19, the "Python runtime fixes in src/" group, ported by hand from humemai/arcadedb-embedded-python bindings/python (origin/main at a9ed1284cf) with their tests. Based on current main; black, isort and bandit run with this repo's config.

What is ported

  • Free-threaded start_jvm(): start_jvm() holds a re-entrant lock, so threads whose first call is create_database() or open_database() no longer race into jpype.startJVM() (all but one failed with "Failed to start JVM", also with the GIL). Source: humemai#454 (only the start_jvm lock; the JPype proxy-class part belongs to the callbacks feature). Test: tests/test_free_threading.py (many threads on one database and one database each, compared with a sequential run).
  • Heap default: start_jvm(heap_size=None) now adds -XX:MaxRAMPercentage=50 (the container limit inside a container) instead of -Xmx4g. An explicit heap_size always wins; a user -Xmx or other max-heap flag is kept; no -Xms is added; an -Xms larger than the maximum heap raises ValueError before the JVM starts (the JVM would otherwise end the process). Source: humemai#420. Tests: tests/test_jvm.py, tests/test_jvm_args.py.
  • Log directory: the engine log, the crash log and a rendered logging.properties go to a per-user directory (ARCADEDB_LOG_DIR, else the platform state directory, with a temp-dir fallback) instead of ./log in the working directory. jvm_args and ARCADEDB_JVM_ERROR_FILE still win. Source: humemai#407. Test: tests/test_log_dir.py.
  • XML limits: start_jvm(disable_xml_limits=...) now defaults to False (the JDK entity limits stay on). Database.import_documents(disable_xml_limits=True) lifts them for that one call and restores them afterwards; an import that hits a limit raises an ArcadeDBError naming the option. Source: humemai#407. Test: tests/test_xml_limits.py.
  • Read-only open: open_database(path, read_only=True), DatabaseFactory.open(read_only=True) and Database.is_read_only(). Source: humemai#385. Test: tests/test_open_read_only.py.
  • Async executor: the docstrings now say what each completion signal means (per-record callback, on_ok, wait_completion()) and that a WAL-off batch can still be lost in a crash. Source: humemai#405. Tests added to tests/test_async_executor.py (signal order).
  • Polyglot engine off by default: start_jvm() adds -Darcadedb.polyglotEngineEnabled=false (instead of -Dpolyglot.engine.WarnInterpreterOnly=false) unless jvm_args or ARCADEDB_JVM_ARGS set the flag, because the wheel ships no GraalVM language and the engine would only log "no languages found" at the first SQL command. Source: humemai#409 (bc93f97a67). Test: tests/test_jvm_args.py. The matching test_polyglot_engine_is_off_while_no_graalvm_language_ships in tests/test_jar_closure.py arrives with Port the JRE module trim, Jackson/snappy/commons-math3 jars and the jar closure test (#19) #28 (JRE trim and jar closure); this default assumes the jar exclusions of that group, so merge this after Port the JRE module trim, Jackson/snappy/commons-math3 jars and the jar closure test (#19) #28 or together with it.
  • Three example scripts no longer say the heap default is 4g or that logs go to ./log/.

Not included

  • Ctrl-C to KeyboardInterrupt: SIGINT and SIGTERM handling is in fix: give SIGINT and SIGTERM back to Python after the JVM starts #26. This PR leaves interrupt and its docstring as on main, so the two do not conflict.
  • The opt-in Leyden AOT cache (humemai#490): not merged there yet; it will be ported separately.
  • Python functions in SQL and Cypher, record events, Document.save() and delete() wrapping (features group, humemai#391). Because of this, Document.save() on a read-only database raises the engine's own exception, and test_open_read_only.py expects that.
  • Studio wording in server.py, build scripts, docs pages, the native wheel and the rest of the tests (other groups of Tracking: porting the rest of the humemai bindings work #19).
  • In test_free_threading.py the Python-function and event probes are guarded by hasattr(db, "register_function"), so they stay inactive until the features group lands; the proxy-class scenario was left out.

Tests run

Tests need a built wheel, so I overlaid this branch's src/arcadedb_embedded/*.py on an existing 26.11.1.dev0 wheel's unzipped contents (jars, JRE and bridge jar as built) and ran pytest from this checkout with PYTHONPATH pointing at the overlay, as for #26. Python 3.12, Linux amd64.

  • The files for the changed modules (test_jvm, test_jvm_args, test_log_dir, test_xml_limits, test_open_read_only, test_async_executor, test_free_threading, test_importer_api, test_core, test_logging_helper): 133 passed, 1 skipped (107 passed, 1 skipped on the re-run of the JVM-argument, log, XML, read-only, importer and core files after the polyglot commit) (pandas not installed in the overlay environment).
  • The whole tests/ directory except the wheel-size, wheel-tag and jar-provenance tests and the server_wire marker: 1050 passed, 6 failed, 20 skipped. The 6 failures and the skips need pandas, pyarrow or requests, which the overlay environment lacks; none touches the changed code.
  • python -m bandit -c pyproject.toml -r src tests --severity-level low --confidence-level low: no issues. The files parse on Python 3.10.

CI on the real wheel across the matrix is the check for Python 3.10, 3.13, 3.14, Windows and macOS.

🤖 Generated with Claude Code

tae898 and others added 2 commits October 24, 2026 10:37
…g directory, XML limits, read-only open, async docs

Ports the runtime-fix group of #19 from humemai/arcadedb-embedded-python bindings/python, with its tests:

- start_jvm() holds a lock, so threads whose first call is create_database() or open_database() no longer race into jpype.startJVM() (humemai #454)
- default heap is 50 percent of the RAM the JVM sees (container limit inside a container) instead of 4g; an -Xms above the maximum heap raises ValueError before the JVM starts (humemai #420)
- engine log, crash log and a rendered logging.properties go to a per-user log directory (ARCADEDB_LOG_DIR) instead of ./log (humemai #407)
- the JDK XML entity limits stay on by default; import_documents(disable_xml_limits=True) lifts them for one call, and a limit error names the option (humemai #407)
- open_database(read_only=...), DatabaseFactory.open(read_only=...), Database.is_read_only() (humemai #385)
- AsyncExecutor docstrings say what each completion signal means, with tests (humemai #405)

Not included: SIGINT and SIGTERM handling (#26), Python functions and record events, the polyglot engine default, the AOT cache.

Refs #19

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…GraalVM language (humemai#409)

The wheel excludes js-language, truffle, icu4j and regex (scripts/jar_exclusions.txt), so the polyglot engine only logs 'no languages found' at the first SQL command. -Darcadedb.polyglotEngineEnabled=false is added unless the caller sets the flag; the old WarnInterpreterOnly flag goes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The only conflict was src/arcadedb_embedded/jvm.py, where both sides added the
_LOGGER line (main with #26 signals, this branch with the log directory work);
kept one _LOGGER and the blank line before _JVM_CONFIG. The signal restore from
main and this branch's start lock, heap default, log directory and XML limits
all stay.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@tae898
tae898 merged commit 47f9b17 into main Oct 24, 2026
46 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant