Repository navigation
Conversation
… one JPype call per element A vector load spent most of its time on text: Python wrote every float as a decimal number and the engine parsed it back (insert_many), or JPype converted every list element on its own (create_vertices, Document.set). For a property declared ARRAY_OF_FLOATS the engine converts each number of a list to float anyway, so the vectors of a chunk now cross as one float32 matrix (FloatVectors.Matrix, one buffer copy) beside the JSON, and each document gets its own float[] row Java-side. Properties that are not declared ARRAY_OF_FLOATS, and vectors that are ragged, missing or null, keep the exact JSON path. Also: insert_columns accepts a 2-D float32 array as a column of vectors; a numpy array is accepted as a property value everywhere (Document.set, insert_many, create_vertices; it raised TypeError before); the allowed_rids filter of a vector search builds its RID set in one crossing (GraphCalls.ridSet) instead of two per RID. Tests: tests/test_float_vector_bulk.py (39 tests: exact float32 equality with the engine's own list conversion on doubles a float32 cannot hold, which values keep the JSON path, every entry point, parallel mode); tests/test_vector.py rid-conversion test now passes a Java RID object so the list takes the per-RID path; tests/test_python_callbacks.py numpy return value test follows the new conversion. Ported from humemai/arcadedb-embedded-python#496. The user docs and the speed table generated there live in that repository's docs; this repository has no user docs yet. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
8 tasks
No conflicts. Brings in the updated functions-events branch (which now carries main's merged changes through runtime-fixes). Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #29 (
port/functions-events): the base is that branch, notmain, and a merge waiter retargets this PR tomainonce #29 merges. Until then the diff againstmainalso shows #29.Ports humemai/arcadedb-embedded-python#496. A vector load spent most of its time on text: Python wrote every float as a decimal number and the engine parsed it back (
insert_many), or JPype converted every list element on its own (create_vertices,Document.set). For a property declaredARRAY_OF_FLOATSthe engine converts each number of a list to float anyway, so the vectors of a chunk now cross as one float32 matrix (FloatVectors.Matrix, one buffer copy) beside the JSON, and each document gets its ownfloat[]row on the Java side. Properties that are not declaredARRAY_OF_FLOATS, and vectors that are ragged, missing or null, keep the exact JSON path.Also in this change:
Database.insert_columnsaccepts a 2-D float32 array as a column of vectors.Document.set,insert_many,create_vertices); it raisedTypeErrorbefore.allowed_ridsfilter of a vector search builds its RID set in one crossing (GraphCalls.ridSet) instead of two JPype calls per RID.FloatVectors.java;DocumentBatcher,VertexBatcherandGraphCallsgain overloads, the old signatures stay.Tests:
tests/test_float_vector_bulk.py(39 tests: exact float32 equality with the engine's own list conversion on doubles a float32 cannot hold, which values keep the JSON path, every entry point, parallel mode).tests/test_vector.pyandtests/test_python_callbacks.pyfollow the new conversion;test_a_numpy_array_is_a_return_valueneedsregister_function, which is why this sits on #29.Measured numbers: they come from the humemai float-vector-crossing laptop A/B study (
tools/ab.py, ABBA order, one fresh process per run, 10 clean pairs per figure, 128-d float32 vectors on anARRAY_OF_FLOATSproperty, 4 GB heap, one base wheel so the engine jars and the JVM are identical). The rows below are copied by script from the table thatrender_ab_tables.pygenerated from that study'sab.txtin humemai/arcadedb-embedded-python#496; they were not re-measured for this PR, and this repository has no copy of the study. Engine 26.11.1, wheel 26.11.1.dev0, cores 2 (pinned), machine x86-64 laptop, 16 cores.The insert_columns row compares the best case of the earlier bindings (one
float[]per row) with the 2-D column.Not in this PR: user docs and the generated speed table. This repository has no user docs on this base (the Antora component is #23), and
render_ab_tables.pyis not here, so the table is not ported; the docs page can follow once #23 lands.Checked locally: pre-commit (black, isort) and bandit (low severity, low confidence) clean; the touched and related tests (335 tests: float vector bulk, vector, vector SQL, insert_columns, graph batch, bulk insert, bulk JSON paths, numpy, type conversion, callbacks) pass on a wheel built from the 26.11.1.dev0 jars with this branch's bridge sources compiled by javac 25. Not run locally: the full wheel build (Docker, jlink) and the full test suite on all platforms and Python versions; CI covers those.
🤖 Generated with Claude Code