diff --git a/README.md b/README.md index 732b3ba..07e7eda 100644 --- a/README.md +++ b/README.md @@ -1,200 +1,91 @@ -# ArcadeDB Python Bindings +# ArcadeDB Python -Native Python bindings for ArcadeDB - the multi-model database that supports Graph, Document, Key/Value, Search Engine, Time Series, and Vector models. +[![PyPI](https://img.shields.io/pypi/v/arcadedb-embedded)](https://pypi.org/project/arcadedb-embedded/) +[![Python versions](https://img.shields.io/pypi/pyversions/arcadedb-embedded)](https://pypi.org/project/arcadedb-embedded/) +[![Tests](https://github.com/ArcadeData/arcadedb-python/actions/workflows/test-python-bindings.yml/badge.svg)](https://github.com/ArcadeData/arcadedb-python/actions/workflows/test-python-bindings.yml) +[![License](https://img.shields.io/badge/license-Apache--2.0-blue)](https://github.com/ArcadeData/arcadedb-python/blob/main/LICENSE) -**Status**: โœ… Production Ready | **Platforms**: 4 Supported +Python bindings for [ArcadeDB](https://arcadedb.com), the multi-model database (graph, document, key/value, vector and time series, queried with SQL or openCypher). The engine runs inside your Python process through [JPype](https://jpype.readthedocs.io/), with no server and no network hop. The wheel bundles its own Java 25 runtime, so you install no Java. Status: beta. ---- - -## ๐Ÿ“š Documentation - -**[๐Ÿ“– Read the Full Documentation โ†’](https://docs.humem.ai/arcadedb/latest)** - ---- - -## ๐Ÿš€ Quick Start - -### Installation +## Install ```bash -uv add arcadedb-embedded # or: pip install arcadedb-embedded +pip install arcadedb-embedded # or: uv add arcadedb-embedded ``` -**Requirements:** +- Python 3.10 to 3.14. CI builds and tests all five versions on every platform below. +- Linux x86_64 and ARM64, macOS on Apple Silicon, Windows x86_64. One wheel per platform and Python version. +- Import name: `arcadedb_embedded`. Optional extras: `arrow`, `pandas`, `vector`. -- **Python 3.10 to 3.14** (CI runs all five on every supported platform). No Java installation required! -- **Supported Platforms**: Prebuilt wheels for **4 platforms** - - Linux: x86_64, ARM64 - - macOS: Apple Silicon (ARM64) - - Windows: x86_64 - -### 5-Minute Example +## Quick start ```python import arcadedb_embedded as arcadedb -# Create database (context manager for automatic open and close) with arcadedb.create_database("./mydb") as db: - # Create schema (DDL) db.command("sql", "CREATE DOCUMENT TYPE Person") db.command("sql", "CREATE PROPERTY Person.name STRING") db.command("sql", "CREATE PROPERTY Person.age INTEGER") - # Insert data (requires transaction) + # Writes need a transaction with db.transaction(): db.command("sql", "INSERT INTO Person SET name = ?, age = ?", "Alice", 30) - # Query data - result = db.query("sql", "SELECT FROM Person WHERE age > 25") - for record in result: + for record in db.query("sql", "SELECT FROM Person WHERE age > 25"): print(f"Name: {record.get('name')}") ``` -**[๐Ÿ‘‰ See full tutorial](https://docs.humem.ai/arcadedb/latest/getting-started/quickstart/)** - ---- - -## โœจ Features - -- โ˜• **No Java Installation Required**: Bundled JRE (~44 MiB uncompressed) -- ๐ŸŒ **4 Platforms Supported**: Linux (x86_64, ARM64), macOS (ARM64), Windows (x86_64) -- ๐Ÿš€ **Embedded Mode**: Direct database access in Python process (no network) -- ๐ŸŒ **Server Mode**: Optional in-process HTTP server with the Studio web UI: it costs ~8MB of wheel and nothing at runtime until you call `create_server()` -- ๐Ÿ“ฆ **Self-contained**: All dependencies bundled (~64 MiB current Linux wheel) -- ๐Ÿ”„ **Multi-model**: Graph, Document, Key/Value, Vector, Time Series -- ๐Ÿ” **Multiple query languages**: SQL and OpenCypher -- โšก **High performance**: Direct JVM integration via JPype -- ๐Ÿ”’ **ACID transactions**: Full transaction support -- ๐ŸŽฏ **Vector storage**: Store and query vector embeddings with HNSW (JVector) indexing -- ๐Ÿ“ฅ **Data import**: Built-in CSV and ArcadeDB JSONL import - ---- - -## ๐Ÿ“ฆ What's Inside - -The `arcadedb-embedded` package is platform-specific and self-contained: - -**Package Contents (the 26.11.1.dev0 Linux x86_64 wheel; varies by platform and version):** - -- **Wheel size (compressed)**: ~64 MiB -- **ArcadeDB JARs (uncompressed)**: ~42 MiB across 72 JARs -- **Bundled JRE (uncompressed)**: ~44 MiB (platform-specific Java 25 runtime via jlink) -- **Installed package size**: ~86 MiB - -The compressed wheel size is measured from `dist/*.whl`, and the installed package size -is measured from the extracted `site-packages/arcadedb_embedded/` directory. - -Of that, the optional **server stack is 12 JARs, 8.58MB uncompressed**. [Server -Mode](https://docs.humem.ai/arcadedb/latest/guide/server/) breaks the cost down and -explains what you pay at runtime (nothing, until you start a server). - -**Note**: Some JARs are excluded to optimize package size (e.g., gRPC wire protocol). See [`scripts/jar_exclusions.txt`](scripts/jar_exclusions.txt) for details. - -Import: `import arcadedb_embedded as arcadedb` - ---- - -## ๐Ÿงช Testing - -Tests run against the built wheel, installed with its `test` extra into a -virtual environment. Once that is set up, `uv run` uses it without activation: - -```bash -# Once -uv venv - -# After each build: --reinstall-package, as a rebuild keeps the version (26.11.1.dev0) -uv pip install --reinstall-package arcadedb-embedded "$(ls dist/*.whl | head -1)[test]" - -# Run all tests -uv run pytest - -# Run specific test file -uv run pytest tests/test_core.py -v -``` - -See [testing documentation](https://docs.humem.ai/arcadedb/latest/development/testing/) for detailed test documentation. - ---- - -## ๐Ÿ”ง Building from Source (Advanced) +This prints `Name: Alice` (the engine also logs to standard error). `create_database` raises if the path already holds a database, so run it in an empty directory. Everything else goes through the same two calls, `db.command(...)` and `db.query(...)`, in SQL or openCypher. -Linux uses Docker. macOS and Windows use a native Java 25+ JDK with jlink. +## What is included -```bash -# Install uv (one-time) -curl -LsSf https://astral.sh/uv/install.sh | sh - -# Build for your current platform (auto-detected) -./scripts/build.sh -``` +- Embedded mode: the database is a directory, opened by one process at a time. +- Graph, document, key/value, vector (JVector HNSW) and time series models; ACID transactions. +- Import from CSV, XML and ArcadeDB JSONL; export to JSONL. +- Columnar results with `to_columns()`, `to_arrow()` and `to_dataframe()`. +- An optional in-process HTTP server (`create_server()`), off until you call it, for other processes that need the same data. From 26.11.1 the wheels also carry the Studio web UI, which the server then serves. Studio used to bundle ApexCharts, which is no longer open source ([ArcadeData/arcadedb#10579](https://github.com/ArcadeData/arcadedb/issues/10579)), and [ArcadeData/arcadedb#10627](https://github.com/ArcadeData/arcadedb/pull/10627) replaced it with Apache ECharts. The earlier 26.10.x wheels on PyPI have no Studio. -Built wheels will be in `dist/`. To test one, install it together with the -`test` extra into a virtual environment and run `pytest tests`. +For several processes, other languages, HA or replication, run the regular [ArcadeDB server](https://docs.arcadedb.com) and connect with [`arcadedb-driver`](https://pypi.org/project/arcadedb-driver/) (HTTP client, developed in [ArcadeData/arcadedb-drivers](https://github.com/ArcadeData/arcadedb-drivers)). -**[Build instructions](https://docs.humem.ai/arcadedb/latest/getting-started/installation/#building-from-source)** +## Packages on PyPI -> **Developer Note:** See [build architecture docs](https://docs.humem.ai/arcadedb/latest/development/build-architecture/) for comprehensive documentation of the multi-platform build architecture, including how we achieve platform-specific JRE bundling across the supported platforms on GitHub Actions. +| Package | What it is | Status | +|---|---|---| +| [`arcadedb-embedded`](https://pypi.org/project/arcadedb-embedded/) | These bindings: the engine on a bundled JVM. The package this README describes | Release 26.10.2.post1 | +| [`arcadedb-embedded-native`](https://pypi.org/project/arcadedb-embedded-native/) | The same API with the engine compiled ahead of time by GraalVM Community Edition: no JVM, no JPype. Import `arcadedb_native`, Python 3.11 or later | Preview of alpha quality: `pip install --pre arcadedb-embedded-native`, currently 26.10.2.dev1 | +| [`arcadedb`](https://pypi.org/project/arcadedb/) and [`arcadedb-native`](https://pypi.org/project/arcadedb-native/) | Names reserved for the project. Today `arcadedb` 0.0.1 only installs `arcadedb-embedded`, and `arcadedb-native` 26.10.2.dev1 only installs `arcadedb-embedded-native` | Placeholders | -## Development +Two things are not true yet. The native build is not in this repository (porting is tracked in [#19](https://github.com/ArcadeData/arcadedb-python/issues/19)), and releases do not come from this repository: the wheels on PyPI are still built and published from [humemai/arcadedb-embedded-python](https://github.com/humemai/arcadedb-embedded-python). A release workflow for this repository is in review in [#22](https://github.com/ArcadeData/arcadedb-python/pull/22). -The ArcadeDB jars come from the `arcadedata/arcadedb` Docker image: `latest` by -default, which follows ArcadeDB's current snapshot, or the tag in `ARCADEDB_TAG` -to pin one (`ARCADEDB_TAG=26.11.1-SNAPSHOT ./scripts/build.sh`). The package -version is taken from the `arcadedb-engine-.jar` in that image, so -`26.11.1-SNAPSHOT` builds `26.11.1.devN`. CI does the same, and also runs every -night so each new snapshot is tested. Both workflows take an `arcadedb-tag` input -(manual run or `workflow_call`) to build from a pinned image: an exact version tag such as `26.10.2`. See [versioning strategy](https://docs.humem.ai/arcadedb/latest/development/release/#python-versioning-strategy) for details on development vs release mode handling. +## Documentation -## Origin +- Here: [`docs/`](https://github.com/ArcadeData/arcadedb-python/tree/main/docs/modules/ROOT/pages) is an Antora component (AsciiDoc) with Installation, Which package for which mode, Quickstart, Databases and queries, and Concurrency, processes and crashes. It covers only what is specific to the Python bindings. +- The component is not published on docs.arcadedb.com yet; adding it as a content source is open in [ArcadeData/arcadedb-docs#505](https://github.com/ArcadeData/arcadedb-docs/pull/505). +- The guide, API reference and example walkthroughs for the published package are still at [docs.humem.ai/arcadedb](https://docs.humem.ai/arcadedb/latest/) until those pages are ported here ([#19](https://github.com/ArcadeData/arcadedb-python/issues/19)). +- ArcadeDB itself (SQL, openCypher, schema, server): [docs.arcadedb.com](https://docs.arcadedb.com). It is not repeated here. -This code moved here from the `bindings/python` directory of -[ArcadeData/arcadedb](https://github.com/ArcadeData/arcadedb) at -`ee04bd202abce0183c6263a3c4d8a61a4d526960`, with its git history. Issue -references in imported commit messages point at `ArcadeData/arcadedb`. +## Examples ---- +[`examples/`](https://github.com/ArcadeData/arcadedb-python/tree/main/examples) holds 27 runnable scripts, from a first document store to vector search, time series, graph algorithms and server mode. CI runs them ([`test-python-examples.yml`](https://github.com/ArcadeData/arcadedb-python/blob/main/.github/workflows/test-python-examples.yml)). -## ๐Ÿ“‹ Package Structure +## Build and test from source ```bash -arcadedb_embedded/ -โ”œโ”€โ”€ async_executor.py # Asynchronous command/query + record execution -โ”œโ”€โ”€ core.py # Database and DatabaseFactory -โ”œโ”€โ”€ exceptions.py # ArcadeDBError exception -โ”œโ”€โ”€ exporter.py # Data export (JSONL, CSV) -โ”œโ”€โ”€ graph.py # Graph wrappers -โ”œโ”€โ”€ graph_batch.py # GraphBatch high-throughput graph ingest wrapper -โ”œโ”€โ”€ importer.py # Data import (CSV, XML, ArcadeDB JSONL) -โ”œโ”€โ”€ __init__.py # Public API exports -โ”œโ”€โ”€ jvm.py # JVM lifecycle management -โ”œโ”€โ”€ _logging.py # Internal logging helpers -โ”œโ”€โ”€ results.py # ResultSet and Result wrappers -โ”œโ”€โ”€ schema.py # Schema management API -โ”œโ”€โ”€ server.py # ArcadeDBServer for optional HTTP/Studio mode -โ”œโ”€โ”€ transactions.py # TransactionContext manager -โ”œโ”€โ”€ type_conversion.py # Python-Java type conversion utilities -โ””โ”€โ”€ vector.py # Vector search and HNSW (JVector) indexing +./scripts/build.sh # a wheel for your platform in dist/ ``` -**[Architecture details](https://docs.humem.ai/arcadedb/latest/development/architecture/)** - ---- +Linux builds run in Docker. macOS and Windows need a JDK 25 or later. The jars come from the `arcadedata/arcadedb` Docker image: `latest` by default, or the tag in `ARCADEDB_TAG` (for example `ARCADEDB_TAG=26.10.2 ./scripts/build.sh`). The package version is read from the engine jar in that image, so a `26.11.1-SNAPSHOT` image builds `26.11.1.devN`. -## ๐Ÿค Contributing +Tests run against the built wheel, not the source tree: install the wheel with its `test` extra into a virtual environment, then run `pytest`. [`CLAUDE.md`](https://github.com/ArcadeData/arcadedb-python/blob/main/CLAUDE.md) has the exact commands, the build options and the lint setup. -[See our contributing guidelines](https://docs.humem.ai/arcadedb/latest/development/contributing/) +## Contributing -## ๐Ÿ“„ License +Open an [issue](https://github.com/ArcadeData/arcadedb-python/issues) or a pull request. Run `pre-commit run --all-files` first, and add a regression test with every fix. The rest of the bindings work from `humemai/arcadedb-embedded-python` is ported here one pull request at a time; [#19](https://github.com/ArcadeData/arcadedb-python/issues/19) lists what is done and what is next. -Both upstream ArcadeDB (Java) and this ArcadeDB Embedded Python project are licensed under Apache 2.0, fully open and free for everyone, including commercial use. +## Where this code comes from ---- +It moved here from `bindings/python` in [ArcadeData/arcadedb](https://github.com/ArcadeData/arcadedb) at `ee04bd202abce0183c6263a3c4d8a61a4d526960`, with its git history. Issue numbers in older commit messages refer to that repository. [humemai/arcadedb-embedded-python](https://github.com/humemai/arcadedb-embedded-python) is the fork where the bindings kept developing (runtime fixes, JRE trim, native build, benchmarks). Its benchmark tooling stays there. -## ๐Ÿ”— Links +## License -- **Documentation**: -- **PyPI**: -- **GitHub**: -- **ArcadeDB Main Docs**: -- **Issues**: +Apache License 2.0, like ArcadeDB. See [`LICENSE`](https://github.com/ArcadeData/arcadedb-python/blob/main/LICENSE). diff --git a/docs/modules/ROOT/pages/index.adoc b/docs/modules/ROOT/pages/index.adoc index 15631ff..3adb6b7 100644 --- a/docs/modules/ROOT/pages/index.adoc +++ b/docs/modules/ROOT/pages/index.adoc @@ -55,7 +55,7 @@ ArcadeDB reaches Python in two ways. Pick by where the engine should run. Choose the embedded bindings for a single script, a notebook, or an application with one Python process that owns its data. Choose the xref:arcadedb::how-to/connectivity/drivers/python-http.adoc[HTTP driver] when more than one process, another language, or a remote machine must reach the same database, or when the server must outlive your Python process. -The two meet in one case. The bindings bundle an optional HTTP server with the Studio web UI. It is off until you call `create_server()`. The process that owns the database keeps direct access, and other processes connect over HTTP. That server lives and dies with your Python process. For a database that must outlive any one client, or for HA, replication and TLS, run the regular xref:arcadedb::index.adoc[ArcadeDB server] instead. +The two meet in one case. The bindings bundle an optional HTTP server, with the Studio web UI from 26.11.1. It is off until you call `create_server()`. The process that owns the database keeps direct access, and other processes connect over HTTP. That server lives and dies with your Python process. For a database that must outlive any one client, or for HA, replication and TLS, run the regular xref:arcadedb::index.adoc[ArcadeDB server] instead. [[jvm-vs-native]] == JVM build or native build @@ -87,7 +87,7 @@ The preview installs beside the default package. Its public API follows the one == Requirements -* Python 3.11 to 3.14. The wheels' CI runs all four versions on every supported platform. +* Python 3.10 to 3.14. The wheels' CI runs all five versions on every supported platform. * Linux (x86_64 and ARM64), macOS on Apple Silicon, or Windows (x86_64). * No Java installation. diff --git a/docs/modules/ROOT/pages/installation.adoc b/docs/modules/ROOT/pages/installation.adoc index ae370a8..8ce7549 100644 --- a/docs/modules/ROOT/pages/installation.adoc +++ b/docs/modules/ROOT/pages/installation.adoc @@ -22,7 +22,7 @@ The package is self-contained. It carries a Java 25 runtime for its platform, so == Requirements -* Python 3.11 to 3.14. The wheels' CI runs all four versions on every supported platform. +* Python 3.10 to 3.14. The wheels' CI runs all five versions on every supported platform. * One of the four platforms below. Every wheel is platform-specific, built on a native runner of its platform, and carries a JRE for that platform, so pip needs no hint to pick the right one. @@ -44,16 +44,19 @@ Every wheel is platform-specific, built on a native runner of its platform, and | `win_amd64` |=== +[[what-is-included]] == What is included * The ArcadeDB engine with all models (graph, document, key/value, vector, time series) and both query languages, SQL and openCypher. * Vector indexes, and data import for CSV, XML and ArcadeDB JSONL. -* The optional in-process HTTP server and the Studio web UI. They cost nothing at run time until you call `create_server()`. +* The optional in-process HTTP server and, from 26.11.1, the Studio web UI. They cost nothing at run time until you call `create_server()`. * A bundled Java 25 runtime, trimmed with `jlink`. Engine modules the bindings do not use are left out to keep the wheel small. These include the gRPC and MongoDB wire protocols, Raft HA, and Gremlin. -NOTE: *JavaScript is not available.* The GraalJS jars are left out, so `DEFINE FUNCTION ... LANGUAGE js` fails. Define functions in SQL or openCypher, or register a Python function with `register_function()`. +NOTE: *JavaScript is not available.* The GraalJS jars are left out, so `DEFINE FUNCTION ... LANGUAGE js` fails. Define functions in SQL or openCypher. Registering a Python function is planned in https://github.com/ArcadeData/arcadedb-python/pull/29[#29]. + +NOTE: *The Studio web UI is in the wheels from 26.11.1.* Studio used to bundle ApexCharts, which is no longer open source (https://github.com/ArcadeData/arcadedb/issues/10579[ArcadeData/arcadedb#10579]). https://github.com/ArcadeData/arcadedb/pull/10627[ArcadeData/arcadedb#10627] replaced it with Apache ECharts. The earlier 26.10.x wheels on PyPI do not contain Studio. The HTTP API of the in-process server works in all of them. == Verify the installation diff --git a/docs/modules/ROOT/pages/quickstart.adoc b/docs/modules/ROOT/pages/quickstart.adoc index fe3b6a3..77a7cb4 100644 --- a/docs/modules/ROOT/pages/quickstart.adoc +++ b/docs/modules/ROOT/pages/quickstart.adoc @@ -120,7 +120,7 @@ ArcadeDBError: Command failed: com.arcadedb.exception.TransactionException: Tran == Server mode -When another process, another language or a browser needs the same data, start the bundled server. The process that created it keeps full-speed embedded access, and `get_studio_url()` returns the address of the Studio UI: +When another process or another language needs the same data, start the bundled server. The process that created it keeps direct embedded access, and other processes reach the same databases over the HTTP API on the port that `get_http_port()` returns: [source,python,subs="attributes+"] ---- @@ -128,15 +128,15 @@ When another process, another language or a browser needs the same data, start t with arcadedb.create_server("./databases", root_password="password123") as server: db = server.create_database("mydb") # embedded access, in this process - print("Studio:", server.get_studio_url()) # HTTP access, any process + print("HTTP port:", server.get_http_port()) # HTTP access, any process ---- [source,text] ---- -Studio: http://localhost:2480/ +HTTP port: 2480 ---- -The server runs only while your Python process runs. See xref:index.adoc#embedded-vs-http[Embedded bindings or the HTTP driver] for when to run the regular ArcadeDB server instead. +The server runs only while your Python process runs. From 26.11.1 it also serves the Studio web UI, and `get_studio_url()` returns its address. The earlier 26.10.x wheels do not contain Studio: see the note in xref:installation.adoc#what-is-included[What is included]. See xref:index.adoc#embedded-vs-http[Embedded bindings or the HTTP driver] for when to run the regular ArcadeDB server instead. == Next steps diff --git a/examples/README.md b/examples/README.md index e1637ea..50a70ce 100644 --- a/examples/README.md +++ b/examples/README.md @@ -14,7 +14,7 @@ cd examples python 01_simple_document_store.py ``` -See the dataset downloader guide: [docs/examples/download_data.md](../docs/examples/download_data.md). +See the dataset downloader guide: [dataset downloader](https://docs.humem.ai/arcadedb/latest/examples/download_data/). ## Available Examples @@ -223,7 +223,7 @@ Synthetic multi-table ingest comparison harness: be trusted - Current outcome is workload-dependent; SQL import can win on some table-heavy shapes - The async SQL arm is a comparison arm, not a recommendation, and `--async-parallel` - accepts only 1 (see [Bulk Ingest Recommendation](../docs/guide/import.md#bulk-ingest-recommendation) + accepts only 1 (see [Bulk Ingest Recommendation](https://docs.humem.ai/arcadedb/latest/guide/import/#bulk-ingest-recommendation) for why). For bulk document ingest use `db.insert_many(...)` **Learn:** Table-ingest tradeoffs for embedded Python workloads @@ -238,7 +238,7 @@ Synthetic graph ingest comparison harness: vertex/edge data - Includes parity checks on final vertex and edge counts - The async SQL arm is pinned to `--async-parallel 1` (see - [Bulk Ingest Recommendation](../docs/guide/import.md#bulk-ingest-recommendation)). + [Bulk Ingest Recommendation](https://docs.humem.ai/arcadedb/latest/guide/import/#bulk-ingest-recommendation)). GraphBatch is the recommended bulk graph ingest path **Learn:** Graph-ingest tradeoffs for embedded Python workloads @@ -334,7 +334,7 @@ Embedded-first server workflow, all in one process: - reads server metadata over HTTP - creates a database through the server-managed Java API - mixes embedded writes with HTTP queries and updates -- opens Studio against the same in-process database +- prints the Studio URL; the Studio web UI is in the wheels from 26.11.1, and the earlier 26.10.x wheels do not include it (see the [top-level README](../README.md)) **Learn:** How to expose an embedded database over HTTP without running a separate container, and when you should reach for the Docker distribution @@ -427,8 +427,8 @@ full-text and graph context in a single query ## ๐Ÿ”— Learn More - **[ArcadeDB Documentation](https://docs.arcadedb.com/)** -- **[Python API Reference](../docs/api/)** -- **[GitHub Repository](https://github.com/ArcadeData/arcadedb)** +- **[Python API Reference](https://docs.humem.ai/arcadedb/latest/api/database/)** +- **[GitHub Repository](https://github.com/ArcadeData/arcadedb-python)** ---