Skip to content

feat(table): make Table a resolved handle and the home of operator creation - #434

Merged
lxy-9602 merged 6 commits into
apache:mainfrom
lucasfang:dev29
Oct 19, 2026
Merged

lxy-9602 merged 6 commits into
apache:mainfrom
lucasfang:dev29

Conversation

@lucasfang

@lucasfang lucasfang commented Oct 15, 2026 •

Copy link
Copy Markdown
Collaborator

Purpose

Linked issue: close #441

Table held only a schema, an identifier and a UUID: the resolution Catalog::GetTable() had just performed — which file system serves the table, where its root is — was discarded at the door. The four operator builds (about 1400 lines in four separate .cpp files) each re-derived the table's components from a context, Table::NewXxx were thin forwarders to the static Creates, and Table::Create read the schema through a SchemaManager it then threw away, so every operator re-read the same schema file through a manager of its own.

Changes:

  • include/paimon/catalog/table.h: Table now carries the per-table file system, root path, schema, identifier, UUID, and schema/snapshot managers bound to that file system, root path and branch. New accessors GetFileSystem(), GetRootPath(), GetSchemaManager(), GetSnapshotManager(), CurrentBranch(); the constructor takes (file_system, root_path, schema, identifier, uuid, schema_manager). include/paimon/type_fwd.h gains the SchemaManager / SnapshotManager forward declarations.
  • src/paimon/core/table/table.cpp plus the new internal header src/paimon/core/table/table_operators.h: the four operator builds move here verbatim behind CreateTableScan / CreateTableRead / CreateFileStoreWrite / CreateFileStoreCommit. The four operator .cpp files shrink to forwarders, and Table::NewScan/NewRead/NewWrite/NewCommit route through the same functions — one build path, with the table as the entry point. Table::Create reads the schema through the manager it hands to the table, so the open-time read warms the cache the table keeps.
  • Catalogs populate the resolved state: FileSystemCatalog::GetTable opens a system table with the data table's root path; RestCatalog::GetTable reads the UUID, location and schema from one response (avoiding mixed generations after a table recreation) and opens the table with the per-table file system (LoadTableSchema gains a table_path out-parameter); MockVersionManagedCatalog serves the per-table file system the same way.
  • examples/read_write_demo.cpp opens the table via catalog->GetTable and creates all four operators through table->NewXxx(...).

The moved operator bodies are unchanged (pure relocation): the operators still build their components from the context — wiring them to the table-held managers is a follow-up.

Tests

  • New TableTest cases: TestCreateHoldsTheFileSystem (fs + root path retained), TestManagersBoundToTheTable (managers bound to the table's fs/path/branch; the kept schema manager serves the schema Table::Create read, i.e. the warmed cache; a branch table keeps branch-bound managers), TestCurrentBranch (main default, identifier-named branch), TestCatalogOpenedTableServesThePerTableFileSystem (no fs handed out → no managers; per-table fs served when issued), TestCreateBuildsTheOperator (the four Xxx::Create build through a catalog-configured context). FileSystemCatalogTest.TestOptionsSystemTableCatalog and TestCreateTableWithBlob assert the root path a table carries (a system table gets the data table's path).
  • Actual runs: cmake --build build --target paimon-core-test -j 96 built clean; ./build/release/paimon-core-test --gtest_filter='TableTest.*:FileSystemCatalogTest.*' passed 45/45; the full suite ctest -j 96 in build/ (Release) passed 31/31 targets; git diff main HEAD --check clean.
  • RestCatalog::GetTable is not compiled locally: both local build directories have PAIMON_ENABLE_REST=OFF, so REST coverage relies on CI.

API and Format

Yes, a breaking C++ API change in include/paimon/catalog/table.h: the public Table constructor changes from (schema, identifier, uuid) to (file_system, root_path, schema, identifier, uuid = "", schema_manager = nullptr), so direct constructions must be updated (all in-tree ones are). New read-only accessors are added; the signatures and behavior of the four Xxx::Create(context) statics and of Table::NewXxx(context) are unchanged. No storage format or protocol change.

Documentation

No new user-facing option or feature; the resolved-handle semantics and the manager lifetimes are documented in the Table class doc comments.

Generative AI tooling

Generated-by: Qoder

@lucasfang
lucasfang marked this pull request as draft October 15, 2026 01:18
@lucasfang
lucasfang force-pushed the dev29 branch 2 times, most recently from dffd3b7 to 526e68a Compare October 19, 2026 01:56
@lucasfang
lucasfang marked this pull request as ready for review October 19, 2026 01:57
@lucasfang lucasfang changed the title Dev29 feat(table): make Table a resolved handle and the home of operator creation Oct 19, 2026
@lucasfang
lucasfang force-pushed the dev29 branch 2 times, most recently from 0f8f220 to ad4b142 Compare October 19, 2026 06:09
Comment thread include/paimon/read_context.h
Comment thread src/paimon/core/table/table.cpp

@lxy-9602 lxy-9602 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@lxy-9602
lxy-9602 merged commit dddfda5 into apache:main Oct 19, 2026
16 of 18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] Make Table a resolved handle and the single home of operator creation

2 participants