feat(scan): support cooperative append-only planning cancellation

[realAaronWu]

Purpose

Refs #435. Add an optional query-scoped cancellation token to batch metadata
planning, with cooperative checkpoints and reader/Arrow-root cleanup. A cancelled
scan must not affect another scan or publish a partial plan.

Preserve main's selected-row/lazy reads and cache candidate accounting while
carrying cancellation through predicates, deserialization and entry consumers.
Includes selected-row regressions and removes the test-only debugger hook.

Validation Draft: runtime qualification is incomplete; not requesting Ready
or upstream review. Current head 40835b3d includes the maintainer's merge of
main e7ec679a; the earlier reviewed feature head was e8121470.

Validation

• Earlier frozen-head local checks passed: formatting, CMake/packaging, source
  inventory, public-token concurrency, token mutation and Mac token TSan. These
  do not qualify the full planner/library or the new toolchain inputs.
• Linux default-profile configure passed; its build was interrupted during a
  nested Thrift download. Complete native/planner regression, guard mutations
  and scoped Linux sanitizers remain pending, not merely GPU-only pending.
• New-head CPU CI requires maintainer approval. Old-head checks are not reused
  as qualification of this revision. Actual enabled compiler/architecture/backend
  and sanitizer jobs must execute and their evidence must be checked.
• Main reverted the shared Rust1.95 setup: default Rust is1.90, with a separate
  stable channel for Vortex. Actual resolved toolchains need CI evidence; the
  interrupted local default build did not exercise these optional backends.

Boundaries

Cancellation cannot preempt filesystem/decoder calls and has no latency guarantee.
Token planning runs synchronously and bypasses shared caches. Active embedded
indexes with a token are explicitly unsupported. Ordinary no-token planning is
preserved; Arrow error-root cleanup also applies without a token.

Library and consumer need a paired rebuild; default arguments do not establish
binary ABI compatibility. No storage-format change or Paimon GPU scan claim.

Generative AI Tooling

Generated-by: OpenAI Codex (desktop session; runtime version not exposed)

[realAaronWu]

Could a maintainer approve the fork workflow runs for exact PR440 head 8c474664f7d2230e8ec4a0c7c99f606d1c09f646?

These runs currently report action_required with zero jobs, so revised-head CI tests, raw logs and runtime discovery have NOT RUN:

• Build and Test (attempt 1)
• Gcc 8 Test (attempt 1)
• License Check (attempt 1)
• pre-commit (attempt 1)

Independent local review cleared this exact commit for validation-Draft publication only. The PR remains Draft pending actual revised-head CI and final delivery review; prior ff6f275 CI is not a substitute.

This updates the existing approval request rather than adding a duplicate.

[realAaronWu]

CPU CI for the updated validation draft (e8121470, merging latest main) is waiting for maintainer approval. Could a maintainer approve the runs? This is not a Ready or code-review request.

[lucasfang]

Thanks for the submission. However, this change is too invasive to the codebase. As shown in the figure, is it really necessary to have so many steps that can all be cancelled? Could you clarify what specific problem your use case is trying to solve? Also, have you profiled the impact on scan performance?