Repository navigation
PostgreSQL: Display the index operator class before the sort options - #2663
Open
LucaCappelletti94 wants to merge 1 commit into
Open
LucaCappelletti94 wants to merge 1 commit into
LucaCappelletti94 wants to merge 1 commit into
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #2663 +/- ##
=======================================
Coverage 81.28% 81.28%
=======================================
Files 42 42
Lines 33920 33923 +3
Branches 33920 33923 +3
=======================================
+ Hits 27571 27574 +3
Misses 2785 2785
Partials 3564 3564 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
LucaCappelletti94
force-pushed
the
index-column-opclass-order
branch
from
October 18, 2026 14:15
7a1eeb7 to
f0972af
Compare
sabir-akhadov-localstack
added a commit
to localstack/datafusion-sqlparser-rs
that referenced
this pull request
Oct 19, 2026
* Task LAV-2134: SAMPLE / TABLESAMPLE on TABLE(...) table-function factors (FLATTEN, GENERATOR, SPLIT_TO_TABLE) (#2552)
* Task LAV-2134: SAMPLE/TABLESAMPLE on TABLE(...) table-function factors
Attach a sample slot to TableFactor::TableFunction in the vendored parser,
lower percent / n-ROWS sampling through the existing RewriteSample derived-
subquery rewrite (including sampled TABLE(GENERATOR(...)) via a deferred
wrapper), and raise Snowflake's 002325 fixed-size errors for block/seed forms.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Task LAV-2134: reword test comment to drop bare-assert false positive
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* Task LAV-2136: IS [NOT] DISTINCT FROM type-matrix tests, RHS precedence fix, VARIANT-vs-string-literal coercion (#2558)
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* Task LAV-2088: User DDL: DEFAULT_NAMESPACE = db.schema parses; BINARY_INPUT_FORMAT / GEOMETRY_OUTPUT_FORMAT in the user parameter catalog with canonical value casing (#2530)
* Task LAV-2088: parse two-part DEFAULT_NAMESPACE; add BINARY_INPUT_FORMAT/GEOMETRY_OUTPUT_FORMAT to user parameter catalog
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Task LAV-2088: normalize LOGIN_NAME and single-part DEFAULT_NAMESPACE casing
Real Snowflake stores LOGIN_NAME uppercased and folds a single-part
DEFAULT_NAMESPACE to upper case; DESC/SHOW USER echo the canonical
casing. Snapshots captured against real Snowflake.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* task LAV-2088: WIP — commit stranded agent work
* Task LAV-2088: user parameter catalog completeness + DISPLAY_NAME unset clears to NULL
Materialise DISPLAY_NAME to the name at CREATE and stop falling back to the
name in SHOW/DESC USER once it has been UNSET, matching real Snowflake
(SHOW -> NULL, DESC -> 'null'); LOGIN_NAME still reverts to the name.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* WIP: auto-commit before PR gate
* WIP: auto-commit before PR gate
* Task LAV-2088: CREATE USER NETWORK_POLICY populates the column; add TIMESTAMP_DAY_IS_ALWAYS_24H
CREATE USER … NETWORK_POLICY lifts the value out of the property bag into the
network_policy column (matching ALTER USER SET), so SHOW PARAMETERS IN USER
reports it. Add the missing TIMESTAMP_DAY_IS_ALWAYS_24H user parameter.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Task LAV-2088: provision account-level RESTRICTED_ACCESS network policy for terraform tests
The provider's user parameter assertions expect NETWORK_POLICY to inherit
RESTRICTED_ACCESS at ACCOUNT level. Set it in the terraform provisioner
alongside the provisioner roles. Reverts the unused CREATE USER NETWORK_POLICY
column path (not exercised by the suite).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Task LAV-2088: DESC/SHOW USER report the user TYPE (PERSON default, SERVICE)
TYPE is read from the CREATE USER options instead of hard-coded to PERSON, so
service users report TYPE = SERVICE.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Wojciech Padło <wojciech.padlo@localstack.cloud>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* Task LAV-2141: MATCH_RECOGNIZE: ALL ROWS PER MATCH, CLASSIFIER/MATCH_SEQUENCE_NUMBER, RUNNING/FINAL (#2562)
* Task LAV-2141: MATCH_RECOGNIZE ALL ROWS PER MATCH, CLASSIFIER/MATCH_SEQUENCE_NUMBER, RUNNING/FINAL
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* task LAV-2141: fix CI — add semantics field to Measure test constructors
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* Task LAV-2144: MATCH_RECOGNIZE anchors, PERMUTE, and pattern exclusion (#2563)
Widen PATTERN support with `^`/`$` anchors (zero-width partition
start/end checks), PERMUTE (expanded to lexicographic alternation over
permutations, capped at 7 elements), and `{- … -}` exclusion (rows are
matched and navigated but dropped from ALL ROWS output).
Extend the vendored parser so exclusion holds a full sub-pattern
(`{- down+ -}`) rather than a single symbol, and stop treating `{-`
after a symbol as a range quantifier. Also support FIRST/LAST of
CLASSIFIER()/MATCH_NUMBER() in ONE ROW MEASURES.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* Task LAV-385: CREATE OR ALTER DYNAMIC TABLE — in-place alter, no-refresh schema evolution (#2554)
* task LAV-385: WIP — commit stranded agent work
* Task LAV-385: CREATE OR ALTER DYNAMIC TABLE — in-place alter, no-refresh schema evolution, DT grant display
Complete the OR ALTER spine: skip the backing-table cross-kind guard on the
dynamic table's own reentrant create_table call, reconcile SHOW GRANTS
granted_on to DYNAMIC_TABLE for a dynamic table's TABLE-recorded edges, and
verify grant/created_on/scheduling_state preservation via a fully-qualified
SHOW GRANTS. Correct ADR 043 and the dynamic-tables architecture doc: body
change evolves the schema in place without a refresh, REFRESH_MODE can change,
options are declarative.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Task LAV-385: keep the original snowflake_compat import line intact (pipeline static check)
* Task LAV-385: add or_alter to exhaustive CreateTable literals in vendored tests
The new CreateTable.or_alter field broke the three upstream sqlparser
tests that build a CreateTable without `..`, failing the "Vendored
sqlparser tests" CI job.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* Task LAV-2173: honour PATTERN on LIST/LS/REMOVE/RM stage commands (#2609)
Thread the parsed PATTERN through the LIST/REMOVE rewriter and stage UDFs,
filtering staged files by an anchored, case-sensitive regex over each file's
reported name (bucket-relative key for external stages). Empty/absent/NULL
patterns filter nothing; an uncompilable pattern raises Snowflake's
002042/42601 error.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* Task LAV-2189: COPY INTO unload accepts a single-quoted internal-stage target ('@stage/path') (#2615)
* Task LAV-2189: accept single-quoted internal-stage COPY INTO unload target
* Task LAV-2189: source case-sensitive stage uid from fixture (avoid test import edit)
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* Task LAV-2208: DROP ACCOUNT GRACE_PERIOD_IN_DAYS soft-deletes the target account (#2622)
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* Task LAV-2209: UNDROP ACCOUNT restores a soft-dropped account within its grace period (#2626)
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* LAV-2227: CREATE TASK: USER_TASK_TIMEOUT_MS / SUSPEND_TASK_AFTER_NUM_FAILURES not parsed (000089, 000249) (#2647)
* LAV-2227: persist CREATE TASK properties
Extend CREATE TASK parsing and rewriting for timeout, retries, managed warehouse size, error integration, and overlap settings. Persist all task properties, project Snowflake-compatible SHOW/DESC values, and add Cloud-captured compat coverage for warehouse-backed and serverless tasks.
* LAV-2227: render serverless task DDL
Omit the warehouse clause for serverless tasks, preserve the managed initial warehouse size in GET_DDL output, add a Snowflake Cloud snapshot, and document supported task properties and nullable warehouses.
---------
Co-authored-by: root <root@ip-10-42-47-165.eu-west-1.compute.internal>
Co-authored-by: root <root@ip-10-42-34-28.eu-west-1.compute.internal>
Co-authored-by: root <root@ip-10-42-36-77.eu-west-1.compute.internal>
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* Task LAV-2201: accept DEFAULT keyword on the UPDATE/MERGE assignment path (#2654)
Preserve `DEFAULT` as `Expr::Default` when it is the entire RHS of an
UPDATE / MERGE-UPDATE assignment (gated to Snowflake, kept an identifier
inside a larger expression so `SET col = DEFAULT || 'x'` still 904s). Plain
declared defaults ride PG's pg_attrdef; a sibling-referencing default is
rejected extension-side in a post-parse-analyze guard — Snowflake's verbatim
1762 / 0A000 when a referenced column is not also assigned, a distinct
unsupported-feature reject (pinned xfail) when all referenced columns are.
Map the guard's messages to their Snowflake codes; the 1762 wire message is
the detail tail only (StripPrefix) and carries queryContext: null despite an
invalid source location. NOT-NULL-no-default already maps PG 23502 to
Snowflake 100072, so that test lands non-xfail.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* Task LAV-2255: accept parenthesized (CALL …) as a RESULTSET payload (#2663)
Add a SetExpr::Call variant and a supports_call_in_resultset dialect flag to
the vendored parser so `(CALL p(args))` is accepted in the RESULTSET
initializer (DEFAULT / :=, DECLARE and LET) and bare-assignment positions, and
rejected — at the CALL keyword, matching Snowflake's `unexpected 'CALL'` — for
the unparenthesized and CURSOR FOR forms.
Lowering unwraps the CALL payload through call_to_select in query_to_statement,
so every static-inline consumer (RETURN TABLE, FOR loop, cursor) streams the
callee's rows under the callee's own columns; the bare-assignment runtime path
re-parses the stored `CALL …` text through __snowflake$prepare_query.
Also resolve `:param`/`:local` refs in a declared-column RETURN TABLE(rs) query
(the untyped RETURNS TABLE() path already did), so a table callee whose
resultset references its parameters works.
Co-authored-by: Wojciech Padło <wojciech.padlo@localstack.cloud>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* Task LAV-2269: parse CREATE SCOPED TEMPORARY {FILE FORMAT|TABLE|VIEW|STAGE} (#2691)
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* LAV-2128: snowflake_task AFTER/predecessor relations don't round-trip → OnTask_Discussion2877 step 2/4 refresh-plan drift (#2723)
* LAV-2128: round-trip task predecessor relations
Persist fully qualified, lexically ordered task predecessor JSON for CREATE TASK and ALTER TASK ADD/REMOVE AFTER, and project matching SHOW/DESC relation metadata. Add a real-Snowflake snapshot covering create, add, remove, ordering, and root-task empty relations.
* LAV-2128: fix task parser test construction
Construct expected task names through ObjectName's conversion API so the vendored parser test builds with the current multipart-name representation.
* LAV-2128: preserve legacy task predecessor rows
Normalize legacy comma-separated predecessor values when projecting or mutating task relations, while retaining ordered JSON for new writes. Add a Snowflake-captured regression for upgrading and mutating an existing row and document both catalog encodings.
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
## Deviations
- **Catalog representation**: the architecture document described comma-separated storage. The earlier implementation retained JSON storage for new rows; this round documents it and adds transparent compatibility for legacy comma-separated rows instead of migrating catalog data eagerly.
* LAV-2128: qualify two-part legacy predecessors
Qualify two-part legacy task predecessor catalog values with the task database before SHOW/DESC projection and ADD/REMOVE normalization. Exercise the path by injecting a two-part legacy value in the existing Snowflake-captured compat test.
* LAV-2128: update task alter callers and docs
Pass the predecessor argument from SYSTEM$TASK_DEPENDENTS_ENABLE and document the complete ALTER TASK UDF surface, including predecessor mutations.
---------
Co-authored-by: root <root@ip-10-42-44-3.eu-west-1.compute.internal>
Co-authored-by: root <root@ip-10-42-42-116.eu-west-1.compute.internal>
* Task LAV-2297: COPY FILES between internal named stages (#2726)
* task LAV-2297: WIP — implementer interrupted by transient API failure
* Task LAV-2297: COPY FILES keeps empty rowset (no 0-files status row)
Exclude CopyIntoSnowflakeKind::Files from the COPY INTO empty-result
status-row substitution so a zero-match COPY FILES returns an empty
rowset (detailed output) rather than a synthetic status message.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* Task LAV-2308: CREATE / DROP INDEX and SHOW INDEXES on hybrid tables (#2732)
Metadata-only secondary-index catalog for hybrid tables. CREATE / DROP
INDEX and SHOW [TERSE] INDEXES are lowered to catalog UDFs; no PostgreSQL
index is ever created and index DDL against a non-hybrid table is rejected
(0A000) instead of leaking to PG. Index identity is table-scoped
(db, schema, table, name); builds are synchronous (ACTIVE). Dropping or
CREATE OR REPLACE-ing the owning table retires its index rows.
Vendored parser: INDEXES keyword, SHOW [TERSE] INDEXES and
CREATE OR REPLACE INDEX parsing, DROP HYBRID TABLE object type.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* Task LAV-2302: PASSWORD POLICY object lifecycle: CREATE / ALTER / DROP / DESCRIBE / SHOW (#2735)
* task LAV-2302: WIP — commit stranded agent work
* Task LAV-2302: PASSWORD POLICY object lifecycle (CREATE/ALTER/DROP/DESCRIBE/SHOW)
Complete the stranded WIP: add the vendored-parser parse functions for the
five PASSWORD POLICY statements, fix snowflake_varchar/text type mismatches in
the shared policy UDFs, echo the bare object name in the duplicate-name error,
add source-position handling for the ALTER/DROP/DESCRIBE not-found and the
001422 out-of-range-value errors, and add the compat tests with snapshots
captured against real Snowflake.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Wojciech Padło <wojciech.padlo@localstack.cloud>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* LAV-2031: SELECT … FOR UPDATE on hybrid tables (#2744)
* LAV-2031: support FOR UPDATE on hybrid tables
Parse Snowflake WAIT clauses, validate hybrid-table and transaction restrictions, apply clause-local lock timeouts, map conflict errors, and add cloud-captured deterministic compat coverage. Regenerate the feature catalog to mark FOR UPDATE supported.
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
## Deviations
- **Join restriction**: real Snowflake accepts a self-join of one hybrid table but rejects a join across two distinct hybrid tables as 'multiple tables'. The captured test and validation implement the observed multiple-table restriction instead of rejecting every syntactic JOIN.
* LAV-2031: verify FOR UPDATE conflict timing
Require default and explicit NOWAIT conflicts to terminate inside the observation window, and snapshot WAIT 2 as RUNNING before its deadline and error 200011 afterward.
* LAV-2031: expose WAIT path to coverage
Move clause-local lock timeout execution out of the async closure so LLVM coverage attributes the exercised WAIT path correctly, and simplify mode-specific timeout error mapping.
---------
Co-authored-by: root <root@ip-10-42-35-60.eu-west-1.compute.internal>
Co-authored-by: root <root@ip-10-42-34-78.eu-west-1.compute.internal>
* Task LAV-2312: inline INDEX in CREATE HYBRID TABLE and index rendering in GET_DDL (#2752)
Parse inline `index <name>(<cols>) [include (<cols>)]` table elements in the
Snowflake dialect, route them into the metadata-only index catalog on
CREATE HYBRID TABLE (rejecting them on standard tables the way real Snowflake
does), and render each secondary index inline in GET_DDL('TABLE', …) so the
output round-trips.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* Task LAV-2303: SESSION POLICY object lifecycle (CREATE/ALTER/DROP/DESCRIBE/SHOW) (#2748)
Add SESSION POLICY as the second kind over the shared policy machinery:
vendored-parser arms for the five statements (SET is space-separated, UNSET
comma-separated), the kind's DdlStrict classification, transform + show
rewriter handlers reusing the shared UDFs, a Session-policy status label, a
list-value render helper for the two SECONDARY_ROLES properties, DESCRIBE
resolution for the CREATED_ON / OWNER_ROLE_TYPE store columns, and the
SESSION_POLICY property-spec / DESCRIBE-row seeds (defaults, ranges,
descriptions, ordinals transcribed from a real-Snowflake capture). Compat
tests with snapshots captured against real Snowflake.
Co-authored-by: Wojciech Padło <wojciech.padlo@localstack.cloud>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* Task LAV-2305: Policy attachment: ALTER ACCOUNT/USER SET/UNSET policy, the connector table, the drop guard and POLICY_REFERENCES (#2750)
* Task LAV-2305: Policy attachment via one connector table
Attach PASSWORD/SESSION/AUTHENTICATION policies to USER and ACCOUNT
entities through the `policy_attachments` connector table (ADR 093),
visible via POLICY_REFERENCES from both argument forms and undroppable
while attached.
- `policy_attachments` connector table + partial unique index enforcing
one policy of a kind per entity; duplicate attach → verbatim 003549.
- `__snowflake$attach_policy` / `detach_policy` / `policy_attachment_guard`
and the account-scoped attach/detach UDFs; DROP and CREATE OR REPLACE of
an attached policy → verbatim 003531.
- ALTER USER / ALTER ACCOUNT SET/UNSET <kind> POLICY grammar + rewriter;
the `<kind>_POLICY = <p>` property spelling is rejected (001420).
- Entity- and policy-side cascades: user rename/drop update or remove
rows; policy rename cascades to live attachments.
- POLICY_REFERENCES gains a connector branch per argument form; the account
entity is named by CURRENT_ACCOUNT(); a by-entity USER lookup on a missing
user raises 002003 unpositioned.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Task LAV-2305: pin combined SET policy+property rejection (001003)
Real Snowflake rejects a policy leg combined with an object property in a
single ALTER USER SET list; SET <kind> POLICY is its own statement form.
Add a snapshot-pinned rejection test; no combined form is accepted.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Apply cargo fmt
---------
Co-authored-by: Wojciech Padło <wojciech.padlo@localstack.cloud>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* Task LAV-2304: AUTHENTICATION POLICY object lifecycle: nested property groups, merge/replace SET, cross-property rules (#2749)
* task LAV-2304: WIP — commit stranded agent work
* Task LAV-2304: emit queryContext for AUTHENTICATION POLICY validation rejects
The closed-vocabulary 001008 reject (`invalid value ['<v>'] for parameter
'<dotted.path>'`) and the two cross-property rejects (004800 / 003623) carry
`queryContext: null` despite `sourceLocationValid: false`, matching real
Snowflake. Route them through the queryContext-emitting paths.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Wojciech Padło <wojciech.padlo@localstack.cloud>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* Task LAV-2306: SHOW POLICIES ON ACCOUNT/USER — ten-column set_on form with BUILT-IN row (#2761)
Add the entity-scoped SHOW form
`SHOW <kind> POLICIES [LIKE ...] ON {ACCOUNT | USER <name>}` for PASSWORD,
SESSION and AUTHENTICATION policies (ADR 094 §7). It reads policy_attachments,
not the policy catalog, and returns the nine IN-form columns plus set_on.
- Vendored parser: parse the optional `ON {ACCOUNT | USER <name>}` clause into
a new `on_entity` field on the three ShowXPolicies statements.
- Transform: route the ON form to a second UDF `__snowflake$show_policies_on`,
keeping the IN form on `__snowflake$show_policies` (fixed nine-column type).
- Extension: `__snowflake$show_policies_on` reports the effective policy —
USER attachment wins over the inherited ACCOUNT one; ON ACCOUNT sees only the
account attachment — with the synthetic BUILT-IN/SYSTEM row for the kinds
that have one (PASSWORD, AUTHENTICATION; SESSION has none). `options` is
rendered per kind and gains target_scopes only for account-scoped SESSION /
AUTHENTICATION.
- Wire: position `SHOW … ON USER <missing>` errors at the user name.
- Compat tests with snapshots captured against real Snowflake.
Co-authored-by: Wojciech Padło <wojciech.padlo@localstack.cloud>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* LAV-2364: RESAMPLE: parser grammar in vendor/sqlparser (#2767)
* LAV-2364: parse Snowflake RESAMPLE table factors
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* LAV-2364: avoid RESAMPLE parser stack overflow
Box the RESAMPLE increment expression so the new TableFactor variant does not inflate recursive parser stack frames.
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* LAV-2364: shrink RESAMPLE parser stack frames
Box the complete RESAMPLE payload so adding the variant does not increase every recursive TableFactor parser frame. This keeps the existing parentheses overflow test within CI's small thread stack.
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
---------
Co-authored-by: spiral <spiral@localhost>
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* Task LAV-2354: BACKUP POLICY object lifecycle: CREATE / ALTER / DROP / DESCRIBE / SHOW (#2763)
* task LAV-2354: WIP — commit stranded agent work
* Task LAV-2354: BACKUP POLICY object lifecycle (CREATE/ALTER/DROP/DESCRIBE/SHOW)
Fourth policy kind on ADR 094's shared store: catalog-and-read-back only.
Bespoke flat-scalar validator (schedule floor, expire bounds, retention-lock
reject, required-argument and unset-schedule guards) and a new ten-column
read-back shared by SHOW BACKUP POLICIES and DESC BACKUP POLICY.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* LAV-2366: RESAMPLE: numeric USING path + validation errors (#2779)
* LAV-2366: support numeric RESAMPLE and validation errors
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* LAV-2366: close RESAMPLE validation gaps
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* LAV-2366: accept unary-plus numeric increments
Handle UnaryOperator::Plus as a signed numeric literal so positive values enter the numeric RESAMPLE path and zero reaches the 009959 validation.
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
## Deviations
- Reviewer feedback requested snapshots for bare +0 and +1. Real Snowflake currently rejects both bare and parenthesized unary-plus forms with compilation error 001003 before RESAMPLE validation, so no parity snapshot can exercise this AST branch; the transform fix is retained defensively and existing snapshots remain unchanged.
* LAV-2366: reject unary-plus RESAMPLE increments
Reject unary-plus increments in the RESAMPLE grammar before expression parsing, including parenthesized forms, so they surface Snowflake’s positioned 001003 parser error. Add real-Snowflake snapshots for +0 and +1 and remove the now-unreachable transform acceptance.
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
---------
Co-authored-by: spiral <spiral@localhost>
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* LAV-2374: Flaky test: tests/queries/stages/test_credential_redaction.py::test_execute_immediate_from_stage_credentials_redacted (#2785)
* LAV-2367: execute staged SQL files
Implement EXECUTE IMMEDIATE FROM grammar, extension-owned stage reads, server-side fail-fast splitting and sequential execution, DRY_RUN, USING parsing, and compatibility coverage for supported stage kinds and errors.
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
## Deviations
- Real-Snowflake snapshot capture was attempted serially after the proxy fix, but the first PUT remained hung for over six minutes and produced no snapshot. The compat tests are present and the named-stage emulator path was verified through its final statement, but snapshot JSON could not be recorded in this environment.
* LAV-2367: capture staged execution snapshots
Capture and verify the EXECUTE IMMEDIATE FROM compatibility cases against real Snowflake, including the external-stage S3 path. Align staged-file error payloads and parse rendering with the captured wire shapes, and isolate the DRY_RUN side-effect assertion from pooled session state.
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* LAV-2367: report staged parse error token positions
Reject reserved FROM as a Snowflake expression operand so staged scripts fail during whole-file parsing. Map the parser's token location into the staged STATEMENT_ERROR and cover it with a real-Snowflake snapshot plus a fail-fast side-effect check.
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* LAV-2367: canonicalize staged error locations
Uppercase unquoted stage names after removing database and schema qualification, while preserving quoted identifiers. Match Snowflake parse-wrapper formatting and cover a lowercase qualified stage reference with a verbatim real-Snowflake exception snapshot.
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* LAV-2367: preserve dotted quoted stage names
Parse stage qualification separators outside quoted identifiers so staged STATEMENT_ERROR wrappers retain an unqualified quoted name containing dots. Cover the behavior with a verbatim real-Snowflake snapshot.
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* LAV-2367: disambiguate staged path option names
Treat USING and DRY_RUN as trailing options only when their required clause syntax follows. Cover both names as stage path components in parser tests and real-Snowflake compat snapshots.
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* LAV-2367: require complete staged option clauses
Recognize USING and DRY_RUN as EXECUTE IMMEDIATE FROM options only when their complete syntax reaches the statement boundary. Preserve clause-shaped path components and cover their execution through existing real-Snowflake snapshots.
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* LAV-2367: redact staged statements and preserve errors
Redact each staged script chunk before it enters the normal statement pipeline while carrying its captured credentials through the out-of-band channel. Snapshot the full runtime STATEMENT_ERROR wrapper and staged credential redaction behavior against real Snowflake.
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* LAV-2367: scrub staged parse failures
Reparse malformed staged scripts only after conservative credential scrubbing so error-token extraction cannot leak sensitive literals. Cover the staged wrapper with an emulator snapshot that pins the masked token.
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
## Deviations
- Real Snowflake echoes the raw credential literal in this malformed staged-file error. The security acceptance criterion requires redaction, so the regression is emulator-only and snapshots the existing five-smiley conservative mask instead.
* LAV-2367: isolate staged dry-run secrets
Preserve staged parser offsets with width-stable masking while normalizing sensitive error tokens to the standard mask. Return DRY_RUN content as a synthetic result so raw staged source never becomes executable statement text or reaches text sinks.
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
## Deviations
- The new parse-location and DRY_RUN isolation snapshots are emulator-only because real Snowflake exposes credential text in the malformed wrapper and cannot expose emulator logs/history internals. They verify the explicit security criterion while the existing cloud snapshots continue to pin Snowflake-visible behavior.
* LAV-2367: isolate dry-run log probe
Use per-test credential sentinels for the shared server-log assertion so parallel credential tests cannot contaminate the DRY_RUN isolation result.
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* LAV-2374: isolate staged credential history assertion
Filter query history by the test's unique target stage before checking for leaked credentials. This prevents parallel tests using the shared redaction constants from contaminating the assertion.
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
## Deviations
- LAV-2367 is not yet merged into main, so this branch is stacked on origin/LAV-2367, which provides the named test and staged execution implementation.
* LAV-2374: record staged child query history
Record each EXECUTE IMMEDIATE FROM child statement after its inner pipeline completes, so the unique target-stage history row exists and carries only redacted credentials. Strengthen the cloud-captured snapshot with a positive statement-presence witness to prevent vacuous leak checks.
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
## Deviations
- LAV-2367 is not yet merged into main, so this branch remains stacked on origin/LAV-2367, which provides the named test and staged execution implementation.
* LAV-2374: preserve staged parent query history
Give every staged child its own query ID, retain its recorded outcome for the outer statement, and rewrite the returned response to the parent ID. Extend the cloud-captured regression to prove the child row, parent row, and LAST_QUERY_ID all agree.
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
## Deviations
- LAV-2367 is not yet merged into main, so this branch remains stacked on origin/LAV-2367, which provides the named test and staged execution implementation.
- Rust and emulator checks could not run because this worker image has no cargo executable; the focused real-Snowflake snapshot capture passed.
* LAV-2374: finalize staged parent query result
Publish the final staged child's completed payload under the reserved outer query ID before returning. This clears the parent's Running record and makes result polling and same-request replay return the completed parent result. Extend the waived credential-redaction regression to exercise both paths.
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
## Deviations
- LAV-2367 is not yet merged into main, so this branch remains stacked on origin/LAV-2367, which provides the named test and staged execution implementation.
- Rust and emulator checks could not run because this worker image has no cargo executable; Python compilation and git diff checks passed.
* LAV-2367: cover staged execution error branches
Capture and match Snowflake behavior for empty staged files, tokenizer EOF failures, and missing files in user and table stages. Align the emulator's error codes, wire shapes, and stage labels while exercising the previously uncovered branches that caused the patch coverage failure.
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
---------
Co-authored-by: spiral <spiral@localhost>
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* LAV-2328: match remaining MATCH_RECOGNIZE errors (#2789)
Co-authored-by: spiral <spiral@localhost>
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* LAV-2368: EXECUTE IMMEDIATE FROM: nested invocation and relative file paths (#2793)
* LAV-2368: support nested staged script execution
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* LAV-2368: preserve nested stage namespace
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
---------
Co-authored-by: spiral <spiral@localhost>
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* Task LAV-2357: Directory tables: DIRECTORY(@stage) read surface and ALTER STAGE … REFRESH (#2766)
* Task LAV-2357: Directory tables — DIRECTORY(@stage) read surface and ALTER STAGE … REFRESH
Make directory tables observable on named internal stages: a materialized
file registry, not a live listing. SELECT … FROM DIRECTORY(@stage) projects
the persisted registry (stage_directory_files); ALTER STAGE [IF EXISTS]
<name> REFRESH [SUBPATH = '<path>'] diffs a fresh listing against it,
returning (file, status, description) rows (REGISTERED_NEW / UNREGISTERED;
no-op refresh returns zero rows). PUT does not self-register; disabling the
directory table wipes the registry; DESC STAGE's DIRECTORY.LAST_REFRESHED_ON
now reflects the last refresh (absent when disabled). DIRECTORY(@~) /
DIRECTORY(@%t) reject 391706; a not-enabled stage rejects 393901.
Vendored sqlparser gains AlterStageOperation::Refresh and parses @~/@%name
stage refs in expression position. Full compat matrix captured against real
Snowflake.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Task LAV-2357: reword test comment tripping the bare-assert scanner
The pipeline's _scan_bare_asserts uses an unanchored \bassert\s regex, so a
comment reading '# assert only status + description.' was flagged as a bare
assert. Reword it; the tests already record every result via run_snapshot_test.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Wojciech Padło <wojciech.padlo@localstack.cloud>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* LAV-2224: CREATE OR ALTER STAGE — parser rejects the STAGE keyword (000040, 000050, 000055, 000209) (#2824)
* LAV-2224: support CREATE OR ALTER STAGE
Preserve OR ALTER intent through parsing and stage DDL rewriting, then update existing stage metadata in place without deleting files, history, or tags. Add real-Snowflake snapshot coverage for creation, alteration, qualification, customer external-stage properties, and modifier conflicts.
Co-authored-by: Przemek Denkiewicz <przemyslaw.denkiewicz@localstack.cloud>
* LAV-2224: enforce stage alteration restrictions
Reject conflicting CREATE OR ALTER STAGE modifiers defensively in the DDL rewriter and prevent existing external stages from changing cloud provider. Add real-Snowflake snapshot coverage for S3-to-Azure and Azure-to-S3 attempts.
Co-authored-by: Przemek Denkiewicz <przemyslaw.denkiewicz@localstack.cloud>
* LAV-2224: remove unreachable modifier guards
Conflict forms are rejected by the Snowflake parser before stage DDL rewriting, as the real-Snowflake snapshot compat cases verify. Remove the redundant rewriter branches that could never receive coverage.
Co-authored-by: Przemek Denkiewicz <przemyslaw.denkiewicz@localstack.cloud>
* LAV-2224: restore stage modifier validation
Restore reachable CREATE OR ALTER STAGE conflict guards so OR REPLACE and IF NOT EXISTS combinations return the recorded Snowflake compilation errors.
Co-authored-by: Przemek Denkiewicz <przemyslaw.denkiewicz@localstack.cloud>
* LAV-2224: exclude unreachable stage modifier guards
Mark the restored conflict validation as unreachable from the SQL compat harness because the parser rejects these modifier combinations before DDL rewriting. This preserves the defensive checks while excluding the guards from patch coverage.
Co-authored-by: Przemek Denkiewicz <przemyslaw.denkiewicz@localstack.cloud>
---------
Co-authored-by: spiral <spiral@localhost>
Co-authored-by: Przemek Denkiewicz <przemyslaw.denkiewicz@localstack.cloud>
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* Task LAV-2361: Iceberg on object storage: CATALOG_SOURCE = OBJECT_STORE and CREATE ICEBERG TABLE … METADATA_FILE_PATH (#2819)
* task LAV-2361: WIP — commit stranded agent work
* Task LAV-2361: fix vendored sqlparser test build; deterministic + real-captured OBJECT_STORE/DELTA catalog-integration test
- Add metadata_file_path field to CreateTable initializers in vendored
sqlparser test files so the workspace (incl. vendored tests) compiles.
- Make test_catalog_integration_object_store_delta deterministic via
register_identifier; re-capture its snapshot against real Snowflake
(confirms REFRESH_INTERVAL_SECONDS is accepted with TABLE_FORMAT=ICEBERG).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Task LAV-2361: ship object-store Iceberg read path; xfail S3 tests pending real-SF snapshot capture
The CATALOG_SOURCE=OBJECT_STORE / METADATA_FILE_PATH read + refresh path is
implemented and verified against the emulator + LocalStack S3. The three
S3-reading compat tests need a baseline captured against real Snowflake reading
real S3 (TEST_TARGET=SNOWFLAKE_CLOUD,AWS_CLOUD --snapshot-update), which the
pipeline environment cannot do (no AWS/S3 credentials). Mark those three tests
xfail so CI stays green and object_store.rs keeps its patch coverage (the tests
still execute the read path, failing only at the un-captured snapshot assert);
removing the marker + capturing the snapshot completes AC-4/AC-5. Document the
object-store shape in the architecture doc.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Task LAV-2361: reject BASE_LOCATION + METADATA_FILE_PATH at compile time
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Task LAV-2361: run all object-store error queries before snapshot asserts (patch coverage)
The xfail error test aborted at the first snapshot_match_exception (missing
real-SF baseline), so only the first case's query executed. The compile-time
BASE_LOCATION+METADATA_FILE_PATH reject (create_rewriter.rs:1807-1813) and the
object-store load-error path (iceberg/mod.rs:158) — the 4th and 2nd cases —
never ran in CI's LocalStack coverage leg, failing patch coverage (88.7% < 90%).
Collect all four error exceptions up front, then match snapshots, so every
reject/error path executes before the fixture aborts. No S3 capture needed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Wojciech Padło <wojciech.padlo@localstack.cloud>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* LAV-2426: ALTER DYNAMIC TABLE … SWAP WITH is not parsed — syntax error unexpected 'SWAP' (#2876)
* LAV-2426: support dynamic table swap with
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* LAV-2426: preserve metadata across dynamic table swaps
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
---------
Co-authored-by: spiral <spiral@localhost>
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* Task LAV-2439: validate SAMPLE / TABLESAMPLE arguments (#2871)
Enforce Snowflake's documented value space for every SAMPLE / TABLESAMPLE
argument position, producing the verbatim Snowflake errors:
- Range/type checks in the sample rewriter: probability 0..=100 and fixed
size 0..=1,000,000 (001817), seed 0..=2^31-1 (001817) and integer-only
(001007). Fractional fixed sizes truncate toward zero so the generated
LIMIT is integral.
- A new pre-pipeline `sample_arg_check` gate emits Snowflake's positioned
001003 syntax error for a leading `-`, a PERCENT unit, and the
unsupported BUCKET form (the vendored parser over-accepts all three),
which also makes the rewriter's BUCKET bail path unreachable.
- The vendored parser now accepts a sampled VALUES list
(`FROM VALUES (...) SAMPLE (p)`).
Compat tests in test_table_samples.py cover the full value space with
snapshots recorded against real Snowflake.
Co-authored-by: Wojciech Padło <wojciech.padlo@localstack.cloud>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* LAV-2465: Support ALTER DYNAMIC TABLE column comments (#2881)
* LAV-2465: support dynamic table column comments
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* LAV-2465: investigator: retain real-Snowflake wrong-kind behavior
Dispatch 2 captured ALTER DYNAMIC TABLE <ordinary table> ALTER c COMMENT 'changed' against Snowflake Cloud. It returned 'Statement executed successfully.' and DESCRIBE TABLE reported comment 'changed'. The reviewer-added rejection/error-snapshot AC is factually incompatible with Snowflake and must not be implemented. Keep the shared tables-store behavior in 99eaf7ce2 and record this AC as superseded by the captured contract.
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* LAV-2465: retain captured wrong-kind behavior
The implementation remains unchanged after reviewer feedback because the investigator's Snowflake Cloud capture proves ALTER DYNAMIC TABLE against an ordinary table succeeds and persists the column comment.
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
## Deviations
- The reviewer-added criterion requires rejecting a non-dynamic table target. Real Snowflake instead accepts the statement and updates that table's column metadata, so the shared table-store behavior is retained as the snapshot-authoritative contract.
---------
Co-authored-by: spiral <spiral@localhost>
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* LAV-2496: support FROZEN WHERE dynamic table alias (#2888)
Co-authored-by: spiral <spiral@localhost>
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* LAV-2498: Support ALTER DYNAMIC TABLE reclustering controls (#2890)
* LAV-2498: support dynamic table reclustering controls
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* LAV-2498: snapshot reclustering error envelopes
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
---------
Co-authored-by: spiral <spiral@localhost>
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* LAV-2466: Support multi-table dynamic-table refresh and COPY SESSION (#2900)
* LAV-2466: support multi-table dynamic refresh pipelines
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* LAV-2466: close dynamic refresh error gaps
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
---------
Co-authored-by: spiral <spiral@localhost>
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* LAV-2470: Support dynamic-table FROZEN WHERE actions (#2902)
* LAV-2470: support frozen dynamic-table refresh regions
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* LAV-2470: validate frozen predicates structurally
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* LAV-2470: remove unrelated test fixture edit
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
---------
Co-authored-by: spiral <spiral@localhost>
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* LAV-5387: Checkpoint replay and intermediate visitor dependency
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* LAV-2473: support dynamic-table DCM project unset (#2908)
Parse and execute UNSET DCM PROJECT as the state-neutral detach observed on real Snowflake. Cover idempotency, IF EXISTS, missing objects, malformed clauses, and metadata surfaces with cloud-captured snapshots. Also accept Snowflake's spaced DYNAMIC TABLE object type in GET_DDL.\n\n## Deviations\n\n- Neither account exposes a direct SQL attachment action for an existing dynamic table. Configured state can only be produced by deploying a staged DCM source bundle, which would require the broader DCM subsystem the spec excludes; the snapshots therefore cover the constructible unconfigured state on both accounts.
Co-authored-by: spiral <spiral@localhost>
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* LAV-2500: Complete ALTER DYNAMIC TABLE scalar parameters (#2896)
* LAV-2500: persist dynamic table scalar parameters
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* LAV-2500: complete dynamic table scalar parameter matrices
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* LAV-2500: fix scalar option routing and transient retention
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* LAV-2500: fix dynamic scalar CI regressions
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* LAV-2500: route each dynamic table scalar option
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* LAV-2500: remove unreachable scalar persistence path
Dynamic-table ALTER statements are handled by rewrite_alter_dynamic_table before ordinary table SET-option rewriting. Remove the duplicate fallback branch, which was unreachable in compat coverage and could diverge from the per-option routing implementation.
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
---------
Co-authored-by: spiral <spiral@localhost>
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* LAV-2509: Accept emitted CREATE DYNAMIC TABLE property spellings and round trips (#2920)
* LAV-2509: accept emitted dynamic table properties
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* LAV-2509: complete dynamic table round-trip coverage
Assert recreated INITIALIZE and WAREHOUSE metadata for both SHOW and GET_DDL output, and replay the three 000030 migration statement shapes through compat SQL.
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
---------
Co-authored-by: spiral <spiral@localhost>
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* Task LAV-2529: parse SEMANTIC VIEW DDL in the vendored parser (#2937)
Add real grammar for the five SEMANTIC VIEW DDL statements
(CREATE/ALTER/DROP/DESCRIBE/SHOW) as boxed Statement variants that
preserve every clause list in declaration order, with each element
carrying its alias, expression, synonyms, tags, access modifier and
comment. Allow-list all five in the identifier gate (090106 unprobed).
No rewriter arms yet — statements parse and round-trip only.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* LAV-2512: Support ADAPTIVE dynamic-table refresh mode (#2921)
* LAV-2512: support adaptive dynamic table refresh mode
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* LAV-2512: cover deferred custom incremental mode
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* LAV-2512: investigator: remove obsolete refresh-mode parse fallback
CI patch coverage is 36% solely on crates/parser/src/parse.rs:1538-1542,1569-1579. The merge with main retained invalid_dynamic_table_refresh_mode() and its map_parse_error call from the pre-merge implementation, but main now parses arbitrary mode tokens as RefreshModeKind::Invalid and rejects them in crates/transform/ddl/src/create_rewriter.rs. Delete the helper and its call; the existing compat snapshot already exercises the active invalid-value path. Re-run make check and the targeted dynamic-table compat tests.
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* LAV-2512: remove obsolete refresh-mode parse fallback
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
---------
Co-authored-by: spiral <spiral@localhost>
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* LAV-2518: validate CREATE STREAM APPEND_ONLY properties (#2943)
Co-authored-by: spiral <spiral@localhost>
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* LAV-2499: Support ALTER DYNAMIC TABLE column comments (#2892)
* LAV-2499: support dynamic table column comments
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* LAV-2499: reject identifier column comments
Restrict dynamic-table column comments to Snowflake single-quoted and dollar-quoted string literals, with real-Snowflake snapshots for bare and double-quoted identifier errors.\n\n## Deviations\n\n- Reviewer feedback requested that ALTER DYNAMIC TABLE column-comment actions reject ordinary tables. A real-Snowflake snapshot proves this spelling succeeds and updates the ordinary table column comment, so the existing behavior is retained as required by the repository's snapshot precedence rule.
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* LAV-2499: snapshot column comment errors
Route dynamic-table column-comment failures through the wire-level snapshot fixture and align the missing-object IF EXISTS source position with Snowflake.
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
---------
Co-authored-by: spiral <spiral@localhost>
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
* Task LAV-2530: Semantic views: store, CREATE / OR REPLACE / DROP, and SHOW SEMANTIC VIEWS (#2949)
* Task LAV-2530: Semantic view store, CREATE/OR REPLACE/DROP, and SHOW SEMANTIC VIEWS
Add the SEMANTIC VIEW object lifecycle: four normalised store tables, the
CREATE / OR REPLACE / IF NOT EXISTS / DROP / DROP IF EXISTS UDFs, the
ddl_rewriter and show_rewriter arms, and the opaque-clause grammar
(MAX_STALENESS quoted-interval form, AI_SQL_GENERATION,
AI_QUESTION_CATEGORIZATION, AI_VERIFIED_QUERIES, COPY GRANTS, per-dimension
WITH CORTEX SEARCH SERVICE) folded into the vendored parser.
SHOW SEMANTIC VIEWS returns the nine columns captured from real Snowflake;
duplicate-name (002002, echoing the name as written), drop-not-found (002003,
positioned) and duplicate-expression (010202) are each snapshot-pinned. New
compat tests in tests/queries/ddl/test_semantic_views.py with snapshots
captured against real Snowflake.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Task LAV-2530: semantic-view expression engine and validation rejections
Wire the element expressions through the shared transform pipeline (ADR 101
§6): each fact/dimension/metric is inlined (metrics-over-facts resolved, the
DAG walked once), pushed through a synthetic single-expression SELECT over its
base table via ctx.reenter to its stored sql_body_pg, and the create UDF
resolves its data_type by introspecting that body (the ADR 027 aggregate-scale
operation), best-effort.
Add the three validation rejections, each pinned to a real-Snowflake capture:
- 000904 invalid identifier, positioned, when an expression reaches through a
relationship the wrong way (reachability over the child->parent edges);
- 010271 unsupported derived-metric expression when a derived metric references
another metric;
- 010224 cyclic reference between element expressions.
Plus 001003 MAX_STALENESS below the two-minute floor (validated in the UDF).
New compat tests + captured snapshots for the four rejections and an
expression-forms create; the five existing snapshots are unchanged.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* LAV-2478: Support dynamic-table storage lifecycle policy actions (#2910)
* LAV-2478: parse dynamic-table storage lifecycle actions
Implement ADD/DROP parsing and route the actions through the dynamic-table lifecycle surface. Capture Snowflake error precedence, malformed syntax, repeated drop, refresh, and metadata behavior.\n\n## Deviations\n\n- No storage-lifecycle attachment/catalog model exists in the emulator. Per the spec fallback, ADD returns Snowflake's captured post-parse missing-policy response and DROP returns the captured unattached response; policy execution and synthetic attachment state were not invented.\n- The focused local compat run stalled while bootstrapping the extension and was stopped; Snowflake Cloud capture, make check, and the vendored sqlparser suite passed.
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* LAV-2478: honor lifecycle target semantics
Forward target-level IF EXISTS to lifecycle actions and reproduce Snowflake's missing-target versus missing-policy error precedence.
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* LAV-2478: match lifecycle snapshot diagnostics
Preserve policy quoting through canonicalization, report lifecycle errors at Snowflake positions, reject malformed lists at the offending token, and classify zero-row refreshes structurally.
The focused compat test could not start because local extension bootstrap stalled; make check and .spiral/check.sh pass.
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* LAV-2478: investigator: fix refresh no-change classification
CI run 35059234737 job 104676006489 has exactly one failure: test_dynamic_table_storage_lifecycle_policy_actions query-result-13. REFRESH returns statistics {"insertedRows":0,"copiedRows":0,"deletedRows":0}, pipeline count 1, and a volatile timestamp, but the Snowflake snapshot expects 'No new data', count 0, and its captured timestamp. The dce9558 bool_or JSON-field change still evaluates the whole refresh as changed in CI. Reproduce the focused compat test and trace v_changed/v_pipeline_count in __snowflake; do not chase the truncated feedback excerpt, which omitted this summary.
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* LAV-2478: make zero-row refresh classification deterministic
Classify refreshes with an explicit nonzero-result existence check so an all-zero statistics row always produces No new data and a zero pipeline count.\n\nThe focused compat bootstrap stalled while linking the extension; make check passed.
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* LAV-2478: preserve no-change refresh metadata
Classify each refresh result from its own row statistics and retain the previous data timestamp when row counts do not change, preventing parallel batch state from turning an all-zero refresh into a changed result.
make check and .spiral/check.sh passed. The focused compat command stalled while linking the extension and was stopped.
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* LAV-2478: salvage real-Snowflake storage lifecycle captures from LAV-2479
Harvested verbatim from abandoned duplicate branch LAV-2479 @ 404945d0b
(closed as a duplicate of LAV-2478).
These captures are the one thing PR #2910 cannot reproduce: they were
recorded against real Snowflake with an *actually constructed* storage
lifecycle policy --
CREATE STORAGE LIFECYCLE POLICY <n> AS (d DATE, n NUMBER)
RETURNS BOOLEAN -> FALSE
so they pin the successful-attachment surface. #2910's own test only ever
references a policy that does not exist, so every one of its captures is a
missing-policy error and it can say nothing about success behaviour.
New cloud facts these snapshots pin that #2910 does not have:
* validation/query-result-10 -- ALTER DYNAMIC TABLE IF EXISTS <missing>
ADD STORAGE LIFECYCLE POLICY ... => "Statement executed successfully."
This is the error-ordering contract case, and the emulator currently
contradicts it (fixed in the following commit).
* validation/query-error-9 -- missing target without IF EXISTS => 002003 /
42S02 "Table '...' does not exist or not authorized." at line 1 pos 20.
(#2910 already matches this; it is the half of the contract it gets right.)
* validation/query-result-11 -- attaching to an ordinary (non-dynamic)
table SUCCEEDS on Snowflake. Wrong object kind is not an error.
* validation/query-result-4, -7, -8 -- successful attach, successful detach
("Successfully detached storage lifecycle policy '...' from the table."),
and repeat drop ("No storage lifecycle policy attached.").
* validation/query-error-3 -- signature mismatch => 003556 / 42601
"Invalid number of arguments for attaching policy ...".
* validation/query-error-5, -6 -- duplicate attachment => 003549 / 23505
"Object <n> already has a STORAGE_LIFECYCLE_POLICY ...".
* validation/query-error-2 -- missing column => 000904 invalid identifier.
* observability/query-result-1..14 -- SHOW DYNAMIC TABLES, DESCRIBE DYNAMIC
TABLE and GET_DDL are byte-identical before attach, after attach and
after detach. Attachment is invisible in all three, so there is no
metadata surface to model.
* syntax_errors/query-error-4 -- DROP STORAGE LIFECYCLE POLICY <name>
=> 001003 syntax error at pos 53. Not covered by #2910.
Only the tests and snapshots are taken. LAV-2479's parser work is
deliberately dropped: #2910's parser is strictly better (it tracks
policy_name_quoted and rejects a missing policy name before ON). LAV-2479's
rewriter hunk is dropped too -- it raised the unsupported-feature error
before target resolution, which is the defect LAV-2479 was re-scoped to fix.
The two cloud-success tests keep xfail(strict=True) while no attachment
subsystem exists; strict=True is the guard against faking success. The
syntax-error test is not xfailed and runs locally.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* LAV-2478: honor target-level IF EXISTS for ADD STORAGE LIFECYCLE POLICY
The error-ordering contract spelled out on the duplicate LAV-2479 requires
the target to be resolved before anything else is diagnosed:
* missing target, no IF EXISTS -> 002003 object-does-not-exist
* missing target, with IF EXISTS -> success
`__snowflake$alter_dynamic_table` got the first half right and the second
half wrong. In the branch chain the STORAGE_LIFECYCLE_DROP arm and the
SUSPEND_RECLUSTER / RESUME_RECLUSTER arms both guard on
`p_args->>'if_exists'`, but the STORAGE_LIFECYCLE_ADD missing-target arm did
not -- so `ALTER DYNAMIC TABLE IF EXISTS <missing> ADD STORAGE LIFECYCLE
POLICY <p> ON (<c>)` raised "Table '...' does not exist or not authorized."
where real Snowflake returns "Statement executed successfully."
Cloud evidence for the expected response is the snapshot harvested in the
previous commit: ..._validation / query-result-10 in
tests/queries/ddl/test_dynamic_table_storage_lifecycle.snapshot.json.
The short-circuit is placed ahead of policy resolution deliberately: in the
capture the referenced policy existed, and the statement still returned
success without the target being resolvable, so IF EXISTS gates the whole
action rather than just the table lookup.
The missing-target-without-IF-EXISTS path is untouched and still produces
002003 / 42S02, which the existing
tests/queries/test_dynamic_table_storage_lifecycle.py already asserts
(query-error-6).
Still needed, and deliberately not fabricated here: a local, non-xfail
snapshot case covering IF EXISTS + missing target, per LAV-2479's AC 3.
That snapshot must be recorded with TEST_TARGET=SNOWFLAKE_CLOUD.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* LAV-2478: classify single-table refresh and mask its data timestamp
`ALTER DYNAMIC TABLE <one> REFRESH` routes to
`__snowflake$refresh_dynamic_table`, not to the batch function, so the
no-change classification added in 382cc0523 never reached the single-table
path: an all-zero refresh still reported
`{"insertedRows":0,"copiedRows":0,"deletedRows":0}` with a refreshed count
of 1 where real Snowflake reports `No new data` with 0. Move the verdict
into the single-table function and have the batch function relay it,
which also removes the duplicated row-count classification.
382cc0523 additionally made `last_refreshed_at` conditional on the row
count changing. That is right for a table that already has a data
timestamp, but it left a dynamic table whose source is empty with a null
`last_refreshed_at` forever — real Snowflake initializes
LATEST_DATA_TIMESTAMP on the first refresh whether or not rows moved. The
`OR last_refreshed_at IS NULL` arm restores that while keeping the
no-change retention the commit was after.
test_dynamic_table_storage_lifecycle_policy_actions asserts the REFRESH
result, whose data timestamp is the moment of capture and can never match
a later run. It now takes the masking that test_dynamic_tables.py already
applies to every other refresh assertion —
`snapshot.transform.key_value("data_timestamp", reference_replacement=False)`
over a `to_records=True` result — and the snapshot was re-recorded against
real Snowflake accordingly (LAV-2478's volatile-value waiver; main
47a2efb09). The refresh status string and the row-count columns stay
asserted: only the timestamp is masked.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
---------
Co-authored-by: spiral <spiral@localhost>
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
* LAV-2513: Support CREATE FROZEN WHERE with SELECT-based BACKFILL FROM (#2929)
* LAV-2513: support frozen dynamic table backfill
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* LAV-2513: align backfill validation and metadata
Capture and enforce wrong-kind, reordered/extra-column, CREATE OR ALTER, and metadata behavior. Restore the unrelated scalar-parameters test body.\n\n## Deviations\n\n- Reviewer feedback expected CREATE OR ALTER to load a newly supplied backfill. The real-Snowflake snapshot rejects changing the backfill spec with 002008, so the implementation follows the authoritative snapshot and preserves existing rows.
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* LAV-2513: confirm scalar parameter test restoration
The existing test_dynamic_table_scalar_parameters function is byte-for-byte identical to origin/main. The branch diff contains only additive backfill test functions and snapshots.
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
---------
Co-authored-by: spiral <spiral@localhost>
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* LAV-2521: Support CREATE STREAM … CLONE (#2957)
* LAV-2521: support cloning streams
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* LAV-2521: complete stream clone parity
Assign clone ownership to the executing role and cover reverse drop independence plus AT/BEFORE STREAM suffix rejection with Cloud snapshots.
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* LAV-2521: use snapshot runner for clone errors
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* LAV-2521: record clone errors with snapshot runner
Recapture the clone error envelopes through run_snapshot_test and strip transport-only source-location fields that the emulator does not emit.
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
---------
Co-authored-by: spiral <spiral@localhost>
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* LAV-2522: Support COPY GRANTS for CREATE STREAM (#2967)
* LAV-2522: support COPY GRANTS for streams
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* LAV-2522: reproject copied stream grants to database roles
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
---------
Co-authored-by: spiral <spiral@localhost>
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
* LAV-2526: Support CREATE STREAM ON STAGE directory tables (#2965)
* LAV-2526: support streams on stage directory tables
Record directory refresh deltas, expose stage stream rows with independent offsets, and cover refresh, metadata, lifecycle, and error behavior with Cloud snapshots.
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
## Deviations
- Volatile FILE_URL hosts, owner/staleness timestamps, file metadata, and row IDs are masked using the established directory-table masking precedent in tests/queries/stages/test_directory_tables.py; snapshots were recorded against Snowflake Cloud.
* LAV-2526: cover stage stream lifecycle gaps
Consume directory changes before metadata updates and removals, assert stable metadata relationships, and cover qualified names plus replacement, drop, and re-enable lifecycles. Fix the stage-offset catalog invariant and Snowflake error rendering exposed by the expanded snapshots.
Co-authored-by: Sabir Akhadov <sabir.akhadov@localstack.cloud>
## Deviations
- Raw internal-stage SIZE, MD5, and ETAG remain masked because Snowflake reports encrypted upload bytes while the emulator stores the source bytes. The update snapshot now asserts the meaningful Cloud behavior directly: MD5 and ETAG change, size remains equal, and the replacement timestamp is newer.
* LAV-2526: preserv…
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
CREATE INDEX ii ON t(c ops DESC)was displayed asCREATE INDEX ii ON t(c DESC ops), putting the operator class after the sort options, which no longer parses as the same index column. PostgreSQL'sCREATE INDEXsynopsis puts the operator class first, followed byASC/DESCandNULLS(https://www.postgresql.org/docs/current/sql-createindex.html).The sort modifiers of
OrderByExpr(ASC/DESC,NULLS,WITH FILL) are now written by a shared helper, andIndexColumnprints the column expression, then the operator class, then those modifiers. The parser reads operator classes in every dialect, so the test round-tripsCREATE INDEXcolumns with and without an operator class and sort options in all of them, and checks the parsed operator class and sort order.