[coderdan]

Summary

A Go program can now send a whole JSON filter to an encrypted column as one SQL parameter, the way it would call jsonb_path_match(col, $1, $2) on a plaintext jsonb column. Before, each part of the filter was its own Go call, its own parameter and its own cast, and the application had to assemble the WHERE clause by hand. A filter compiler (such as a customer's AIP-160 engine, which turns Google's list-filter syntax into SQL) can now emit one template for both.

Terms used below:

• A JsonSearch field is a Go struct, map or json.RawMessage field stored as one encrypted JSON document in a public.eql_v4_json_search column.
• A jsonpath predicate is PostgreSQL's SQL/JSON path filter text, such as $.country == $val && $.weight < $max. Its variables (vars) are the values named $val and $max.
• The engine is the Rust code (Stack Encrypt), run inside the Go SDK as a WebAssembly guest, that derives every search term.
• An ordered path is a path declared in the field's tag (ordered=$.weight) whose numbers and strings get an ordering term, so the database can compare them without decrypting.

This is the Go half of #1441 and the last issue of the "JSONB path match on encrypted JSON" milestone. The work it builds on (#1444, #1445, #1446 and #1451) is merged into next, so this PR's diff is its own three commits. It does not change the engine or the EQL SQL.

Changes

Public Go API

• On every generated JsonSearch field: Fields.<Field>.Match(ctx, cipher, predicate string, vars any, opts encrypt.JSONReadOptions) (eql.JSONSearchPredicateQuery, error).
• The result binds as col @@ $1::eql_v4.query_json_predicate, or eql_v4.json_tree_match(col, $1::eql_v4.query_json_predicate). Both forms return the same rows.
• vars is any value encoding/json writes as a JSON object: a map, a struct, or a json.RawMessage of one. It is written by the same gensupport.JSONText the other methods use.
  • They answer as the same Go value bound as plaintext jsonb_path_match's vars through pgx or database/sql answers. A nil value (nil, or a nil map, slice, pointer or json.RawMessage) is SQL NULL there, so plaintext is NULL for every row: Match returns an empty operand, which binds as NULL, and @@ and json_tree_match (both STRICT) match no row. The predicate is still compiled, so text the grammar does not read is still refused; a variable it names is not, as plaintext does not look for one. A value that writes the JSON null (json.RawMessage("null"), a pointer to a nil map) is refused as json_input, as plaintext refuses vars that are not an object. An empty map is the way to say "no variables".
  • Any other text goes to the engine, so an empty or whitespace-only json.RawMessage is refused as json_syntax, as { is.
  • A Go string is refused, as it is for a Contains needle, because it writes one JSON string, not an object. One helper, isGoString, holds that rule for both.
• opts.WildcardBound works as it does for Elements.<Array>.Match: 0 means the engine's default, 64.
• The Go side reads nothing of the predicate. The engine parses it and compiles it into a tree whose leaves are the operands EqAt, Selector, At.<Path> and Elements.<Array>.Match already derive.

Errors

• A predicate the engine cannot answer comes back as an *encrypt.Diagnostic wrapping encrypt.ErrEncoding, with code stack_encrypt::dynamic_predicate. It is refused; it is never compiled to a tree that silently drops a leaf.
• New accessors Diagnostic.Construct() and Diagnostic.At() read the two fields the engine already sent. Construct() names the refused construct in the grammar's own fixed words (`like_regex`, a position ([2])). At() is the byte offset in the predicate where that construct starts; it reads the offset in any integer width the transport carries, or a float holding a whole number, and refuses a negative or fractional one. Reason() says why (json_predicate_unsupported and the others). None of them carries text from the predicate or a variable's value.
• An ordering at a path the tag does not order comes back as encrypt.ErrTerm with code stack_encrypt::json_path_not_ordered, as the per-path orderings report it.

Guest and wiring

• The guest (encrypt/guest/src/targets.rs) now names the eql_v4.query_json_predicate domain. The engine therefore lists match (input predicate) on every JsonSearch field, with or without ordered paths.
• parseOperatorName and the operator input parser in checker.go accept match and predicate. So does stashgen's jsonOperators.
• The guest's status.rs maps Error::Predicate to STATUS_ENCODING (bad input). Before this, it fell into the catch-all and a refused predicate was reported as ErrInternal.
• stashgen generates Match on every JsonSearch field. It stops with an error if the engine lists no match for a document field: that would mean the generator and its engine disagree, the same rule feat(go): test several members of one array element of a JsonSearch field #1451 applies to a missing element_match.

Generated EQL types print redacted (outside the issue's strict scope; see Review notes)

• Every type in encrypt/eql now has String, Format and LogValue, which all write one text: eql.TextEq{[redacted]}, or eql.TextEq{} when empty, and never the bytes. Format answers every verb, %#v included, so no GoString is generated. Before this, every one of them was a bare []byte to fmt and slog.
• eql-codegen writes the methods for every type, so no type can be missed. The three small helpers they call are hand-written in eql.go.
• This is listed under Breaking: %s used to give a value's JSON. To get the JSON, convert the value (string(v), json.RawMessage(v)) or call MarshalJSON; Value is unchanged. To print or log the JSON on purpose, call Reveal() at that call (slog.Debug("stored", "email", v.Reveal())): every type has it, and it returns the JSON as a string, so fmt and every slog handler print it as text.

Tests

• Hermetic Go tests (json_match_test.go, deterministic guest):
  • A fixed predicate compiles to the expected tree, and(contains, exists, not(contains), cmp.lt), and each leaf equals the operand its own method derives (EqAt, At.Weight.Selector, EqAt with nil, At.Weight.Lt).
  • Six more predicates are pinned by shape. A filter of two conditions equals Elements.ExtraEmergencyContacts.Match at the same bound.
  • vars given as a map, a struct, a pointer to a struct, a json.RawMessage and a map[string]json.RawMessage all derive the same operand. With literals in place of variables, {} derives it too. Every kind of nil vars (untyped, map, pointers, json.RawMessage, []byte, slice) gives the empty operand, which binds as NULL, for a predicate with variables or without; with nil vars, a syntax error and an unsupported construct are still refused. TestJSONPredicateMatchInPostgres checks that the empty operand finds no row through @@ and json_tree_match. The comparison treats a containment needle's sv/el lists as sets, because the engine shuffles them on every query.
  • Fifteen grammar refusals, each checked for its sentinel, reason, construct, byte offset, field and code: json_predicate_syntax, json_predicate_unsupported (5 constructs), json_predicate_unknown_variable, json_predicate_value_type (2), json_predicate_not_equal_at_wildcard, json_predicate_too_deep, json_path_position, json_path_no_wildcard, json_path_wildcard and json_syntax.
  • Also refused: vars that are not an object (json_input); vars that are not JSON, an empty or whitespace-only json.RawMessage included (json_syntax); the JSON null, padded or not, and a pointer to a nil map (json_input); json_path_not_ordered for a range and for an unordered filter member; a Go string vars, before the engine is asked; and json_wildcard_bound_zero, through QueryTerm.
  • No error holds the predicate text or a variable's value. The operand prints and logs redacted.
• PostgreSQL (TestJSONPredicateMatchInPostgres, added to go:test:postgres, whose tests are now named once in a TESTS array that both the -run pattern and the pass-by-name check come from): the seven filters of the customer's request, with the predicates from the RC2 note written verbatim. Each runs four ways: @@, json_tree_match, the hand-written per-operator query, and jsonb_path_match over the plaintext (stored beside each row). All four return the expected rows.
  • The element-scoped case returns the row where one contact is both a sibling and called more than five times. It does not return the two rows where those two facts belong to different contacts.
  • A bound of 1 fails the query with SQLSTATE 22023.
• Guest:
  • The renamed test a_document_names_the_leaf_element_and_predicate_domains asserts the predicate domain is named and match is listed on fields with and without ordered paths.
  • The status test adds Error::Predicate → STATUS_ENCODING.
  • A new ops test runs a match through the query exports: one predicate derives eql_v4.query_json_predicate, and a like_regex one is STATUS_ENCODING.
  • native_ops's catalog listing now includes match.
• stashgen: Match is generated on an ordered and an unordered document field, and an engine that answers no match is refused.
• eql: every type prints and logs redacted, by every verb, inside a struct and through slog. The codegen test checks all four methods on every declared type.
• testusers.Entity gains the RC2 note's members (score, weight, address, country, preferred languages), and $.weight is ordered. customer_stash.go, subscriber_stash.go, entity_stash.go and the example's customer_stash.go are regenerated.

Docs

• cmd/stashgen/README.md, "Encrypted JSON documents": a Match row with both SQL forms, and a section covering:
  • the grammar and the refusals;
  • the five places the answer differs from jsonb_path_match (lax unwrapping not applied; lax wrapping of a non-array under [*] not applied; a string compared with a number is false under == and true under !=; orderings and multi-condition filters compare folded terms, so such a filter's == is of folded values; and an ordering of a JSON null is NULL rather than false except under an odd number of !, which returns the same rows, since feat(eql): one-parameter jsonpath predicates on json_tree documents (json_tree_match, @@, match) #1446 sends the null needle only there);
  • the 22023 bound;
  • the index cost: the GIN and btree indexes engage for @> and ->, not inside json_tree_match, so a query that needs an index uses EqAt, Contains and At.<Path>.
• encrypt/README.md and encrypt/doc.go say the same in short. The typed-document example's README and main.go add a Match call.
• languages/golang/CHANGELOG.md gains Added entries for Match and the Diagnostic accessors, and a Breaking entry for the redacted types. The stashgen README lists every Reason() a refusal can give, json_syntax and json_input included, one line each. The Go module records changes there, not in changesets.
• Skills: none mention the Go SDK's JSON queries, so there is nothing to change.

Verification

• mise run wasm:guest:test (before the later restacks, which changed no guest source): fmt, clippy, rustdoc; nextest 73 passed (default features) and 78 passed (eql).
• On this head, with all four Go guests rebuilt from the engine as merged (wasm:guest:build, wasm:guest:build:eql and both deterministic builds): go vet ./..., go test ./... (every package ok), gofmt -l . (empty), mise run go:lint (0 issues), and go generate ./... leaves the committed tree unchanged.
• mise run go:test:postgres against a local PostgreSQL 15 with the EQL v4 bundle built from this tree: all 7 tests passed, including TestJSONPredicateMatchInPostgres, which also checks that nil vars match no row through @@ and json_tree_match.
• eql-codegen (before Reveal was added; after it, cargo test -p eql-codegen and mise run types:check passed at every commit): cargo nextest run -p eql-codegen 202 passed, cargo clippy -p eql-codegen --all-targets --all-features -D warnings and cargo fmt --check clean. mise run types:check and mise run codegen:parity (from packages/eql) passed after commit.
• Not run:
  • The example program against real ZeroKMS (no credentials here). It compiles, and its predicate uses only paths the example orders.
  • Mutation testing: no stack-auth or stack-encrypt lines changed, and eql-codegen is not in the mutants set.

Related

Closes #1443

Review notes

• Scope beyond the issue: redacted EQL types. The issue asks that the returned predicate type print redacted "like the other EQL value types". None of them did: every generated type printed its bytes. Per the instruction to fix it in the generator, not by hand, all generated types now redact. A program that printed an EQL value with %s to see its JSON now sees eql.X{[redacted]}; it converts the value (string(v)) or calls MarshalJSON instead. Listed under Breaking. Reveal() is the named call for printing or logging the JSON, added after review: redacted by default, revealed only where a program asks for it by name.
• No range-path offset. An ordering at a path the tag does not order is still the engine's TermError::JsonPathNotOrdered. It carries no offset and no path text, as feat(eql): one-parameter jsonpath predicates on json_tree documents (json_tree_match, @@, match) #1446 left it, so At() is false there; the docs name its sentinel and code. Wrapping it in Error::Predicate with the path's offset is a small engine follow-up, if wanted.
• nil vars mirror plaintext. Decided in review: checked against pgx v5.11 and database/sql (pgx stdlib) over a plaintext column, where every Go nil binds as SQL NULL (so jsonb_path_match is NULL and no row matches), the JSON null is refused ("vars" argument is not an object), and an empty json.RawMessage is a syntax error. Match gives the same three answers.
• Where to look first: encrypt/gensupport/json.go (JSONMatch), stashgen/emit.go (jsonMatch, and the match check in planJSON), encrypt/guest/src/status.rs, and json_match_postgres_test.go.

[changeset-bot]

⚠️ No Changeset found

Latest commit: 8579e95

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

Run configuration

• Source mode: Auto
• Repository: Unavailable
• Branch: Unavailable
• File: No configuration file from this repository applied
• Commit: Unavailable
• Layers: Organization UI; Enforced overrides: Organization
• Review profile: CHILL
• Plan: Advanced
• Run ID: bb9fafe7-1119-42c6-9839-eb578a3329b0

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• Trigger review

[github-actions]

Mutation testing (cargo-mutants, --in-diff, stack-auth + stack-encrypt)

No mutants were generated for the changed lines.

[coderdan]

Rebased onto #1451's new head (cb6a190); new head be8052a. Rebase only: no change to this PR's own commits.

Why: the PRs below changed under review (#1444's feedback, the move of the JSON tree SQLx tests onto documents Stack Encrypt seals, and the clippy fix in #1445 that gates JsonSubtreeKey::below on the dynamic feature).

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-30T10:46:52.371394Z	be8052a	Draft marked ready
🔒 Security Review	✅ Completed 2026-10-30T10:49:21.090905Z	be8052a	Draft marked ready

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[cipherstash-agent]

Review attestation

Reviewed at head be8052a by Claude Code: /code-review medium, then company-skills:cross-check-review-findings against the existing review threads.

Critical or blocking issues: none.

New findings (2, both minor, inline):

• gensupport/json.go:345: a typed-nil *string as vars is refused, though the doc says a nil pointer sends no variables.
• internal/guest/diagnostic.go:197: wholeFloat is not bounded to math.MaxInt, so a large float offset overflows on 32-bit builds.

Not repeated (already covered by earlier threads): the redacted Format change is listed under Breaking in the CHANGELOG, the LogValue consistency, the whitespace vars handling, and the TESTS array in mise.toml.

Dropped after verification: the stashgen version-skew concern, the json.RawMessage coupling concern, and the nil/empty print diagnosability concern are speculative or design preferences, not defects.

Process note: the changeset-bot reports no changeset. Go releases are recorded in languages/golang/CHANGELOG.md per the PR description, so this looks intentional.

Codex code and security reviews were still running when this review was posted and are not reflected here.

---

Generated by Claude Code

[coderdan]

New head: 6f533d4, rebased onto #1451 at 52802ef. JSONMatch now treats a nil pointer of any type as "no variables", and Diagnostic.At refuses a float offset past math.MaxInt (see the two review threads).

https://claude.ai/code/session_01MrxG73fnX1p7oiizh3GRG5