Skip to content

Use signed i64 generated tie breakers + bump Tantivy - #6882

Merged
Mallets merged 3 commits into
mainfrom
mallets/tiebreaker-i64
Oct 7, 2026
Merged

Mallets merged 3 commits into
mainfrom
mallets/tiebreaker-i64

Conversation

@Mallets

@Mallets Mallets commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • Pin tantivy and tantivy-common to latest Tantivy main at 2e87c2eb0c8577c96c82a777f345bbf236a985fe, which includes the merged Generate tie-breakers as i64 values tantivy#3158. Lockfile changes are limited to the Tantivy git revision.
  • Treat tie_breaker fields as i64 when parsing term/range queries and converting values to JSON.
  • Update the field documentation and validate signed sort values, the generated i32 range, term/range queries, and one-event search_after pagination in both directions.

Motivation

Downstream cursors represent tie-breakers as signed integers. Tantivy #3158 maps generated values onto the signed i32 range while retaining 32 bits of entropy, allowing generated and legacy explicit i64 tie-breakers to share the same representation.

Backward compatibility

#6859 has been merged few days ago, I don't expect tiebreakers are used by any existing deployments .

Validation

  • cargo check -p quickwit-doc-mapper -p quickwit-query -p quickwit-search: passed.
  • cargo test -p quickwit-query: 184 passed.
  • cargo test -p quickwit-search --lib: 226 passed.
  • cargo test -p quickwit-query -p quickwit-doc-mapper -p quickwit-search tie_breaker: 9 passed.
  • cargo test -p quickwit-doc-mapper: 232 passed; test_concatenate_multiple_field fails due to concatenated string ordering. Reproduced the identical failure in a clean worktree at origin/main (8952853b8) using the original Tantivy dependency. No unrelated test changes made.
  • cargo clippy -p quickwit-doc-mapper -p quickwit-query -p quickwit-search --all-targets -- -D warnings: passed.
  • make fmt (formatting, license headers, log format) and git diff --check: passed.

Pin Tantivy to quickwit-oss/tantivy#3158 and treat tie_breaker fields as signed integers in query parsing and JSON conversion. Validate the signed generation range, term/range queries, and pagination in both sort directions.
@Mallets
Mallets force-pushed the mallets/tiebreaker-i64 branch from e0dfa4f to 58c303a Compare October 7, 2026 08:05
@Mallets Mallets changed the title Use signed i64 generated tie breakers Use signed i64 generated tie breakers + bump Tantivy Oct 7, 2026
@Mallets
Mallets marked this pull request as ready for review October 7, 2026 09:55
@Mallets
Mallets requested a review from a team as a code owner October 7, 2026 09:55
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 7, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-07T09:58:01.253166Z 4d9767e Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@Mallets
Mallets added this pull request to the merge queue Oct 7, 2026
Merged via the queue into main with commit d140cfc Oct 7, 2026
18 checks passed
@Mallets
Mallets deleted the mallets/tiebreaker-i64 branch October 7, 2026 11:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants