Repository navigation
Document full-text paging and configurable search limits - #739
kylebernhardy wants to merge 50 commits into
Conversation
There was a problem hiding this comment.
Code Review
This pull request introduces comprehensive documentation for the new Full-Text Paging feature (continuation paging) in version 5.4.0, including a new dedicated guide, configuration options, REST header details, and sidebar integration. The review feedback suggests improving the formatting and placement of the <VersionBadge> component across several files to avoid awkward usage in running prose, recommending plain text or standalone placement under headings instead.
🚀 Preview DeploymentYour preview deployment is ready! 🔗 Preview URL: https://preview.harper-documentation.harperfabric.com/pr-739 This preview will update automatically when you push new commits. |
🚀 Preview DeploymentYour preview deployment is ready! 🔗 Preview URL: https://preview.harper-documentation.harperfabric.com/pr-739 This preview will update automatically when you push new commits. |
🚀 Preview DeploymentYour preview deployment is ready! 🔗 Preview URL: https://preview.harper-documentation.harperfabric.com/pr-739 This preview will update automatically when you push new commits. |
🚀 Preview DeploymentYour preview deployment is ready! 🔗 Preview URL: https://preview.harper-documentation.harperfabric.com/pr-739 This preview will update automatically when you push new commits. |
🚀 Preview DeploymentYour preview deployment is ready! 🔗 Preview URL: https://preview.harper-documentation.harperfabric.com/pr-739 This preview will update automatically when you push new commits. |
🚀 Preview DeploymentYour preview deployment is ready! 🔗 Preview URL: https://preview.harper-documentation.harperfabric.com/pr-739 This preview will update automatically when you push new commits. |
kriszyp
left a comment
There was a problem hiding this comment.
Cool, looks good!
🤖 Reviewed with Claude
Proposed inline comments (anchors failed):
reference/logging/configuration.md:342: [nit] The "Complete Example" still setsauditLog: false. This PR changes the example inoptions.mdtotrueand adds full-text indexing to the features that need the transaction log (reference/database/transaction.md:26). Anyone who copies the complete example disables audit logging and can no longer register full-text indexes (outside-this-diff)
|
Thanks, Kris. All seven inline comments are now addressed in f1f55da with individual source-grounded replies. I also fixed the unanchored logging example nit: both auditLog snippets now use true, including Complete Example, and the text explains the full-text audit prerequisite. Updated logging examples. The standalone production build passes (410 documents). The exact combined tree with Document native full-text search passes a 414-document build, formatting, typecheck and all companion-workflow tests. These are documentation corrections to the existing implementation; no runtime or dependency change was made. Comment generated by kAIle (GPT-6) |
🚀 Preview DeploymentYour preview deployment is ready! 🔗 Preview URL: https://preview.harper-documentation.harperfabric.com/pr-739 This preview will update automatically when you push new commits. |
🚀 Preview DeploymentYour preview deployment is ready! 🔗 Preview URL: https://preview.harper-documentation.harperfabric.com/pr-739 This preview will update automatically when you push new commits. |
⊙ Problem
The earlier draft described a separate full-text continuation API. Fill full-text and autocomplete pages using Harper's existing paging now fills ordinary limit/offset pages and removes that API.
💡 Solution
Explain paging, limits and recovery in the existing references. There are no continuation functions, tokens, close operations or retained searches, and no separate paging page or sidebar entry. Each page runs a new query and can see later changes.
⚖️ Alternatives
Document native full-text search owns the broader declaration/query guide. The reconciliation qualifies its historical 100-record prefix limit as pre-v5.4 and gives both guides the same independent-page and work-budget semantics. The two branches merge cleanly and their combined documentation build passes; numeric defaults and restart requirements remain in the existing configuration references. The broader guide also removes unsupported filterFields declarations and consolidates its duplicate 5.3 release-note sections. Publish this reference-owner change before the broader guide. Only the broader guide carries a Depends-on marker, pointing here; no reciprocal marker is added.
🔧 Changes
reference/configuration/options.md — Owns all three request defaults and six native process budgets; corrects auditLog to the source default true.
reference/configuration/operations.md — Owns the request-worker versus native-process restart contract, states that full-text updates use individual parameter names rather than a whole storage_fullText object, and uses the actual configuration response text.
reference/configuration/overview.md — Links file-edit restart guidance to its reference owner.
reference/database/storage-tuning.md — Adds a versioned full-text tuning pointer before Related, without repeating numeric defaults.
reference/database/transaction.md — Includes full-text indexing among transaction-log consumers and warns that per-table audit opt-out prevents full-text index registration.
reference/operations-api/operations.md — Makes full-text comparators discoverable in search_by_conditions, documents full_text_indexes/readiness directly under describe_table, and names runtime budgets in the replicated full-process restart exception.
reference/resources/query-optimization.md — Explains independent offset requests, an offset-20/limit-10 example counting eligible records, filtering, raw/result/time budgets, native capture, freshness condition fields, common error statuses and recovery with describe_table readiness inspection, audit retention and concurrency costs; also covers response-envelope overflow from multi-hit ordinary batches.
reference/resources/resource-api.md — Lists positive and all six negated full-text comparators, requires a positive anchor for negation, and removes a promise of a missing Table.search example.
reference/rest/querying.md — Documents positive and negated comparator syntax, filtered page filling, concurrent-write caveats and the difference between exhausted matches and budget errors, plus count behavior including the absence of an exact total for full-text and autocomplete queries.
release-notes/v5-lincoln/5.4.md — Describes full-text paging, the three request controls and native process-capacity budgets without introducing a continuation API.
reference/logging/configuration.md — Keeps both auditLog examples enabled, including Complete Example, and explains the full-text per-table audit prerequisite.
✅ Verification
Standalone and combined documentation builds, formatting, typecheck and workflow tests pass. CI is green. Publish this reference-owner PR before the native full-text guide.
Verification evidence and review history
Main integration verified October 26
Merged current main into this branch and resolved the v5.4 release-note conflict by retaining both the full-text paging section and main's pointer-compression addon section. Paging behavior, limits and restart requirements are unchanged.
Standalone build (410 documents), repository-wide formatting, typecheck and workflow tests pass. The exact updated native-guide and paging branches merge cleanly and pass the combined build (414 documents), formatting, types and workflow tests. Updated core verification passes 128 focused tests (two LMDB-only skips), 32 real-native tests, source TypeScript worker loading, and all 21 deployed full-text/example tests independently under npm 11.
The final Claude/Composer review and domain adjudication completed at this merge head (13 rounds / 10 full cumulatively). No conflict-resolution blocker was identified; the canonical result remains non-converged with minor completeness/prose concerns. The declaration and example contract is supplied by the coordinated native guide. Existing freshness/recovery discoverability and native-limit ownership tradeoffs remain recorded below; unrelated console-default and archived-v4 questions are outside this update. The previously adopted framing dispositions are retained, not replaced with a new planning-clearance claim.
The overlapping Document nested relationship filters and pagination uses the same inclusive-start/exclusive-end range semantics; current core computes the two-argument limit as end minus offset. The schema-sharding companion also corrects the audit default; retain both its placement guidance and the full-text audit prerequisite when integrating it. Publication ordering remains one-way: this reference-owner PR before the native guide. No PR merge or new review request was made.
Earlier verification and review history
Kris approved the prior head and left seven inline follow-ups plus an unanchored audit-example nit. The follow-up corrects all eight: replicated restart exceptions, 400 versus 503 budget failures, bare unavailable/wait-path 503s, all six negated comparators, a directly available describe_table readiness reference, rejected pre-v5.4 prefix windows, verified native version wording, and audit-enabled logging examples. Each inline thread has a source-grounded response and is resolved. No approval of this newer head is inferred from the earlier review.
Review closeout on the paired guide clarifies the explicit audit prerequisite, consumes freshness results without buffering, separates generic describe_table metadata from full-text metadata, removes obsolete prerelease guidance and explains POST versus GET and streaming failures. The exact final guide and paging branches pass the combined production build (414 documents), formatting and typecheck. Fresh unchanged-core verification passes 119 focused tests (two LMDB-only skips in RocksDB mode) and 21 deployed full-text/example integration tests without skips.
Production build passes and processes 410 documents; repository-wide formatting passes after the paging and configuration clarification. The clarification and nine defaults were checked against the final paging core commit, including the accepted-record window, replay work and explicit budget-error contract. Default values and request/runtime names were checked against core's defaultConfig, validation, policy resolver and native binding. Error statuses were checked against query error mapping; the table is explicitly common failures, with 413 / other 500 behavior described separately. Local anchors and the public v0.6.0 native-limits link were checked; the build alone is not claimed to validate every anchor.
Earlier core verification covered 101 focused tests and all 14 deployed HTTP tests on both Node 24.16 and CI Bun 1.4.3. Its runtime fixture separately verifies unchanged captured budgets across replacement workers after configuration changes. The HTTP tests alone are not claimed to prove that negative restart boundary.
Earlier coverage comprised 10 rounds, eight full, with Claude, Cursor Composer/Muse and completed domain adjudication. Native Gemini CLI supplemented that earlier state and raised no documentation defects. The current cumulative canonical coverage is 12 rounds, nine full, as recorded above and in the generated footer. The last planning review did not clear automatically: its concrete placement, error, audit and reservation findings were adopted as described above; a generic build-source checker and product-wide error redesign were deferred rather than claimed complete. Native Gemini is supplemental coverage, not a canonical Gemini receipt.
Verify maxHeaderSize defaults across Harper HTTP configuration references remains an unrelated accuracy question. No version4 docs or ambiguous console default were changed. Before this main update, the core and documentation heads passed their active CI checks; all three bot review threads are resolved, and no new actionable feedback was found. The paging overlap with the broader full-text guide has been reconciled locally; the native-guide review comments have since been addressed with code-grounded replies and all inline threads resolved. Its older changes-requested review still requires a fresh human review; thread resolution is not an approval. Kyle authorized moving this PR out of Draft after readiness checks clear. Neither PR may be merged by this task.
Comment generated by kAIle (GPT-6)
Release target: main — reference documentation for v5.4 full-text paging and limits.
Related PRs: #691 overlaps (Document native full-text search, coordinated publication), #736 overlaps (Document nested relationship filters and pagination, consistent range semantics), #753 overlaps (Document schema-defined sharding and regional placement, shared audit-default correction), 3 others independent
Complexity: medium
Review-Coverage: authored=codex; ran=cursor-composer,cursor-muse,claude; adjudicated=domain; declined=gemini,cursor-grok,cursor-kimi; rounds=13; full=10 @ b5294b0
Review-Attention: skim ~2m (decisions: companion-guide-ordering, native-limit-source, configuration-surface) @ b5294b0