Skip to content

feat(stovepipe): store queue state and read models - #855

Merged
mnoah1 merged 4 commits into
mnoah1/stovepipe-queue-state-apifrom
mnoah1/stovepipe-queue-state-storage
Oct 16, 2026
Merged

mnoah1 merged 4 commits into
mnoah1/stovepipe-queue-state-apifrom
mnoah1/stovepipe-queue-state-storage

Conversation

@mnoah1

@mnoah1 mnoah1 commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

Why?

Queue enablement needs durable internal state and separate read models so orchestration updates do not alter enablement history.

What?

Extend the existing queue snapshot, add immutable QueueLog and derived QueueSummary records, and implement key-oriented stores with conditional writes. Add queue and read-model schemas, with backend tests for byte-exact queue identity and independent projection versions.

Test Plan

Run the Stovepipe storage unit and MySQL integration tests.

Revert Plan

Revert this layer together with any dependent layers in the stack.

Issue Links

None — this is a standalone queue-status proposal.

Stack

Review in this order; each PR targets the preceding branch.

  1. API and RFC
  2. Storage (this PR)
  3. Transitions and materialization
  4. Lookup

@behinddwalls behinddwalls left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stores and schema look clean. They're key-only, have no secondary indexes, and queue leads the primary keys. One question about rollout, and one housekeeping item.

@mnoah1
mnoah1 force-pushed the mnoah1/stovepipe-queue-state-storage branch 3 times, most recently from 703bd88 to f5c6597 Compare October 14, 2026 22:49
@mnoah1

mnoah1 commented Oct 14, 2026

Copy link
Copy Markdown
Contributor Author

One question about rollout, and one housekeeping item.

Were there meant to be some inline comments for these?

@mnoah1
mnoah1 marked this pull request as ready for review October 14, 2026 23:07
@mnoah1
mnoah1 requested review from a team and sbalabanov as code owners October 14, 2026 23:07
@mnoah1 mnoah1 changed the title feat(stovepipe): [DRAFT] store queue state and read models feat(stovepipe): store queue state and read models Oct 14, 2026
@mnoah1
mnoah1 force-pushed the mnoah1/stovepipe-queue-state-storage branch from f5c6597 to df1f9a2 Compare October 16, 2026 14:26
Summary:
Queue enablement needs durable internal state and separate read models so orchestration updates do not alter enablement history.

Extend the existing queue snapshot, add immutable QueueLog and derived QueueSummary records, and implement key-oriented stores with conditional writes. Include the one-time migration for existing queue tables and backend tests for byte-exact queue identity and independent projection versions.

Part 2 of 4, built on the API and RFC. This layer supplies storage for transitions and lookup.

Test Plan:
Run the Stovepipe storage unit and MySQL integration tests. Migration coverage verifies existing operational values survive. Before deploying to an existing database, apply queue_enablement.sql once and create the queue log and summary tables.
@mnoah1
mnoah1 force-pushed the mnoah1/stovepipe-queue-state-storage branch from df1f9a2 to 91d2c33 Compare October 16, 2026 17:25
@mnoah1
mnoah1 added this pull request to the merge queue Oct 16, 2026
Merged via the queue into main with commit 1f6af7e Oct 16, 2026
19 checks passed
@mnoah1
mnoah1 deleted the mnoah1/stovepipe-queue-state-storage branch October 16, 2026 17:43
DivyanshGarg380 pushed a commit to DivyanshGarg380/submitqueue that referenced this pull request Oct 17, 2026
## Why?

Queue enablement needs a dedicated lookup: validation results and the
current queue row do not identify the state covering an arbitrary
commit.

## What?

Define GetQueueStatus with an optional change URI and an `enablement`
field of type `QueueEnablementStatus`, commit generated bindings, and
document inclusive boundaries, intermediate commits and error semantics
in the indexed RFC. The RPC remains unimplemented until the final layer.

## Test Plan

Build the published bindings and run the Stovepipe server tests. The
final stack also passes all 27 Stovepipe unit, MySQL integration,
service integration and end-to-end targets.

## Revert Plan

Revert this layer together with any dependent layers in the stack.

## Stack

Review in this order; each PR targets the preceding branch.

1. [API and RFC](uber#854) (this PR)
2. [Storage](uber#855)
3. [Transitions and
materialization](uber#856)
4. [Lookup](uber#857)
DivyanshGarg380 pushed a commit to DivyanshGarg380/submitqueue that referenced this pull request Oct 17, 2026
…r#856)

## Why?

Queue enablement changes must preserve coordination fields and remain
recoverable when history or summary writes fail.

## What?

Add an internal TransitionController with Initialize, Prepare and Apply,
plus queuelog.Materializer.PersistLog. Retain the previous occurrence
before replacing internal enablement and materialize the committed
winner afterward. Prepared retries preserve the accepted boundary;
optimistic locking handles operational races. Startup registers
configured queues as disabled without overwriting registered enablement.

## Test Plan

Unit, service integration and end-to-end tests cover competing
decisions, operational races, projection races and failed writes. Run
make local-stovepipe-start for local startup with schemas applied before
registration. Changing enablement uses the internal transition
controller; this layer adds no mutation RPC or CLI.

## Revert Plan

Revert this layer together with any dependent layers in the stack.

## Issue Links

None — this is a standalone queue-status proposal.

## Stack

Review in this order; each PR targets the preceding branch.

1. [API and RFC](uber#854)
2. [Storage](uber#855)
3. [Transitions and
materialization](uber#856) (this PR)
4. [Lookup](uber#857)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants