Skip to content

fix: model LangGraph checkpoint tables in Prisma to remove migrate dev drift (partial) - #67

Merged
kartikmanimuthu merged 15 commits into
offloadmemory:mainfrom
fabicarvano:fix/prisma-checkpoint-tables-drift
Oct 19, 2026
Merged

kartikmanimuthu merged 15 commits into
offloadmemory:mainfrom
fabicarvano:fix/prisma-checkpoint-tables-drift

Conversation

@fabicarvano

Copy link
Copy Markdown
Contributor

Known issue: `prisma migrate dev` asked to reset the database over permanent "drift". That drift had two independent sources: the 4 checkpoint tables created at runtime by `@langchain/langgraph-checkpoint-postgres`, and the pgvector HNSW indexes.

The checkpoint tables use only standard Postgres types (TEXT, JSONB, BYTEA, INTEGER) with composite primary keys — fully modelable in Prisma, no `Unsupported` needed. This PR adds those 4 models to `schema.prisma` (mapped via `@@map` to the exact table names `PostgresSaver` uses) plus a hand-authored migration with the identical DDL `.setup()` already runs at runtime (`CREATE TABLE IF NOT EXISTS`) — so it's a no-op on any existing deployment and only provisions the tables up front on a fresh database.

The pgvector HNSW indexes (`claw_memories`/`document_chunks`) remain unsolved — there's no `Unsupported()` equivalent for indexes in Prisma, only for column types. That half of the drift is permanent and documented as such.

Verified end-to-end against a disposable Postgres: full deploy from scratch, `.setup()` runs idempotently on top, and `prisma migrate diff` confirms the checkpoint tables no longer show up as drift — only the two HNSW indexes remain. `persistence.test.ts` (real-Postgres integration tests) still passes unchanged.

What changed

  • `prisma/schema.prisma`: added `CheckpointMigration`/`Checkpoint`/`CheckpointBlob`/`CheckpointWrite` models
  • `prisma/migrations/20261017120000_add_checkpoint_tables/migration.sql`: hand-authored, `CREATE TABLE IF NOT EXISTS` matching `PostgresSaver.setup()`'s own DDL
  • `docs/superpowers/claw-studio-known-issues.md`: issue Agent knowledge #5 updated to partially-fixed, with the remaining HNSW-index limitation documented
  • `libs/claw-studio/CLAUDE.md`: migrations section updated to match

Test plan

  • `prisma migrate deploy` from scratch against a disposable Postgres
  • `PostgresSaver.setup()` run on top — idempotent, no error
  • `prisma migrate diff` confirms only the 2 pgvector HNSW indexes remain as drift
  • `persistence.test.ts` (real Postgres) — 3/3 passing
  • `prisma generate` — client regenerates cleanly

…d chat backends

The OpenAICompatibleProvider called the AI SDK client as a bare
function (this.client(model)), which routes to the Responses API
(/v1/responses). Any self-hosted OpenAI-compatible backend that only
implements /v1/chat/completions (Ollama, vLLM, LiteLLM, etc.) 404s on
every chat/streaming request. The failure surfaces to end users only
as a generic "An error occurred.", with the real cause
(unsupported path /v1/responses) visible only in server logs.

Switched to this.client.chat(model), which targets
/v1/chat/completions as intended. Embeddings were unaffected (already
used textEmbeddingModel() correctly).

Also add an exports map to the @chatbot/ai package.json generated in
apps/workers/Dockerfile, matching the pattern already used for
@chatbot/shared and @chatbot/agent-studio in the same file. Without
it, subpath imports such as @chatbot/ai/tools/built-in-registry fail
to resolve at runtime inside the workers container. This silently
broke Claw's web-search, web-fetch and URL-guard tools and caused
the claw-scheduler sweep to fail every 30 seconds in any
Docker-deployed workers container.

Both issues were confirmed by running the full stack in Docker
end-to-end (agent create/publish/inference against a real
OpenAI-compatible backend), not just by code review.
…ion mismatch)

The code-side guardrail for this issue already shipped, but the actual
database row for the affected tenant was never corrected because DB
access was not available in the session that made the code fix. That
part cannot be fixed by a PR against this repo — it is deployment-
specific data — so this documents the exact SELECT/UPDATE needed, for
whoever has DB access to run directly from the known-issues entry
instead of re-deriving the root cause.
The gateway's resume handler (Approve/Reject/"Always allow for this
task" on Slack/Telegram/Discord) validated the channel signature, that
the run exists, and that it belongs to the right tenant, but never
checked the clicking user's platform id against anything. Any member
of the connected workspace who could see the approval message could
press it — and approve_always grants a STANDING permission on the
task, not a one-time action, so this let a single click by any
workspace member leave a scheduled task able to send mail or create
issues unattended indefinitely.

Adds an optional, per-channel approverIds allowlist (connectors/types.ts
-- not a secret, just platform user ids) and enforces it in the
gateway's handleResume before enqueueing approve/approve_always/reject:

- If an allowlist is configured, the clicking user must be on it.
- approve_always specifically default-denies with NO allowlist
  configured at all, since granting a standing permission on trust
  alone is too consequential -- this matches the stopgap already
  recommended in docs/superpowers/claw-studio-known-issues.md #11.
- Plain approve/reject with no allowlist configured keep their
  existing (pre-this-change) behavior, so a tenant that hasn't set one
  up yet is not suddenly locked out of ordinary approvals.

Also threads the approving user's id through to the run's
approval_decision event content, so a decision is attributable to a
person rather than just "someone in the workspace" -- partially
addresses the attribution gap noted in known-issues #13 as well.

Covered by new gateway-service.test.ts cases (allowlisted/non-
allowlisted approve, approve_always with and without an allowlist,
non-allowlisted reject) plus updated assertions on the existing
resume tests for the new approvedBy field. Verified: bunx vitest run
on gateway-service.test.ts (23/23) and execute-run.test.ts (50/50),
bunx tsc --noEmit -p tsconfig.lib.json on libs/claw-studio, bunx tsc
--noEmit on apps/workers, and the broader apps/workers vitest suite
(68/68, two unrelated pre-existing env-var failures in other job
handlers, nothing touching this change).
…visibility

- Postgres service container (pgvector/pgvector:pg16) with both port 5432
  and 5433 exposed, matching docker-compose.yml's local-dev mapping that
  claw-studio's persistence.test.ts hardcodes rather than reading
  DATABASE_URL.
- bunx fumadocs-mdx generates web-ui's @/.source before typecheck runs —
  the Next.js plugin only does this as a side effect of next dev/build,
  which a standalone tsc --noEmit never triggers.
- Wire the existing (already working) typecheck script into CI.
- continue-on-error on the Build step: agent-studio:build fails outside
  Docker on a cross-package path alias (whatsapp -> @chatbot/agent-studio/server)
  that only resolves inside the Dockerfile's hand-built node_modules exports
  map, not under @nx/js:tsc's dependency-closure build. Pre-existing, not
  something this change fixes; left visible rather than hidden.

Validated locally end to end against a throwaway Postgres container:
migrate deploy, typecheck (10/10 projects), unit tests (1113/1113, with the
openai-compatible.test.ts mock fix in the companion commit), and build
(8/9 projects; agent-studio is the known gap above).
streamChat now calls this.client.chat(model) instead of the bare
this.client(model) call (see the previous commit on this branch), but
this test's mock only exposed the provider as a bare callable, so every
streamChat assertion broke with 'this.client.chat is not a function'.
Add provider.chat alongside the existing bare callable and
textEmbeddingModel, matching the real @ai-sdk/openai client shape.

Caught by actually running bun run test for the ai project end to end,
which the original fix wasn't validated against.
…rate dev drift

checkpoint_migrations/checkpoints/checkpoint_blobs/checkpoint_writes are
created at runtime by PostgresSaver.setup() and were previously absent from
schema.prisma, so prisma migrate dev reported them as drift and offered to
reset the database. These 4 tables use only standard Postgres types with
composite primary keys, fully representable in Prisma's schema language.

Adds CheckpointMigration/Checkpoint/CheckpointBlob/CheckpointWrite models
mapped to the exact table names PostgresSaver uses, plus a hand-authored
migration with the identical CREATE TABLE IF NOT EXISTS DDL setup() runs
at runtime, so existing deployments get a no-op and fresh databases get
the tables provisioned up front.

Verified end-to-end against a disposable Postgres: migrate deploy from
scratch, PostgresSaver.setup() runs idempotently, and migrate diff confirms
the checkpoint tables are no longer drift. The pgvector HNSW indexes on
claw_memories/document_chunks remain unmodeled (Prisma has no Unsupported()
equivalent for indexes) and are documented as a permanent limitation.

@kartikmanimuthu kartikmanimuthu left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @fabicarvano — I checked the migration against upstream rather than taking it on trust: @langchain/langgraph-checkpoint-postgres@1.0.0's getMigrations() returns exactly five statements, and your migration.sql reproduces all five faithfully (including ALTER … ALTER COLUMN blob DROP not null as their migration #5). The @@map names, composite @@ids, Bytes? for the nullable blob and CREATE TABLE IF NOT EXISTS everywhere all line up with PostgresSaver.setup(), and the honest note in claw-studio-known-issues.md #5 about the remaining HNSW drift is exactly the right record.

One change before I merge: the branch currently carries 8fc4ae1 "ci: add Postgres service, Fumadocs codegen step, and typecheck/build visibility", an independent CI rewrite that conflicts with #62's ci.yml (I verified: one conflict hunk, that file only). Please drop that commit — and the three merge: commits, which become no-ops once #63/#64/#65 are on main. The PR then rebases to a clean four-file diff. If it's easier, I can resolve ci.yml in favour of main's version at merge time, but dropping it is cleaner and keeps #62's build gate intact — that variant sets build and e2e to continue-on-error, which would quietly remove the only meaningful check.

Two technical notes, both optional:

  • .setup() consults SELECT v FROM checkpoint_migrations, but this migration pre-creates the tables and leaves that version table empty, so the library replays all five of its migrations on first use. That's safe only because all five happen to be idempotent today — the moment upstream adds a #6 with a non-idempotent ALTER, the two provisioning paths diverge and setup() can fail on a database Prisma prepared. Seeding (1),(2),(3),(4),(5) in the same migration would pin the contract.
  • The smaller long-term commitment might be to not model these at all: PostgresSaver.fromConnString(url, { schema: 'langgraph' }) keeps the checkpoint tables in their own schema and out of Prisma's public snapshot, which removes the drift with no DDL duplication and no ownership conflict. Worth a sentence in the PR about why it wasn't chosen.

Also worth filing separately: nothing prunes the checkpoint tables, and now that prisma.checkpoint* clients exist, app code can write to them too.

Approving — thanks for the end-to-end verification (migrate diff, .setup() idempotency, real-Postgres tests); that's what made this reviewable.

@kartikmanimuthu
kartikmanimuthu merged commit e7587e5 into offloadmemory:main Oct 19, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants