diff --git a/docs/superpowers/claw-studio-known-issues.md b/docs/superpowers/claw-studio-known-issues.md index 85896ed6..ebddfa44 100644 --- a/docs/superpowers/claw-studio-known-issues.md +++ b/docs/superpowers/claw-studio-known-issues.md @@ -149,14 +149,14 @@ mock. ~2 lines. --- -## 🟠 5. `prisma migrate dev` wants to drop the database +## 🟡 5. PARTIALLY FIXED (2026-10-17) — `prisma migrate dev` wants to drop the database **Where:** repo-wide Prisma workflow. -**Problem:** `migrate dev` reports drift and offers only a reset. The drift is permanent and by design: -the `checkpoint_*` tables are created at runtime by `@langchain/langgraph-checkpoint-postgres` and are -deliberately absent from `schema.prisma`, and the `embedding` indexes are pgvector indexes Prisma -cannot represent (`Unsupported("vector(1024)")`). +**Problem:** `migrate dev` reports drift and offers only a reset. Two independent sources of drift: +the `checkpoint_*` tables, created at runtime by `@langchain/langgraph-checkpoint-postgres` and +previously absent from `schema.prisma`; and the `embedding` indexes, pgvector indexes Prisma cannot +represent (`Unsupported("vector(1024)")`). **Verified:** `migrate dev` output listed exactly those tables/indexes as drift and demanded a reset. @@ -164,8 +164,26 @@ cannot represent (`Unsupported("vector(1024)")`). **Fix / workaround (in use):** hand-author `prisma/migrations/_/migration.sql` and apply with `prisma migrate deploy`, which ignores drift and is what `bun run setup` and container start already -use. Documented in `libs/claw-studio/CLAUDE.md`. A real fix would add the checkpoint tables and -pgvector indexes to the schema (via `Unsupported`/raw SQL migrations) so drift is genuinely empty. +use. Documented in `libs/claw-studio/CLAUDE.md`. + +**Partially fixed (2026-10-17):** the checkpoint-table half is genuinely fixable and now fixed — all 4 +tables (`checkpoint_migrations`, `checkpoints`, `checkpoint_blobs`, `checkpoint_writes`) use only +standard Postgres types (TEXT, JSONB, BYTEA, INTEGER) with composite primary keys, fully expressible in +Prisma's schema language with no `Unsupported` needed. Added `CheckpointMigration`/`Checkpoint`/ +`CheckpointBlob`/`CheckpointWrite` models to `schema.prisma` (mapped via `@@map` to the exact table +names `PostgresSaver` uses) and a hand-authored `prisma/migrations/20261017120000_add_checkpoint_tables/ +migration.sql` that creates them with the identical `CREATE TABLE IF NOT EXISTS` DDL +`PostgresSaver.setup()` runs at runtime — so this is a no-op on every existing deployment (where +`.setup()` already created the tables) and provisions them up front on a fresh database. Verified +end-to-end against a disposable Postgres: deployed all migrations from scratch, ran `.setup()` +(idempotent, no error), then confirmed via `prisma migrate diff --from-url ... --to-schema-datamodel` +that the checkpoint tables are no longer reported as drift at all — only the two pgvector HNSW indexes +remain. `libs/claw-studio`'s `persistence.test.ts` (real Postgres) still passes unchanged. + +**Still open:** the pgvector HNSW index half is a genuine, permanent Prisma limitation — there is no +`Unsupported()` (or anything else) for an index, only for a column type, so `prisma migrate dev` will +always report `claw_memories_embedding_hnsw` and `idx_document_chunks_embedding` as drift. The existing +`migrate deploy` + hand-authored-migration workflow remains required for those two indexes. --- diff --git a/libs/claw-studio/CLAUDE.md b/libs/claw-studio/CLAUDE.md index d8b93b94..f1c61252 100644 --- a/libs/claw-studio/CLAUDE.md +++ b/libs/claw-studio/CLAUDE.md @@ -29,15 +29,23 @@ rather than fighting it. ## Migrations: never use `prisma migrate dev` -It wants to **reset the database**. The drift it reports is permanent and by design: - -- `checkpoint_*` tables are created at runtime by `@langchain/langgraph-checkpoint-postgres` and are - deliberately absent from `schema.prisma`. -- The `embedding` indexes are pgvector indexes Prisma cannot represent (`Unsupported("vector(1024)")`). - -Hand-author the migration SQL under `prisma/migrations/_/migration.sql` with a -round timestamp (matching the recent claw migrations) and apply with `bunx prisma migrate deploy`, -which ignores drift and is what `bun run setup` and container start already use. +It wants to **reset the database**. One source of drift was fixed (see below); one is permanent and +by design: + +- ~~`checkpoint_*` tables created at runtime by `@langchain/langgraph-checkpoint-postgres`~~ — **fixed**. + `CheckpointMigration`/`Checkpoint`/`CheckpointBlob`/`CheckpointWrite` in `schema.prisma` now model + these 4 tables (mapped to `checkpoint_migrations`/`checkpoints`/`checkpoint_blobs`/`checkpoint_writes`), + and `prisma/migrations/20261017120000_add_checkpoint_tables/migration.sql` creates them with the exact + `CREATE TABLE IF NOT EXISTS` DDL `PostgresSaver.setup()` itself runs — so an existing deployment where + `.setup()` already created them gets a no-op, and a fresh database gets them up front. `prisma migrate + dev` no longer reports these as drift. +- The `embedding` indexes are still pgvector HNSW indexes Prisma cannot represent — there is no + `Unsupported()` (or anything else) for an index, only for a column type, so this half is permanent. + +For this remaining half, hand-author the migration SQL under +`prisma/migrations/_/migration.sql` with a round timestamp (matching the recent +claw migrations) and apply with `bunx prisma migrate deploy`, which ignores drift and is what `bun run +setup` and container start already use. ## The agent loop diff --git a/prisma/migrations/20261017120000_add_checkpoint_tables/migration.sql b/prisma/migrations/20261017120000_add_checkpoint_tables/migration.sql new file mode 100644 index 00000000..b78a8940 --- /dev/null +++ b/prisma/migrations/20261017120000_add_checkpoint_tables/migration.sql @@ -0,0 +1,56 @@ +-- Models checkpoint_migrations, checkpoints, checkpoint_blobs and checkpoint_writes +-- in schema.prisma so `prisma migrate dev` no longer reports them as drift. +-- +-- This DDL is copied verbatim from the DDL that +-- `@langchain/langgraph-checkpoint-postgres`'s `PostgresSaver.setup()` already +-- runs at runtime (node_modules/@langchain/langgraph-checkpoint-postgres/dist/migrations.js), +-- including its own `CREATE TABLE IF NOT EXISTS` guards. A tenant whose gateway +-- has already run `.setup()` (every existing deployment) gets a no-op migration; +-- a fresh database gets the tables provisioned up front instead of waiting for +-- the first LangGraph checkpoint write to create them lazily. +-- +-- Known limitation that remains after this migration: pgvector's HNSW indexes on +-- claw_memories.embedding and document_chunks.embedding are still unmodeled — +-- Prisma's schema language cannot express an index, only a column type via +-- Unsupported(). `prisma migrate dev` will still report those 2 indexes as +-- drift. See docs/superpowers/claw-studio-known-issues.md #5 and +-- libs/claw-studio/CLAUDE.md. + +CREATE TABLE IF NOT EXISTS "checkpoint_migrations" ( + v INTEGER PRIMARY KEY +); + +CREATE TABLE IF NOT EXISTS "checkpoints" ( + thread_id TEXT NOT NULL, + checkpoint_ns TEXT NOT NULL DEFAULT '', + checkpoint_id TEXT NOT NULL, + parent_checkpoint_id TEXT, + type TEXT, + checkpoint JSONB NOT NULL, + metadata JSONB NOT NULL DEFAULT '{}', + PRIMARY KEY (thread_id, checkpoint_ns, checkpoint_id) +); + +CREATE TABLE IF NOT EXISTS "checkpoint_blobs" ( + thread_id TEXT NOT NULL, + checkpoint_ns TEXT NOT NULL DEFAULT '', + channel TEXT NOT NULL, + version TEXT NOT NULL, + type TEXT NOT NULL, + blob BYTEA, + PRIMARY KEY (thread_id, checkpoint_ns, channel, version) +); + +CREATE TABLE IF NOT EXISTS "checkpoint_writes" ( + thread_id TEXT NOT NULL, + checkpoint_ns TEXT NOT NULL DEFAULT '', + checkpoint_id TEXT NOT NULL, + task_id TEXT NOT NULL, + idx INTEGER NOT NULL, + channel TEXT NOT NULL, + type TEXT, + blob BYTEA NOT NULL, + PRIMARY KEY (thread_id, checkpoint_ns, checkpoint_id, task_id, idx) +); + +ALTER TABLE "checkpoint_blobs" ALTER COLUMN blob DROP NOT NULL; diff --git a/prisma/schema.prisma b/prisma/schema.prisma index 5986fa5d..c5b70681 100644 --- a/prisma/schema.prisma +++ b/prisma/schema.prisma @@ -1956,3 +1956,62 @@ model TranscriptionAwsCredential { @@map("transcription_aws_credentials") } + +// ─── LangGraph checkpoint tables ─────────────────────────────────────────── +// These 4 models are NOT created by Prisma. They mirror the exact DDL that +// `@langchain/langgraph-checkpoint-postgres`'s `PostgresSaver.setup()` already +// runs at runtime against the "public" schema (libs/claw-studio/src/agent/persistence.ts +// calls `PostgresSaver.fromConnString(databaseUrl)` with no schema override, and +// that package defaults to "public" — see node_modules/@langchain/langgraph-checkpoint-postgres/dist/index.js). +// Modeling them here is purely to stop `prisma migrate dev` from reporting them +// as drift and offering to drop them; the migration that introduces them must +// use `CREATE TABLE IF NOT EXISTS` (see prisma/migrations), matching what +// `.setup()` itself does, so neither side fights the other over who owns creation. +// pgvector's HNSW indexes on ClawMemory/DocumentChunk remain unmodeled — Prisma's +// schema language has no way to express an index, only a column type via +// Unsupported(), so that half of the known drift is permanent (see +// docs/superpowers/claw-studio-known-issues.md #5 and libs/claw-studio/CLAUDE.md). +model CheckpointMigration { + v Int @id + + @@map("checkpoint_migrations") +} + +model Checkpoint { + threadId String @map("thread_id") + checkpointNs String @default("") @map("checkpoint_ns") + checkpointId String @map("checkpoint_id") + parentCheckpointId String? @map("parent_checkpoint_id") + type String? + checkpoint Json + metadata Json @default("{}") + + @@id([threadId, checkpointNs, checkpointId]) + @@map("checkpoints") +} + +model CheckpointBlob { + threadId String @map("thread_id") + checkpointNs String @default("") @map("checkpoint_ns") + channel String + version String + type String + blob Bytes? + + @@id([threadId, checkpointNs, channel, version]) + @@map("checkpoint_blobs") +} + +model CheckpointWrite { + threadId String @map("thread_id") + checkpointNs String @default("") @map("checkpoint_ns") + checkpointId String @map("checkpoint_id") + taskId String @map("task_id") + idx Int + channel String + type String? + blob Bytes + + @@id([threadId, checkpointNs, checkpointId, taskId, idx]) + @@map("checkpoint_writes") +}