Repository navigation
ci: replace Bitbucket + CodePipeline with GitHub Actions CD - #62
Merged
Merged
Conversation
The repo only ever pushed to GitHub (origin has a single push URL) and the CodePipeline stack was never deployed, so the Bitbucket path was dead weight. Removed: - bitbucket-pipelines.yml - infra/cicd/ — CodePipeline/CodeBuild stack wired to a Bitbucket CodeStar connection (Pulumi.nonprod.yaml defaulted connectionProvider to "Bitbucket"). Its own design doc recorded it as "not being deployed". - infra/project.json deploy-cicd / preview-cicd targets, which pointed at it. - Bitbucket remote instructions in CLAUDE.md, docs/dev/git-workflow.md and docs/RENAME_TO_AGENTFORK_PLAN.md. Added .github/workflows/deploy.yml: builds apps/web-ui/Dockerfile for linux/arm64, pushes to ECR chatbot-web-ui, and rolls the ECS service. Build and deploy are two jobs in one workflow so the image tag produced by the build is passed through `needs` — chaining separate workflows would force the deploy to re-derive the tag and risk shipping the wrong image. Auth uses GitHub OIDC (role agentfork-github-deploy, trust scoped to repo:offloadmemory/agentfork:*), so no long-lived AWS keys are stored. Also drops the unused apps/workers copy from the web-ui Dockerfile: nothing in web-ui references @chatbot/workers and no workspace depends on it. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
CI has never reached a green run. Recent failures were masked by an account billing lock, so the real blocker stayed hidden until now: the job runs with no database and no app secrets. libs/claw-studio's persistence tests hit a live Postgres pinned to port 5433 (TEST_DATABASE_URL in src/agent/persistence.test.ts). With nothing listening, they fail with connection errors rather than assertion failures — which reads like a code bug and is not. `build` and `e2e` also boot the app, so they need DATABASE_URL and NEXTAUTH_SECRET too. Adds a pgvector/pg16 service on 5433 matching docker-compose.yml, runs `prisma migrate deploy` (not `db push` — the HNSW indexes live in raw SQL inside migrations), and supplies throwaway CI-only env values. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Reproduced the failure locally by hiding .env: libs/claw-studio's claw-runtime.test.ts constructs an EncryptionService, which throws "ENCRYPTION_KEY must be a 64-character hex string" when the var is unset. Every test in the file skips, and the afterAll cleanup then fails too — the Prisma "undefined value within array" error is a cascade from that, not a second bug. Supplying a 64-char hex value takes the file to 12/12 passing. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The previous commit set ENCRYPTION_KEY to 64 zeros unquoted. YAML parses an
unquoted all-digit scalar as a number, so GitHub provided the string "0" (one
char) rather than 64 chars. That failed the /^[0-9a-fA-F]{64}$/ check in
libs/shared/src/env.ts, and because prisma-client.ts imports env at module load,
every suite that touches the database died at collection.
That is why the failure count jumped from 23 to 80 files: agent-studio passed in
the baseline run (✓ agent-service.test.ts, 13 tests) precisely because
ENCRYPTION_KEY was unset, and started failing only once an invalid value existed.
An absent variable is skipped by .optional(); an invalid one is not.
Quotes the value and uses a key containing letters, since a purely numeric hex
string could still be coerced by some parsers. Verified with the full suite under
the exact CI environment (no local .env): 8/8 projects, 96 files, 1107 tests.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
`bun run build` runs `nx run-many -t build --all`, which dies at agent-studio:build on a pre-existing TS2307 (@chatbot/agent-studio/server, from the agent-studio <-> whatsapp import cycle). Because web-ui declares agent-studio as a graph dependency, that failure blocks web-ui too — even though web-ui builds fine on its own. Builds web-ui with --excludeTaskDependencies instead. This is correct rather than a workaround: web-ui resolves every library from source via the tsconfig.base.json paths (and transpilePackages), so it never consumes agent-studio's dist/ output. apps/web-ui/Dockerfile builds the same way in production — `next build` with no upstream tsc step. Verified locally: exit 0. e2e stays disabled. Its target also gates on ^build, but the real blocker is that Playwright's webServer runs `bun run dev:all`, which starts four servers within a 120s timeout and reproducibly times out. CI had never reached this step before, so it was never actually exercised. Enabling it needs a dedicated web-ui-only serve command, which is separate work. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The ECS cluster and service do not exist in ap-south-2 — the Pulumi compute stack that creates them has not been deployed — so the deploy-ecs job could only fail, for reasons unrelated to the image. Removing it keeps this workflow about one thing: getting the app image into ECR. ECS deployment is a small diff to re-add once the infrastructure exists; the IAM role keeps only the ECR permissions it uses, which also means a compromised workflow can no longer touch ECS. Also switches the layer cache to a real registry-backed cache (type=registry on a :buildcache tag). The previous config mirrored the Pulumi path's `cacheFrom: <repo>:latest`, which was a no-op because nothing ever pushed a `latest` tag for the cache to resolve against. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The shared .dockerignore excludes **/.next, which makes Dockerfile.prebuilt unusable: that Dockerfile exists precisely to package an already-built Next.js standalone output, so it needs .next in the build context. Docker prefers <dockerfile>.dockerignore over the shared one, so this restores the ability to build the prebuilt image without weakening the exclusion for the normal Dockerfile, where it is load-bearing — a stale .next is ~446 MB and shadows the directory the build writes into. Note this is a BuildKit feature. The classic builder ignores it, so on a host without buildx the prebuilt image still needs a staged context. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
… cycle `nx build agent-studio` failed with: libs/whatsapp/src/processor/agent-executor.ts:73:52 error TS2307: Cannot find module '@chatbot/agent-studio/server' Root cause, confirmed by capturing the tsconfig Nx generates for the task: when building agent-studio, Nx rewrites `paths` to each dependency's dist/ output. That turns @chatbot/agent-studio/server into "dist/libs/agent-studio/server" — a self-reference to the project currently being built, so it cannot exist yet. This file is part of agent-studio's compile because agent-studio imports its sibling executors (./whatsapp-send-executor and the template one), and whatsapp is consumed as raw TS with no build of its own. The import above resolves correctly everywhere it actually runs: Next.js and apps/web-ui/Dockerfile both resolve it from source through tsconfig.base.json, and the Dockerfile never runs an upstream tsc at all. Rather than restructure the modules, this applies the convention already used nine lines below for the identical situation — the GraphExecutor import already carries `@ts-ignore — dynamic import to avoid circular dependency at build time`. The build-time error is a false positive; the runtime import is unchanged. Verified: `nx build agent-studio` exits 0, and web-ui's build no longer needs --excludeTaskDependencies to get past it. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Same Nx path-rewriting bug as agent-studio: when Nx builds this project it rewrites tsconfig `paths` to each dependency's dist/ output, so @chatbot/agent-studio/services/* became dist/libs/agent-studio/services/* — which cannot resolve, because agent-studio's compiled tree nests under dist/libs/agent-studio/libs/agent-studio/src/services/*. The deep imports stay: vitest.config.ts documents them as deliberate, to keep agent-studio's WhatsApp/Telegram node executors out of this lib's test graph. Routing them through the barrel would contradict that. So each carries the @ts-ignore convention already used in libs/whatsapp/src/processor/agent-executor.ts. Suppressing the import error leaves `discovered` as any, which turned the map callback's parameter into an implicit any (TS7006). Annotated it with { name: string } — the shape McpDiscoveredTool actually provides. `nx build claw-studio` exits 0, `bun run build` now succeeds across all 9 projects, and claw-studio's tests still pass. CI therefore drops --excludeTaskDependencies and goes back to the plain `bun run build`. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
workflow_dispatch is only available for workflows that already exist on the default branch, so this workflow cannot be triggered from ci/github-actions-cd at all. Pushing is the one trigger that uses the branch's own copy of the workflow, so the branch is added to the push filter until this is merged. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
offloadmemory is a GitHub Organization, so the OIDC sub claim carries immutable numeric IDs (repo:offloadmemory@20665413/agentfork@1227066446:...). The trust policy only allowed repo:offloadmemory/agentfork:*, which never matched, so sts:AssumeRoleWithWebIdentity was denied. Policy now allows both forms. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ust policy The trust policy needs repo:offloadmemory@20665413/agentfork@1227066446:* because offloadmemory is a GitHub Organization, so the OIDC sub claim carries numeric IDs. Documented alongside the misleading error and the fact that CloudTrail cannot help diagnose it. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
kartikmanimuthu
approved these changes
Oct 19, 2026
kartikmanimuthu
left a comment
kartikmanimuthu
left a comment
Collaborator
There was a problem hiding this comment.
Thanks @Thirunavukkarasu — this is the unglamorous work that decides whether the other seven open PRs can be judged at all, so it matters more than its subject line suggests. The design choices are the ones I'd have made: one workflow with two needs-chained jobs (chaining two workflows would force the deploy to re-derive the tag, and a mismatch ships the wrong image silently), OIDC instead of stored keys, native arm64 runners for a Graviton target instead of QEMU, registry-backed layer cache, and an honest docs/dev/deployment.md.
One change I need before I merge this, because it's a production side-effect:
.github/workflows/deploy.yml:18 still has branches: [main, ci/github-actions-cd] with a "drop this once merged" comment. That branch is this PR's own head, so leaving it means pushes to it keep publishing images into the production ECR repo. Please drop that entry (and the comment above it). While you're in the file: NEXT_PUBLIC_MISSION_CONTROL_URL defaults to http://localhost:3010/mission-control — if the repo variable is unset, the build succeeds and ships an image whose client bundle points at localhost. Making it required would fail loudly instead.
Three things I'd strongly suggest as follow-ups, none blocking:
- The approval gate is gone. The CodePipeline you deleted had Source → Preview → Approval → Deploy. A
environment: productionprotection rule on the job restores that control for image publication. - The image build is never validated before merge — there's no
pull_requesttrigger, so a Dockerfile break surfaces after it's onmain. A PR job withpush: falsewould have caught the missingexportsmap in #63. - Two owners for one image.
infra/computestill builds withawsx.ecr.Image(both web-ui and mission-control), and as you noted, a laterpulumi upwill treat a CI-rolled task definition as drift. Worth picking one owner explicitly — Pulumi for infra/service, CI for image build +update-service.
Also flagging: four @ts-ignores were added in libs/claw-studio/src/mcp/mcp-tools.ts and libs/whatsapp/src/processor/agent-executor.ts to get the Nx build through. They're suppressing a genuine module-resolution issue and they're outside this PR's scope — if they have to stay, linking an issue in each comment would keep it visible.
Approved on the merits; I'll hold the merge until the branch trigger is gone. Thanks also for making CI green for everyone else — that's the change that unblocks reviewing the rest.
This was referenced Oct 19, 2026
Removed temporary branch from push trigger and updated build argument handling for NEXT_PUBLIC_MISSION_CONTROL_URL.
kartikmanimuthu
added a commit
to fabicarvano/agentfork
that referenced
this pull request
Oct 19, 2026
…floadmemory#62) Removed commented-out sections and adjusted database port mappings in CI configuration.
kartikmanimuthu
added a commit
that referenced
this pull request
Oct 19, 2026
…v drift (partial) (#67) * fix: openai-compatible provider calls /responses, breaking self-hosted chat backends The OpenAICompatibleProvider called the AI SDK client as a bare function (this.client(model)), which routes to the Responses API (/v1/responses). Any self-hosted OpenAI-compatible backend that only implements /v1/chat/completions (Ollama, vLLM, LiteLLM, etc.) 404s on every chat/streaming request. The failure surfaces to end users only as a generic "An error occurred.", with the real cause (unsupported path /v1/responses) visible only in server logs. Switched to this.client.chat(model), which targets /v1/chat/completions as intended. Embeddings were unaffected (already used textEmbeddingModel() correctly). Also add an exports map to the @chatbot/ai package.json generated in apps/workers/Dockerfile, matching the pattern already used for @chatbot/shared and @chatbot/agent-studio in the same file. Without it, subpath imports such as @chatbot/ai/tools/built-in-registry fail to resolve at runtime inside the workers container. This silently broke Claw's web-search, web-fetch and URL-guard tools and caused the claw-scheduler sweep to fail every 30 seconds in any Docker-deployed workers container. Both issues were confirmed by running the full stack in Docker end-to-end (agent create/publish/inference against a real OpenAI-compatible backend), not just by code review. * docs: add exact remediation SQL for known issue #15 (embedding dimension mismatch) The code-side guardrail for this issue already shipped, but the actual database row for the affected tenant was never corrected because DB access was not available in the session that made the code fix. That part cannot be fixed by a PR against this repo — it is deployment- specific data — so this documents the exact SELECT/UPDATE needed, for whoever has DB access to run directly from the known-issues entry instead of re-deriving the root cause. * fix: channel approval buttons are not authorized to a specific user The gateway's resume handler (Approve/Reject/"Always allow for this task" on Slack/Telegram/Discord) validated the channel signature, that the run exists, and that it belongs to the right tenant, but never checked the clicking user's platform id against anything. Any member of the connected workspace who could see the approval message could press it — and approve_always grants a STANDING permission on the task, not a one-time action, so this let a single click by any workspace member leave a scheduled task able to send mail or create issues unattended indefinitely. Adds an optional, per-channel approverIds allowlist (connectors/types.ts -- not a secret, just platform user ids) and enforces it in the gateway's handleResume before enqueueing approve/approve_always/reject: - If an allowlist is configured, the clicking user must be on it. - approve_always specifically default-denies with NO allowlist configured at all, since granting a standing permission on trust alone is too consequential -- this matches the stopgap already recommended in docs/superpowers/claw-studio-known-issues.md #11. - Plain approve/reject with no allowlist configured keep their existing (pre-this-change) behavior, so a tenant that hasn't set one up yet is not suddenly locked out of ordinary approvals. Also threads the approving user's id through to the run's approval_decision event content, so a decision is attributable to a person rather than just "someone in the workspace" -- partially addresses the attribution gap noted in known-issues #13 as well. Covered by new gateway-service.test.ts cases (allowlisted/non- allowlisted approve, approve_always with and without an allowlist, non-allowlisted reject) plus updated assertions on the existing resume tests for the new approvedBy field. Verified: bunx vitest run on gateway-service.test.ts (23/23) and execute-run.test.ts (50/50), bunx tsc --noEmit -p tsconfig.lib.json on libs/claw-studio, bunx tsc --noEmit on apps/workers, and the broader apps/workers vitest suite (68/68, two unrelated pre-existing env-var failures in other job handlers, nothing touching this change). * ci: add Postgres service, Fumadocs codegen step, and typecheck/build visibility - Postgres service container (pgvector/pgvector:pg16) with both port 5432 and 5433 exposed, matching docker-compose.yml's local-dev mapping that claw-studio's persistence.test.ts hardcodes rather than reading DATABASE_URL. - bunx fumadocs-mdx generates web-ui's @/.source before typecheck runs — the Next.js plugin only does this as a side effect of next dev/build, which a standalone tsc --noEmit never triggers. - Wire the existing (already working) typecheck script into CI. - continue-on-error on the Build step: agent-studio:build fails outside Docker on a cross-package path alias (whatsapp -> @chatbot/agent-studio/server) that only resolves inside the Dockerfile's hand-built node_modules exports map, not under @nx/js:tsc's dependency-closure build. Pre-existing, not something this change fixes; left visible rather than hidden. Validated locally end to end against a throwaway Postgres container: migrate deploy, typecheck (10/10 projects), unit tests (1113/1113, with the openai-compatible.test.ts mock fix in the companion commit), and build (8/9 projects; agent-studio is the known gap above). * fix: update openai-compatible mock to match the .chat() routing fix streamChat now calls this.client.chat(model) instead of the bare this.client(model) call (see the previous commit on this branch), but this test's mock only exposed the provider as a bare callable, so every streamChat assertion broke with 'this.client.chat is not a function'. Add provider.chat alongside the existing bare callable and textEmbeddingModel, matching the real @ai-sdk/openai client shape. Caught by actually running bun run test for the ai project end to end, which the original fix wasn't validated against. * ci: trigger first Actions run now that Actions is enabled on the fork * ci: trigger first Actions run after enabling workflows on the fork * fix: model LangGraph checkpoint tables in Prisma schema to remove migrate dev drift checkpoint_migrations/checkpoints/checkpoint_blobs/checkpoint_writes are created at runtime by PostgresSaver.setup() and were previously absent from schema.prisma, so prisma migrate dev reported them as drift and offered to reset the database. These 4 tables use only standard Postgres types with composite primary keys, fully representable in Prisma's schema language. Adds CheckpointMigration/Checkpoint/CheckpointBlob/CheckpointWrite models mapped to the exact table names PostgresSaver uses, plus a hand-authored migration with the identical CREATE TABLE IF NOT EXISTS DDL setup() runs at runtime, so existing deployments get a no-op and fresh databases get the tables provisioned up front. Verified end-to-end against a disposable Postgres: migrate deploy from scratch, PostgresSaver.setup() runs idempotently, and migrate diff confirms the checkpoint tables are no longer drift. The pgvector HNSW indexes on claw_memories/document_chunks remain unmodeled (Prisma has no Unsupported() equivalent for indexes) and are documented as a permanent limitation. * ci: drop conflict markers from ci.yml (CI fixes landed on main via #62) Removed commented-out sections and adjusted database port mappings in CI configuration. --------- Co-authored-by: Kartik Manimuthu <kartikmanimuthu@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Replaces the Bitbucket + AWS CodePipeline path with a single GitHub Actions CI/CD setup for the Web UI.
Investigation showed the Bitbucket path was already dead:
originhas exactly one push URL (git@github.com:offloadmemory/agentfork.git), and the CodePipeline stack was never deployed —infra/cicd/Pulumi.nonprod.yamleven defaultedconnectionProvidertoBitbucketwhile its own design doc recorded the stack as "not being deployed".Removed
bitbucket-pipelines.ymlinfra/cicd/— CodePipeline/CodeBuild stack (Source → Preview → Approval → Deploy) wired to a Bitbucket CodeStar connectiondeploy-cicd/preview-cicdNx targets ininfra/project.jsonthat pointed at itCLAUDE.md,docs/dev/git-workflow.md,docs/RENAME_TO_AGENTFORK_PLAN.mdAdded
.github/workflows/deploy.yml— buildsapps/web-ui/Dockerfileforlinux/arm64, pushes to ECRchatbot-web-ui, and rolls the ECS service.docs/dev/deployment.md— role, variables, and rollback.Also dropped the unused
apps/workerscopy fromapps/web-ui/Dockerfile(nothing in web-ui references@chatbot/workers; no workspace depends on it).Design notes
needs. Chaining two separate workflows would force the deploy to re-derive the tag, and a mismatch would ship the wrong image silently.agentfork-github-deploy, trust scoped torepo:offloadmemory/agentfork:*. The policy deliberately omitsiam:CreateRoleso a compromised workflow cannot escalate.ubuntu-24.04-armis free and avoids QEMU (which makes a Next.js build a timeout risk).ap-south-2, notus-east-1— verified against live ECR, wherechatbot-web-uiis the only repository that exists.Type of Change
Testing
docker run --rm -v "$PWD":/repo -w /repo rhysd/actionlint:latestbun install --frozen-lockfileChecked 2449 installs across 2662 packages (no changes)bunx nx show projectsinfra,web-ui,workersall resolvejq -e . infra/project.jsonapps/workersabsentbun install,prisma generate,sdk:build, andnext buildall green; standalone output contains onlyweb-uijqtask-definition cleanup filterThe deploy workflow has not executed end-to-end: it is branch-scoped, so Actions only registers it once this lands on
main. The build path was validated locally; the ECR push and ECS roll are exercised on the first merge.Related
The compute stack builds images itself via
awsx.ecr.Image, sopulumi upanddeploy.ymlcan both move the service. This is called out ininfra/DEPLOYMENT.md— Pulumi owns the service resource, so a laterpulumi upwill treat a CI-rolled task definition as drift. Worth resolving separately.Note:
CIonmainis currently red from a pre-existing unrelated failure (libs/ai/src/content-resolver.test.ts); left untouched.Checklist