Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 8 additions & 3 deletions docs/guides/how-it-works.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
Resources are scoped by organization, with names you choose (not fixed `dev`/`stg`/`prod`). Each org gets:

- `.env.<org>` — private API key and base URL
- `.vapi-state.<org>.json` — resource name ↔ UUID mappings (nothing else — committed)
- `.vapi-state.<org>.json` - committed resource name-to-UUID mappings and optional observed Vapi versions
- `.vapi-state-hash/<org>/<uuid>` — last-seen platform content hash per resource, used for drift detection (per-developer, gitignored)
- `resources/<org>/` — all resource files

Expand Down Expand Up @@ -136,7 +136,7 @@ Tracks resource ID ↔ Vapi UUID mappings per org:

```json
{
"assistants": { "my-assistant": { "uuid": "9c0f3f42-…" } },
"assistants": { "my-assistant": { "uuid": "9c0f3f42-...", "latestVersion": "v8" } },
"credentials": { "my-cred": { "uuid": "2f6db611-…" } },
"squads": { "my-squad": { "uuid": "51a9e1c7-…" } },
"tools": { "my-tool": { "uuid": "d4b8a2e0-…" } }
Expand All @@ -145,6 +145,11 @@ Tracks resource ID ↔ Vapi UUID mappings per org:

Every resource type has a section. Keys are sorted, so diffs stay readable.

`latestVersion` is optional Vapi metadata recorded in the state file, not desired resource configuration.
Pull and successful writes update the observed version. The CLI excludes it from resource files and content hashes,
and from update payloads for versioned resources, so a version-only change does not create drift. For example,
v6 and v8 can have identical configuration even when v7 contained an intervening change.

## What Vapi sees

Every API request uses the org's private key and identifies the tool with a
Expand All @@ -161,7 +166,7 @@ requests themselves; there is no separate telemetry.
| Path | What it is |
| --- | --- |
| `resources/<org>/` | Your resources, one folder per org |
| `.vapi-state.<org>.json` | Name → UUID mappings per org (committed) |
| `.vapi-state.<org>.json` | Name → UUID mappings and optional Vapi version metadata per org (committed) |
| `.env.<org>` | API key and generated binding IDs (gitignored) |
| `promotion.yml`, `vapi-checks.yml` | Promotion pipelines and PR checks (copy from the `*.example.yml` files) |
| `resources/<org>/.vapi-ignore` | Platform resources this repo shouldn't manage (see `resources/.vapi-ignore.example`) |
Expand Down
16 changes: 13 additions & 3 deletions docs/learnings/sync-behavior.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ function of which of them exist and whether their contents agree.
| Artifact | Where | Committed? | Meaning |
|---|---|---|---|
| **Local file** (`L`) | `resources/<org>/<type>/<name>.md\|yml` | yes | The declarative source of truth you edit |
| **State entry** (`S`) | `.vapi-state.<org>.json` → `{ "<name>": { "uuid": … } }` | yes | The ONLY thing state stores: which platform UUID a local name is bound to |
| **State entry** (`S`) | `.vapi-state.<org>.json` -> `{ "<name>": { "uuid": "...", "latestVersion": "vN" } }` | yes | UUID binding and optional observed Vapi version; version metadata is excluded from content hashes |
| **Baseline** (`B`) | `.vapi-state-hash/<org>/<uuid>` | **no** (per-developer, gitignored) | sha256 of the last platform content *you* saw (last pull or push) |
| **Dashboard resource** (`D`) | Vapi platform | n/a | What's live |

Expand Down Expand Up @@ -59,6 +59,14 @@ UUIDs → names):

---

## Version metadata and drift

`latestVersion` records the most recent Vapi version observed by pull or a
successful write. It is stored in the committed state mapping, not in resource
configuration or the per-developer content baseline. If the assistant config
at v6 is later restored and published as v8 after a different v7, identical v6
and v8 configs hash the same and do not create a conflict.

## Content scenarios (resource exists everywhere: L + S + B + D)

### 1. Nothing changed (`clean`)
Expand Down Expand Up @@ -202,8 +210,10 @@ sweep, which consume the resources today without consuming the verdict.

`pull`, `push`, and `apply` all **hard-refuse** with
`Run \`npm run migrate\` first.` The migration (no org argument, no token
needed) slims every `.vapi-state.<org>.json` to `name → { uuid }` and seeds
each org's `.vapi-state-hash/` from the legacy hashes. Idempotent.
needed) normalizes each `.vapi-state.<org>.json` mapping to
`{ uuid, latestVersion? }` and seeds each org's `.vapi-state-hash/` from
the legacy hashes. Existing version metadata is retained and does not
participate in content drift. Idempotent.

### G. Local file renamed

Expand Down
53 changes: 49 additions & 4 deletions improvements.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,7 @@ you which stack PR closes the row.**
| 36 | `cleanup` deletes resources excluded by `.vapi-ignore` | A destructive cleanup can delete resources another team owns | None | RESOLVED 2026-10-03 |
| 37 | Resource validation ran only at deploy time, after merge | A config `apply` refuses could merge and block deploys and promotion | #32 | RESOLVED 2026-10-03 (#76) |
| 38 | Promotion can't carry a simulation that uses a stock personality by UUID | Gated orgs, and any promoted tests, need local personality files | None | Open |
| 39 | `latestVersion` metadata can cause false content drift | Matching configs can appear drifted across published versions | None | Open |

**Active backlog after cleanup:** `#2`, `#6`, `#8`, `#12`, `#20`, `#24–#26`, `#31`, and the open remainder of `#27` (wiring the listing-completeness verdict into push/delete/audit, and moving `cleanup.ts` onto the shared pager). Resolved entries stay in this file as historical incident notes per the maintenance directive; stale superseded backlog rows are not duplicated.

Expand Down Expand Up @@ -2061,12 +2062,56 @@ dependency check, as `check-payload.ts` already does.

---

## 39. `latestVersion` metadata can cause false content drift

**Discovered:** 2026-10-10, while tracing version-only drift after a push.

### Problem

Vapi can advance a resource version even when its effective configuration later
returns to an earlier value. For example, an assistant can be published as v6,
changed and published as v7, then reverted and published as v8. The v6 and v8
configurations are identical, but a local v6 resource can conflict with the
platform v8 resource if the version number is treated as configuration.

### Current behavior (upstream/main before this change)

The state mapping records the resource UUID but not its observed Vapi version.
`latestVersion` is included in resource data and content hashes, while a
successful POST or PATCH refreshes the hash baseline without recording the
returned version in state. A version-only difference can therefore look like
content drift. The affected paths are `src/types.ts`, `src/canonical.ts`,
`src/resources.ts`, `src/pull.ts`, and `src/push.ts`.

### Risk

A version-only difference can trigger an unnecessary conflict even when the
local and dashboard configurations are identical.

### Current mitigation

There is no version-aware CLI workaround on the base branch. Compare the
resource configuration manually before resolving a reported conflict.

### Possible fix

Store the observed Vapi version as state metadata, exclude it from resource
files and content hashes, and persist the version returned by successful API
writes. The implementation in this PR follows that approach; see
`src/types.ts:20-23`, `src/pull.ts:65-71`, `src/push.ts:149-162`, and
`src/resources.ts:42-48`.

### Status

**Open.**

---

## Out of scope (intentionally not improvements)

- **State file is identity-only and not git-ignored.** It's intentionally
committed so all collaborators share the same local→UUID mapping.
The proposal in #4 is *additive* — keep identity mappings, add
content hashes.
- **State file is committed; content hashes are not.** It shares resource
UUID mappings and observed Vapi version metadata across collaborators.
Content baselines remain per-developer in the gitignored hash store.
- **`push -- <env>` does not require an interactive confirmation prompt.**
That's a UX choice — adding a prompt would break automation. The right
place to add friction is `--dry-run` (#5).
Expand Down
1 change: 1 addition & 0 deletions src/canonical.ts
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ export const EXCLUDED_FIELDS = [
"orgId",
"createdAt",
"updatedAt",
"latestVersion", // Vapi version metadata is tracked in the state file
"analyticsMetadata",
"isDeleted",
// Computed/derived fields that shouldn't be synced back
Expand Down
6 changes: 3 additions & 3 deletions src/config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -302,10 +302,10 @@ export const STATE_FILE_PATH = join(BASE_DIR, `.vapi-state.${VAPI_ENV}.json`);
// ─────────────────────────────────────────────────────────────────────────────

export const UPDATE_EXCLUDED_KEYS: Record<ResourceType, string[]> = {
tools: ["type"],
assistants: [],
tools: ["type", "latestVersion"],
assistants: ["latestVersion"],
structuredOutputs: ["type"],
squads: [],
squads: ["latestVersion"],
personalities: [],
scenarios: [],
simulations: [],
Expand Down
40 changes: 29 additions & 11 deletions src/migrate-hash-store.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
// One-time migration from the legacy fat state file to the slim
// `name → { uuid }` state file + `.vapi-state-hash/<org>/<uuid>` baseline store.
// `name → { uuid, latestVersion? }` state file +
// `.vapi-state-hash/<org>/<uuid>` baseline store.
//
// This whole module is a REMOVABLE SEAM. When the migration is no longer
// needed, delete this file + `migrate-cmd.ts`, remove the `assertStateMigrated`
Expand Down Expand Up @@ -33,14 +34,17 @@ export interface MigrationResult {
orgs: OrgMigrationResult[];
}

// A legacy entry is anything that isn't already exactly `{ uuid: string }`:
// a bare string (oldest form) or an object carrying extra fields
// (lastPulledHash / lastPulledAt / lastPushedHash / platformVersionId).
// A legacy entry is a bare string (oldest form) or an object carrying fields
// other than the current state metadata (`uuid`, `latestVersion`).
function isLegacyEntry(value: unknown): boolean {
if (typeof value === "string") return true;
if (value && typeof value === "object" && !Array.isArray(value)) {
const keys = Object.keys(value as Record<string, unknown>);
return keys.some((k) => k !== "uuid");
const record = value as Record<string, unknown>;
return (
keys.some((k) => k !== "uuid" && k !== "latestVersion") ||
("latestVersion" in record && typeof record.latestVersion !== "string")
);
}
return false;
}
Expand All @@ -65,8 +69,9 @@ function uuidOf(value: unknown): string | undefined {
}

// Treat a top-level value as a "section" if it's a plain object whose values
// look like state entries (string or { uuid }). This avoids hard-coding the
// section list, so the migration keeps working if a section is added later.
// look like state entries (string or `{ uuid, latestVersion? }`). This avoids
// hard-coding the section list, so migration keeps working if a section is
// added later.
function isSection(value: unknown): value is Record<string, unknown> {
return !!value && typeof value === "object" && !Array.isArray(value);
}
Expand All @@ -93,7 +98,10 @@ async function migrateOne(
let seeded = 0;
let skipped = 0;
let needsSlim = false;
const slim: Record<string, Record<string, { uuid: string }>> = {};
const slim: Record<
string,
Record<string, { uuid: string; latestVersion?: string }>
> = {};

for (const [sectionKey, section] of Object.entries(raw)) {
if (!isSection(section)) {
Expand All @@ -102,7 +110,10 @@ async function migrateOne(
(slim as Record<string, unknown>)[sectionKey] = section;
continue;
}
const slimSection: Record<string, { uuid: string }> = {};
const slimSection: Record<
string,
{ uuid: string; latestVersion?: string }
> = {};
for (const [resourceId, value] of Object.entries(section)) {
if (isLegacyEntry(value)) needsSlim = true;
const uuid = uuidOf(value);
Expand All @@ -118,7 +129,14 @@ async function migrateOne(
skipped++;
}
}
slimSection[resourceId] = { uuid };
const latestVersion =
value && typeof value === "object"
? (value as { latestVersion?: unknown }).latestVersion
: undefined;
slimSection[resourceId] = {
uuid,
...(typeof latestVersion === "string" ? { latestVersion } : {}),
};
}
slim[sectionKey] = slimSection;
}
Expand Down Expand Up @@ -151,7 +169,7 @@ export async function migrateAll(
}

// Guard called at the entry of pull/push/apply. Throws if the given state file
// is still in the legacy format (any entry that isn't exactly `{ uuid }`). A
// is still in the legacy format (hash/timestamp fields or bare-string entries). A
// missing or empty file is new-style by definition (fresh repos / --bootstrap
// must not be blocked). Shape-only check — never inspects the hash store.
//
Expand Down
68 changes: 53 additions & 15 deletions src/pull.ts
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,15 @@ const ENDPOINT_MAP: Record<ResourceType, string> = {
evals: "/eval",
};

function stateMappingForResource(resource: VapiResource): ResourceState {
return {
uuid: resource.id,
...(typeof resource.latestVersion === "string"
? { latestVersion: resource.latestVersion }
: {}),
};
}

// ─────────────────────────────────────────────────────────────────────────────
// Git Helpers (detect locally changed files to skip during pull)
// ─────────────────────────────────────────────────────────────────────────────
Expand Down Expand Up @@ -918,9 +927,9 @@ export async function pullResourceType(
});
if (driftCounts) driftCounts[direction]++;

// The drift baseline now lives in the hash store, NOT in the state
// section that gets wholesale-replaced at end-of-loop. So the
// preserve branches just write the `{ uuid }` mapping and leave the
// The content baseline now lives in the hash store, NOT in the state
// section that gets wholesale-replaced at end-of-loop. Preserve
// branches refresh version metadata while leaving the
// `.vapi-state-hash/<org>/<uuid>` file untouched — the baseline
// survives by construction, no carry-forward needed.
if (direction === "both-diverged") {
Expand All @@ -932,14 +941,15 @@ export async function pullResourceType(
platformHash,
lastPulledHash: baseline,
});
// Preserve the `{ uuid }` mapping. `resolveBothDivergedResources`
// Preserve the resource mapping. `resolveBothDivergedResources`
// will rewrite the baseline to the resolve-mode-appropriate hash
// after the per-type loop; if the operator doesn't pass --resolve,
// the baseline file is left intact so the next pull still sees drift.
const existing = state[resourceType][resourceId];
if (existing) {
newStateSection[resourceId] = existing;
}
upsertState(
newStateSection,
resourceId,
stateMappingForResource(resource),
);
skipped++;
continue;
}
Expand All @@ -964,7 +974,11 @@ export async function pullResourceType(
console.log(
` ⬆️ ${resourceId} (local ahead of dashboard) ${formatDriftLabel(direction)}`,
);
upsertState(newStateSection, resourceId, { uuid: resource.id });
upsertState(
newStateSection,
resourceId,
stateMappingForResource(resource),
);
skipped++;
continue;
}
Expand Down Expand Up @@ -1002,7 +1016,11 @@ export async function pullResourceType(
changedFiles.has(yamlPath)
) {
console.log(` ✏️ ${resourceId} (locally modified, preserving)`);
upsertState(newStateSection, resourceId, { uuid: resource.id });
upsertState(
newStateSection,
resourceId,
stateMappingForResource(resource),
);
skipped++;
continue;
}
Expand All @@ -1029,7 +1047,11 @@ export async function pullResourceType(
const stateMtime = statSync(stateFilePath).mtimeMs;
if (localMtime > stateMtime) {
console.log(` ✏️ ${resourceId} (locally modified, preserving)`);
upsertState(newStateSection, resourceId, { uuid: resource.id });
upsertState(
newStateSection,
resourceId,
stateMappingForResource(resource),
);
skipped++;
continue;
}
Expand All @@ -1051,7 +1073,11 @@ export async function pullResourceType(
console.log(
` 🗑️ ${resourceId} (deleted locally, intent in state — add to .vapi-ignore to stop tracking)`,
);
upsertState(newStateSection, resourceId, { uuid: resource.id });
upsertState(
newStateSection,
resourceId,
stateMappingForResource(resource),
);
skipped++;
continue;
}
Expand Down Expand Up @@ -1113,7 +1139,11 @@ export async function pullResourceType(
` ⚠️ ${resourceType}/${resourceId}: failed to hash post-write disk form; falling back to in-memory hash (may produce phantom drift on next pull)`,
);
}
upsertState(newStateSection, resourceId, { uuid: resource.id });
upsertState(
newStateSection,
resourceId,
stateMappingForResource(resource),
);
await writeBaseline(
VAPI_ENV,
resource.id,
Expand Down Expand Up @@ -1209,7 +1239,11 @@ async function resolveBothDivergedResources(options: {
// No file write — local is preserved. Set the baseline to the platform
// state at resolve time so the next classifyDrift correctly reports
// `local-ahead` instead of re-flagging `both-diverged`.
upsertState(section, entry.resourceId, { uuid: entry.resource.id });
upsertState(
section,
entry.resourceId,
stateMappingForResource(entry.resource),
);
await writeBaseline(VAPI_ENV, entry.resource.id, entry.platformHash);
continue;
}
Expand All @@ -1235,7 +1269,11 @@ async function resolveBothDivergedResources(options: {
` ⚠️ ${entry.resourceType}/${entry.resourceId}: failed to hash post-write disk form; falling back to in-memory hash (may produce phantom drift on next pull)`,
);
}
upsertState(section, entry.resourceId, { uuid: entry.resource.id });
upsertState(
section,
entry.resourceId,
stateMappingForResource(entry.resource),
);
await writeBaseline(
VAPI_ENV,
entry.resource.id,
Expand Down
Loading