Implemented in crates/sentinel-api (HTTP/1.1 on the crate's own bounded acceptor — 64 connections (at most 16 from one peer address, an IPv6 /64 counted as one; a configured reverse proxy is exempt), 8 handler permits of which at most 3 go to transfer bodies, and a separate budget of 32 parked long polls (at most 8 of one user, whatever credential it presents) that hold a connection but give their handler permit back while they wait, so 5 permits always serve everything else; deadline-bounded heads/bodies/writes) and sentinel api … (the CLI client in crates/sentinel). sentinel server opens the API on api_listen (default 127.0.0.1:7080) beside the worker link. The human web interface is a separate process in front of it (web interface).
Everything a person, the CLI, the web interface or an agent does goes through the same routes, authenticated the same way and authorized by the same store predicates. The web interface is not a second path into the controller: it renders pages and relays updates by calling /api/v1/… with the browser's own session cookie (web interface).
Plain HTTP on loopback by default; TLS and exposure are a reverse proxy's job. The session cookie is __Host-sentinel_session (Secure, HttpOnly, SameSite=Strict), which a browser accepts from localhost or over HTTPS only — deliberately.
A response body that fails mid-stream, or ends shorter than its declared length, closes the connection (Request::respond sets close and, on a failed write, shuts the socket) instead of reusing it for another request head. Otherwise the client would read the next response out of the middle of the unfinished body, a framing desync; keep that branch when changing respond.
| Credential | How | Mutations |
|---|---|---|
Authorization: Bearer sntl_… (API credentials) |
tokens::authenticate: one digest lookup; scope is a ceiling on the account's live membership |
allowed; no CSRF (nothing sends a bearer ambiently) |
| session cookie (local authentication) | local_auth::authenticate |
require the x-sentinel-csrf header with the session's CSRF secret, else forbidden |
Authorization: Bearer sntl_at_… (OAuth) |
oauth::authenticate_access: one statement (token, grant and account by key); the grant's scopes are the ceiling |
allowed; no CSRF |
The credential yields a Principal; every route then asks the store — auth::require_repo for anything under a run, auth::list_repos, tenancy::pools_for_tenant and workers::in_pool under an Authority — so a resource the caller may not see is not_found, never revealed. A bearer credential can never step up. Every route also requires the scope in its row below: an OAuth access token carries exactly its grant's scopes, while a session or sntl_ credential carries the scopes its permissions imply (read → every :read scope and secrets:metadata, run → runs:write and cache:write, the administrative bits one to one), so their authority is unchanged. A missing scope is 403 forbidden with details.scope and WWW-Authenticate: Bearer error="insufficient_scope", scope="…"; every 401 from /api/v1 carries WWW-Authenticate: Bearer realm="sentinel", resource_metadata="…" naming the RFC 9728 location of {issuer}/api/v1 (OAuth), plus error="invalid_token" when a token was presented (the login and hook routes excepted). A credential the store is too busy to check is rate_limited (429, details.retry_after_ms), never 401, so a client does not take overload for a dead credential. A session's idle deadline slides (one write per session per five minutes) as it is used.
All under /api/v1, JSON in and out, errors as sentinel.error/1 (protocol); bodies bounded before they are read, each route class by its own limit (body limits); the intake routes bound their own bodies (intake).
| Route | Auth | Scope | Does |
|---|---|---|---|
GET /health |
none | — | {ok:true, github_sign_in}: whether this deployment offers GitHub sign-in (what a sign-in page shows) |
GET /ready |
none | — | readiness: 200 {ready:true, checks, degraded} or 503 while shutting down or when the store cannot read or commit; checks has stopping, store_reads, store_commits; degraded lists codes only (external_copy_backlog, disk_below_watermark, external_copy, backup_failing, backup_incomplete, no_workers_connected) (monitoring) |
GET /metrics (also /metrics outside /api/v1) |
platform admin | platform:admin |
the Prometheus text exposition (text/plain; version=0.0.4): HTTP, auth refusals, queue, workers, link, storage, external copy, backup and GitHub check publication figures; no tenant or repository labels (monitoring) |
POST /login {username,password} |
none | — | password login (username is a login name or a verified email address, any spelling of it; an unverified or unknown address is the same refusal as a wrong password) → Set-Cookie session (Max-Age is the session's absolute life; the idle deadline is enforced and slid by the server), body {user, csrf}; every non-accepted outcome is one unauthenticated. Each client (counted as the other unauthenticated budgets are) gets a burst of 10 attempts and then one per 3 s, all clients together 10 per second; past it the answer is 429 rate_limited with details.retry_after_ms, before any password hash. A client that fails 10 times against one account is refused for the lockout window without a hash, a writer transaction or an audit row, and one failure after that window locks it again; other clients still reach the account, so a stranger cannot lock the super admin out by guessing. The account-wide lock of local authentication stays as the ceiling for many clients guessing together, at four times the per-client threshold. The body is read without a handler permit (below). Against login CSRF the body must be content-type: application/json (else invalid_request) and a present Origin must be the issuer's origin (else forbidden) |
POST /logout |
session + CSRF | — | ends the session, clears the cookie |
GET /setup |
none | — | {open: bool, mail_configured: bool} and nothing else about the instance: whether first-run setup is still available (no super admin yet) and whether the controller has a Resend key and sender (mail_configured is false once setup is closed). no-store; once closed it stays closed (one atomic read, no budget). While setup is open the route draws on its own per-client budget (burst 10, then one per 2 s; 20 per second across clients) and answers 429 rate_limited with details.retry_after_ms past it; an open answer comes from a one-second cache, so a closure by sentinel admin bootstrap (another process) can show up to a second late, while the wizard closing it in the same process shows at once. Five wrong guesses on POST /setup lock the wizard, not this route: the escape is sentinel admin bootstrap |
POST /setup {token, email, password, registration, instance_name, display_name?, tenant_creation?, base_url?, personal_namespace?, organization?} |
the one-time setup token from the host; application/json and this deployment's Origin; per client burst 5, then one per 10 s (deployment 1/s) |
— | complete first-run setup in one commit (admission): the super admin (address stored verified, so it signs in at once), the registration policy (closed, invite-only or approval-required) and tenant_creation (super-admin-only default, or approved-users), the instance name and base URL, and an optional personal namespace and first organization; closes the bootstrap latch, kills the token and signs the administrator in → 201 {user, csrf, organization, personal_namespace, mfa_enrollment:"available"} with the session cookie. A wrong, expired, absent or locked-out token is one 403 "setup token refused"; setup already complete is 404; a taken address or name 409; a bad field 400. Second-factor enrollment is offered, not forced: POST /mfa/enroll from the returned session |
POST /setup/mail-test {token, to} |
the setup token; same rules and budget as POST /setup |
— | send one test message to to through the controller's configured Resend key and sender (nothing is stored; the fixed Resend host is the only destination). 200 {sent:true} or 200 {sent:false, failure:"not_configured"|"invalid_settings"|"transient"|"permanent", reason}: not_configured when no key and sender are set, invalid_settings for a to that is not an address, transient for a network error, 429 or 5xx, permanent for a key problem (401/403) or a refused message (422, an unverified sender domain). reason is short and carries at most the status and Resend's error name, never the key or an address. A failure never blocks setup. Unknown fields are 400. One test at a time (429 while another runs) |
POST /orgs {slug} |
browser session + CSRF; per client burst 5, then one per 10 s (deployment 2/s) | — | create an organization the caller administers when the tenant-creation policy is approved-users and the account is approved (a platform administrator always may) → 201 {slug, role:"admin"}. 400 for a reserved or malformed name (2–63 bytes of a-z, 0-9, -), 409 for a taken one or for the name of a GitHub account the App is installed on (the slug check inherently says so; the budget bounds probing; held names), 403 when the policy or the account does not allow it, quota_exceeded past 3 an hour or 10 in all |
GET /tenants/{slug}/settings |
member | runs:read |
the namespace's kind (personal or organization), membership (app, or github when GitHub confirms it), github_installation, the caller's role and created_ms; for a GitHub-backed organization the caller's own github_sync {confirmed_ms (when GitHub last confirmed the membership, or null), check_due (a check is waiting to run), unanswered (passes in a row GitHub could not answer: rate limit, outage, missing App permission — existing access is kept for the grace period and nothing new is granted)}; for an administrator also auto_join {applies, domains}: read-only, the rule being the operator's SENTINEL_AUTO_JOIN_DOMAINS and never applying to a personal or GitHub-backed namespace. A namespace the caller cannot see is not_found like every slug route |
POST /register {email, password, display_name?, invitation?} |
none; application/json and this deployment's Origin; per client burst 5, then one per 20 s (deployment 2/s) |
— | self-service sign-up by email when the registration policy allows it (invitation is an sntl_… secret). 202 {status:"accepted", verification:"email"|"operator"} for a new address, an address already verified by another account and a merely claimed one alike; 403 when registration is closed, an invitation is required or the invitation cannot be used; 400 for a malformed address or password. The address signs in only once verified. verification says whether the link is mailed or the operator hands it over (admission) |
POST /email/verify {token} |
none (the link is opened from a mail client); same JSON/Origin rules; per client burst 8, then one per 15 s |
— | spend a verification token: single-use, 24 h, tied to its address → {verified:true, joined} (joined when it also redeemed an email invitation); an unknown, spent, expired, voided or "verified elsewhere first" token is one invalid_request |
POST /email/resend {email} |
none; as above | — | the "I never got it" form: always 202 whatever the address, with the per-account and per-recipient mail limits applied silently |
POST /me/email {email} |
session + CSRF | — | add an address to the account or ask to change it: the new address is a claim until its link is opened and the old verified one stays the identity until then; 202 for every outcome (another account holds the address, the allowance is spent, a link was queued), so the reply never reveals whether an address is taken; only the per-client budget answers 429; 400 for a bearer or OAuth credential |
POST /me/email/resend |
session + CSRF | — | send the pending link again; 409 when nothing is pending, 429 when the allowance is spent |
POST /invitations/accept {token} |
session + CSRF | — | present an invitation as the signed-in account: a link-form one joins (200 {status:"joined", tenant}); an email-addressed one joins at once when the account holds that address verified, otherwise 202 {status:"verification_sent"} and the join happens when the link is opened; already_member leaves it unspent; unknown, spent, expired, revoked, addressed elsewhere or for a GitHub-backed organization is one not_found; so is a host-local invitation to a tenant presented by a platform administrator, which stays unspent (admission) |
POST /tenants/{slug}/invitations {email, role?, expires_in_secs?, reason?} |
tenant admin | tenant:admin |
invite a person by email to an app-created organization (role reader by default, expires_in_secs 1 s–30 d, default 7 d) → 201 {id, role, expires_ms, delivery:"email"}, or delivery:"link" with the one-time token when this deployment sends no mail; 400 for a GitHub-backed organization, a bad address, role or lifetime; 50 per organization per hour |
GET /tenants/{slug}/invitations |
tenant admin | tenant:admin |
the organization's invitations, newest first: id, email, role, expires_ms, state (pending, redeemed, revoked, expired); never a secret |
DELETE /tenants/{slug}/invitations/{inv} |
tenant admin | tenant:admin |
revoke an unspent invitation of this organization (optional body {reason}); another organization's is not_found. Creating or revoking as a platform administrator who is not an administrator-member needs fresh step-up and a reason (step-up), audited as platform-override |
GET /me |
any | — | who the credential is and how: user, username (local login name or null), email and email_verified (the account's address, verified or not; null/false without one), via (bearer, session, oauth), super_admin, tenant/repo narrowing, scopes (names), stepped_up (a session inside its step-up window), mfa (the account has TOTP), and for an OAuth token its grant and expires_ms |
GET /tenants |
any | runs:read |
the tenants the caller is a member of, by slug, with its role in each, the namespace's kind (personal, organization) and its membership source (app, github) ({tenants:[{slug, role, kind, membership}], super_admin}); a credential narrowed to a tenant lists that one only |
POST /step-up {method, code} |
session + CSRF | — | a browser session proves presence again: method is totp, recovery or password (the last only for an account without TOTP) → {stepped_up:true, until_ms}; a wrong proof is one forbidden with details.step_up, counted by the store against the session and the factor; a bearer or OAuth token is invalid_request (they never step up); needs the deployment master key |
GET /me/security |
browser session | — | what an account page shows, metadata only: password (the account has a local password), mfa {enrolled, pending (an enrollment begun and not confirmed), recovery_codes_remaining}, identities [{provider, linked_ms}] (never a subject), github_link (the deployment offers linking) |
POST /mfa/enroll {password?} |
session + CSRF | — | begin enrolling an authenticator app → {provisioning_uri} (an otpauth:// URI carrying the seed, shown once; no-store). An account with a password must send it (invalid_request without; a wrong one is forbidden enrollment refused, counted against the session, five revoke it); an account without one (GitHub-only) needs no body but a session issued within the step-up window, else forbidden with details.reauthenticate. 409 conflict when a factor is already confirmed (replace it by removing it under step-up first); a repeat before confirming supersedes the earlier seed. Nothing about authentication changes until confirmed. Draws on the sign-in admission budget when a password is hashed; needs the deployment master key |
POST /mfa/enroll/confirm {code} |
session + CSRF | — | a current six-digit code from the app proves it holds the seed → {recovery_codes:[10 strings], stepped_up:true, until_ms}; the recovery codes are shown once and the session is stepped up. A wrong code is forbidden (code refused), audited as a failed proof and counted against the session — five revoke it, and the enrollment stays pending for the next sign-in; a body that is not 1–64 bytes is invalid_request; none begun is not_found. The code's time step is spent, so the same code cannot step up afterwards |
POST /mfa/recovery-codes |
session + CSRF, stepped-up | — | replace the recovery codes with ten new ones → {recovery_codes}, shown once; the old set stops working. A cold session gets forbidden with details.step_up; no enrolled factor is not_found |
POST /password {current, new} |
session + CSRF | — | change the local password. The current password is always required; with a second factor enrolled a fresh step-up is too, asked for before the password is judged (forbidden with details.step_up). Success → {ok:true, signed_out:true}, every session and OAuth grant of the account is revoked (this one included) and the response clears the cookie, so sign in again. A wrong current password — or an account with no local password — is one forbidden (password change refused); wrong passwords count against the session (five revoke it) and never against the second factor. A new password outside 12–256 bytes is invalid_request. Draws on the sign-in admission budget before any hash |
POST /identities/github/link |
session + CSRF, stepped-up, GitHub sign-in configured | — | begin linking a GitHub account to the signed-in account → {authorize_url, expires_in} and the __Host-sentinel_signin cookie; the browser navigates to authorize_url, and the rest is the sign-in round trip ending on / with the session unchanged. A cold session gets forbidden with details.step_up; the callback hop links only in the session that began it, and an identity another account holds is refused, never moved. Rate-limited per client like the sign-in routes |
GET /grants |
browser session, or a token | tenant:admin unless the token lists its own grant |
the caller's own OAuth grants as metadata ({grants:[{id, user, client_id, kind, scope, tenant, repo, name, created_ms, expires_ms, last_used_ms, revoked}]}, newest first, at most 100); never a token (OAuth). A browser session or a token holding tenant:admin lists every grant of the account; any other OAuth token lists only the grant it was issued under, so a runs:read token cannot enumerate its siblings. A sntl_ credential without tenant:admin has no grant of its own and gets forbidden naming the scope |
DELETE /grants/{grt} |
owner, the home tenant's admin for a service grant, or platform admin | tenant:admin, unless the token revokes its own grant (a browser session needs no scope) |
revoke the grant (reason 7, audited) → {grant, revoked:true}; a token without the scope that names another grant is forbidden with details.scope; anyone else without authority over it not_found; its access tokens are then 401 and refresh tokens invalid_grant |
POST /tenants/{slug}/service-accounts {name, role?, reason?} |
tenant admin, stepped-up session | tenant:admin |
create a service principal confined to the tenant (role reader or operator, default operator) → 201 {user, name, role}; audited ServiceAccountCreated. Minting credentials needs a browser session that proved presence within the step-up window, re-checked in the writing transaction: a bearer or OAuth token, even a tenant administrator's, is forbidden with details.step_up (step-up). reason (1–100 characters) is required of a platform administrator who does not administer the tenant, and audited as platform-override |
PUT /tenants/{slug}/service-accounts/{usr}/repos/{name} {access:["read","run"], reason?} |
tenant admin, stepped-up session | tenant:admin |
set the account's access to one repository of the tenant ([] withdraws) → {user, repo, access}; same step-up and reason rule, withdrawal included; an id that is not an active service account of the tenant (a person, another tenant's account, nobody) is the same not_found on this and the two routes below, so the route cannot probe which usr_ ids exist |
POST /tenants/{slug}/service-accounts/{usr}/grants {name, scope, repo?, expires_in_ms?, reason?} |
tenant admin, stepped-up session | tenant:admin |
issue a service grant (1 h..90 d, default 30 d; never tenant:admin/platform:admin) → 201 {grant, refresh_token, expires_ms, scope}; the sntl_rt_ token is shown this once; same step-up and reason rule. Grants a person minted are revoked when that person's sessions are revoked (step-up) |
GET /tenants/{slug}/service-accounts/{usr}/grants |
tenant admin | tenant:admin |
the account's grants as metadata; a revoked grant stays listed until the next maintenance purge (about ten minutes) |
GET /github/app |
any signed-in credential | — | whether a GitHub App is in force and its public facts: {configured, source ("files", "manifest" or null), app, webhook_reachable, reason, can_create, install_url}; a platform administrator also sees public_url, webhook_url and create_refusal (GitHub App) |
POST /github/app/manifest {owner:{kind:"user"} or {kind:"organization", login}, public?} |
browser session + CSRF, platform administrator, stepped-up | — | begin one-click App creation → {action_url, manifest, state, expires_ms}; 409 with details.reason (files, configured, no_public_url, public_url_has_path, no_master_key), 403 with details.step_up, 400 for a bad owner; the body is at most 4 KiB |
POST /github/app/conversion {code, state, tenant?, reason?} |
browser session + CSRF, platform administrator | — | spend the single-use state, exchange GitHub's code, seal and store the App and put it in force without a restart → {app, install_url}; 404 for a refused state, 409, 502 upstream_failed |
POST /github/app/installation {installation_id, state, setup_action} |
browser session + CSRF | — | bind an installation GitHub reports to the tenant the state names, after the App itself confirms it → {tenant, account}; 404 for a refused state, 403 with details.reason, 409 (bound_elsewhere, not_configured), 502 upstream_failed |
POST /tenants/{slug}/github/installation-url {reason?} |
browser session + CSRF, tenant admin (a platform administrator outside the tenant: fresh step-up and a reason) |
— | mint a one-time installation link → {install_url, expires_ms}; 404 unknown tenant, 409 not_configured, 400 when the reason is missing |
POST /hooks/github |
App webhook signature | — | GitHub webhooks (intake): raw-body HMAC-SHA256, delivery dedup, push and pull_request intake, ping probe. not_found until <data_dir>/github-webhook.json exists |
POST /intake/{repo} |
repository hook secret | — | generic ref updates (intake): bounded JSON, dedup, durable acceptance → 202 with the delivery id |
GET /tenants/{slug}/repos |
member | runs:read |
repositories visible to the caller; a tenant the caller has no part in is not_found, exactly like one that does not exist |
GET /tenants/{slug}/repos/{name}/runs?limit&before&ref|pr|sha |
read |
runs:read |
newest runs first, a keyset page over (created_ms, id) (limit default 100, max 500): before=run_… continues after that run of the same repository (another repository's run is not_found), and next is the cursor for the following page — null on the last one, so no empty follow-up request is needed. Pages stay stable when runs share a millisecond or new runs arrive meanwhile. At most one filter, each a range on its own index (migration 45): ref exactly (a value without refs/ means refs/heads/<value>; a pull request's ref is its base branch), pr (a pull request number), sha (a lowercase commit prefix of 7 to 64 hex digits); another combination, a repeated parameter or a shorter prefix is invalid_request. Each run carries id, sha, created_ms, state, trigger, ref, pr |
GET /tenants/{slug}/sync?since_ms&after&limit |
member | runs:read |
GitHub synchronization per repository the caller may read (repository-id keyset, limit at most 100): check publications still pending and the oldest one's oldest_pending_ms (its last change), refused since since_ms (default the last 24 h) with last_refused_ms, last_published_ms, and webhook or generic deliveries not yet resolved (open_deliveries, oldest_delivery_ms); plus now_ms. Each figure is one probe of an index over rows in that state (migration 45) |
POST /tenants/{slug}/repos/{name}/runs {pipeline, source:{repo,sha,ref}} |
run |
runs:write |
compile, pin, create the run and its jobs, resolve every digest-pinned image, record manual provenance, wake the dispatcher → 201 run status. Idempotency-Key replays the same run (200) and refuses a different body (idempotency_mismatch). A write the controller stopped waiting for is 503 outcome_unknown (not retryable as is: the run may still be created); repeat it only with the same key, or list runs first. Every image must be pinned by digest. A pipeline declaring secrets, secret_files or registry_auth is 403 forbidden (details.reason manual_run_secrets): manual runs never receive secrets (secrets). On a bound repository allowed_refs gates the ref name and the remote, not the commit: RUN is effectively read of any commit the binding's credential can fetch by hash (sources) |
GET /runs/{id} |
read |
runs:read |
the run and its jobs: state, failure class, trigger (push, tag, pull_request, manual), cancel flag, newest attempt, log_state (pending/incomplete/complete of that attempt's durable log end, null until an attempt exists), fence, phase timestamps |
GET /runs/{id}/pipeline |
read |
runs:read |
repository-authorized sentinel.explain/1 view of the immutable compiled pipeline in that run's stored spec; includes declared jobs, budgets, caches, artifacts, secret names and unresolved expressions, never secret values |
GET /runs/{id}/wait?since&timeout_ms |
read |
runs:read |
long poll for a change → {version, changed, finished, run} with run the same document as GET /runs/{id}. version is 16 hex digits (64-bit FNV-1a over the run's cancel flag and each job's id, state, fence, cancel flag and newest attempt's log state). Answers at once without since, when since differs, or when every job is terminal; the answer's version, finished and run come from one read snapshot, so finished: true always describes the run beside it; otherwise parks on the store's commit notifier — re-reading only the version, at most every 10 ms — until something visible changes or timeout_ms (1–25000, default 25000) passes (changed:false). At most 32 subscribers (run waits and logs?wait=1 together) park at once, and at most 8 of one user; beyond that the poll is rate_limited with details.retry_after_ms. A parked wait re-authorizes when the tenant's authorization epoch moves and ends when its client hangs up (live updates) |
POST /runs/{id}/cancel |
run |
runs:write |
cancel_run (cancellation) |
POST /jobs/{id}/cancel |
run |
runs:write |
cancel → terminal, requested or alreadyterminal |
POST /jobs/{id}/rerun |
run |
runs:write |
a new attempt of a finished job, which also returns the jobs downstream of it that ended skipped to blocked so they are decided again, and clears the run's cancel flag (rerun semantics); conflict for a running or cancelled one |
GET /attempts/{id}/logs?after|cursor&limit&wait=1&step |
read |
logs:read |
frames after a position from the segmented store the controller writes (logs) → {attempt, frames:[{seq, step, stream, text}], complete, gaps, next_after, next}. The position is after (a sequence number) or cursor (the versioned c1 cursor of protocol, bound to the tenant and the attempt — the answer's next); not both. A malformed after, limit or step is invalid_request; a malformed, foreign-tenant or other-attempt cursor is invalid_cursor. A page holds at most limit frames (default 100, max 500) and 1 MiB of frame payload, and decodes at most 16 MiB past the position; a page cut by any bound carries next_after (continue from it — next is the same position as a cursor), null when the page reached the end of what is stored. step serves one step only; wait=1 parks up to 25 s for more — on the log store's append notifier, re-reading only when this attempt's log grew — holding one of the 32 subscriber slots it shares with run waits (beyond them, or beyond the user's share of 8, a parked poll is rate_limited with details.retry_after_ms); a poll with frames to return, a complete log or a finished step never parks; step_done says, with a step filter, that the log has moved past that step (no frame of it will come); complete and gaps say when the log is closed (a live log's gaps include the holes stored so far); legacy flat logs still read; a log retention removed (retention) answers at once with no frames, complete: true and expired_ms, when it was removed — as does its search |
GET /attempts/{id}/logs/search?q&after&limit&carry |
read |
logs:read |
bounded literal search: q is 1–256 bytes, percent-decoded (+ is a space), matched byte for byte with memmem, one report per line. A string split across consecutive frames of the same step and stream is found too, once, and reported with the frame it ends in — whatever lies between the pieces: other frames, a sealed log segment, or the cut between two requests (the scan keeps each stream's open line as its last len(q) − 1 bytes). Frames past after are scanned in stored order, at most 4 MiB of payload (a frame that would pass it waits for the next request) and limit matching lines (default 100, max 500; checked between frames) per request → {attempt, matches:[{seq, step, stream, text}], next_after, next_carry, complete}; text is the matching line, at most 512 bytes around the match (from the match itself when it began in an earlier frame). When a bound stopped the request, next_after and next_carry (opaque hex, at most 1,066 characters) are where it resumes: pass them back as after and carry with the same q. A carry for another after or q length, or malformed, is invalid_request. Without carry the state is rebuilt from the segment holding after (the sealed one when after ends it), so a line begun before that segment can be missed or repeated — older clients only. complete means the log is finished and was scanned to its end |
GET /attempts/{id}/failure?budget&limit&after|cursor |
read |
logs:read |
bounded failure context → {schema:"sentinel.failure/1", authoritative, failed_step, timings_ns, cache, reports, artifact_reports, tail, truncated, omitted_diagnostics, log_complete, log_gaps, next_cursor}. The authoritative job state is returned only while this attempt still owns the job: its fence is the job's, the job is leased under it and not queued. From the moment of a rerun (before any new lease) or a lapsed offer, the older attempt reports current_attempt:false with no job state. With no cursor, Go test/rustc/cargo JSON parsing starts at sequence 0 and reads at most 4 MiB or 1,024 frames; when that page is cut, the failed step's newest window (at most 4 MiB/1,024 frames, sought through the sparse index so a later always() step cannot displace it) is parsed too and is the excerpt's source. Each log report carries window (first/last frame sequence parsed). Legacy flat logs have no sparse index, so their excerpt begins at the available scan position and pages forward. With artifacts:read, report files the attempt published as artifacts are parsed as advisory reports with an artifact {name, path} (diagnostics); without it artifact_reports.status is scope_required. A parser report carries frame evidence references and SHA-256 provenance; it is incomplete unless its selected stream was completely covered; an invalid diagnostic is dropped and counted in parse.malformed_records, never the whole report. Failures and errors are budgeted before warnings and informational records. Text defaults to 8 KiB, budget accepts 1–65,536, and limit defaults to 20 diagnostics (max 20). With no budget the whole compact JSON body is at most 8 KiB: tail frames go first, oldest first, and a frame larger than the remaining excess keeps its newest bytes and carries cut:true; then diagnostic messages longer than 256 bytes are shortened from the end, last diagnostic first, keeping at least their first 256 bytes and carrying message_cut:true; whole diagnostics are dropped (and counted in omitted_diagnostics) only after that. With an explicit budget the body never exceeds 64 KiB. after or a tenant-and-attempt-bound c1 cursor pages forward through the indexed log, never both; next_cursor continues a cut scan page. log_complete is false for a live log or any declared gap. Control characters other than newline and tab become U+FFFD in excerpts and diagnostic messages alike; a run of binary frames is one {binary:true, sequence, end_sequence, bytes} entry that spends no text budget; report text is untrusted data |
GET /attempts/{id}/steps |
read |
runs:read |
the attempt's terminal summary as timing: timings_ns (checkout, checkout_fetch, checkout_materialize, image_pull, container_start, steps, finalize — each only when measured), image_present, guest_image ({digest, rootfs_bytes, built, build_ns} for a microVM attempt: the pinned digest its guest root file system was built from and checked against; null otherwise), each step's index, id, outcome, exit_code/signal and duration_ns, and detail; {present:false} before the attempt reported |
GET /attempts/{id}/summary |
read |
cache:read |
the attempt's cache records from its terminal summary → {attempt, present:true, image_present, caches:[{name, class, outcome, lookup_ns, lock_wait_ns, clone_ns, first_touch_ns, files, bytes, copied_bytes, reflink, commit_ns, staged_bytes, reused_bytes, dirty_bytes, publish, costly_hit}]}, or {attempt, present:false} before the attempt reported |
GET /runs/{id}/artifacts |
read |
artifacts:read |
{tenant, artifacts}: the owning tenant's slug (for the objects route) and every artifact row of the run: name, job, attempt, captured/absent/failed, entries, bytes, retention and creation (storage) |
GET /runs/{id}/artifacts/{arf} |
read |
artifacts:read |
one row with the owning tenant slug plus, when captured, its immutable manifest: version, digest, payload length and each entry's path/digest/len/mode; entry bytes download via GET /tenants/{slug}/objects/{digest} |
GET /workers?tenant=slug |
member | runs:read |
the pools the tenant may use and their workers, each with its negotiated protocol and the software it announced, connected from the live fleet and, for a connected protocol-7 worker, transport: path (unknown, direct or relay), reconnects, bytes_in, bytes_out (cumulative over the session's control and bulk connections; throughput is their rate between two reads), and rtt_ns and helper_version only when measured — never an address or a key. From the store, each worker's reported capacity (cpu_millis, memory_bytes, disk_bytes; absent until reported), held_attempts and held (the reservations of the attempts it holds, as totals — which tenants hold a shared worker is not reported), free (capacity left after every reservation on its host), host_workers (workers sharing its host, when known), labels, cache_bytes (when reported), draining and draining_since_ms. For protocol 7 and later, disk_bytes: 0 explicitly means no scratch disk and no job placement; legacy workers below protocol 7 omit zero disk |
GET /queue?tenant=slug&limit |
member | runs:read |
the waiting jobs in repositories the caller may read, oldest first, each with its run, repo, age_ms and the reason it is not running: dependency, policy (with its detail), no_matching_worker (with cpu_short/memory_short), disk_short (with disk_short, the missing scratch bytes, as a number), worker_offline, capacity, and the fleet constraints arch_mismatch, label_missing, runtime_missing (no worker in reach runs the os and isolation the job states; both named, isolation null when only the OS is stated), drain, concurrency_limit, fairness_hold and locality_wait, and ready (nothing holds it and a connected worker has the room now; the next dispatch pass places it unless older work takes that room first), or storage for such a job while the controller places nothing (with detail disk_below_watermark or external_copy_backlog, s3); placement_paused carries the same detail for the whole page, null when placement runs. Quantities are numeric fields, never text to parse; a reason the store gains before the API names it still reports its snake-case code and its fields. Bounded to limit (default 100, max 500): the limit is applied in the store query and only the returned jobs are explained, total is the number of those visible waiting jobs (an index count with the same repository-visibility predicate as the page) and truncated marks total exceeding the page, so a ten-thousand-job queue is neither a ten-thousand-job document nor ten thousand explanations |
POST /workers/{wrk}/drain |
platform admin | platform:admin |
the worker keeps the attempts it already holds and takes no new offers → {worker, draining:true}; work it could have taken stays queued with reason drain |
POST /workers/{wrk}/undrain |
platform admin | platform:admin |
the worker is offered work again → {worker, draining:false} |
GET /admin/tenants?after&limit |
platform admin | platform:admin |
every tenant by slug (keyset after, at most 100): id, slug, kind, active, created_ms, members, usage_bytes, quota_bytes (the effective quota, 0 = unlimited) and quota_set (its own row, not the default) |
POST /admin/tenants {slug} |
platform admin | platform:admin |
a new organization → 201 {id, slug} (audited); it has no members until an administrator is added through the members route; a taken slug is conflict |
POST /admin/tenants/{slug}/suspend|reactivate |
platform admin, stepped up | platform:admin |
suspension revokes the tenant's credentials and invitations and cancels its jobs in one transaction → the counts; both are privileged: without a session stepped up in the window, forbidden with details.step_up |
PUT|DELETE /admin/tenants/{slug}/quota {bytes} |
platform admin | platform:admin |
set the tenant's storage quota, or return it to the deployment default → {quota_bytes, quota_set} |
PUT /admin/tenants/{slug}/storage {quota_bytes?, log_retention_secs?, artifact_retention_secs?} |
platform admin | platform:admin |
replace the tenant's storage policy; an absent or null field inherits the deployment's configuration, all absent removes the policy; retentions are 1 h to 366 d; a lowered retention applies to logs and artifacts already stored → {policy, effective} (retention) |
GET /admin/storage |
platform admin | platform:admin |
the deployment's disk: filesystem_bytes, free_bytes, metadata_bytes, admission_open, inflight_bytes, watermarks (reserve_bytes in force, configured_reserve_bytes, low_watermark_bytes, high_watermark_bytes, log_floor_bytes), stored_bytes (every tenant's objects, uploads and logs) the configured limits (quota_bytes, tenant_quota_bytes, log_retention_ms, artifact_retention_ms, run_artifact_bytes), and s3, the external copy's replicator (s3; null without [s3]) |
GET /version |
any credential | any | the server version, api (v1), the schema of the database and of the binary, and the worker_protocol range (min, max) it accepts (upgrades) |
GET /admin/diagnostics?integrity |
platform admin | platform:admin |
the sanitized diagnostic bundle sentinel.diagnostics-bundle/1: schema, row counts by state, workers (name, arch, protocol, software, ages, drain), queue age, the last day's audit events as counts by code, and the runtime (version, process, readiness, link counters, HTTP counters, storage, external copy, backups without their target); no names of tenants, repositories or people, addresses, paths, credentials or log text. integrity=1 adds SQLite's quick_check (monitoring) |
GET /admin/backups |
platform admin | platform:admin |
configured, the scheduler (target, interval_secs, keep, running, last_success_ms, last_backup counts, last_failure_ms, last_failure) and every backup, newest first: id, started_ms, took_ms, schema, version, metadata_bytes, objects, objects_remote_only, key_required, key_ids (backup) |
POST /admin/backups |
platform admin | platform:admin |
take a backup now on the scheduler's thread → 202 {started:true}; conflict while one runs or without [backup] |
GET /admin/registrations?limit |
platform admin | platform:admin |
pending applications, oldest first |
POST /admin/registrations/{usr}/approve|reject |
platform admin | platform:admin |
decide an application; rejecting an already approved account is privileged |
GET|PUT /admin/policy |
platform admin (PUT stepped up) | platform:admin |
the admission policy: registration (closed, invite_only, approval_required), tenant_creation (super_admin_only, approved_users), installation_binding (super_admin_only, tenant_admins) |
GET /admin/pools |
platform admin | platform:admin |
every pool with kind, owner (a dedicated pool's tenant), grants (tenant slugs of a shared pool), workers and active |
POST /admin/pools {name, kind, tenant?} |
platform admin | platform:admin |
a shared pool, or a dedicated one owned by tenant → 201 {id, name} |
PUT|DELETE /admin/pools/{pool}/grants/{slug} |
platform admin | platform:admin |
grant a shared pool to a tenant, or withdraw it |
POST /admin/worker-enrollments {pool, transport?, lifetime_ms?, cpu_millis?, memory_bytes?, disk_bytes?, priority?} |
browser session, fresh step-up | platform:admin |
issue a single-use, pool-bound worker enrollment (worker bootstrap) → 201 {id, pool, transport, expires_ms, host, release_version, bootstrap_path, bootstrap_sha256, redeem, secret, tailcat_address}. bootstrap_sha256 is the SHA-256 (64 hex) of the exact bytes bootstrap_path serves, so a page can show that its copy is intact; redeem is https when [worker_bootstrap] api_url is set (the worker redeems at the API) and link otherwise (the worker presents the secret with its first link hello and a Tailcat worker needs node-key below). transport is tailcat (the default) or direct; lifetime_ms defaults to an hour, at most a day; the ceilings and priority (-100..100) are the controller's approval, never the worker's. secret (and, for Tailcat, tailcat_address) is returned once and never served again; the response is no-store. 409 when [worker_bootstrap] is not configured, when a Tailcat enrollment is asked of a controller with no Tailcat helper pinned or no address yet, 404 for an unknown or inactive pool, 403 with step_up without a fresh step-up (a bearer token never steps up), 429 past 5 enrollments in a burst and one per 6 s per administrator. Audited as worker-enrollment-issued |
POST /worker-enrollments/redeem {secret, worker, name, fingerprint, arch, node_key?} |
none: the enrollment secret is the credential, in the body only | none | a machine redeems its enrollment over HTTPS and, in one transaction, binds its TLS fingerprint (64 hex) and, for a Tailcat enrollment, its node_key (nodekey:<64 hex>) to the enrollment's pool → 200 {worker, pool, transport, host, controller, controller_fingerprint, tailcat_address} (no-store; tailcat_address is a credential and is null for direct TLS). 404 unless [worker_bootstrap] api_url is set; 413 for a body over 2 KiB, 400 for a non-JSON content type or malformed fields, 403 for a browser-origin request and for every refused secret (unknown, spent, expired, withdrawn, wrong pool or transport, deactivated pool) with one uniform message, 409 when the worker identity, fingerprint or node key already belongs to another worker (the enrollment stays unspent), 429 past 5 calls in a burst and one per 12 s per client address, past 20 and one per 3 s overall, and while a Tailcat controller has no address yet. The same worker id, fingerprint and key may present the same secret again for 10 minutes and get the same answer. Refusals write at most 20 worker-enrollment-refused audit rows an hour; success is audited as before (worker bootstrap) |
GET /admin/worker-enrollments?pool&before&limit |
platform admin | platform:admin |
enrollments newest first, optionally for one pool → {enrollments:[{id, pool, state, transport, created_ms, expires_ms, host}], next_before}; each entry is the terms and state as in the single read, never a secret or a Tailcat address. next_before (the last entry's created_ms, null on the last page) is passed back as before; an unparsable pool or before is 400 |
GET /admin/worker-enrollments/{tok} |
platform admin | platform:admin |
the enrollment's terms and state (pending, redeemed, revoked, expired); never the secret |
DELETE /admin/worker-enrollments/{tok} |
platform admin, rate limited | platform:admin |
withdraw an unspent enrollment → 204; 404 once it is spent or already withdrawn. Audited as worker-enrollment-revoked |
GET /admin/worker-enrollments/{tok}/bootstrap.sh |
platform admin, rate limited | platform:admin |
the worker-only bootstrap script for that enrollment's transport, as an attachment (text/x-shellscript, nosniff, no-store). It holds the controller's public link fingerprint and the SHA-256 pins of the release artifacts, no secret and no Tailcat address. Its bytes are the same for every enrollment of a transport, rendered once at start, and their SHA-256 is the bootstrap_sha256 of the issue answer and of GET /admin/worker-bootstrap. 409 unless the enrollment is still pending (or when the controller cannot render it); at most 10 in a burst and one per 2 s per administrator. Each download is audited as worker-bootstrap-downloaded |
GET /admin/worker-bootstrap |
platform admin | platform:admin |
what an enrollment page offers, with no credential in it → {configured:false} without [worker_bootstrap], else {configured:true, release_version, redeem ("https" or "link"), archs, transports:{direct:{available, bootstrap_sha256}, tailcat:{available, address_ready, helper_version, bootstrap_sha256}}}. tailcat.available means a helper is pinned and an address file is configured; address_ready that the controller has written its address; helper_version and bootstrap_sha256 are null without a pinned helper. No address, secret or URL appears in it |
POST /admin/worker-enrollments/{tok}/node-key {worker, node_key} |
browser session, fresh step-up, rate limited | platform:admin |
for a pending Tailcat enrollment whose controller has no api_url: name the worker identifier and the Tailcat node key (nodekey:<64 hex>, the line sh bootstrap identity prints) the machine will present, so the controller admits that key to its Tailcat network before the machine has a tunnel → 204. Idempotent for the same pair. 409 for a spent, withdrawn or expired enrollment, a second different pair, and a worker id or key that already belongs to a worker; 404 for an unknown enrollment; 400 for a direct enrollment or malformed fields; 403 with step_up without a fresh step-up (a bearer token never steps up); at most 5 in a burst and one per 6 s per administrator (shared with issuing). Audited as worker-node-key-authorized, without the key |
GET /admin/audit?before&limit |
platform admin | platform:admin |
the authentication and administration audit, newest first, keyset by seq (next is the following page's before); events by their stable names |
GET /tenants/{slug}/members?after&limit |
tenant admin | tenant:admin |
members by user id (keyset): user, display_name, username, kind (person, service), role, active, source (app, or github for a membership derived from GitHub) and synced_ms (when GitHub last confirmed it) |
PUT /tenants/{slug}/membership-source {source, reason?} |
tenant admin, fresh step-up | tenant:admin |
make an organization tenant GitHub-backed (github) or app-managed (app) (GitHub sign-in); conflict without a bound GitHub-organization installation, invalid_request when the administrator has not linked GitHub. In a GitHub-backed tenant adding, changing, removing and granting people by hand is forbidden |
PUT|DELETE /tenants/{slug}/members/{user} {role, reason?} |
tenant admin | tenant:admin |
add or change a member (reader, operator, admin), or remove one (its tenant-narrowed credentials are revoked, and domain auto-join will not re-add the person); {user} is a usr_… id, or the local sign-in name of an existing member — a name is never resolved outside the tenant, so a non-member's name and an unknown name are the same not_found and the route cannot probe which accounts exist; adding a new member takes the id the person gave. An id that names nobody usable (unknown, suspended, another tenant's service account) is not_found and removing an id that is not a member is not_found, so those answers do not separate existing accounts from missing ones; adding an existing active person succeeds, because adding is the act asked for, and ids are random 122-bit values. On a GitHub-backed tenant every id but the tenant's own service accounts gets the same forbidden, whether or not it names an account. A platform administrator who is not an administrator of the tenant (platform scope carries no ambient data access) needs a stepped-up session and a reason (1–100 characters, audited as platform-override) to add or change a member; a tenant administrator needs neither. Taking the admin role from the tenant's last administrator is conflict |
POST /tenants/{slug}/repos {name} |
tenant admin | tenant:admin |
a repository record → 201 {id, name}; its source binding and hook secret are set host-locally, where credentials never cross the API |
GET /tenants/{slug}/repos/{name}/source |
read |
runs:read |
the source binding as metadata — remote, allowed_refs, pipeline_path, trust_pinned, version, revoked, github_repository_id — or {bound:false}; never a credential |
PUT /tenants/{slug}/repos/{name}/grants/{user} {access, reason?} |
tenant admin | tenant:admin |
a member's repository access, any of read, run, secrets ([] withdraws); membership stays the ceiling, so a grant to a non-member is not_found whether or not the id names an account; {user} resolves as for members (an id, or a member's sign-in name); the same step-up and reason rule as members for a platform administrator who does not administer the tenant |
GET /tenants/{slug}/storage |
tenant admin | tenant:admin |
the tenant's usage (object_bytes, log_bytes), its own policy, the effective limits (tenant_quota_bytes, repo_quota_bytes, log_retention_ms, artifact_retention_ms) and every repository's usage (artifact_bytes, log_bytes), policy and effective (retention) |
PUT /tenants/{slug}/repos/{name}/storage {quota_bytes?, log_retention_secs?, artifact_retention_secs?} |
tenant admin | tenant:admin |
replace a repository's storage policy: it may only narrow the tenant's — a retention above the tenant's effective one or a quota above the tenant's quota is invalid_request → {policy, effective} |
GET /tenants/{slug}/audit?before&limit |
tenant admin | tenant:admin |
the tenant's run control audit (cancel run, cancel job, rerun job), newest first by seq, with target, actor, via and the OAuth client |
POST /tenants/{slug}/uploads {len, digest?, ttl_ms?} |
member (operator+) | runs:write |
open a resumable upload session → 201 {upload, received, ranges, expires_ms} (storage); len is at most 4 GiB (invalid_request beyond); quota_exceeded when the tenant's budget cannot take the declared length, storage_full below the disk watermarks |
GET /uploads/{upl} |
member (operator+) | runs:write |
the durable resume state: held byte ranges and expiry |
PUT /uploads/{upl}?offset=N |
member (operator+) | runs:write |
one chunk, raw body ≤ 8 MiB; re-sent ranges merge, so retries are safe. A chunk (or commit) for a session past its expiry retires the session durably and answers invalid_request |
POST /uploads/{upl}/commit |
member (operator+) | runs:write |
tile check + digest verification → publish the object → {digest}; repeating returns the same digest |
DELETE /uploads/{upl} |
member (operator+) | runs:write |
abort and drop the staged bytes |
GET /tenants/{slug}/objects/{digest} |
read on a repository with an artifact referencing the digest, or member (operator+) with RUN for an uploaded object |
artifacts:read |
stream a committed object; Range: bytes=a-b/a-/-n → 206 with Content-Range; invalid ranges are invalid_request. Anyone else — a reader without a grant on the referencing repositories, a platform admin who is no member — gets the same not_found as for an unknown digest. A whole-object body is rehashed while it streams and ends short of its content-length (the connection closes) if the stored bytes no longer match the digest; range reads are not verified. At most three transfer bodies are in flight at once, counted until the body is written — the next is rate_limited |
The OAuth authorization server's endpoints are at the deployment's root, not under /api/v1, and answer the RFC 6749 error shape {error, error_description} instead of sentinel.error/1 (OAuth):
| Route | Auth | Does |
|---|---|---|
GET /.well-known/oauth-authorization-server |
none | RFC 8414 metadata: the issuer, the endpoints below (registration_endpoint only while policy allows DCR), grant types, S256, the scopes, client_id_metadata_document_supported per policy |
GET /.well-known/oauth-protected-resource/api/v1 |
none | RFC 9728: resource {issuer}/api/v1, its authorization server and scopes |
GET /.well-known/oauth-protected-resource/mcp |
none | RFC 9728 for the remote MCP resource {issuer}/mcp and the MCP scopes (MCP) |
GET, POST /oauth/authorize |
session cookie (else the embedded password sign-in, and "Sign in with GitHub" when configured) | authorization code + PKCE consent; approve or deny → 303 to the client's registered redirect (loopback for the CLI, HTTPS or loopback for a registered MCP client) with code/error, state, iss; a registered client's errors redirect only after sign-in |
POST /oauth/token |
public client (client_id) |
authorization_code, refresh_token (rotation with a 60 s lost-response grace, replay revokes the grant) and urn:ietf:params:oauth:grant-type:device_code grants → TokenResponse; cache-control: no-store |
POST /oauth/register |
none; per client 2/s (deployment 10/s) | RFC 7591 registration of a public MCP client while the instance policy is open → 201; otherwise 403 access_denied (OAuth) |
POST /oauth/revoke |
public client | RFC 7009: either token kind revokes its whole grant; always 200 |
POST /oauth/device_authorization |
public client | RFC 8628 device request → {device_code, user_code, verification_uri, verification_uri_complete, expires_in, interval}; 429 slow_down at 1024 pending |
GET, POST /device |
session cookie (else the embedded password sign-in, and "Sign in with GitHub" when configured) | enter a user code, then approve (narrowing scopes, tenant, repository) or deny; five wrong codes in ten minutes lock the account out of the page for the rest of the window |
POST, GET, DELETE /mcp |
MCP-audience OAuth access token only | Streamable HTTP MCP on revision 2025-11-25 (GET is 405); errors are sentinel.error/1 with the MCP WWW-Authenticate challenges (MCP) |
GitHub web sign-in, present only when <data_dir>/github-sign-in.json configures it (configuration, GitHub sign-in); both answer HTML pages, never JSON:
| Route | Auth | Does |
|---|---|---|
GET /auth/github/start?return_to=<path> |
none; per client burst 10, then one per 2 s (deployment 20/s) | return_to must be /, /oauth/authorize?… or /device[?…] (issuer-relative, URL-safe characters only, at most 2048 bytes; absent means /), else 400 with no state and no cookie. Records a single-use 10-minute state with that destination, sets __Host-sentinel_signin (SameSite=Lax, HttpOnly, Secure, Path=/, Max-Age=600) and answers 303 to GitHub with the client ID, the exact redirect URI {issuer}/auth/github/callback, the state and no scope. Over the limit 429 with retry-after |
GET /auth/github/finish |
the sign-in cookie of a verified callback; a browser request must be Sec-Fetch-Site: same-origin (else 403, nothing spent) |
takes the hand-off once (400 without one), revokes the session cookie the browser sent and issues the ordinary session cookie (a fresh secret) in one transaction, clears the sign-in cookie and answers a page that continues to the recorded destination with its parameters intact |
GET /auth/github/callback?code&state |
the sign-in cookie, equal to state |
a missing, foreign or replayed state is 400 and spends nothing it does not own; a GitHub denial 403; then the code exchange and GET /user, and the account linked to that immutable GitHub ID — 403 "no active Sentinel account" for an unlinked, pending, rejected or suspended one, with no session. Success issues nothing yet: it parks a 60-second hand-off keyed by the sign-in cookie and answers a page that continues to /auth/github/finish (the browser's Strict session cookie is not sent on this cross-site navigation). At most one exchange runs at a time; a second is 503 with its state still pending, so a reload completes |
Secret mutations are raw-byte writes; only metadata is returned. Authorization requires both the OAuth scope below and the store's current tenant/repository permission.
| Route | Store authority | Scope | Does |
|---|---|---|---|
GET /tenants/{slug}/secrets?repo&after&limit |
member or secret writer | secrets:metadata |
keyset page of names, versions, active state and timestamps; limit is 1–100; no ciphertext or value |
GET /tenants/{slug}/secrets/{name}?repo |
member or secret writer | secrets:metadata |
one metadata record; no value read-back |
PUT /tenants/{slug}/secrets/{name}?repo |
secret writer | secrets:write |
raw body 1–65,536 bytes (values below 4 bytes are stored but refused for job delivery); required If-Match current version (0 creates) and Idempotency-Key; atomic sealed write → metadata |
DELETE /tenants/{slug}/secrets/{name}?repo |
secret writer | secrets:write |
required If-Match and Idempotency-Key; revokes all versions and reserves the name |
POST /tenants/{slug}/secrets/import?repo |
secret writer | secrets:write |
strict env-file bytes ≤ 1 MiB; If-Match lists every NAME=version; one transaction creates/rotates every entry or none; requires Idempotency-Key |
GET /tenants/{slug}/secrets/{name}/allow?after&limit |
member or tenant administrator | secrets:metadata |
the repositories (ID and name) a tenant secret is allowed in, keyset by repository ID |
PUT|DELETE /tenants/{slug}/secrets/{name}/allow?repo |
tenant administrator | secrets:write |
allow the tenant secret in that repository, or stop (removing its bindings there); idempotent |
POST /tenants/{slug}/secrets/{name}/versions/{version}/revoke?repo |
secret writer | secrets:write |
revoke one version permanently → the secret's metadata; idempotent; a version above the current one is not_found |
GET /tenants/{slug}/secret-bindings?repo&after&limit |
member or secret writer | secrets:metadata |
the repository's bindings (job, step, name, source secret ID, override_tenant); after is the previous page's next, JOB/STEP/NAME |
PUT /tenants/{slug}/secret-bindings/{name}?repo&job&step |
secret writer | secrets:write |
optional body {"from_tenant": bool, "override_tenant": bool}; binds the repository's secret (or the allowed tenant secret) at that selector, replacing what was there; idempotent |
DELETE /tenants/{slug}/secret-bindings/{name}?repo&job&step |
secret writer | secrets:write |
removes that binding; an absent one is not an error |
Every secret route checks the path, query, headers and body shape first, then resolves the tenant slug through the caller's membership and the repository name inside the same transaction that authorizes the operation, so a foreign tenant or repository answers exactly like a missing one. repo is percent-decoded (repo=Acme%2Fapp). A reused Idempotency-Key with a different request is idempotency_mismatch (422), distinct from a stale If-Match (conflict, 409). The retry record stores a keyed MAC of the request under a subkey of the master key, never a digest that could be recomputed from the database.
The tenant-facing GET /workers never shows ceilings, priority or why a machine is clamped; the approved-versus-reported view below (host.approved, host.reported, host.effective, host.over_cap) is the administrators' route for that.
Host policy (Q12) is the controller's alone: the ceilings and priority a worker is approved for come from the enrollment and from these routes, never from the worker's hello.
| Route | Who | Scope | Result |
|---|---|---|---|
GET /admin/workers?pool |
platform admin | platform:admin |
every live worker, optionally of one pool → {workers:[{id, pool, name, arch, enrolled_ms, last_seen_ms, connected, software, transport ("tailcat" or "direct"), uninstall_requested_ms, host:{approved, reported, effective, reserved, over_cap}}]}, with host in the shape of the policy read below. A worker whose uninstall was requested stays listed, marked, until it settles. No fingerprint, node key or enrollment secret is part of it |
GET /admin/workers/{wrk}/policy |
platform admin | platform:admin |
{approved:{cpu_millis, memory_bytes, disk_bytes, priority}, reported:{cpu_millis, memory_bytes, disk_bytes}, effective:{…}}: what the controller approved (null = no ceiling), what the worker last stated (0 = not stated) and what it is offered (the report under the ceilings), plus reserved:{cpu_millis, memory_bytes, disk_bytes} (what the unreleased attempts of the worker's whole machine hold) and over_cap:{over, cpu_millis, memory_bytes, disk_bytes}: whether, and by how much, the accepted work exceeds the offered size after a ceiling was lowered (over is false and the amounts 0 when it fits; a size the controller does not enforce is never over). 404 for an unknown or revoked worker |
GET /admin/placement |
platform admin | platform:admin |
{order, workers:[{position, worker, priority, expected_finish_ns, ahead_because}]}: the configured order (fastest or priority, placement_order in configuration) and the connected free workers in the order the next placement round uses, each with its approved priority, the expected finish of a reference job on its machine and the first factor that puts it ahead of the next one (priority, expected_finish, best_fit or identity; null for the last). It explains the order of hosts; whether a host may take a given job is hard admission, explained per job by the queue's reason |
PUT /admin/workers/{wrk}/policy {cpu_millis?, memory_bytes?, disk_bytes?, priority?} |
browser session, fresh step-up, rate limited | platform:admin |
replace the whole approved policy (an absent or null ceiling removes it, an absent priority is 0; ceilings are positive, priority −100…100, unknown fields 400) and answer the new view. A reduction stops new oversubscription at once and a raise is used by the next placement pass, never above the worker's report; running attempts are untouched. Audited as worker-policy-changed |
POST /admin/workers/{wrk}/uninstall |
browser session, fresh step-up, rate limited | platform:admin |
drain the worker now and revoke it, with its fingerprint and Tailcat node key, once its accepted attempts have settled → 202 {worker, uninstall:"requested"}; idempotent. Audited as worker-uninstall-requested (and worker-uninstalled when it settles) |
export SENTINEL_SERVER=http://127.0.0.1:7080
sentinel api --token-file ~/.sentinel/token me
sentinel api --token-file ~/.sentinel/token run --tenant acme --repo app \
--pipeline .sentinel.yml --source https://github.com/acme/app.git --sha <full sha> --ref main --idempotency-key push-1
sentinel api --token-file ~/.sentinel/token status run_…
sentinel api --token-file ~/.sentinel/token logs att_… --follow
sentinel api --token-file ~/.sentinel/token cancel --run run_…
sentinel api --token-file ~/.sentinel/token rerun job_…
sentinel api --token-file ~/.sentinel/token workers --tenant acme --json
sentinel api --token-file ~/.sentinel/token queue --tenant acme --limit 50
sentinel api --token-file ~/.sentinel/token drain wrk_…
sentinel api --token-file ~/.sentinel/token undrain wrk_…--json prints the server's document, and a failure as one sentinel.error/1 line on stderr; text output is for people. --token-file/--token/SENTINEL_TOKEN accept an sntl_ credential or an sntl_at_ access token. Exit codes are the shared table in CLI: 0 success, 1 remote fault or malformed answer, 2 usage, 3 unauthenticated/forbidden, 4 not_found, 5 conflict/idempotency_mismatch, 6 busy or unreachable after retries. --token-file keeps the secret out of the process list; --token and SENTINEL_TOKEN exist for tooling that already protects its environment.
Draining needs a platform-admin credential on the API; on the controller's own host the same change is sentinel admin worker drain --id wrk_… --data-dir … (and undrain), which acts on the store directly like the rest of sentinel admin.
The controller serves no page of its own besides the OAuth consent and device pages: GET / is not_found. The web interface is sentinel-web (web interface), a client of these routes.
The per-client rate limits on the unauthenticated routes, the sign-in lockout pair (client, account) and the public-body slot all use one client key:
| Peer | Key |
|---|---|
Not in trusted_proxies |
its own address (an IPv6 /64 counts as one); every header is ignored |
In trusted_proxies (loopback when the key is absent), X-Forwarded-For ends in a valid address |
that last address; a proxy appends the real peer, so the entry before it is whatever the client wrote and is never read |
In trusted_proxies, no X-Forwarded-For or a last entry that is not an address |
the proxy's connection: its address and source port |
The last row is the dependency on the proxy. A proxy that does not send X-Forwarded-For used to give every client behind it one key: one guesser exhausted the allowance of all, and five wrong passwords for an account from the shared key locked that account out for the owner. Now each upstream connection is a client of its own, so a guesser throttles and locks out the connections it uses, not the people behind other ones. That is the finest identity the controller has without the header, and it is weaker than a real client address: a client that reaches the controller through a new upstream connection for every request (a proxy without keep-alive) gets a fresh key each time. The following still bound it, and none depends on the header: the global ceiling of every budget, the 8 handler and 4 public-body slots, the 10 s body deadline, and the account-wide lock, which locks the account for everybody after four times the per-client failures (ACCOUNT_LOCK_FACTOR) whatever address they come from. Unauthenticated per-client limits, the lockout and the per-client body slot are therefore only fully effective when the proxy overwrites or appends X-Forwarded-For with the address it saw, as the web interface does with SENTINEL_CLIENT_IP_HEADER (web interface). A proxy must be listed in trusted_proxies for its header to be read, and nothing else may reach the API port: a client that reaches it from a trusted address could name any client.
Quotas that reconnecting must not mint are still keyed on the proxy's address alone, so everything behind a proxy that names no client shares them: the dynamic client registrations per registrant (OAuth). A trusted-proxy peer is exempt from the per-peer connection cap, which is why a proxy that names no client is not capped per connection.
Every message the controller queues (an address verification or an email invitation) is bounded per recipient address, whoever asked for it, on top of the per-account and per-client limits: at most 5 messages to one address per hour and at least 60 seconds between two messages of the same kind to it. A request refused by this allowance sends nothing and leaves the link already mailed valid (the link is minted only when the message can be queued). The bounds are shared by every requester, so they cap the mail a stranger can make the deployment send to a victim, but they also mean a stranger can spend a victim's allowance: while it is spent, the holder's own resend waits (at most until the oldest message leaves the hour; POST /me/email/resend answers 429, every unauthenticated or address-taking route still answers 202), and an operator can still mint a link on the host with sentinel admin account email-link (admission). An invitation is not delayed by a verification message, or the reverse; both count toward the hourly 5. Delivered messages keep counting for seven days of outbox retention, so delivery does not reset the allowance. Email-registered accounts that never verify their address are retired after seven days (admission).
A body longer than its limit is 413 payload_too_large (details.limit_bytes; the OAuth endpoints answer 413 invalid_request in their own error shape). A declared Content-Length over the limit is refused from the request head alone; a body without a length is cut off while it is read, one byte past the limit.
| Route class | Limit | Source |
|---|---|---|
authenticated JSON (/api/v1, the default) |
1 MiB | MAX_API_BODY_BYTES in sentinel-protocol (a pipeline file inside it is at most 256 KiB, MAX_PIPELINE_FILE_BYTES) |
PUT /uploads/{upl} chunk |
8 MiB | MAX_UPLOAD_CHUNK |
| secrets: value / import | 64 KiB / 1 MiB | secrets |
POST /login |
16 KiB | MAX_LOGIN_BODY_BYTES |
POST /setup and the email routes, public and signed-in (/me/email, /invitations/accept, a tenant's invitations) |
8 KiB | MAX_BODY_BYTES in routes/setup.rs and routes/email.rs |
| account security routes | 4 KiB | MAX_BODY_BYTES in routes/account.rs |
| GitHub App routes | 4 KiB | MAX_BODY_BYTES in github_app.rs |
POST /worker-enrollments/redeem |
2 KiB | MAX_REDEEM_BODY_BYTES (worker enrollment) |
POST /hooks/github |
4 MiB | MAX_WEBHOOK_BODY_BYTES |
POST /intake/{repo} |
64 KiB | MAX_HOOK_BODY_BYTES |
POST /mcp message |
2 MiB | mcp::MAX_MESSAGE_BYTES (larger than the API limit; MCP responses are capped separately at 8 MiB, MCP) |
/oauth/* forms and the device page |
8 KiB | MAX_OAUTH_FORM_BYTES (the consent form answers 413 invalid_request in the OAuth error shape, or 503 temporarily_unavailable when no body slot frees up; the signed-in device page answers 413 as an HTML page; any other unreadable form is 400) |
POST /oauth/register |
16 KiB | OAuth |
The unauthenticated classes read their body through the public body slots below. every_route_class_enforces_its_own_body_limit (crates/sentinel-api/tests/api.rs) sends one byte over and exactly the limit to the API, upload, login, setup, GitHub webhook, intake, MCP, OAuth token and consent routes, the signed-in device page, the account security, signed-in email and invitation routes and the GitHub App routes, and a chunked login body and a chunked device-page form over the limit (the enrollment redeem route has its own test in worker_enrollment.rs; oauth_code.rs an_open_registration_cuts_an_oversized_body_off_at_16_kib covers POST /oauth/register).
A handler permit is never held while a client with no credential yet sends its body. The login, the sign-up, setup and public email routes, both webhooks (/hooks/github, the generic hook) and the OAuth forms and registration read their whole body first: if it is not already in the connection's buffer, the handler permit is given back, the read waits for one of 4 body slots (sentinel_api::PUBLIC_BODIES) and must finish within 10 seconds, slot wait included (rate_limited, or 503 temporarily_unavailable on the OAuth routes, when no slot frees up in time), and only then is a handler permit taken again. Each client (the admission key: peer address, or the X-Forwarded-For address behind a trusted proxy, or the proxy's connection when it names none; client keys) may hold only one of those slots at a time, and a second body from the same client is turned away at once, so one host cannot fill the pool and block other people's sign-in or webhook delivery. A client that trickles a body therefore delays only its own other unauthenticated bodies, and never a run read, a token refresh, a health check or another client's sign-in. A trusted-proxy peer is exempt from the per-peer connection cap, so the clients it forwards are bounded in the slots instead: together they may hold at most 3 of the 4 (sentinel_api::PROXY_BODIES), each for at most 5 seconds, slot wait included (sentinel_api::PROXY_BODY_TIME), and one slot is left to direct clients however many addresses the proxy names; a forwarded reader that finds its share spent waits for it within those 5 seconds and is then turned away (rate_limited, or 503 on the OAuth routes). Many distinct forwarded addresses can still take the 3 for five seconds each; that is bounded by the admission budgets, not by this pool. Authenticated routes keep the 120 s body deadline: their sender has already presented a credential. Bodies are also bounded in size (login 16 KiB, OAuth forms and registration by their own limits, webhooks by the intake limits), and Content-Length and chunk sizes are plain digits (decimal or hexadecimal); a sign or any other character is 400.
GET /runs/{id}/wait and GET /attempts/{id}/logs?wait=1 are the live paths every client uses — the CLI's wait and --follow, MCP, and the web interface's event streams. A parked poll:
- takes one of 32 subscriber slots (8 per user), holds its connection, and gives its handler permit back while it waits, so parked polls never starve requests doing work (
sentinel_api::SUBSCRIBERS,SUBSCRIBERS_PER_USER,RESERVED_HANDLERS); - re-authorizes whenever the tenant's authorization epoch moves (a membership, role or grant change, a suspension) and then answers what a fresh request would (
not_found), rather than keep serving a reader who lost access until its deadline; - ends as soon as its client hangs up (a non-blocking
peekevery 250 ms), giving the slot back, so a closed tab or an aborted upstream request never holds a slot to the 25 s deadline.
Tests: crates/sentinel-api/tests/api.rs (authentication, dispatch and idempotency, run and log reads, cancel and rerun, worker status, login and CSRF over loopback HTTP), web.rs (the web interface's routes and the long-poll rules above), account.rs (account-security routes), github_sign_in.rs (GitHub linking), oauth_core.rs (OAuth access tokens, per-route scopes and WWW-Authenticate challenges; OAuth), email.rs and setup.rs (admission). crates/sentinel/tests/cli.rs (Linux, both role features) runs sentinel api against the real sentinel server with a credential issued by admin token issue.