Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -9,3 +9,4 @@ target/
.trezor-user-env/
__pycache__/
.marketplace/
.shop-order/
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Added
- Run the Shop's marketplace on Synonym's staging with our own Paykit Server v0.1.0-rc11 behind a Cloudflare quick tunnel (`./shop-mixed`, profile `shop-mixed`)
- Run a paid Shop order on our own Paykit Server, Lock Server, marketplace service and Shop with Synonym's staging homeserver (`./shop-order`, profile `shop-order`)
- Rebuild the Paykit fixtures on the paykit-rs version a Bitkit pull request pins with `scripts/follow-app-paykit`
- Withhold and restore the payment request issuer's endpoints with `POST /endpoints`, so a journey can make the app's request resolution fail and recover
- Route the apps' homeserver traffic through `homeserver-proxy`, whose control port (23298) delays or fails one identity's homeserver requests by path
Expand Down
18 changes: 18 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -164,6 +164,7 @@ Every host port is 1024 or above. The ports Bitkit, the Pubky SDK or Bitkit's UI
| Paykit Server (`marketplace`) | 23101 (`MARKETPLACE_PAYKIT_PORT`) | 3001 |
| `fixture-issuer`, `rc56-peer` (`payment-requests`) | 23012, 23013 | 3012, 3013 |
| Paykit Server, quick tunnel metrics (`shop-mixed`) | 23110, 23111 | 3001, 23111 |
| Lock Server, marketplace service, Shop, and their tunnel metrics (`shop-order`) | 23120, 23130, 23140; 23121, 23131, 23141 | 3000, 8080, 3000 |

## API Examples

Expand Down Expand Up @@ -638,6 +639,23 @@ outbound internet.
The headless seller and buyer of the `marketplace` profile need the local testnet and chain, so `seed`, `fund`, `receive`, `pay`,
`peers` and `verify` refuse here. See [docs/shop-mixed.md](docs/shop-mixed.md) for what runs where and how the pins are checked.

#### Shop Order on Our Own Servers

The `shop-order` profile adds the rest of the Shop to `shop-mixed`, so a Bitkit buyer can pay a Pubky Ring seller's digital listing
and the same marketplace order becomes paid: the Lock Server v0.1.0-rc10, the marketplace service (pubky-marketplace-service#98,
`005b0707`) and the Shop (pubky-marketplace#154, `fbe3babb`), the commits the Android and iOS order-paid acceptance ran on, with
Postgres, a quick tunnel per public URL, and keys made once and kept. Synonym's staging Shop stays the first route; use this when it
cannot serve the build under test. Needs Docker, `curl`, `jq`, `git` and `python3`, and about 6 GB free for the Shop's build.

```bash
SHOP_ORDER_PROVISIONING=1 ./shop-order fetch # pinned sources, plus the seller's provisioning page
./shop-order up # starts everything; prints the URLs, readiness and Paykit's trusted keys
./shop-order lock --session-file seller-locks-session.json --content product.txt --sats 1000 # the listing's digitalLock
./shop-order down --keep-data # stop; keeps databases, keys and the seller's setup
```

See [docs/shop-order.md](docs/shop-order.md) for the seller and buyer steps, the pass criteria and every setting.

#### Bech32 LNURL Pay

- in `Env.{kt,swift}`, use for REGTEST electrum server: `"tcp://localhost:60001"`
Expand Down
162 changes: 158 additions & 4 deletions docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -626,7 +626,7 @@ services:
# Drive it with ./shop-mixed; see docs/shop-mixed.md. Nothing of the local chain or Pubky testnet runs for it.
# ---------------------------------------------------------------------------
shop-mixed-postgres:
profiles: [shop-mixed]
profiles: [shop-mixed, shop-order]
image: postgres:16-alpine
restart: "no"
environment:
Expand All @@ -642,7 +642,7 @@ services:
retries: 30

shop-mixed-paykit:
profiles: [shop-mixed]
profiles: [shop-mixed, shop-order]
image: bitkit-docker/paykit-server:${SHOP_MIXED_PAYKIT_SERVER_TAG:-v0.1.0-rc11}
build:
context: ./marketplace/shop-mixed
Expand Down Expand Up @@ -681,7 +681,7 @@ services:
# answers /quicktunnel with the hostname (./shop-mixed url). Built from the pinned cloudflared image, so a project whose
# images are never pulled (the QA lanes) fetches it with `up --build`.
shop-mixed-tunnel:
profiles: [shop-mixed]
profiles: [shop-mixed, shop-order]
image: bitkit-docker/shop-mixed-tunnel:2026.9.3
build:
context: ./marketplace/shop-mixed
Expand All @@ -696,7 +696,7 @@ services:
# The driver's service writes the Paykit config, issuer and master keys (`init`) and stays up, so `up --wait` sees no exited
# one-shot container; ./shop-mixed runs the driver's other commands with `compose run`.
shop-mixed-driver:
profiles: [shop-mixed]
profiles: [shop-mixed, shop-order]
image: bitkit-docker/shop-mixed-driver:local
build:
context: ./marketplace/driver
Expand All @@ -711,6 +711,159 @@ services:
entrypoint: ["node", "/app/driver.mjs"]
command: ["init-and-stay"]

# ---------------------------------------------------------------------------
# A Shop order paid end to end on our own servers with Synonym's staging homeserver (profile "shop-order", on top of
# shop-mixed's Paykit Server rc11): the Lock Server v0.1.0-rc10, the marketplace service and the Shop at the commits that ran
# the Android and iOS order-paid acceptance (10 Oct), Postgres for the three, and a quick tunnel for each URL a phone or the
# seller's browser opens. ./shop-order fetches each source at its exact commit into .shop-order/src and refuses another
# revision; docs/shop-order.md has the flow. Secrets are made by shop-order-keys in the state volume, never committed.
# ---------------------------------------------------------------------------
shop-order-keys:
profiles: [shop-order]
image: node:22.11.0-alpine3.20
restart: "no"
environment:
SHOP_ORDER_FRESH: ${SHOP_ORDER_FRESH:-0}
volumes:
- shop_mixed_state:/state
- ./marketplace/shop-order:/order:ro
command: ["node", "/order/keys.mjs"]

shop-order-postgres:
profiles: [shop-order]
image: postgres:16-alpine
restart: "no"
environment:
POSTGRES_USER: shop
POSTGRES_DB: postgres
volumes:
- shop_order_postgres_data:/var/lib/postgresql/data
- shop_mixed_state:/state:ro
- ./marketplace/shop-order/postgres-init.sh:/docker-entrypoint-initdb.d/10-shop-order.sh:ro
entrypoint:
- /bin/sh
- -c
- |
set -a; . /state/order/postgres.env; set +a
exec docker-entrypoint.sh postgres
healthcheck:
test: ["CMD-SHELL", "pg_isready -U shop -d postgres"]
interval: 2s
timeout: 5s
retries: 30

shop-order-locks:
profiles: [shop-order]
image: bitkit-docker/locks-server:${SHOP_ORDER_LOCKS_REV:-01cfeca14c7b5d385d8c3536c0cb4e1af81b458c}
build:
context: ./.shop-order/src/locks
restart: "no"
user: "0:0" # reads its secrets in the state volume (mode 0600)
depends_on:
shop-order-postgres:
condition: service_healthy
shop-mixed-paykit:
condition: service_started
volumes:
- shop_mixed_state:/state:ro
- ./marketplace/shop-order:/order:ro
- ./.shop-order/run:/run/order:ro
entrypoint: ["/bin/sh", "/order/start.sh", "locks"]
ports:
- "127.0.0.1:23120:3000"

shop-order-service:
profiles: [shop-order]
image: bitkit-docker/marketplace-service:${SHOP_ORDER_SERVICE_REV:-005b0707e6047b388ce032f4b51a2e0ed9e3a752}
build:
context: ./.shop-order/src/service
restart: "no"
user: "0:0" # reads its secrets in the state volume (mode 0600)
depends_on:
shop-order-postgres:
condition: service_healthy
volumes:
- shop_mixed_state:/state:ro
- ./marketplace/shop-order:/order:ro
- ./.shop-order/run:/run/order:ro
entrypoint: ["/bin/sh", "/order/start.sh", "service"]
ports:
- "127.0.0.1:23130:8080"

# The Shop BFF's own tables (db/bff in the Shop source), applied once to the bff database.
shop-order-bff-migrate:
profiles: [shop-order]
image: postgres:16-alpine
restart: "no"
depends_on:
shop-order-postgres:
condition: service_healthy
volumes:
- shop_mixed_state:/state
- ./.shop-order/src/shop/db/bff:/bff:ro
entrypoint:
- /bin/sh
- -c
- |
set -eu
[ -e /state/order/bff-migrated ] && { echo 'bff tables already applied'; exit 0; }
set -a; . /state/order/postgres.env; set +a
for file in /bff/*.sql; do
PGPASSWORD="$$POSTGRES_PASSWORD" psql -v ON_ERROR_STOP=1 -h shop-order-postgres -U shop -d bff -f "$$file"
done
date -u > /state/order/bff-migrated

shop-order-shop:
profiles: [shop-order]
image: bitkit-docker/pubky-marketplace:${SHOP_ORDER_SHOP_REV:-fbe3babba0c3c05990571221b5d4dc0c31788e68}
build:
context: ./.shop-order/src/shop
restart: "no"
user: "0:0" # reads the BFF secrets in the state volume (mode 0600)
depends_on:
shop-order-service:
condition: service_started
volumes:
- shop_mixed_state:/state:ro
- ./marketplace/shop-order:/order:ro
- ./.shop-order/run:/run/order:ro
entrypoint: ["/bin/sh", "/order/start.sh", "shop"]
ports:
- "127.0.0.1:23140:3000"

shop-order-locks-tunnel:
profiles: [shop-order]
image: bitkit-docker/shop-mixed-tunnel:2026.9.3
build:
context: ./marketplace/shop-mixed
dockerfile: tunnel.Dockerfile
restart: "no"
command: ["tunnel", "--metrics", "0.0.0.0:23121", "--url", "http://shop-order-locks:3000"]
ports:
- "127.0.0.1:23121:23121"

shop-order-service-tunnel:
profiles: [shop-order]
image: bitkit-docker/shop-mixed-tunnel:2026.9.3
build:
context: ./marketplace/shop-mixed
dockerfile: tunnel.Dockerfile
restart: "no"
command: ["tunnel", "--metrics", "0.0.0.0:23131", "--url", "http://shop-order-service:8080"]
ports:
- "127.0.0.1:23131:23131"

shop-order-shop-tunnel:
profiles: [shop-order]
image: bitkit-docker/shop-mixed-tunnel:2026.9.3
build:
context: ./marketplace/shop-mixed
dockerfile: tunnel.Dockerfile
restart: "no"
command: ["tunnel", "--metrics", "0.0.0.0:23141", "--url", "http://shop-order-shop:3000"]
ports:
- "127.0.0.1:23141:23141"

volumes:
trezor_emulator_state:
trezor_emulator_logs:
Expand All @@ -724,5 +877,6 @@ volumes:
marketplace_state:
shop_mixed_postgres_data:
shop_mixed_state:
shop_order_postgres_data:

networks: {}
104 changes: 104 additions & 0 deletions docs/shop-order.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,104 @@
# Shop order on our own servers (`shop-order`)

A Shop order paid end to end: a Pubky Ring seller lists a digital product, a Bitkit buyer pays the original request once, Paykit
Server sees `amount_matched`, the Lock Server completes the bundle, and the same marketplace order becomes `paid`. Our own Paykit
Server, Lock Server, marketplace service and Shop run here; Synonym's staging homeserver, relay, Shop indexer and Blocktank's staging
regtest chain do the rest, so Bitkit staging builds (devDebug, iOS Debug) work unchanged.

Synonym's staging Shop stays the first route to try. Use this profile when the staging Shop cannot serve the Bitkit build under
test, for example a Paykit version the staging Paykit Server does not run.

## Revisions

| Component | Revision | Built from |
| --- | --- | --- |
| Shop | pubky/pubky-marketplace#154, fbe3babba0c3c05990571221b5d4dc0c31788e68 | the repository's Dockerfile |
| Marketplace service | pubky/pubky-marketplace-service#98, 005b0707e6047b388ce032f4b51a2e0ed9e3a752 | the repository's Dockerfile |
| Lock Server | pubky/locks v0.1.0-rc10, 01cfeca14c7b5d385d8c3536c0cb4e1af81b458c | the repository's Dockerfile |
| Paykit Server | v0.1.0-rc11, 662dca0619a9aa2962bcd677bd5ddd4563cd2784 (paykit-rs ad3c7224 = rc72) | `marketplace/shop-mixed` |

The Shop was tested at fbe3babb; pubky/pubky-marketplace#154 later moved to 84934516 for review changes to the seller-readiness
source and the buyer key-record gate, with the Locks payment path unchanged. The pin stays at the tested commit until #154 merges.

`./shop-order fetch` checks out each source at exactly that commit and refuses another. These are the commits the Android
(aa330959, devDebug) and iOS (bb02e835, Debug) order-paid acceptance ran on, on 10 Oct.

## Run

```bash
SHOP_ORDER_PROVISIONING=1 ./shop-order fetch # the pinned sources, plus the seller's provisioning page
./shop-order up # keys, Paykit rc11, tunnels, Postgres, Lock Server, service, Shop
./shop-order urls # SHOP_URL, SERVICE_URL, LOCKS_URL, PAYKIT_URL
./shop-order health # readiness, revisions, Paykit's trusted keys
```

The first build compiles three Rust services and the Shop, and the Shop's build needs about 6 GB of memory. Each start gets new
quick-tunnel URLs; `up` writes them to `.shop-order/run/urls.env` and every origin setting is filled from them.

## Seller

1. Open `SHOP_URL`, sign in with Pubky Ring, **Connect Locks**, reload the Shop, then do the Bitkit/Paykit setup. The Shop sends no
`creator` parameter (`PUBKY_RUNTIME_PAYKIT_SETUP_CREATOR_PARAM=false`). This order matters: the setup reads the seller's Lock
Server session, so Locks comes first.
2. Create the listing in the Shop studio as usual. The marketplace service registers it as revision 1.
3. Create its Lock with the seller's Locks session (the frontend session the Shop holds after Connect Locks, saved as
`{"session_token": ..., "creator": ...}`; keep the file private):

```bash
./shop-order lock --session-file seller-locks-session.json --content product.txt --sats 1000
```

It registers the content, creates a content Lock whose criterion is a Paykit payment of that price to the seller, and prints
the listing's `digitalLock`. It writes no order and no payment.
4. Open `SHOP_URL/marketplace/shop-order-provision` as the seller, enter the listing id and the `digitalLock`, and publish. That
publishes revision 2 through the Shop's normal path (`schema.parse`, then `CommerceApplication.commitUpsertListing`), which
emits the service's `listing.sync`. The service takes Lock terms only from the seller-signed homeserver record, so
`prepare_locks` needs that revision; the Shop's own publish flow works the same way (register, then sync).
5. Check the service holds the Lock snapshot, the current revision and stock before any checkout. Then remove the provisioning
page: `./shop-order fetch` (without `SHOP_ORDER_PROVISIONING`) and `./shop-order up`; keys and data are kept.

The listing keeps two different hashes: its cover image's `contentHash`, which the service registers, and the guarded content's
`resourceHash` (BLAKE3) in `digitalLock`. Physical listings are not supported yet: they need pubky-marketplace-service#89 and
Paykit Server #67/#68.

## Buyer

1. Fund the Bitkit buyer on Blocktank's staging regtest chain (`./shop-mixed mine` confirms). Use one wallet per simultaneous payer,
and keep only one running instance of a restored wallet.
2. Sign in to `SHOP_URL` with Bitkit and save the seller as a contact first.
3. Start a fresh order. Checkout still asks for a shipping address for a Locks-backed listing.
4. Press **Pay**, then **Request payment in your wallet** on the order page. The Shop runs `prepare_locks`, creates the Lock bundle
and registers it. Check the payer, seller, request and amount in Bitkit, then pay once.

Pass: the original Paykit invoice has `amount_matched: true`, the original SDK request reaches `proof_submitted`, the Lock Server
completes the original bundle, the same service order is `paid`, one transaction of the listed amount exists, and the buyer's
authenticated content access returns the content. If registration needs a retry, keep the browser profile and the stored bundle
and retry that same payment; never create a new invoice or pay again to settle a doubt.

## Configuration

Every secret is made by `shop-order-keys` on the first start, kept in the project's state volume (mode 0600) and reused after, so
a rerun keeps its databases, the seller's Paykit setup and the Lock Server identity (`SHOP_ORDER_FRESH=1 ./shop-order up` makes new
ones). Nothing is committed. What each service gets:

- **Shop** (runtime `PUBKY_RUNTIME_*`): `PAYKIT_SERVER_API=upstream` and `PAYKIT_SETUP_CREATOR_PARAM=false` together,
`COMMERCE_ADAPTER_MODE=locks-paykit`, `MARKETPLACE_GRANT_FLOW_ENABLED=true`, the service, Lock Server and Paykit setup URLs, and
the staging homeserver, homegate, relay, pkarr relays and Nexus. Its BFF: `SHOP_BFF_GRANT_FLOW_ENABLED`, allowed and public
origins, the service URL, its grant-state database (tables from the Shop's `db/bff`, applied once), `CRON_SECRET`, and its
assertion and service-request signing keys.
- **Marketplace service**: upstream Paykit with its request-signing key, which Paykit trusts; the Lock Server URL with bundle and
lookup keys and a 3,600-second payment window; digital-delivery, private-data and Stripe encryption keys (the Stripe key is
needed even for Bitcoin); the grant flow with its encryption and result keys and the Shop's assertion and request verifying
keyrings; and refusal audit with its own writer and retention logins, made on Postgres's first start. Its three refusal-audit
attestations are set because this database is disposable: it has no backup or replica, and its audit rows are not evidence.
- **Lock Server**: grant-connect for `shop.pubky.app` returning to `SHOP_URL`, credentials up to 3,600 seconds (the payment
window), Paykit at our rc11, and `network = "mainnet"`, which is where it resolves Pubky keys (the staging homeserver is
published there); payments stay on the staging regtest chain.
- **Paykit Server**: `[signed_services]` trusts the driver's issuer, the Lock Server's signer and the service's request key. Trust
is merged, never replaced, and each start reads the list back (`./shop-order health` prints it).

## Ownership and cleanup

The stack runs under its compose project, never a QA lane's: a lane's cleanup removes that lane's project. `./shop-order down`
removes the containers and volumes; `./shop-order down --keep-data` keeps the databases, keys and the seller's setup for a rerun.
On a QA box, a lease on the seat keeps the VM up while it runs.
2 changes: 1 addition & 1 deletion marketplace/driver/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -11,4 +11,4 @@ COPY --from=paykit /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ca-certific
# Paykit Server rc65 accepts a setup claim only after the identity published its Paykit noise key authorization; the payment request
# fixture binary publishes it for the headless seller under this name (a Bitkit wallet does it in its own Paykit setup).
COPY --from=fixture /usr/local/bin/fixture /usr/local/bin/paykit-key-authorization
COPY driver.mjs ./
COPY driver.mjs trust.mjs ./
2 changes: 1 addition & 1 deletion marketplace/driver/Dockerfile.staging
Original file line number Diff line number Diff line change
Expand Up @@ -4,5 +4,5 @@ FROM node:22-bookworm-slim@sha256:d649c27dae7ba0137b3cef5dd75baa422c08dc3d9e3fc0
WORKDIR /app
COPY package.json package-lock.json ./
RUN npm ci --omit=dev --ignore-scripts --no-audit --no-fund
COPY driver.mjs ./
COPY driver.mjs trust.mjs ./
ENV MARKETPLACE_BACKEND=staging
Loading