fix(storage)!: a record write naming a file its caller may not read is refused at the field

.changeset/26043-unreadable-file-refused-at-field.md

@@ -0,0 +1,21 @@
+---
+'@objectstack/service-storage': patch
+---
+
+fix(storage)!: a record write naming a file its caller may not read is refused at the field
+
+A record write that names a file its caller may not read is now refused at that field, instead of saving the id as written. The answer is one `400 VALIDATION_FAILED` whose `fields[]` names each such field with the ADR-0114 code `invalid_value`, the same envelope the public form door gives an anonymous submission that carries a file. It names the field only. An id that names no file of the acting organization gets the same answer, so the write cannot tell the two apart. Nothing is claimed, copied or written. System writes and writes with no caller context reference files as before, and an authorization-store outage still fails the write with its declared envelope.
+
+A file governed by a record's rule also gets the record-governed link lifetime at the download door.
+
+Clause-②: no (narrowing) — a record write that names a file its caller may not read, or an id no file of the organization has, was saved and is now refused. Nothing that was refused becomes accepted.
+
+Metadata-protocol: none (a write-time refusal on file fields; nothing a metadata author writes changes)
+
+<!-- adr-0087: not-required (no-migration-prescription) No metadata author's surface moves: no spec key or authorable spelling is added, removed, renamed or re-shaped, and no exported declaration is removed or narrowed, so there is nothing for `objectstack migrate meta` to rewrite. What changes is which record writes the file-reference lifecycle accepts: a write naming a file its caller may not read is refused at the field. The remedy is naming a file the caller may read, or leaving the field empty, never a rewrite of a package source or a stored row. The other categories are closed on facts: the package publishes (not unpublished); no ADR-0087 id is named or touched (not registered or already-registered); and nothing exported is removed or narrowed (not runtime-interface-only, type-surface-only or runtime-export-retired). -->
+
+**BREAKING** for a caller that writes a file id it may not read, shipped as `patch` because the metadata protocol does not move.
+
+- **Before.** The write saved, and the field kept an id its readers could not use.
+- **Now.** The write is refused with `400 VALIDATION_FAILED`, naming the field with `invalid_value`.
+- **Fix.** Name a file the caller may read (upload it, or choose one they can open), or leave the field empty.

docs/adr/0104-field-runtime-value-shape-contract.md

@@ -28,6 +28,12 @@
   kept as written and never claimed or copied, the answer #25682 gave across
   organizations. This qualifies the 2026-07-27 addendum's worst-case row for
   exclusive ownership. See the 2026-10-26 addendum.
+  · **Amended** (2026-10-26, #26043) — that kept id is replaced by a
+  refusal at the field: a write naming a file its caller may not read, or an
+  id the acting organization cannot see, fails with one `400
+  VALIDATION_FAILED` naming each such field with the ADR-0114 code
+  `invalid_value`, and nothing is written. See "The field refusal" in the
+  2026-10-26 addendum.
 - **Date**: 2026-07-22
 - **Issue**: design follow-up generalizing #3405 / #3406 (inline lookup param
   silently stripped); relates #3407 (silently dropped writes), #1878 / #1891
@@ -1464,17 +1470,20 @@ exists in the acting organization. One file had two access rules.
   collected (#26046); a member of the organization for a file no record has
   owned; a reader of any parent for an attachments-scope file; anyone for a
   `public_read` file.
-- **The quiet answer.** At the write door, a file the writer may not read is
-  answered as one the organization-scoped lookup cannot see: the id is kept
-  as written, nothing is claimed or copied, and constraint verdicts do not run
-  ahead of the read check. #25682 gave this answer across organizations; this
-  addendum states it within one. Unless the record already owned the file,
-  the kept id gives no reader anything through it: a reader refused
-  `sys_file` read gets the refused marker (`metadataRefused`), and the
-  download door judges the file by its own rule, never by this record. A slot
-  holding a file it does not own is a state that step 5's
-  `verify-references` classifies as blocking. #26043 tracks replacing this
-  answer with a field-level refusal.
+- **The field refusal.** *(Amended 2026-10-26, #26043. It replaces the quiet
+  answer this item first stated, which kept the id as written and so left a
+  slot holding a file it does not own, the state step 5's
+  `verify-references` classifies as blocking.)* At the write door, a write
+  naming a file its caller may not read is refused at the field, before the
+  driver runs: one `400 VALIDATION_FAILED` whose `fields[]` names each such
+  field with the ADR-0114 code `invalid_value`, the envelope the form door
+  gives an anonymous submission that carries a file (#25682). Nothing is
+  claimed, copied or written, and constraint verdicts do not run ahead of the
+  read check. An id the organization-scoped lookup cannot see gets the same
+  answer, and the answer names the field only, never the file or the record
+  that holds it, so the write door cannot tell "may not read" from "does not
+  exist in this organization". A write with no caller (below) keeps such an
+  id as written, as before.
 - **A write's own copy** is claimed by the same write without being asked
   again.
 - **Outage.** An authorization-store outage fails the write with its declared

packages/qa/dogfood/test/field-zoo-roundtrip.dogfood.test.ts

@@ -18,6 +18,7 @@
 
 import { describe, it, expect, beforeAll, afterAll } from 'vitest';
 import { SECRET_MASK } from '@objectstack/objectql';
+import { FILE_REFERENCE_TYPES } from '@objectstack/spec/data';
 import { type VerifyStack } from '@objectstack/verify';
 import { bootShowcase } from './showcase-boot.js';
 
@@ -27,6 +28,8 @@ describe('dogfood: field-type capability matrix round-trips over HTTP (#2004)',
   let record: Record<string, unknown>;
   /** Resolved reference ids, keyed by field — the assertions compare to these. */
   const referenceIds: Record<string, string> = {};
+  /** Uploaded `sys_file` ids, keyed by media field — the assertions compare to these. */
+  const fileIds: Record<string, string> = {};
 
   beforeAll(async () => {
     stack = await bootShowcase();
@@ -54,13 +57,38 @@ describe('dogfood: field-type capability matrix round-trips over HTTP (#2004)',
       referenceIds[target.field] = id as string;
     }
 
+    // [#26043] Upload a REAL file for each media field. A write naming an id
+    // that is no file of the organization is now refused at the field, as one
+    // naming a file the writer may not read is, so the matrix's stand-in ids
+    // would be refused. What this file proves for the media family is
+    // unchanged: the stored id round-trips as the same file.
+    for (const c of MATRIX) {
+      if (!FILE_REFERENCE_TYPES.has(c.type) || !('write' in c.check) || c.check.write === undefined) continue;
+      const bytes = `zoo ${c.field}`;
+      const presign = await stack.apiAs(token, 'POST', '/storage/upload/presigned', {
+        filename: `${c.field}.txt`, mimeType: 'text/plain', size: bytes.length,
+      });
+      expect(presign.status, `presign for ${c.field}`).toBe(200);
+      const { data } = (await presign.json()) as { data: { uploadUrl: string; fileId: string; headers?: Record<string, string> } };
+      const put = await stack.raw(String(data.uploadUrl).replace(/^https?:\/\/[^/]+/, ''), {
+        method: 'PUT', headers: data.headers ?? { 'content-type': 'text/plain' }, body: bytes,
+      });
+      expect(put.status, `upload for ${c.field}`).toBeLessThan(300);
+      const complete = await stack.apiAs(token, 'POST', '/storage/upload/complete', { fileId: data.fileId });
+      expect(complete.status, `complete for ${c.field}`).toBe(200);
+      fileIds[c.field] = String(data.fileId);
+    }
+
     // Build the create body from every entry that carries a `write` value
     // (+ required name). `present`/`computed` server-owned fields are skipped.
-    // A reference placeholder resolves to the id seeded above.
+    // A reference placeholder resolves to the id seeded above, and a media
+    // field to the file uploaded above.
     const body: Record<string, unknown> = { name: 'zoo-roundtrip' };
     for (const c of MATRIX) {
       if (!('write' in c.check) || c.check.write === undefined) continue;
-      body[c.field] = c.field in referenceIds ? referenceIds[c.field] : c.check.write;
+      body[c.field] = c.field in referenceIds
+        ? referenceIds[c.field]
+        : c.field in fileIds ? fileIds[c.field] : c.check.write;
     }
 
     const created = await stack.apiAs(token, 'POST', '/data/showcase_field_zoo', body);
@@ -88,6 +116,13 @@ describe('dogfood: field-type capability matrix round-trips over HTTP (#2004)',
       const actual = record[c.field];
       switch (c.check.kind) {
         case 'equal':
+          // A media field reads back in its expanded form, the file object
+          // the stored id resolves to (ADR-0104 D3 wave 2), so its `id` is
+          // the stored id.
+          if (c.field in fileIds) {
+            expect(actual).toMatchObject({ id: fileIds[c.field] });
+            break;
+          }
           expect(actual).toEqual(
             c.field in referenceIds ? referenceIds[c.field] : c.check.write,
           );

packages/qa/dogfood/test/field-zoo.matrix.ts

@@ -138,9 +138,10 @@ export const MATRIX: FieldCase[] = [
   { field: 'f_record', type: 'record', check: { kind: 'equal', write: { home: '+1', work: '+2' } } },
   // media — the STORED form is an opaque sys_file id (ADR-0104 D3 wave 2); the
   // inline `{url, …}` blob is the expanded READ form, derived by the resolver
-  // rather than written. These ids match no sys_file row here, which is the
-  // point: the round-trip must return the stored id verbatim when there is
-  // nothing to expand it into.
+  // rather than written. The `write` values are stand-ins in the stored shape
+  // (the value-shape test parses them): [#26043] a write naming an id that is
+  // no file of the organization is refused, so the HTTP suite uploads a real
+  // file for each and asserts the read form carries its id.
   { field: 'f_video', type: 'video', check: { kind: 'equal', write: 'file_zoo_video' } },
   { field: 'f_audio', type: 'audio', check: { kind: 'equal', write: 'file_zoo_audio' } },
   { field: 'f_composite', type: 'composite', check: { kind: 'equal', write: { label: 'x', n: 1 } } },

packages/services/service-storage/src/file-reference-lifecycle.test.ts

@@ -1081,6 +1081,9 @@ describe('File Reference Ownership (ADR-0104 D3 wave 2)', () => {
  * file ids; `file-reference-read-verdict.integration.test.ts` pins the real
  * binding against the door. Each refusal pin below is ablated in the PR that
  * introduced it.
+ *
+ * [#26043] A file the verdict refuses refuses the write at the field, before
+ * the driver runs; an id that names no file gets the same answer.
  */
 describe('a write references only a file its caller may read', () => {
   const CALLER = { userId: 'u_writer', tenantId: 'org_1' };
@@ -1146,34 +1149,45 @@ describe('a write references only a file its caller may read', () => {
   const ownedByP1 = () => file({ ref_object: 'product', ref_id: 'p1', ref_field: 'image' });
   const sysFileWrites = (engine: Engine) =>
     engine.calls.filter((c) => (c.op === 'update' || c.op === 'insert') && c.object === 'sys_file');
+  const refusedBy = (p: Promise<unknown>) => p.then(() => null, (e) => e);
+
+  /** [#26043] The field refusal, as the HTTP boundary answers it: one `invalid_value` entry per refused field. */
+  const expectFieldRefused = (err: unknown, fields: string[]) => {
+    const answered = resolveThrownHttpError(err);
+    expect({ status: answered.status, code: answered.code }).toEqual({ status: 400, code: 'VALIDATION_FAILED' });
+    expect((answered.details?.fields as any[]).map((f) => [f.field, f.code]))
+      .toEqual(fields.map((f) => [f, 'invalid_value']));
+  };
 
   describe('a file another record owns', () => {
-    it('is not copied for a caller the verdict refuses: the value is kept as written', async () => {
+    it('is refused at the field for a caller the verdict refuses, and not copied', async () => {
       const engine = fakeEngine({ files: [ownedByP1()], records: { product: [{ id: 'p1', image: 'file_a' }] } });
       const { authorizeRead } = verdict([]);
       const { storage } = installWith(engine, authorizeRead);
 
-      const row = await insertAs(engine, 'product', { image: 'file_a' }, 'p2');
+      const err = await refusedBy(insertAs(engine, 'product', { image: 'file_a' }, 'p2'));
 
-      expect(row.image).toBe('file_a');
+      expectFieldRefused(err, ['image']);
+      expect((engine.tables.product ?? []).map((r) => r.id)).toEqual(['p1']);
       expect(storage.download).not.toHaveBeenCalled();
       expect(storage.upload).not.toHaveBeenCalled();
       expect(engine.tables.sys_file).toHaveLength(1);
       expect(engine.tables.sys_file[0]).toMatchObject({ id: 'file_a', ref_object: 'product', ref_id: 'p1', ref_field: 'image' });
       expect(sysFileWrites(engine)).toEqual([]);
     });
 
-    it('is not copied by an update for a caller the verdict refuses', async () => {
+    it('is refused at the field by an update for a caller the verdict refuses, and not copied', async () => {
       const engine = fakeEngine({
         files: [ownedByP1()],
         records: { product: [{ id: 'p1', image: 'file_a' }, { id: 'p2', image: null }] },
       });
       const { authorizeRead } = verdict([]);
       const { storage } = installWith(engine, authorizeRead);
 
-      const row = await updateAs(engine, 'product', 'p2', { image: 'file_a' });
+      const err = await refusedBy(updateAs(engine, 'product', 'p2', { image: 'file_a' }));
 
-      expect(row?.image).toBe('file_a');
+      expectFieldRefused(err, ['image']);
+      expect(engine.tables.product.find((r) => r.id === 'p2')?.image).toBeNull();
       expect(storage.download).not.toHaveBeenCalled();
       expect(engine.tables.sys_file).toHaveLength(1);
       expect(engine.tables.sys_file[0]).toMatchObject({ ref_id: 'p1' });
@@ -1199,25 +1213,28 @@ describe('a write references only a file its caller may read', () => {
   });
 
   describe('a file no record owns', () => {
-    it('is not claimed for a caller the verdict refuses', async () => {
+    it('is refused at the field for a caller the verdict refuses, and not claimed', async () => {
       const engine = fakeEngine({ files: [file()] });
       const { authorizeRead } = verdict([]);
       installWith(engine, authorizeRead);
 
-      const row = await insertAs(engine, 'product', { image: 'file_a' }, 'p1');
+      const err = await refusedBy(insertAs(engine, 'product', { image: 'file_a' }, 'p1'));
 
-      expect(row.image).toBe('file_a');
+      expectFieldRefused(err, ['image']);
+      expect(engine.tables.product ?? []).toEqual([]);
       expect(engine.tables.sys_file[0].ref_id ?? null).toBeNull();
       expect(sysFileWrites(engine)).toEqual([]);
     });
 
-    it('is not claimed by an update for a caller the verdict refuses', async () => {
+    it('is refused at the field by an update for a caller the verdict refuses, and not claimed', async () => {
       const engine = fakeEngine({ files: [file()], records: { product: [{ id: 'p1', image: null }] } });
       const { authorizeRead } = verdict([]);
       installWith(engine, authorizeRead);
 
-      await updateAs(engine, 'product', 'p1', { image: 'file_a' });
+      const err = await refusedBy(updateAs(engine, 'product', 'p1', { image: 'file_a' }));
 
+      expectFieldRefused(err, ['image']);
+      expect(engine.tables.product[0].image).toBeNull();
       expect(engine.tables.sys_file[0].ref_id ?? null).toBeNull();
       expect(sysFileWrites(engine)).toEqual([]);
     });
@@ -1227,7 +1244,7 @@ describe('a write references only a file its caller may read', () => {
       const { authorizeRead } = verdict([]);
       installWith(engine, authorizeRead);
 
-      await insertAs(engine, 'product', { image: 'file_a' }, 'p1');
+      expectFieldRefused(await refusedBy(insertAs(engine, 'product', { image: 'file_a' }, 'p1')), ['image']);
 
       expect(engine.tables.sys_file[0]).toMatchObject({ status: 'deleted', deleted_at: '2026-01-01T00:00:00.000Z' });
       expect(engine.tables.sys_file[0].ref_id ?? null).toBeNull();
@@ -1243,21 +1260,55 @@ describe('a write references only a file its caller may read', () => {
       expect(engine.tables.sys_file[0]).toMatchObject({ ref_object: 'product', ref_id: 'p1', ref_field: 'image' });
     });
 
-    it('each id of a multiple:true field is judged on its own', async () => {
+    it('one id of a multiple:true field the caller may not read refuses the field, and none of its ids is taken', async () => {
       const engine = fakeEngine({ files: [file(), file({ id: 'file_b', key: 'user/file_b.png' })] });
       const { authorizeRead } = verdict(['file_b']);
       installWith(engine, authorizeRead);
 
-      const row = await insertAs(engine, 'product', { gallery: ['file_a', 'file_b'] }, 'p1');
+      expectFieldRefused(await refusedBy(insertAs(engine, 'product', { gallery: ['file_b', 'file_a'] }, 'p1')), ['gallery']);
 
-      expect(row.gallery).toEqual(['file_a', 'file_b']);
       expect(engine.tables.sys_file.map((f) => [f.id, f.ref_id ?? null])).toEqual([
         ['file_a', null],
-        ['file_b', 'p1'],
+        ['file_b', null],
       ]);
     });
   });
 
+  it('[#26043] an id that names no file gets the answer a file the caller may not read gets', async () => {
+    const engine = fakeEngine({ files: [file()] });
+    const { authorizeRead } = verdict([]);
+    installWith(engine, authorizeRead);
+
+    const unreadable = await refusedBy(insertAs(engine, 'product', { image: 'file_a' }, 'p1'));
+    const nowhere = await refusedBy(insertAs(engine, 'product', { image: 'file_nowhere' }, 'p2'));
+
+    expectFieldRefused(nowhere, ['image']);
+    expect(resolveThrownHttpError(nowhere)).toEqual(resolveThrownHttpError(unreadable));
+    expect(engine.tables.product ?? []).toEqual([]);
+  });
+
+  it('[#26043] every refused field is named, in one answer', async () => {
+    const engine = fakeEngine({ files: [file()] });
+    const { authorizeRead } = verdict([]);
+    installWith(engine, authorizeRead);
+
+    const err = await refusedBy(insertAs(engine, 'product', { image: 'file_a', gallery: ['file_nowhere'] }, 'p1'));
+
+    expectFieldRefused(err, ['image', 'gallery']);
+  });
+
+  it('[#26043] a refused field fails the write before another field\'s file is copied', async () => {
+    const engine = fakeEngine({ files: [ownedByP1()], records: { product: [{ id: 'p1', image: 'file_a' }] } });
+    const { authorizeRead } = verdict(['file_a']);
+    const { storage } = installWith(engine, authorizeRead);
+
+    const err = await refusedBy(insertAs(engine, 'product', { image: 'file_a', gallery: ['file_nowhere'] }, 'p2'));
+
+    expectFieldRefused(err, ['gallery']);
+    expect(storage.download).not.toHaveBeenCalled();
+    expect(engine.tables.sys_file).toHaveLength(1);
+  });
+
   it('constraint verdicts do not run ahead of the read check', async () => {
     const engine = fakeEngine({
       files: [file({ mime_type: 'application/pdf', size: 10_000_000 })],
@@ -1269,9 +1320,10 @@ describe('a write references only a file its caller may read', () => {
     const { authorizeRead } = verdict([]);
     installWith(engine, authorizeRead);
 
-    const row = await insertAs(engine, 'product', { image: 'file_a' }, 'p1');
+    const err = await refusedBy(insertAs(engine, 'product', { image: 'file_a' }, 'p1'));
 
-    expect(row.image).toBe('file_a');
+    // The read refusal, not the constraint's `ERR_FILE_CONSTRAINT`.
+    expectFieldRefused(err, ['image']);
     expect(engine.tables.sys_file[0].ref_id ?? null).toBeNull();
   });
 
@@ -1298,7 +1350,7 @@ describe('a write references only a file its caller may read', () => {
     installWith(engine, authorizeRead);
     const context = { ...CALLER, permissions: ['member_default'], __operationPrivate: 'dropped' };
 
-    await insertAs(engine, 'product', { image: 'file_a' }, 'p2', { session, context });
+    await refusedBy(insertAs(engine, 'product', { image: 'file_a' }, 'p2', { session, context }));
 
     expect(asked.length).toBeGreaterThan(0);
     for (const q of asked) {