Skip to content
Merged
9 changes: 9 additions & 0 deletions .changeset/15953-s8-cli-dashboard-windows-flag.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
---
'@objectstack/cli': patch
---

fix(cli): `os verify --dashboard-windows` describes the measures it proves: counts, sums and derived measures

Clause-②: no (the flag, its default and what it accepts are unchanged; only its help text moves)

The `--dashboard-windows` flag description ended "(date and datetime fields; count measures)". The date-window family in `@objectstack/verify` now also proves sums and derived measures over counts or sums, so it reads FROM "(date and datetime fields; count measures)" TO "(date and datetime fields; count, sum and derived measures)". Nothing else about the flag changes.
16 changes: 16 additions & 0 deletions .changeset/15953-s8-window-sum-derived.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
---
'@objectstack/verify': patch
---

fix(verify): the dashboard date-window family also proves sums and derived measures, and says why an average stays out

Clause-②: no (no export, option or type moves: the family reads the measures it already read, and a widget it used to exclude can now be proven or contradicted)

`runDashboardWindowProofs` (and `os verify --dashboard-windows`) now judges every verdict in fixture rows, so it compares every measure whose difference can be read back as rows:

- **A `sum`** is probed. Each fixture row holds the summed field to the value the CRUD derivation synthesizes for it, or to the value the widget's own selection already holds there. The sum's difference over that value is the rows the window took in. FROM: a sum-only widget was `excluded` (`no-count-measure`), and a sum beside a count was an `excluded` measure (`not-a-count`). TO: `proven` or `contradicted` like a count. A sum of a relationship path, a summed field with no synthesizable value, and a held value that is not a non-zero number are `excluded` as `count-field-unsynthesizable`, and the reason names which.
- **A `derived` measure** (`ratio`, `sum`, `difference`, `product`) is probed through its operands, which the widget's own answer carries beside it. Each operand must be a count or a sum of the same dataset, and the fixture rows meet every operand's filter. Under each preset every operand must take in exactly the rows the window selects. The value the widget shows must equal `evaluateDerivedMeasures` (the server's own evaluator, from `@objectstack/service-analytics`) over that row's operands, or the preset is `contradicted` and the detail names both values. A derived measure over a derived one, or over a measure that is not a count or a sum, stays `excluded` (`not-a-count`), and the reason names the operand.
- **An `avg`, `min`, `max` or `count_distinct`** stays `excluded` (`not-a-count`, or `no-count-measure` for a widget with nothing else), with a reason that says why. What a fixture row does to it depends on rows the app already holds in the window: their number for an average, their extreme for a minimum or maximum, their values for a distinct count. Nothing the widget answers reveals that, so deriving it would mean guessing the data. The reason text changes: FROM `"avg_effort" aggregates avg of effort, not a row count` TO `"avg_effort" averages effort: an average is not additive — …`.
- The widget-level `no-count-measure` reason reads `no measure the widget selects can be compared (…)`. The `excludedKind` values are unchanged.
- **A new `not-run` cause.** The family gives up on a preset when a sum's own total is so large that its floating-point difference cannot resolve one fixture row. That needs 2^34 times the held value, and the detail says so.
- **Upgrading:** on an app whose dashboards bind sums or derived measures, `os verify --dashboard-windows` now probes those widgets. A window that does not reach them now fails the run (`hardFailures`), where before they were only counted as `excluded`.
10 changes: 6 additions & 4 deletions content/docs/deployment/cli.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -2209,8 +2209,9 @@ os verify --dashboard-windows # Also prove each dashboard date
`filterBindings.dateRange` names, or on the dashboard's `dateRange.field`
(`created_at` when unauthored), resolved against the object behind the
widget's dataset, and must be a `date` or `datetime` field; a widget with
`filterBindings: { dateRange: false }` is reported `unbound`. For each count
measure of a bound widget, fixture rows are written through the seed door
`filterBindings: { dateRange: false }` is reported `unbound`. For each
measure of a bound widget that is a count, a sum, or a derived measure over
those, fixture rows are written through the seed door
on every preset's first and last day and the day either side (never after
today), stamped at UTC noon, owned by the caller and holding every column
the widget's own filter, the dataset's filter and the measure's filter
Expand All @@ -2230,8 +2231,9 @@ os verify --dashboard-windows # Also prove each dashboard date
probe whose fixtures do not meet the widget's own selection, or that cannot
be written or read back as written, is reported inconclusive, never proven.
A held column is an equality, an `$in`, a `$nin` or `$ne` on a field that
declares its options, or `{current_user_id}`; an average, a sum or a derived
measure, and a filter the ground truth cannot model (`$or`, a range, a
declares its options, or `{current_user_id}`; an average, a minimum or
maximum, a distinct count, or a derived measure over one, and a filter the
ground truth cannot model (`$or`, a range, a
relationship path, a condition on the window's own field), are reported
`excluded`.
`--multi-tenant` boots org-scoped so tenant-isolation RLS policies
Expand Down
2 changes: 1 addition & 1 deletion packages/cli/src/commands/verify.ts
Original file line number Diff line number Diff line change
Expand Up @@ -155,7 +155,7 @@ export default class Verify extends Command {
default: false,
}),
'dashboard-windows': Flags.boolean({
description: "Also prove, for every dashboard that declares a dateRange, that each widget it binds counts, under each date-range preset of the picker, the rows that window selects: fixture rows written on known days, the widget asked through the dataset door with and without the window (date and datetime fields; count measures)",
description: "Also prove, for every dashboard that declares a dateRange, that each widget it binds counts, under each date-range preset of the picker, the rows that window selects: fixture rows written on known days, the widget asked through the dataset door with and without the window (date and datetime fields; count, sum and derived measures)",
default: false,
}),
'parent-write-reach': Flags.boolean({
Expand Down
10 changes: 6 additions & 4 deletions packages/verify/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,8 @@ derived from your own metadata:
master.
- **Dashboard date windows** (opt-in) — *every widget a dashboard's `dateRange`
binds counts, under each preset of the picker, the rows that window selects*:
per bound widget and count measure, fixture rows are written on known days
per bound widget and measure (a count, a sum, or a derived measure over
those), fixture rows are written on known days
around every preset's edges, and the widget is asked through the dataset door
with no window and under each of the 13 presets, before and after the
fixtures. Each difference must equal the fixture rows the window selects; a
Expand Down Expand Up @@ -502,18 +503,19 @@ compute one, so it is not read as "no grant anywhere". `summary.hardFailures` is
`contradicted + failed` objects plus personas that could not be provisioned.

**Dashboard date windows** (`runDashboardWindowProofs`), per widget a
dashboard's `dateRange` binds. Probes per count measure: the widget with no
dashboard's `dateRange` binds. Probes per measure that is a count, a sum or
derived over those: the widget with no
window, then each of the 13 presets, each counted before and after the
fixture rows:

| status | meaning |
| --- | --- |
| `proven` | under every preset the window counted exactly the fixture rows it selects |
| `contradicted` | under some preset the widget showed nothing where the window selects fixture rows (`blank`), or counted another number of them; a window that answers what no window answers under every preset is also named `inert` **(failure)** |
| `contradicted` | under some preset the widget showed nothing where the window selects fixture rows (`blank`), or counted another number of them, or a derived measure showed another value than the server's evaluator makes of its operands; a window that answers what no window answers under every preset is also named `inert` **(failure)** |
| `failed` | the door refused or faulted on a windowed query while the widget's own query answered **(failure)** |
| `inconclusive` | a fixture row could not be written or did not read back as written (an app hook rewrote it, or the read omits the column), the widget with no window did not count every fixture row (its own selection does not meet the fixtures), its own query is refused, the UTC day turned after the run started, or the caller's reference timezone is not UTC |
| `unbound` | the widget opts out of the window (`filterBindings: { dateRange: false }`) |
| `excluded` | the family cannot compare it: no count measure, a filter the ground truth cannot model, a window on a field that is missing or is not `date` / `datetime`, a dataset or dashboard that is undeclared or does not parse, or an object the app does not declare or the CRUD derivation cannot synthesize (`excludedKind` names which) |
| `excluded` | the family cannot compare it: no measure it can compare (an `avg`, `min`, `max` or `count_distinct` is not additive), a filter the ground truth cannot model, a window on a field that is missing or is not `date` / `datetime`, a dataset or dashboard that is undeclared or does not parse, or an object the app does not declare or the CRUD derivation cannot synthesize (`excludedKind` names which) |

`member-visible` everywhere usually means the app is single-tenant; pass
`--multi-tenant` (or `{ multiTenant: true }`) to register org-scoping so tenant
Expand Down
Loading
Loading