diff --git a/.claude/commands/mendix/lint.md b/.claude/commands/mendix/lint.md index 3c4bec2cb2..e672b95816 100644 --- a/.claude/commands/mendix/lint.md +++ b/.claude/commands/mendix/lint.md @@ -39,7 +39,7 @@ mxcli lint -p app.mpr --exclude System --exclude Administration | MPR004 | quality | ValidationFeedback - Validation feedback with empty message | | MPR005 | quality | ImageSource - IMAGE widgets with no source configured | | MPR006 | quality | EmptyContainer - Empty layout containers | -| MPR007 | security | PageNavigationSecurity - Navigation pages need allowed roles (CE0557); every user role must be able to open its home page (else CE2729 per widget) | +| MPR007 | security | PageNavigationSecurity - Navigation pages need allowed roles (CE0557); every user role must be able to open its home page (at Production security, else CE2729 per widget) | | MPR012 | correctness | LegacyImageWidget - staticimage/dynamicimage are unsupported by the React client (CE0582) | | SEC001 | security | NoEntityAccessRules - Persistent entities need access rules | | SEC002 | security | WeakPasswordPolicy - Password minimum length should be 8+ | diff --git a/.claude/commands/mxcli-dev/review.md b/.claude/commands/mxcli-dev/review.md index da4df5772a..fc9ae79710 100644 --- a/.claude/commands/mxcli-dev/review.md +++ b/.claude/commands/mxcli-dev/review.md @@ -52,6 +52,7 @@ proactively. Add a row after every review that surfaces something new. | 21 | A registry that reads only one source (an `embed.FS`) has no extension point, so the only way to customise its output is to hand-edit the generated block — which a digest fence then refuses to touch on the next run. The feature ends up hostile to the exact case it was built for | API design | When a guard refuses a user's edit, ask whether the thing being edited should have been authorable. Introduce a source abstraction (`fs.FS` + root) and let a project-local directory shadow the embedded set — the walk functions usually already take a root, so the change is contained | | 22 | A test helper that expands a template cross-products every list against every reference (e.g. every `@each $weight` list against every `url()`), inventing artefacts no asset ever shipped and failing on correct input | Test coverage | Scan positionally: a reference must be expanded against the loop it actually sits under. Prove the helper both fires on a real break (delete one shipped file) and stays quiet on correct input — a helper only checked against green code has not been shown to detect anything | | 23 | A copy-to-scaffold path renames files but not the identifiers built from the name (`@mixin mxcli--`, `@import "mxcli-"`), so two artefacts collide the moment both exist — and the symptom is a rule that silently compiles to nothing | Code correctness | Assert the structural contract on the *generated* artefact, not just the shipped ones: factor the built-in's contract test into a helper and run the scaffold through it. Verify once end to end against the real toolchain and record it | +| 24 | A stacked PR's base merged at an older head: commits pushed to the base branch after its merge never reach `main`, while both PRs read as fine (ako/mxcli#1108 lost its last two commits this way) | Merge hygiene | Before merging any stacked PR, and after pushing to a branch whose PR may already be merged, check each commit with `git merge-base --is-ancestor origin/main`; retarget the next PR to `main` so it carries the stranded commits, and say so in its description | | 24 | A writer leaves out a property Studio Pro always writes (an empty `Documentation`, the empty half of an attribute/association pair) or writes one Mendix does not declare. `mx check`, the build and the runtime stay green, but Mendix's merge/diff engine compares property NAMES and throws, so the document becomes unmergeable once Studio Pro saves it (mendixlabs/mxcli#1373) | BSON writing | The doctype gate's `checkPropertySets` catches it per script: `mx convert -p` re-serializes what is not canonical. Fix the writer, or record Mendix's value with `MXCLI_PROPERTY_SETS_RECORD` + `internal/propertysets/cmd/propdefaults`; never guess a default | | 24 | A skill or command still instructs work into a package the repo has deleted (`sdk/mpr` test locations, a symptom table moved to `findings/*.jsonl` years prior) — the doc reads as authoritative and every instruction in it is a compile error or a no-op | Docs quality | When a package is deleted or a doc is restructured, grep `.claude/` for its name in the same PR. A deletion that leaves the guidance behind is worse than no guidance, because the reader trusts it | | 25 | A doc or skill shows a CLI invocation nobody ran — a command name that does not exist (`mxcli dump-bson` for `mxcli bson dump`), or a flag form the parser rejects (`--compare "A" "B"` where `--compare` is a StringSlice needing `"A,B"`). Worst when copied FROM the command's own `--help`, which had the same error, so the doc looks sourced | Docs quality | Run every command a doc shows, against a real project, before committing it. If it came from `--help`, run that form too — the example in the help text is not evidence that it works | diff --git a/.claude/skills/fix-issue/findings/cmd-mxcli/2026-10-11-project-tree-offers-nodes-describe-cannot-answer.json b/.claude/skills/fix-issue/findings/cmd-mxcli/2026-10-11-project-tree-offers-nodes-describe-cannot-answer.json new file mode 100644 index 0000000000..b17fd0218a --- /dev/null +++ b/.claude/skills/fix-issue/findings/cmd-mxcli/2026-10-11-project-tree-offers-nodes-describe-cannot-answer.json @@ -0,0 +1 @@ +{"area": "cmd/mxcli", "date": "2026-10-11", "symptom": "Clicking a JavaScript action (or the Navigation root) in a project-tree client fails: `mxcli describe javascriptaction X` is 'Unknown type', and `describe navigation Navigation` looks for a profile named Navigation", "cause": "project-tree and describe each kept their own idea of the node types: the tree emits javascriptaction, pagetemplate, domainmodel and ~20 container types, describe's CLI map had no javascriptaction keyword (the executor and grammar did), and the tree's navigation node carries the qualified name 'Navigation', which is not a profile. Every client (vscode-mdl's openElement fallback list) papered over it with its own guess list", "file": "`cmd/mxcli/project_tree.go` (`treeTypesWithoutSource`, `markDescribable`), `cmd/mxcli/cmd_describe.go` (`describeMDLCommand`)", "insight": "Two hand-kept lists over the same set drift; make one side derive from the other and test the source, not a copy. project-tree now marks each node `describable` from describe's own dispatch, and the guard reads every `Type: \"x\"` literal out of project_tree.go so a new node type fails the test until someone decides whether it has MDL. Revert check: removing the javascriptaction case fails the guard with the exact type named. Measured on Evora: 5748 nodes, 4611 describable, 1137 containers/no-source", "refs": []} diff --git a/.claude/skills/fix-issue/findings/mdl-backend/2026-10-11-empty-widgets-list-omitted-on-new-owner.json b/.claude/skills/fix-issue/findings/mdl-backend/2026-10-11-empty-widgets-list-omitted-on-new-owner.json new file mode 100644 index 0000000000..4d21bb16d2 --- /dev/null +++ b/.claude/skills/fix-issue/findings/mdl-backend/2026-10-11-empty-widgets-list-omitted-on-new-owner.json @@ -0,0 +1 @@ +{"area": "mdl/backend", "date": "2026-10-11", "symptom": "a freshly built Forms$TabPage with no widgets (also DivContainer, GroupBox, LayoutGridColumn, ScrollContainerRegion, NavigationListItem) was written with no Widgets key at all, where Studio Pro stores Widgets: [2]", "cause": "the codec's new-element path emits only dirty properties; an untouched empty PartList is never dirty, so it was omitted. Only Forms$DataView had a per-type TypeDefaults MandatoryListMarkers entry for Widgets", "file": "modelsdk/codec/defaults.go + encoder.go (RegisterMandatoryPropertyList: a property-keyed marker that is also emitted empty on a new element); mdl/backend/modelsdk/widget_list_markers.go registers it for the 15 Widgets owners", "insight": "Sweep presence, not just values: a scan of every $Type with a Widgets key in TestApp + PedApp found it absent 0 times (TabPage 89/89, 20 empty; LayoutGridColumn 732/732; DivContainer 838/838; WidgetValue 20,775/20,775), so the list is mandatory per owner. Key it on owner+property in the codec rather than adding per-type RegisterTypeDefaults entries, which replace (not merge) an existing registration and clobbered defaults once already (#812). The control is a type Studio Pro never stores the key on (TabControl, ScrollContainer) - it must not gain one."} diff --git a/.claude/skills/fix-issue/findings/mdl-backend/2026-10-11-layoutgrid-column-previewwidth-reset-on-rewrite.json b/.claude/skills/fix-issue/findings/mdl-backend/2026-10-11-layoutgrid-column-previewwidth-reset-on-rewrite.json new file mode 100644 index 0000000000..144d386b0c --- /dev/null +++ b/.claude/skills/fix-issue/findings/mdl-backend/2026-10-11-layoutgrid-column-previewwidth-reset-on-rewrite.json @@ -0,0 +1 @@ +{"area": "mdl/backend", "date": "2026-10-11", "symptom": "describe page -> exec on a Studio Pro-built project rewrote Forms$LayoutGridColumn.PreviewWidth from a stored 1 to -1 (ako/mxcli-ledger FINDINGS #170)", "cause": "layoutGridColumnToGen hardcodes SetPreviewWidth(-1) and nothing carried the stored value; PreviewWidth is a Studio Pro editor-canvas value with no MDL spelling", "file": "mdl/backend/modelsdk/widget_layoutgrid_carry.go (carryLayoutGridPreviewWidth, wired in UpdatePage/UpdateSnippet beside carryNoActionExecution)", "insight": "The hardcode matched 731 of 732 stored columns in TestApp+PedApp, so a fixture sweep never showed it; one real project did. A nameless element is carried by position inside its nearest named ancestor (grid name, row i, column j), and only when the stored row has the same column count, so an edited row falls back to the new-column default instead of mispairing. Set only a differing value: an equal set dirties a kept element (#721 L4)."} diff --git a/.claude/skills/fix-issue/findings/mdl-backend/2026-10-11-open-link-static-address-drops-null-attributeref.json b/.claude/skills/fix-issue/findings/mdl-backend/2026-10-11-open-link-static-address-drops-null-attributeref.json new file mode 100644 index 0000000000..fb68a13c02 --- /dev/null +++ b/.claude/skills/fix-issue/findings/mdl-backend/2026-10-11-open-link-static-address-drops-null-attributeref.json @@ -0,0 +1 @@ +{"area": "mdl/backend", "date": "2026-10-11", "symptom": "describe page -> exec on a Studio Pro-built project dropped `AttributeRef: null` from the Forms$StaticOrDynamicString Address of every static OPEN_LINK button (Forms$OpenLinkClientAction), so an unchanged page was rewritten (ako/mxcli-ledger FINDINGS #170)", "cause": "staticAddressToGen never set AttributeRef and no type default emitted it; its comment conflated the stored AttributeRef key with gen's extra, never-stored `Attribute` key, so the omission looked deliberate", "file": "mdl/backend/modelsdk/widget_write.go (init registering Forms$StaticOrDynamicString NullFields, staticAddressToGen)", "insight": "Two near-identical key names, one stored and one invented by gen: the earlier 'never invent a key' reasoning was right about `Attribute` and wrongly extended to `AttributeRef`. A key-set sweep of the references (21 of 21 static addresses carry AttributeRef: null) settles it in seconds; a NullFields default is safe for the dynamic branch because it only fills an unset key. Grepped first that no other RegisterTypeDefaults existed for the type, since a second registration replaces the first (#812). The default is per $Type, so it also reaches a navigation menu item's address; the #980 test's hand-built menu fixture lacked the key and failed. `mx convert -p` (11.15.0) on a menu item written without it adds AttributeRef: null, so the fixture, not the writer, was wrong: when no stored reference exists for a context, convert is the measurement."} diff --git a/.claude/skills/fix-issue/findings/mdl-backend/2026-10-11-rest-inline-value-mapping-writes-isdefaulttype.json b/.claude/skills/fix-issue/findings/mdl-backend/2026-10-11-rest-inline-value-mapping-writes-isdefaulttype.json new file mode 100644 index 0000000000..f8c926284b --- /dev/null +++ b/.claude/skills/fix-issue/findings/mdl-backend/2026-10-11-rest-inline-value-mapping-writes-isdefaulttype.json @@ -0,0 +1 @@ +{"area": "mdl/backend", "date": "2026-10-11", "symptom": "Projects written by mxcli carry IsDefaultType on ImportMappings$ValueMappingElement (x49) and ExportMappings$ValueMappingElement (x1), a key Mendix declares only on the object mapping element, so mx diff / merge throws while mx check and the build stay green", "cause": "#882 removed the key from the standalone import/export-mapping writer (mapping_write.go), but the consumed-REST inline mapping writer (restInlineMappingElementToGen, used for both Body: MAPPING and Response: MAPPING) builds its own value elements and still wrote addBool(v, \"IsDefaultType\", false). The legacy modelsdk/mpr serialize_mappings.go / serialize_web_services.go had the same line but had no caller anywhere, tests included", "file": "mdl/backend/modelsdk/consumed_rest_write.go restInlineMappingElementToGen; test mdl/backend/modelsdk/consumed_rest_isdefaulttype_test.go", "insight": "A per-type property fix is only done when every builder of that $Type is fixed: grep the storage name (\"$ValueMappingElement\") and the key across the repo, not the writer named in the report. Mapping elements have three builders here (import mapping, export mapping, REST inline), and the inline one is the high-count source in real projects because every REST operation with a mapped body or response gets one. Assert on the encoded BSON (codec.Encoder), walking nested children and counting visited value elements, so an empty tree cannot pass. Delete dead legacy twins rather than patch them, after grepping their exported entry points for callers", "refs": ["ako/mxcli#1097", "#882"]} diff --git a/.claude/skills/fix-issue/findings/mdl-backend/2026-10-11-template-grid-contents-widgets-marker-measured.json b/.claude/skills/fix-issue/findings/mdl-backend/2026-10-11-template-grid-contents-widgets-marker-measured.json new file mode 100644 index 0000000000..3cb7365237 --- /dev/null +++ b/.claude/skills/fix-issue/findings/mdl-backend/2026-10-11-template-grid-contents-widgets-marker-measured.json @@ -0,0 +1 @@ +{"area": "mdl/backend", "date": "2026-10-11", "symptom": "Forms$TemplateGridContents.Widgets was written with marker 3 when its first child had no child-type registration (e.g. a TabControl), when emptied, and on selective rebuild, and was omitted when new and empty; Studio Pro writes [2]", "cause": "TemplateGridContents was left off the Widgets property-marker owner list because neither reference app has a template grid", "file": "mdl/backend/modelsdk/widget_list_markers.go (Forms$TemplateGridContents added to widgetsListOwners)", "insight": "No reference document is not no evidence: mx convert (11.15.0) re-serializes every unit, so injecting a deliberately wrong marker ([3]) and reading the converted BSON measures Studio Pro's serializer directly - with a control (a DivContainer written [3] comes back [2]) proving the unit was rewritten. A bare injected element is also filled out by convert, which shows mandatory keys. The same probe settles which gen types are dead: convert refuses Forms$TableCell, Forms$VerticalFlow and NativePages$NativeLayoutCallArgument/NativePage ('type cache does not contain a type'), and an 11.x table cell is Forms$DbTableCell, which gen does not model (decoded opaque), so registering any of them would be inert."} diff --git a/.claude/skills/fix-issue/findings/mdl-backend/2026-10-11-widgets-list-marker-follows-first-child-type.json b/.claude/skills/fix-issue/findings/mdl-backend/2026-10-11-widgets-list-marker-follows-first-child-type.json new file mode 100644 index 0000000000..90ede464e1 --- /dev/null +++ b/.claude/skills/fix-issue/findings/mdl-backend/2026-10-11-widgets-list-marker-follows-first-child-type.json @@ -0,0 +1 @@ +{"area": "mdl/backend", "date": "2026-10-11", "symptom": "describe page -> exec on a Studio Pro-built project rewrote the typed-array marker of a LayoutGridColumn.Widgets list whose first child is a Forms$TabControl from 2 to 3 (ako/mxcli-ledger FINDINGS #170)", "cause": "codec partListMarker derives a list's marker from its first child's $Type via RegisterListMarker, defaulting to 3; Forms$TabControl had no registration. The same 3 default also hit an emptied Widgets list and the selective-rebuild path (a stored list with one dirty child), which never consult child-type markers", "file": "mdl/backend/modelsdk/widget_list_markers.go (RegisterPropertyListMarker(owner, \"Widgets\", 2) for the 15 evidenced owners)", "insight": "The marker belongs to the property, not to the child: 22,000+ Widgets lists across TestApp and PedApp, every owner, empty or not, are 2, while TabControl.TabPages is 3 (32/32). Per-child registration is whack-a-mole that leaves every unregistered widget type and both the empty and the selective paths on 3; a one-file sweep of markers grouped by owner.key and first child settles it. Register by STORAGE owner name (Forms$FormCallArgument, not gen's LayoutCallArgument)."} diff --git a/.claude/skills/fix-issue/findings/mdl-executor/2026-10-11-create-modify-edit-inside-loop-body-refused.json b/.claude/skills/fix-issue/findings/mdl-executor/2026-10-11-create-modify-edit-inside-loop-body-refused.json new file mode 100644 index 0000000000..8c6b9b846d --- /dev/null +++ b/.claude/skills/fix-issue/findings/mdl-executor/2026-10-11-create-modify-edit-inside-loop-body-refused.json @@ -0,0 +1 @@ +{"area": "mdl/executor", "date": "2026-10-11", "symptom": "ako/mxcli#1106: under `mdl 1;` a re-run `create or modify microflow` whose only change is one expression in a `set` inside a `loop` body is refused: \"the Loop at (x, y) changes inside its body; the splice does not edit inside a loop, and replacing the whole loop would rebuild every node it holds. Nothing was written. [MDL-V1-REBUILD]\" — so one script cannot both provision and update a flow that loops; the advised `alter … replace loop` means stating the body twice or giving up provisioning", "cause": "patchDiff.gap refused as soon as sameLoopShell held and the bodies differed (loopBodyChanged), without asking how they differed; alter's applyTo refused any in-loop target (enclosingLoop); the mutator's graph.node refused any node with n.loop set. The stated reason (body coordinates are relative to the loop box) only applies to changes that add, remove or move a node", "file": "mdl/executor/cmd_flow_modify.go (loopBody, inLoopBody, editableLoopBody, storedLocator.within, lcsStatements), mdl/executor/cmd_alter_flow.go (applyTo in-loop guard, upstreamOf/iteratorsAround/loopsAround), mdl/backend/mfmutator/splice.go (replaceInLoop)", "fix": "A loop body of the same shape — same length, each statement as stored, a nested loop of the same shape, or the same kind of plain activity (no if/return/break/continue, no custom error handler) whose annotations state nothing the stored one lacks — is diffed into in-place replaces, located among the loop's own objects (body @positions are loop-relative, so the top-level locator cannot be used). The mutator swaps the activity's document in the loop collection's slot, at the stored RelativeMiddlePoint and Size, and repoints the flows in/out (the last body activity has no flow out — removable() rejected it). alter accepts the same one-for-one replace; scope inside a body adds the enclosing loops' iterators and what runs before each loop. Such loops are also paired by lcsStatements so a change beside the loop does not merge with it into a replace", "test": "mdl/executor/cmd_flow_loop_body_edit_pedapp_test.go TestCreateOrModify_EditInsideLoopBody (only the edited activity's $ID changes, replacement at the same position, re-run Unchanged, iterator-reading edit, change beside the loop + inside it, controls insert/drop/swap/kind-change still refused with nothing written), TestCreateOrModify_EditInsideNestedLoopBody; mfmutator splice_test.go TestSplice_ReplaceInsideALoopIsInPlace (byte-exact expected unit, first and last body activity, resized stored Size kept, two-for-one refused). Fix reverted: both executor tests fail with 'changes inside its body; the splice does not edit inside a loop', the mutator test with 'alter does not splice inside a loop yet'. Real 11.14.0 app (mxcli new): old binary refuses with MDL-V1-REBUILD under mdl 1 and 'Replaced' (rebuild) under mdl 0; fixed binary 'Modified … (spliced: 1 replaced)', re-run in sync; mx check 0 errors on both copies", "insight": "A refusal justified by geometry (loop-relative coordinates) should be asked of the changes that actually have geometry: an in-place replace of one activity by one of the same kind moves nothing, so recurse into the body when it has the same shape and keep refusing only shape changes. Two traps on the way: describe's body @positions are relative to the loop box, so locating a body statement must use a locator over that loop's objects only; and the last activity of a loop body has no outgoing flow, so the top-level 'exactly one flow out' removability check does not apply there. When validating with the CLI binary, rebuild it after the fix — a bin/mxcli built before the change reproduced the refusal and looked like the fix failing on a real app", "refs": ["ako/mxcli#1106", "ako/mxcli#942"]} diff --git a/.claude/skills/fix-issue/findings/mdl-executor/2026-10-11-source-attribute-change-leaves-view-entities-out-of-sync.json b/.claude/skills/fix-issue/findings/mdl-executor/2026-10-11-source-attribute-change-leaves-view-entities-out-of-sync.json new file mode 100644 index 0000000000..cfcadfc88d --- /dev/null +++ b/.claude/skills/fix-issue/findings/mdl-executor/2026-10-11-source-attribute-change-leaves-view-entities-out-of-sync.json @@ -0,0 +1 @@ +{"area": "mdl/executor", "date": "2026-10-11", "symptom": "Changing a source attribute's Localize (`DateTime` -> `DateTime not localized`) with `alter entity … modify attribute` or `create or modify entity` leaves every view entity selecting it with the old LocalizeDate; `mxcli check`/`exec` are silent and `mx check` reports CE6770 \"View Entity is out of sync with the OQL Query\" once per view. A String length / kind / enumeration change on the source breaks a pass-through view column the same way", "cause": "A view column's LocalizeDate was derived from its source (viewDateTimeLocalize, #1297) only when the view statement itself ran; nothing on the source-entity write paths looked at the views reading it", "file": "mdl/executor/oql_view_resync.go (resyncViewsOverSource), called from execAlterEntity modify attribute and the create-or-modify branch of execCreateEntity in mdl/executor/cmd_entities.go", "insight": "A property derived from another document at write time goes stale when that other document changes: every derivation needs a re-derivation hook on the source's write paths, not only on the deriving statement's. Diff type signatures (kind, length, enum ref, localize) before/after the source write, resolve `alias.Attr` refs per view column through extractAliasMap, rewrite only a column whose derived value differs (repeat run writes nothing; a view not reading the changed attribute is the control), chain to views over the resynced view. What the view text declares (length, kind, enum) cannot be re-derived without rewriting the user's statement, so it is an exec-time warning naming view and column — check cannot see it because check never applies the source change. Ledger FINDINGS #169", "refs": ["ako/mxcli-ledger FINDINGS #169", "#1297"]} diff --git a/.claude/skills/fix-issue/findings/mdl-executor/2026-10-11-submit-behaviour-and-grid-width-reset-on-rewrite.json b/.claude/skills/fix-issue/findings/mdl-executor/2026-10-11-submit-behaviour-and-grid-width-reset-on-rewrite.json new file mode 100644 index 0000000000..2b6d6c7f22 --- /dev/null +++ b/.claude/skills/fix-issue/findings/mdl-executor/2026-10-11-submit-behaviour-and-grid-width-reset-on-rewrite.json @@ -0,0 +1 @@ +{"area": "mdl/executor", "date": "2026-10-11", "symptom": "`describe page` → `exec` over a Studio Pro page rewrites a text box's / text area's `SubmitBehaviour` WhileEditing → OnEndEditing (and `SubmitOnInputDelay` to 300) and a layout grid's `Width` FixedWidth → FullWidth, `mx check` clean (ako/mxcli-ledger FINDINGS #170). Worse, `check` passed `SubmitBehaviour: WhileEditing` (MDL-WIDGET07 is only a warning) and exec dropped it, and `Width: FixedWidth` on a layoutgrid drew not even the warning", "cause": "All four layers missing at once: the writer hard-coded the three values (widget_write.go TextBox, TextArea, LayoutGrid cases), sdk/pages had no field, describe neither read nor printed them, the builder consumed nothing. `Width` was silent because it sits in `staticWidgetKnownProps`, the cross-widget union, for the pixel widths of other widgets", "file": "`mdl/executor/input_submit_width.go` (read, validate, describe), `cmd_pages_builder_v3_widgets.go` / `_layout.go`, `cmd_pages_describe_parse.go` / `_output.go`, `validate_widgets.go`, `sdk/pages/submit_width.go`, `mdl/backend/modelsdk/widget_write.go`, `mdl/backend/pagemutator/mutator.go` (alter set), `mdl/backend/mcp/page_widgets.go`", "insight": "**Grep the writer for literal setter arguments** (`Set…(\"…\")`, `Set…(300)`) — every one is a stored value MDL cannot express and a describe → exec reset waiting for a Studio Pro page that differs; PedApp already had both (FeedbackModule.ShareFeedback: three WhileEditing inputs; MyFirstModule.Home_Web: two FixedWidth grids), so the Studio Pro fixture is the test, no hand-patched BSON needed. **A key in the known-props union is not evidence it is consumed**: `Width` passed check for layoutgrid because image/container widths put it there. Keep a per-type key OUT of the union (exempt it per type like the #968 formatting keys, reading and emitting it from a file the vocabulary test does not scan) so the same key on another widget still warns. One validation function feeds builder, check (MDL-WIDGET18) and the sdk/pages canonicaliser feeds alter set, so the three cannot disagree. Revert proof: restoring the three hard-coded setters fails the PedApp round trip with exactly the reported WhileEditing → OnEndEditing / FixedWidth → FullWidth", "refs": ["ako/mxcli-ledger FINDINGS #170", "#550", "#968"]} diff --git a/.claude/skills/fix-issue/findings/mdl-other/2026-10-11-mpr007-home-page-message-claims-ce2729-at-prototype.json b/.claude/skills/fix-issue/findings/mdl-other/2026-10-11-mpr007-home-page-message-claims-ce2729-at-prototype.json new file mode 100644 index 0000000000..c60fdb9bce --- /dev/null +++ b/.claude/skills/fix-issue/findings/mdl-other/2026-10-11-mpr007-home-page-message-claims-ce2729-at-prototype.json @@ -0,0 +1 @@ +{"area": "mdl/linter", "date": "2026-10-11", "symptom": "MPR007's home-page warning said a user role 'cannot open' its home page and that 'mxbuild reports CE2729 for each widget on it' on a project at Prototype security, where `mx check` reports zero CE2729 and the role opens the page fine (HTTP 200, no banner) — only the data and actions it lacks access to are withheld (ako/mxcli-rest FINDINGS #68)", "cause": "The message was written from the one project that motivated the check (ChipCoV6, Production security). CE2729 is an entity-access error, and only Production (CheckEverything) checks entity access, so the consequence was level-dependent while the message stated it unconditionally", "file": "`mdl/linter/rules/page_navigation_security.go` (homePageConsequence); test `TestPageNavigationSecurity_PrototypeLevelClaimsNoCE2729`", "insight": "A lint message has two halves: the model fact (no module role of the user role is granted the page — true at every level) and the consequence (level-dependent). Measure the consequence at each level the rule fires at before naming a CE number; the rest project's `playwright check --role User --fresh-login` is what separated the two"} diff --git a/.claude/skills/fix-issue/findings/modelsdk/2026-10-11-removed-key-closeformaction-numberofpages-unmergeable.json b/.claude/skills/fix-issue/findings/modelsdk/2026-10-11-removed-key-closeformaction-numberofpages-unmergeable.json new file mode 100644 index 0000000000..a1104fc422 --- /dev/null +++ b/.claude/skills/fix-issue/findings/modelsdk/2026-10-11-removed-key-closeformaction-numberofpages-unmergeable.json @@ -0,0 +1 @@ +{"area": "modelsdk", "date": "2026-10-11", "symptom": "After `mxcli fix property-sets` completed every missing key on an 11.15 project (ChipCoV6 copy), Mendix's merge engine still saw a property-name mismatch: 4 `Microflows$CloseFormAction` elements carried `NumberOfPages`, which `mx convert -p` strips (ako/mxcli#1097 follow-up)", "cause": "An older mxcli wrote the int NumberOfPages (removed from the metamodel in 8.11; the current writer no longer does, #1373). The strip pass only dropped keys introduced AFTER the project's version, never keys removed BEFORE it, so stored units kept the stale name", "file": "`modelsdk/canon/propertyfloor.go` (measuredRemoved), `modelsdk/canon/studiopro_property_removed.json`; tests `TestStripUndeclared_KeyDeletedBeforeItsVersionIsDropped`, `TestStripUndeclared_KeyBeforeItsDeletionStays`", "insight": "Do NOT drive removal from the generated metamodel's Deleted versions: they are wrong for keys Studio Pro still writes (LayoutGridRow.Columns 'deleted 8.0', JavaAction.Parameters 'deleted 7.21'), and trusting them made `fix property-sets --dry-run` want to strip ~297,000 keys from the Studio Pro-authored pedapp fixture. The control that caught it — dry-run over Studio Pro-authored projects must report 0 units — is the one to run before adding any row to the property tables. Removals are measured rows only, like the floors table"} diff --git a/.claude/skills/fix-issue/findings/modelsdk/2026-10-11-stored-units-keep-property-name-gaps-no-rewrite-reaches.json b/.claude/skills/fix-issue/findings/modelsdk/2026-10-11-stored-units-keep-property-name-gaps-no-rewrite-reaches.json new file mode 100644 index 0000000000..a00cc46e55 --- /dev/null +++ b/.claude/skills/fix-issue/findings/modelsdk/2026-10-11-stored-units-keep-property-name-gaps-no-rewrite-reaches.json @@ -0,0 +1 @@ +{"area": "modelsdk/mpr", "date": "2026-10-11", "symptom": "A project built by an older mxcli stays unmergeable after the property-set completion shipped: `mx diff` still throws `Objects with ID … of type Microflows$RestOperationCallAction do not have the same properties` (ErrorHandlingType missing), and Projects$ModuleSettings, ChangeActionItem.Attribute and the rest stay incomplete even after re-running the whole MDL corpus (16 of 64 affected units rewritten) (ako/mxcli#1097)", "cause": "Reach, not elision. The suspicion was that normalizePropertySets running before canon.Reconcile made a rewrite of an incomplete unit compare equal and get elided; measured, it does not — canon.Equal compares the canonical form strictly, so the normalized bytes (with the key) differ from the stored ones (without it) and the write lands. The gaps live in units NO statement rewrites: module settings, documents whose script is not re-run, and any unit a statement reaches only by reading", "file": "`modelsdk/mpr/writer_property_sets.go` (`Writer.CompleteStoredPropertySets`, `Reader.StoredPropertySetGaps`, `normalizePropertySetsFor` shared with `normalizePropertySets`), `cmd/mxcli/cmd_fix_property_sets.go` (`mxcli fix property-sets [--dry-run]`); tests `modelsdk/mpr/writer_property_set_repair_test.go` (incl. `TestStoredUnitMissingAKeyIsCompletedByAnyRewrite`, the measurement of the premise), `cmd/mxcli/cmd_fix_property_sets_test.go` ; user-role GUID fill: `modelsdk/canon/userroleguid.go` (`FillMissingUserRoleGUIDs`, called from `normalizePropertySetsFor`, so every write path and the repair apply it), tests `modelsdk/canon/userroleguid_test.go`, `modelsdk/mpr/writer_userrole_guid_test.go`", "insight": "**Measure the premise before building the bypass.** The brief asked for an elision bypass; a ten-line writer test (stored unit missing a table key, UpdateRawUnit of the same bytes, read the file) showed the write lands, so the repair goes through the ORDINARY update path — Reconcile, the storage-GUID guard and every other guard — and needed no bypass at all. A write-path change that would have been justified by a false premise is the one that breaks rule 2 of 'Writes Are Conditional'. **A per-write fix only reaches what is written**: anything normalized at the choke point needs a sweep for the units already stored, or the fix is invisible on every existing project. **Decide on property NAMES, not bytes**: the sweep compares per-node key sets (by containment path — normalization never reorders a list), so it is idempotent by construction and a second run offers nothing. **The control is a Studio Pro-authored project**: `fix property-sets --dry-run` on testdata/pedapp (11.13.0), expr-checker (11.6.6) and testapp-views (11.14.0) reports 0 units — a table row adding a key Studio Pro does not write would show up there first **A per-element identity is not a table default**: `Security$UserRole.GUID` cannot be a CompletePropertySets row (the value differs per role), so it is filled by a dedicated pass with GUID = $ID — the create path's value — on ABSENT keys only. Measured on 11.15.0 + PostgreSQL: with no GUID mxbuild invents a random one per build; the runtime matches system$userrole by modelguid first, then name, and never refreshes modelguid on a name match; GUID = $ID on a GUID-less role kept the row id and every user-role link (a random GUID is also harmless — the control), while changing an EXISTING GUID would break the primary match. **The fill only protects databases first synced after it**: an existing database keeps the stale random modelguid and is still matched by name, so a later role rename there deletes the row and drops the non-demo user assignments. The storage-GUID guard does not refuse the fill — it compares GUIDs present on both sides.", "refs": ["ako/mxcli#1097", "mendixlabs/mxcli#1373"]} diff --git a/.claude/skills/mendix/agents/SKILL.md b/.claude/skills/mendix/agents/SKILL.md index a9e8e87574..4e27900396 100644 --- a/.claude/skills/mendix/agents/SKILL.md +++ b/.claude/skills/mendix/agents/SKILL.md @@ -2,7 +2,23 @@ name: agents description: "Author Mendix AI agent documents in MDL — Model, Knowledge Base, Consumed MCP Service and Agent, with variables, tools and multi-line prompts. Use when adding GenAI features to a Mendix app, wiring an LLM model or knowledge base, or listing/describing existing agent documents. Requires AgentEditorCommons and Mendix 11.9+." --- - + +## Find it fast + +`sed -n ',p' SKILL.md` one section rather than reading the file; +`./mxcli syntax ` answers *how do I write this* for a fraction of either. + +| lines | section | +|---|---| +| 24-50 | Overview | +| 51-137 | Syntax | +| 138-164 | Altering existing documents | +| 165-193 | Gotchas | +| 194-215 | Common Patterns | +| 216-251 | Calling Agents from Microflows | +| 252-254 | AI agent documents | + + # Agents ## Overview diff --git a/.claude/skills/mendix/alter-page/SKILL.md b/.claude/skills/mendix/alter-page/SKILL.md index c352ff1a5c..89ffc00eb3 100644 --- a/.claude/skills/mendix/alter-page/SKILL.md +++ b/.claude/skills/mendix/alter-page/SKILL.md @@ -2,7 +2,27 @@ name: alter-page description: "Modify an existing page or snippet's widget tree in place with ALTER PAGE / ALTER SNIPPET — SET, INSERT, DROP, REPLACE and SET Layout. Use when changing a caption, style or property, adding or removing a widget, or reordering a form, instead of rewriting the whole page with CREATE OR REPLACE. The only safe way to change a page or snippet authored in Studio Pro." --- - + +## Find it fast + +`sed -n ',p' SKILL.md` one section rather than reading the file; +`./mxcli syntax ` answers *how do I write this* for a fraction of either. + +| lines | section | +|---|---| +| 28-31 | Overview | +| 32-56 | When to Use | +| 57-106 | Syntax | +| 107-446 | Operations | +| 447-525 | Examples | +| 526-616 | DataGrid 2 columns: how to address them, and what you can set | +| 617-628 | Common Mistakes | +| 629-650 | Limitations — prefer binding at page creation (ledger finding #45) | +| 651-658 | Validation Checklist | +| 659-668 | Related Commands | +| 669-673 | Related Skills | + + # ALTER PAGE / ALTER SNIPPET - Modify Existing Pages and Snippets ## Overview diff --git a/.claude/skills/mendix/analyze-runtime/SKILL.md b/.claude/skills/mendix/analyze-runtime/SKILL.md index d8033e9ddd..ebb9dbfae3 100644 --- a/.claude/skills/mendix/analyze-runtime/SKILL.md +++ b/.claude/skills/mendix/analyze-runtime/SKILL.md @@ -2,7 +2,26 @@ name: analyze-runtime description: "Find out what a running Mendix app actually does — logs, Prometheus metrics, OpenTelemetry traces and the model catalog, joined across sources. Use when profiling a slow page or microflow, finding what hits the database, chasing an error that shows only a generic dialog, or correlating runtime cost with model shape." --- + +## Find it fast +`sed -n ',p' SKILL.md` one section rather than reading the file; +`./mxcli syntax ` answers *how do I write this* for a fraction of either. + +| lines | section | +|---|---| +| 27-41 | Overview | +| 42-51 | When to Use This Skill | +| 52-136 | 1. Logs — the first stop for errors | +| 137-151 | 2. Metrics — throughput and database pressure | +| 152-175 | 3. Traces — where the time goes | +| 176-193 | 4. Model shape — the catalog | +| 194-221 | 5. The app warehouse — join the signals (external DuckDB) | +| 222-229 | Known gap: logs ↔ traces | +| 230-239 | Decision guide | +| 240-245 | See Also | + + # Analyze an App's Runtime Behavior — Logs, Metrics, Traces, Catalog ## Overview diff --git a/.claude/skills/mendix/assess-migration/SKILL.md b/.claude/skills/mendix/assess-migration/SKILL.md index 2560e6031e..e76cff373d 100644 --- a/.claude/skills/mendix/assess-migration/SKILL.md +++ b/.claude/skills/mendix/assess-migration/SKILL.md @@ -2,7 +2,30 @@ name: assess-migration description: "Investigate an existing non-Mendix application (Java, .NET, Python, Node, PHP, …) and produce a structured migration assessment for Mendix. Use when asked to analyse a legacy codebase for migration, build a migration inventory, or size the effort before any MDL is written." --- - + +## Find it fast + +`sed -n ',p' SKILL.md` one section rather than reading the file; +`./mxcli syntax ` answers *how do I write this* for a fraction of either. + +| lines | section | +|---|---| +| 33-40 | When to Use This Skill | +| 41-295 | Investigation Process | +| 296-302 | Assessment Report Template | +| 303-309 | Executive Summary | +| 310-323 | Inventory Summary | +| 324-326 | data model | +| 327-329 | business Logic and rules | +| 330-332 | pages and UI | +| 333-335 | Integrations | +| 336-338 | security | +| 339-347 | Migration Risks | +| 348-357 | Recommended Migration Phases | +| 358-365 | Tips | +| 366-371 | Related Skills | + + # Migration Assessment: Investigating Non-Mendix Projects This skill guides the investigation of existing non-Mendix applications to produce a structured migration assessment for Mendix. diff --git a/.claude/skills/mendix/assess-quality/SKILL.md b/.claude/skills/mendix/assess-quality/SKILL.md index 3fc2ed3e69..e3ff4b8522 100644 --- a/.claude/skills/mendix/assess-quality/SKILL.md +++ b/.claude/skills/mendix/assess-quality/SKILL.md @@ -2,7 +2,26 @@ name: assess-quality description: "Audit a Mendix project against best practices for naming, security, performance, maintainability and architecture, and report a scored result. Use when asked to evaluate project quality, run a pre-go-live health check, or get oriented in an unfamiliar app." --- - + +## Find it fast + +`sed -n ',p' SKILL.md` one section rather than reading the file; +`./mxcli syntax ` answers *how do I write this* for a fraction of either. + +| lines | section | +|---|---| +| 29-36 | When to Use This Skill | +| 37-96 | Assessment Workflow | +| 97-351 | Quality Guidelines Reference | +| 352-362 | Assessment Report Template | +| 363-375 | Executive Summary | +| 376-386 | project Metrics | +| 387-399 | Automated Findings | +| 400-419 | Manual Review Findings | +| 420-435 | Recommendations | +| 436-446 | Sources | + + # Assess Mendix Project Quality This skill guides a comprehensive quality assessment of a Mendix project, evaluating it against established best practices across naming, security, performance, maintainability, architecture, and more. @@ -152,7 +171,7 @@ After reviewing automated results, assess the following areas manually. The guid |-----------|------|----------| | 1:1 mapping between module roles and user roles | CONV008 | High | | Pages in navigation must have allowed roles | MPR007 | High | -| Every user role can open its home page (else CE2729) | MPR007 | High | +| Every user role can open its home page (else CE2729 at Production security) | MPR007 | High | | No guest/anonymous access to sensitive data | SEC004 | Critical | | Strict security mode enabled | SEC005 | High | | No demo users in production | SEC003 | Critical | diff --git a/.claude/skills/mendix/atlas-design/SKILL.md b/.claude/skills/mendix/atlas-design/SKILL.md index 92538a2d81..81210ac8c6 100644 --- a/.claude/skills/mendix/atlas-design/SKILL.md +++ b/.claude/skills/mendix/atlas-design/SKILL.md @@ -2,7 +2,29 @@ name: atlas-design description: "Make a Mendix app look designed rather than default-Atlas: layout, spacing, typography, colour and design properties that reach a finished standard. Use when asked to make an app look professional, branded or less bland, when styling pages, or when matching a design mock." --- + +## Find it fast +`sed -n ',p' SKILL.md` one section rather than reading the file; +`./mxcli syntax ` answers *how do I write this* for a fraction of either. + +| lines | section | +|---|---| +| 30-43 | Reference files | +| 44-59 | When to Use This Skill | +| 60-77 | Contents | +| 78-97 | The thesis: be Atlas-first | +| 98-156 | The 4-layer architecture | +| 157-217 | The workflow: discover → inspect → use | +| 218-235 | Brand re-tune (Layer 1) — where most of the win is | +| 236-321 | Layer 1 in practice — start from the shipped theme | +| 322-362 | Dark mode — Mendix 11 makes this cheap | +| 363-405 | Verify at runtime — this is mandatory | +| 406-470 | Gotchas catalog | +| 471-488 | Validation Checklist | +| 489-496 | Related skills | + + # Atlas Design — Make a Mendix App Look Designed, Not Bland ## Reference files diff --git a/.claude/skills/mendix/bootstrap-app/SKILL.md b/.claude/skills/mendix/bootstrap-app/SKILL.md index c1bea4f568..6e58fb6fb0 100644 --- a/.claude/skills/mendix/bootstrap-app/SKILL.md +++ b/.claude/skills/mendix/bootstrap-app/SKILL.md @@ -2,7 +2,22 @@ name: bootstrap-app description: "Provision a Mendix project in a repo that has none — interview, `mxcli new`, session hook, project brief, first commit, boot and quality baseline. Use when the repository is empty or has no .mpr yet, typically from the empty-repo seed prompt." --- - + +## Find it fast + +`sed -n ',p' SKILL.md` one section rather than reading the file; +`./mxcli syntax ` answers *how do I write this* for a fraction of either. + +| lines | section | +|---|---| +| 23-43 | When to Use This Skill | +| 44-130 | Step 0 — interview, and WAIT for the answers | +| 131-263 | Provision | +| 264-333 | If this is a solution (several apps in one repo) | +| 334-391 | Then propose the model — do not build it yet | +| 392-455 | After bootstrap — the inner loop | + + # Bootstrap a Mendix App in an Empty Repo ## When to Use This Skill @@ -220,6 +235,29 @@ drop the `./` if it came pre-installed on `PATH`. normal local run. `--hub` ships in the **Linux** build only (a cloud session is a Linux container, so it works there); on a native Windows/macOS mxcli it fails with an explanatory message — continue as a normal local run. +11. **Hand over, and offer the next step.** Report what is ready in a few lines (the + app, the boot URL, the baseline), then two things: + + - **The project explorer.** `mxcli init` installed the *mxcli companion* in + `.claude/skills/mxcli-companion/`: a Claude Code pane that shows the app's + structure and the MDL of any document. It was written during this session, and + Claude Code only loads plugins it finds at session start, so tell the user to + run **`/reload-plugins`** and then **`/mxcli-project`** (or to start a new + session). Only the user can run those; do not try. The pane draws in the + terminal or the desktop app that runs the session; over remote control, on the + web or on a phone it may not show — say so rather than promise it. A non-Claude + tool has no companion: skip this point there. + - **A choice of what next.** Ask one question with your harness's question tool + (`AskUserQuestion` in Claude Code), and wait for the answer: + + | Option | What it leads to | + |---|---| + | Propose the domain model *(recommended)* | the next section | + | Import requirements from a document (a PRD, notes) | read it, record each requirement with `./mxcli brain capture "" --slice `, then propose the model from them | + | Open the project explorer | repeat the two commands above and stop | + | Something else | ask what | + + With no question tool, ask the same as a numbered list. --- diff --git a/.claude/skills/mendix/browse-integrations/SKILL.md b/.claude/skills/mendix/browse-integrations/SKILL.md index 73d09617c1..fdb0172526 100644 --- a/.claude/skills/mendix/browse-integrations/SKILL.md +++ b/.claude/skills/mendix/browse-integrations/SKILL.md @@ -2,7 +2,22 @@ name: browse-integrations description: "Discover the external services a project consumes and browse their cached contracts through the MDL CATALOG — OData entities and actions, REST clients and operations, AsyncAPI channels and messages. Use when asked what integrations exist, what an OData service offers, or what a business-event contract contains." --- - + +## Find it fast + +`sed -n ',p' SKILL.md` one section rather than reading the file; +`./mxcli syntax ` answers *how do I write this* for a fraction of either. + +| lines | section | +|---|---| +| 27-34 | When to Use This Skill | +| 35-62 | Discovery: List All Services | +| 63-91 | Contract Browsing: OData $metadata | +| 92-106 | Contract Browsing: AsyncAPI (Business Events) | +| 107-143 | Catalog Queries (requires REFRESH CATALOG) | +| 144-195 | Workflow: Import Entities from a Contract | + + # Browse Integration Services and Contracts This skill covers discovering external services, browsing cached contracts, and querying integration assets via the **MDL CATALOG** (local project metadata). diff --git a/.claude/skills/mendix/bulk-widget-updates/SKILL.md b/.claude/skills/mendix/bulk-widget-updates/SKILL.md index 22eabe5af6..13a81363f7 100644 --- a/.claude/skills/mendix/bulk-widget-updates/SKILL.md +++ b/.claude/skills/mendix/bulk-widget-updates/SKILL.md @@ -2,7 +2,21 @@ name: bulk-widget-updates description: "Discover and bulk-change widget properties across many pages with LIST WIDGETS and UPDATE WIDGETS. Use when the same widget setting has to change on many pages at once. Experimental — always DRY RUN first and back the project up." --- + +## Find it fast +`sed -n ',p' SKILL.md` one section rather than reading the file; +`./mxcli syntax ` answers *how do I write this* for a fraction of either. + +| lines | section | +|---|---| +| 36-42 | Prerequisites | +| 43-82 | LIST WIDGETS - Discover Widgets | +| 83-163 | UPDATE WIDGETS - Modify Properties | +| 164-201 | Important Notes | +| 202-223 | Workflow Example | + + # Bulk Widget Property Updates > **EXPERIMENTAL**: These commands are an untested proof-of-concept. diff --git a/.claude/skills/mendix/business-events/SKILL.md b/.claude/skills/mendix/business-events/SKILL.md index b5ddd4c842..5d19a5ba4e 100644 --- a/.claude/skills/mendix/business-events/SKILL.md +++ b/.claude/skills/mendix/business-events/SKILL.md @@ -2,7 +2,23 @@ name: business-events description: "Define event-driven APIs over Kafka with Mendix business event services — publish and subscribe contracts, CREATE/DROP/DESCRIBE. Use when building publish/subscribe integration between apps, or inspecting the business event services a project already has." --- - + +## Find it fast + +`sed -n ',p' SKILL.md` one section rather than reading the file; +`./mxcli syntax ` answers *how do I write this* for a fraction of either. + +| lines | section | +|---|---| +| 24-31 | When to Use This Skill | +| 32-97 | Commands | +| 98-113 | Message Definition Syntax | +| 114-121 | Service Properties | +| 122-128 | Operations | +| 129-165 | Publishing Events from Microflows | +| 166-177 | Checklist | + + # Business Events ## When to Use This Skill diff --git a/.claude/skills/mendix/catalog-search/SKILL.md b/.claude/skills/mendix/catalog-search/SKILL.md index 1fb7e33df4..d0167e6526 100644 --- a/.claude/skills/mendix/catalog-search/SKILL.md +++ b/.claude/skills/mendix/catalog-search/SKILL.md @@ -2,7 +2,26 @@ name: catalog-search description: "Search the Mendix Catalog platform service registry (catalog.mendix.com) from the CLI to find services published across an organisation. Use when looking for an existing published service to consume. This is the external registry, not the MDL CATALOG keyword — for that see browse-integrations." --- - + +## Find it fast + +`sed -n ',p' SKILL.md` one section rather than reading the file; +`./mxcli syntax ` answers *how do I write this* for a fraction of either. + +| lines | section | +|---|---| +| 31-41 | Authentication Required | +| 42-61 | Basic Usage | +| 62-93 | Output Formats | +| 94-106 | Pagination | +| 107-132 | Common Use Cases | +| 133-144 | Flags | +| 145-161 | Error Handling | +| 162-170 | Future Features | +| 171-188 | Disambiguation: Two Different "Catalogs" | +| 189-193 | Related | + + # Catalog Search (Mendix Platform Service Registry) Search and discover services registered in **Mendix Catalog** (catalog.mendix.com) programmatically. diff --git a/.claude/skills/mendix/cheatsheet-errors/SKILL.md b/.claude/skills/mendix/cheatsheet-errors/SKILL.md index e7c8e4b255..3dc7c7d633 100644 --- a/.claude/skills/mendix/cheatsheet-errors/SKILL.md +++ b/.claude/skills/mendix/cheatsheet-errors/SKILL.md @@ -2,7 +2,23 @@ name: cheatsheet-errors description: "Quick fixes for the MDL syntax errors that come up most: undeclared variables, wrong retrieve source, list operations, expression mistakes. Use when `mxcli check` reports an error and you want the fix rather than the reference." --- - + +## Find it fast + +`sed -n ',p' SKILL.md` one section rather than reading the file; +`./mxcli syntax ` answers *how do I write this* for a fraction of either. + +| lines | section | +|---|---| +| 26-78 | Variable Errors | +| 79-130 | Expression Errors | +| 131-178 | Control Flow Errors | +| 179-216 | Syntax Errors | +| 217-275 | Reference Errors | +| 276-291 | Studio Pro Error Code Reference | +| 292-303 | Quick Validation Checklist | + + # MDL Common Errors Cheatsheet Quick fixes for common MDL syntax errors. diff --git a/.claude/skills/mendix/cheatsheet-variables/SKILL.md b/.claude/skills/mendix/cheatsheet-variables/SKILL.md index c32d29ad79..7ec33c4705 100644 --- a/.claude/skills/mendix/cheatsheet-variables/SKILL.md +++ b/.claude/skills/mendix/cheatsheet-variables/SKILL.md @@ -2,7 +2,22 @@ name: cheatsheet-variables description: "One-page reference for declaring and assigning variables in MDL microflows — every type, with the exact syntax. Use when writing a `declare` and unsure of the spelling, or when a declaration error needs fixing fast." --- - + +## Find it fast + +`sed -n ',p' SKILL.md` one section rather than reading the file; +`./mxcli syntax ` answers *how do I write this* for a fraction of either. + +| lines | section | +|---|---| +| 25-37 | Declaration Syntax | +| 38-45 | Key Rules | +| 46-88 | Common Mistakes | +| 89-97 | Special Values | +| 98-117 | Parameter vs Variable | +| 118-131 | Variable Scope | + + # MDL Variable Cheatsheet Quick reference for variable declarations in MDL microflows. diff --git a/.claude/skills/mendix/check-syntax/SKILL.md b/.claude/skills/mendix/check-syntax/SKILL.md index 27c024c117..7b20e4474c 100644 --- a/.claude/skills/mendix/check-syntax/SKILL.md +++ b/.claude/skills/mendix/check-syntax/SKILL.md @@ -2,7 +2,26 @@ name: check-syntax description: "Validate MDL with `mxcli check` before presenting or executing it, including reference resolution against a project. Use ALWAYS before showing MDL to a user, running `mxcli exec`, or committing a .mdl file — exec refuses exactly what check rejects." --- + +## Find it fast +`sed -n ',p' SKILL.md` one section rather than reading the file; +`./mxcli syntax ` answers *how do I write this* for a fraction of either. + +| lines | section | +|---|---| +| 29-35 | When to Use This Skill | +| 36-287 | `exec` refuses what `check` rejects | +| 288-423 | Pre-Flight Validation Checklist | +| 424-474 | Two rules that only real validation used to catch | +| 475-521 | Validation Workflow | +| 522-542 | Script Execution Behavior | +| 543-603 | Script Organization | +| 604-656 | Troubleshooting Parse Errors | +| 657-714 | Studio Pro MCP — Verification Only | +| 715-720 | Related Skills | + + # MDL Syntax Validation Skill This skill ensures MDL scripts are validated before presenting them to users or executing them. @@ -276,6 +295,7 @@ A script without the header is checked and executed as `mdl 0`, the alpha langua where some statements mean something else. Write `mdl 1;` as the first line of every new script. To edit a headerless one, upgrade it first (`mxcli fmt --upgrade --header -p app.mpr -w script.mdl`) — never mix the two in one file; see `choose-edit-mode`. +Readiness gate for the header: `check --deprecations=error --v1=error` (either alone lets the other's codes through). ### 1. Check Supported Syntax diff --git a/.claude/skills/mendix/choose-edit-mode/SKILL.md b/.claude/skills/mendix/choose-edit-mode/SKILL.md index bd0cc81640..dc107ecd3f 100644 --- a/.claude/skills/mendix/choose-edit-mode/SKILL.md +++ b/.claude/skills/mendix/choose-edit-mode/SKILL.md @@ -2,7 +2,20 @@ name: choose-edit-mode description: "Decide how to change a document that already exists: ALTER, describe → edit → CREATE OR MODIFY, or a new document. The answer depends on who owns the document (your MDL scripts or Studio Pro). Use before editing anything in an existing app, before editing a project script that has no `mdl 1;` header, before re-executing DESCRIBE output, and before touching a Studio Pro-authored microflow or nanoflow." --- + +## Find it fast +`sed -n ',p' SKILL.md` one section rather than reading the file; +`./mxcli syntax ` answers *how do I write this* for a fraction of either. + +| lines | section | +|---|---| +| 30-49 | The rule | +| 50-78 | Scripts are `mdl 1` — upgrade a headerless file before you edit it | +| 79-94 | Which document types have `alter` | +| 95-112 | Types with no `alter`: microflows, nanoflows, menus, Java actions | + + # Choose the Edit Mode by Who Owns the Document MDL has two ways to change a document, and each is safe in a different situation. diff --git a/.claude/skills/mendix/connect-rapidminer-graph/SKILL.md b/.claude/skills/mendix/connect-rapidminer-graph/SKILL.md index eb2d6f8a4d..58c00dbe85 100644 --- a/.claude/skills/mendix/connect-rapidminer-graph/SKILL.md +++ b/.claude/skills/mendix/connect-rapidminer-graph/SKILL.md @@ -2,7 +2,25 @@ name: connect-rapidminer-graph description: "Fetch data from a RapidMiner graph mart or any SPARQL 1.1 HTTP endpoint (AnzoGraph and friends) and surface it as Mendix entities. Use when a graph database with a SPARQL endpoint has to feed a Mendix app read-only." --- - + +## Find it fast + +`sed -n ',p' SKILL.md` one section rather than reading the file; +`./mxcli syntax ` answers *how do I write this* for a fraction of either. + +| lines | section | +|---|---| +| 28-33 | When to Use | +| 34-62 | Endpoint shape | +| 63-83 | SPARQL JSON result shape | +| 84-103 | The full pipeline | +| 104-243 | Step-by-step template | +| 244-283 | Gotchas (things that burned an hour during development) | +| 284-319 | Exploring the graph | +| 320-323 | Credential management | +| 324-329 | Related skills | + + # Connecting Mendix to RapidMiner / AnzoGraph via SPARQL Use this skill when you need to fetch data from a RapidMiner graph mart (or any SPARQL 1.1 HTTP endpoint like AnzoGraph) and surface it in a Mendix app. diff --git a/.claude/skills/mendix/create-custom-widget/SKILL.md b/.claude/skills/mendix/create-custom-widget/SKILL.md index bedb94fb43..7bc6ab05ed 100644 --- a/.claude/skills/mendix/create-custom-widget/SKILL.md +++ b/.claude/skills/mendix/create-custom-widget/SKILL.md @@ -2,7 +2,28 @@ name: create-custom-widget description: "Build a Mendix pluggable widget from scratch with React and TypeScript and package it as an .mpk. Use when no marketplace or built-in widget covers what is needed and a custom React component has to be written." --- - + +## Find it fast + +`sed -n ',p' SKILL.md` one section rather than reading the file; +`./mxcli syntax ` answers *how do I write this* for a fraction of either. + +| lines | section | +|---|---| +| 31-35 | Prerequisites | +| 36-117 | Step 1: Scaffold the Project | +| 118-252 | Step 2: Define Widget Properties (widget.xml) | +| 253-313 | Step 3: Write the Entry Component | +| 314-339 | Step 4: Write the React Component | +| 340-368 | Step 5: Editor Config (optional but recommended) | +| 369-380 | Step 6: CSS Styles | +| 381-390 | Step 7: Build | +| 391-398 | Step 8: Install to Mendix Project | +| 399-422 | Common Widget Patterns | +| 423-435 | Checklist Before Build | +| 436-443 | Troubleshooting | + + # Create Custom Pluggable Widget Build a Mendix pluggable widget from scratch using React + TypeScript. Produces a `.mpk` file ready for Studio Pro. diff --git a/.claude/skills/mendix/create-page/SKILL.md b/.claude/skills/mendix/create-page/SKILL.md index 70611b0969..af7d94f478 100644 --- a/.claude/skills/mendix/create-page/SKILL.md +++ b/.claude/skills/mendix/create-page/SKILL.md @@ -2,7 +2,27 @@ name: create-page description: "CREATE PAGE syntax reference — parameters, variables, layouts, and the full widget vocabulary. Use before writing any CREATE PAGE statement, or when a widget's property spelling needs checking." --- - + +## Find it fast + +`sed -n ',p' SKILL.md` one section rather than reading the file; +`./mxcli syntax ` answers *how do I write this* for a fraction of either. + +| lines | section | +|---|---| +| 28-43 | Reference files | +| 44-46 | Overview | +| 47-193 | Syntax | +| 194-264 | Basic Examples | +| 265-286 | Modifying Existing Pages | +| 287-374 | Conditional Visibility and Editability | +| 375-516 | Known Limitations | +| 517-525 | Tips | +| 526-552 | Related Commands | +| 553-577 | PLUGGABLEWIDGET Escape Hatch | +| 578-581 | See Also | + + # CREATE PAGE - MDL Syntax Guide ## Reference files @@ -70,6 +90,8 @@ Both are optional and can be changed later with `alter page … { set (Class: ' | Attribute binding | `attribute: AttrName` | `textbox txt (label: 'Name', attribute: Name)` | | Attribute over an association | `attribute: Assoc/Attr` (bare association name, multi-hop OK) | `textbox txt (label: 'Rule', attribute: RuleAction_BusinessRule/Name)` | | Password field | `Password: true` | `textbox tbPw (attribute: Secret, Password: true)` | +| Submit while typing | `SubmitBehaviour: WhileEditing` (+ `SubmitOnInputDelay: `) on textbox / textarea | `textbox tbSearch (attribute: Query, SubmitBehaviour: WhileEditing, SubmitOnInputDelay: 500)` — default `OnEndEditing` / `300` | +| Layout grid width | `Width: FullWidth \| FixedWidth` on layoutgrid | `layoutgrid lg (Width: FixedWidth) { ... }` — default `FullWidth` | | Widget validation | `Validation: ''` + `ValidationMessage: ''` | `Validation: 'length(toString($value)) > 0'` — quoted, not `[bracketed]` | | Variable binding | `datasource: $Var` | `dataview dv (datasource: $Product) { ... }` | | Action binding | `action: type` | `actionbutton btn (caption: 'Save', action: save changes)` | @@ -463,6 +485,14 @@ it the field renders the value in plaintext, and before ako/mxcli#550 a `describe page` → `exec` round trip silently turned every stored password field into an ordinary one — so copying a login or change-password page lost it. +`SubmitBehaviour: WhileEditing` makes a text box or text area submit as the +user types (after `SubmitOnInputDelay` milliseconds, default 300) instead of on +leaving the field — the setting behind live search boxes and "on change while +typing" handlers. `Width: FixedWidth` on a layout grid is Studio Pro's "Fixed +width" container. Both are printed by `describe page` only when not the default, +and before ako/mxcli-ledger FINDINGS #170 a describe → exec round trip reset +them to the defaults. An unknown value is refused (MDL-WIDGET18). + An input widget can also *traverse* an association to show a value from the other side: `attribute: Assoc/Attr` binds the far attribute and stores the hops, which is what Studio Pro does. It works on textbox, textarea, datepicker, diff --git a/.claude/skills/mendix/custom-widgets/SKILL.md b/.claude/skills/mendix/custom-widgets/SKILL.md index 5073ff1b55..c321514000 100644 --- a/.claude/skills/mendix/custom-widgets/SKILL.md +++ b/.claude/skills/mendix/custom-widgets/SKILL.md @@ -2,7 +2,26 @@ name: custom-widgets description: "MDL syntax for pluggable widgets in CREATE PAGE / ALTER PAGE — any installed widget is named by its own name (`htmlelement frame (…) { … }`), with object lists and child slots read from its definition. Covers GALLERY, COMBOBOX, DataGrid2, charts and third-party widgets: datasource and column forms, child slots (TEMPLATE/FILTER), the `pluggablewidget ''` fallback, and adding a widget via .def.json. Use when placing a pluggable widget on a page, or when `mxcli widget describe` output needs interpreting. For the widgets THIS project has, read the generated `widgets` skill." --- - + +## Find it fast + +`sed -n ',p' SKILL.md` one section rather than reading the file; +`./mxcli syntax ` answers *how do I write this* for a fraction of either. + +| lines | section | +|---|---| +| 27-148 | Any installed widget is named by its own name | +| 149-250 | Built-in Pluggable Widgets | +| 251-339 | Charts (Mendix Charts.mpk) | +| 340-487 | Adding a Third-Party Widget | +| 488-513 | Authoring over MCP (live Studio Pro) | +| 514-616 | .def.json Reference | +| 617-677 | Engine Internals | +| 678-693 | Verify & Debug | +| 694-705 | Common Mistakes | +| 706-717 | Key Source Files | + + # Custom & Pluggable Widgets in MDL ## Any installed widget is named by its own name diff --git a/.claude/skills/mendix/database-connections/SKILL.md b/.claude/skills/mendix/database-connections/SKILL.md index 1057f9289b..03af9bb6c6 100644 --- a/.claude/skills/mendix/database-connections/SKILL.md +++ b/.claude/skills/mendix/database-connections/SKILL.md @@ -2,7 +2,29 @@ name: database-connections description: "Connect a Mendix app to an external database over JDBC with the External Database Connector, and define the queries microflows run against it. Use when the app must read or write Oracle, PostgreSQL, MySQL or SQL Server directly from a microflow." --- - + +## Find it fast + +`sed -n ',p' SKILL.md` one section rather than reading the file; +`./mxcli syntax ` answers *how do I write this* for a fraction of either. + +| lines | section | +|---|---| +| 30-32 | Purpose | +| 33-49 | When to Use This Skill | +| 50-84 | Prerequisites | +| 85-193 | Database Connection Syntax | +| 194-255 | Query Definition Syntax | +| 256-353 | Complete Examples | +| 354-369 | Viewing Connections | +| 370-390 | Best Practices | +| 391-404 | Troubleshooting | +| 405-416 | Related Commands | +| 417-542 | Executing Queries from Microflows | +| 543-557 | Importing Data from External Databases | +| 558-561 | References | + + # Skill: Create External Database Connections ## Purpose diff --git a/.claude/skills/mendix/debug-bson/SKILL.md b/.claude/skills/mendix/debug-bson/SKILL.md index c1b0d21a97..a3617182e0 100644 --- a/.claude/skills/mendix/debug-bson/SKILL.md +++ b/.claude/skills/mendix/debug-bson/SKILL.md @@ -2,7 +2,30 @@ name: debug-bson description: "Diagnose BSON serialization problems in the MPR file — missing properties, wrong storage names, widget definitions Studio Pro rejects. Use when something created through MDL does not appear correctly in Studio Pro, or when a CE error points at a document mxcli wrote." --- - + +## Find it fast + +`sed -n ',p' SKILL.md` one section rather than reading the file; +`./mxcli syntax ` answers *how do I write this* for a fraction of either. + +| lines | section | +|---|---| +| 33-42 | When to Use This Skill | +| 43-51 | Overview | +| 52-111 | Part 1: The Debugging Workflow | +| 112-159 | Part 2: Common BSON Patterns | +| 160-212 | Part 3: Key Structures Reference | +| 213-252 | Part 4: Debugging Tools | +| 253-265 | Part 5: Checklist for BSON Fixes | +| 266-280 | Part 6: Common Issues and Solutions | +| 281-330 | Part 7: Nested WidgetObjects (Critical) | +| 331-412 | Part 8: Property Value Types (Critical) | +| 413-451 | Part 9: Texts$Translation vs Texts$TextItem | +| 452-584 | Part 10: CE0463 TextTemplate Troubleshooting | +| 585-591 | Related Documentation | +| 592-626 | Quick Reference | + + # BSON Serialization Debugging Skill This skill provides guidance for debugging BSON serialization issues when implementing or fixing Mendix SDK writers. diff --git a/.claude/skills/mendix/debug-microflows/SKILL.md b/.claude/skills/mendix/debug-microflows/SKILL.md index 49465e03f3..6587e1c14c 100644 --- a/.claude/skills/mendix/debug-microflows/SKILL.md +++ b/.claude/skills/mendix/debug-microflows/SKILL.md @@ -2,7 +2,25 @@ name: debug-microflows description: "Drive the Mendix runtime's microflow and nanoflow debugger from the command line with `mxcli debug` — breakpoints by name, variable inspection, step and continue. Use when a microflow produces the wrong result and reading it is not enough." --- - + +## Find it fast + +`sed -n ',p' SKILL.md` one section rather than reading the file; +`./mxcli syntax ` answers *how do I write this* for a fraction of either. + +| lines | section | +|---|---| +| 26-37 | Overview | +| 38-47 | When to use this skill | +| 48-61 | Prerequisites | +| 62-81 | The loop | +| 82-101 | Commands | +| 102-132 | Nanoflows (client-side) | +| 133-149 | Gotchas | +| 150-157 | Validation checklist | +| 158-160 | Microflow/nanoflow debugger (`mxcli debug`) | + + # Debug Microflows — `mxcli debug` ## Overview diff --git a/.claude/skills/mendix/demo-data/SKILL.md b/.claude/skills/mendix/demo-data/SKILL.md index be43231631..482105e9f7 100644 --- a/.claude/skills/mendix/demo-data/SKILL.md +++ b/.claude/skills/mendix/demo-data/SKILL.md @@ -2,7 +2,27 @@ name: demo-data description: "Seed a Mendix app's PostgreSQL database directly, bypassing the runtime — Mendix's internal ID system, association storage, and bulk `import from` an external database. Use when populating test or demo data, when data is needed before the UI exists, or for imports too large for the runtime." --- - + +## Find it fast + +`sed -n ',p' SKILL.md` one section rather than reading the file; +`./mxcli syntax ` answers *how do I write this* for a fraction of either. + +| lines | section | +|---|---| +| 28-33 | Purpose | +| 34-42 | When to Use This Skill | +| 43-76 | Step 1: Get Database Settings | +| 77-103 | Step 2: Connect to the Database | +| 104-165 | Step 3: Understand the Mendix ID System | +| 166-253 | Step 4: Check Association Storage and Optimistic Locking | +| 254-355 | Step 5: Insert Demo Data | +| 356-401 | Important Caveats | +| 402-447 | Quick Reference | +| 448-494 | Automated Alternative: IMPORT FROM | +| 495-499 | Related Skills | + + # Skill: Connect to Application Database and Generate Demo Data ## Purpose diff --git a/.claude/skills/mendix/docker-workflow/SKILL.md b/.claude/skills/mendix/docker-workflow/SKILL.md index 6a7904910b..2611d4e36c 100644 --- a/.claude/skills/mendix/docker-workflow/SKILL.md +++ b/.claude/skills/mendix/docker-workflow/SKILL.md @@ -2,7 +2,26 @@ name: docker-workflow description: "Build, run and validate a Mendix app in Docker from inside a devcontainer. Use when asked to run the app in Docker or produce a deployable image. For the faster Docker-free loop, prefer run-local." --- - + +## Find it fast + +`sed -n ',p' SKILL.md` one section rather than reading the file; +`./mxcli syntax ` answers *how do I write this* for a fraction of either. + +| lines | section | +|---|---| +| 29-36 | When to Use This Skill | +| 37-52 | Prerequisites | +| 53-68 | Architecture | +| 69-85 | Quick Start: One Command | +| 86-113 | Creating an Empty Mendix App | +| 114-272 | Step-by-Step Workflow | +| 273-303 | Common Workflow: Edit, Rebuild, Test | +| 304-415 | Hot Reload | +| 416-434 | Environment Variables | +| 435-454 | Troubleshooting | + + # Docker Build & Run Skill This skill guides you through building, running, and testing a Mendix application using Docker inside a devcontainer. diff --git a/.claude/skills/mendix/download-marketplace-content/SKILL.md b/.claude/skills/mendix/download-marketplace-content/SKILL.md index 6a03971761..1ef0b887d1 100644 --- a/.claude/skills/mendix/download-marketplace-content/SKILL.md +++ b/.claude/skills/mendix/download-marketplace-content/SKILL.md @@ -2,7 +2,29 @@ name: download-marketplace-content description: "The full lifecycle of Mendix Marketplace modules and widgets from the CLI — search, download, install, detect local edits, and update. Use when adding a marketplace module or widget, upgrading one, or asking what an upgrade would overwrite." --- + +## Find it fast +`sed -n ',p' SKILL.md` one section rather than reading the file; +`./mxcli syntax ` answers *how do I write this* for a fraction of either. + +| lines | section | +|---|---| +| 34-42 | When to Use This Skill | +| 43-59 | Prerequisites: Authenticate | +| 60-93 | Step 1 — Discover | +| 94-103 | Step 2 — Download a `.mpk` to disk (optional) | +| 104-223 | Step 3 — Install into a project | +| 224-283 | Step 4 — Repair the model after the install (required, headless) | +| 284-340 | Step 5 — Before updating: has the module been edited? | +| 341-399 | Step 6 — Update an installed module | +| 400-445 | Worked example: the agent-editor stack on a vanilla app | +| 446-453 | Notes | +| 454-457 | Platform authentication (`mxcli auth login/logout/status/list`) with PAT scheme for marketplace-api | +| 458-461 | Marketplace download/install (`mxcli marketplace download/install`) — the content API now exposes a per-version downloadUrl (303→public CDN); install is type-aware (widget→widgets/, new module→`mx module-import`); existing-module updates are reported, not applied (entity-ID/local-edit safety — see PROPOSAL_marketplace_modules | +| 462-466 | Marketplace drift detection (`mxcli marketplace diff -p app.mpr [--to VERSION] [--json]`) | + + # Download, Install and Update Marketplace Content This skill covers the full lifecycle of Mendix Marketplace content (modules and widgets) diff --git a/.claude/skills/mendix/fragments/SKILL.md b/.claude/skills/mendix/fragments/SKILL.md index 6b0ce1fbb1..829dbf66df 100644 --- a/.claude/skills/mendix/fragments/SKILL.md +++ b/.claude/skills/mendix/fragments/SKILL.md @@ -2,7 +2,23 @@ name: fragments description: "Define reusable widget groups with CREATE FRAGMENT and place them with USE FRAGMENT. Use when the same widget pattern repeats across pages and should be written once." --- - + +## Find it fast + +`sed -n ',p' SKILL.md` one section rather than reading the file; +`./mxcli syntax ` answers *how do I write this* for a fraction of either. + +| lines | section | +|---|---| +| 24-32 | When to Use This Skill | +| 33-40 | What Are Fragments? | +| 41-189 | Syntax Reference | +| 190-259 | Common Patterns | +| 260-306 | Common Mistakes | +| 307-316 | Validation Checklist | +| 317-322 | Related Documentation | + + # Mendix Fragments Skill ## When to Use This Skill diff --git a/.claude/skills/mendix/generate-domain-model/SKILL.md b/.claude/skills/mendix/generate-domain-model/SKILL.md index 4644a92b41..7da40347fe 100644 --- a/.claude/skills/mendix/generate-domain-model/SKILL.md +++ b/.claude/skills/mendix/generate-domain-model/SKILL.md @@ -2,7 +2,25 @@ name: generate-domain-model description: "Generate a complete Mendix domain model in MDL — entities, attributes, associations, enumerations — and validate it. Use when asked for a domain model for a business area (e-commerce, HR, CRM, …) rather than a single entity." --- - + +## Find it fast + +`sed -n ',p' SKILL.md` one section rather than reading the file; +`./mxcli syntax ` answers *how do I write this* for a fraction of either. + +| lines | section | +|---|---| +| 28-40 | Reference files | +| 41-47 | When to Use This Skill | +| 48-59 | Modelling for the client: widgets bind members, not expressions | +| 60-96 | Documentation Best Practices | +| 97-197 | Step-by-Step Process | +| 198-214 | Testing the Script | +| 215-233 | Checklist | +| 234-242 | References | +| 243-252 | Tips for AI Assistants | + + # Creating Mendix Domain Model MDL Scripts Use this skill to generate Mendix domain model scripts in MDL (Mendix Definition Language) format and validate them with the linter. diff --git a/.claude/skills/mendix/graph-analysis/SKILL.md b/.claude/skills/mendix/graph-analysis/SKILL.md index c8ff50e2db..914b1e7b80 100644 --- a/.claude/skills/mendix/graph-analysis/SKILL.md +++ b/.claude/skills/mendix/graph-analysis/SKILL.md @@ -2,7 +2,29 @@ name: graph-analysis description: "Query the project as a dependency graph to assess the blast radius of a change, find dead or risky elements, and evaluate module health. Use when planning a refactor, judging what a change will break, or looking for unused documents." --- - + +## Find it fast + +`sed -n ',p' SKILL.md` one section rather than reading the file; +`./mxcli syntax ` answers *how do I write this* for a fraction of either. + +| lines | section | +|---|---| +| 32-41 | When to Read This Skill | +| 42-77 | Setup: Two-Step Refresh | +| 78-112 | Use Case 1: Pre-Change Impact Assessment | +| 113-139 | Use Case 2: Dead Code Identification | +| 140-178 | Use Case 3: Module Health (Cohesion & Coupling) | +| 179-210 | Use Case 4: Community Analysis (Bounded Context Validation) | +| 211-239 | Use Case 5: Finding Bridges (Centrality) | +| 240-253 | Use Case 6: Entity Hotspots | +| 254-299 | Use Case 7: Dependency Cycles | +| 300-331 | Use Case 8: Monolith-to-Multi-App Planning | +| 332-347 | Topological Layer Reference | +| 348-367 | Quick Reference: Which Table Answers What | +| 368-373 | Related Skills | + + # Graph Analysis: Impact Assessment & Architecture Evaluation Use this skill when assessing the impact of a change, identifying risky elements, evaluating module health, or planning a structural refactor. The graph tables model the entire project as a dependency graph and expose topology metrics on top of it. diff --git a/.claude/skills/mendix/java-actions/SKILL.md b/.claude/skills/mendix/java-actions/SKILL.md index 3912177386..9e92094487 100644 --- a/.claude/skills/mendix/java-actions/SKILL.md +++ b/.claude/skills/mendix/java-actions/SKILL.md @@ -2,7 +2,25 @@ name: java-actions description: "Create and call custom Java actions — extending Mendix with server-side Java, and invoking those actions from microflows in MDL. Use when logic needs a Java library, an algorithm microflows cannot express, or an integration only available in Java." --- + +## Find it fast +`sed -n ',p' SKILL.md` one section rather than reading the file; +`./mxcli syntax ` answers *how do I write this* for a fraction of either. + +| lines | section | +|---|---| +| 28-39 | Reference files | +| 40-48 | When to Use This Skill | +| 49-55 | Overview | +| 56-330 | Part 2.5: Creating Java Actions in MDL | +| 331-524 | Part 3: Calling Java Actions from MDL | +| 525-538 | Validation Checklist | +| 539-549 | Common Errors | +| 550-557 | Related Documentation | +| 558-639 | Quick Reference | + + # Mendix Java Actions Skill This skill provides comprehensive guidance for creating and using custom Java actions in Mendix projects. diff --git a/.claude/skills/mendix/java-dependencies/SKILL.md b/.claude/skills/mendix/java-dependencies/SKILL.md index 68bf8ab3d9..694cd2868a 100644 --- a/.claude/skills/mendix/java-dependencies/SKILL.md +++ b/.claude/skills/mendix/java-dependencies/SKILL.md @@ -2,7 +2,27 @@ name: java-dependencies description: "Add, update, remove and audit a module's Maven/JAR dependencies in MDL — Studio Pro's Module Settings → Java dependencies tab. Use when a Java action needs an external library, when bumping a library version, or when auditing which modules pull what." --- - + +## Find it fast + +`sed -n ',p' SKILL.md` one section rather than reading the file; +`./mxcli syntax ` answers *how do I write this* for a fraction of either. + +| lines | section | +|---|---| +| 34-44 | When to Use This Skill | +| 45-56 | Listing Dependencies | +| 57-64 | Describing a Dependency | +| 65-80 | Adding a Dependency | +| 81-93 | Updating a Dependency | +| 94-108 | Managing Transitive Exclusions | +| 109-116 | Removing a Dependency | +| 117-132 | Jackson Version Gotcha | +| 133-143 | After Adding a Dependency | +| 144-154 | Coordinate Format | +| 155-158 | Related Skills | + + # Managing Maven / JAR Dependencies This skill explains how to add, update, and remove Maven/JAR dependencies in a diff --git a/.claude/skills/mendix/javascript-actions/SKILL.md b/.claude/skills/mendix/javascript-actions/SKILL.md index c81b51c6ef..a8869513ee 100644 --- a/.claude/skills/mendix/javascript-actions/SKILL.md +++ b/.claude/skills/mendix/javascript-actions/SKILL.md @@ -2,7 +2,26 @@ name: javascript-actions description: "Create, call and drop JavaScript actions — client-side logic called from nanoflows, not microflows. Use when behaviour must run in the browser or on the device: DOM, fetch, geolocation, or wrapping a marketplace JS action." --- - + +## Find it fast + +`sed -n ',p' SKILL.md` one section rather than reading the file; +`./mxcli syntax ` answers *how do I write this* for a fraction of either. + +| lines | section | +|---|---| +| 31-39 | When to Use This Skill | +| 40-51 | Overview | +| 52-85 | Syntax | +| 86-147 | Examples | +| 148-159 | Calling from a nanoflow | +| 160-184 | The generated .js file | +| 185-193 | Inspecting | +| 194-202 | Validation Checklist | +| 203-211 | Common Errors | +| 212-216 | Related Documentation | + + # Mendix JavaScript Actions Skill Guidance for creating, calling, and dropping **JavaScript actions** in Mendix diff --git a/.claude/skills/mendix/json-structures-and-mappings/SKILL.md b/.claude/skills/mendix/json-structures-and-mappings/SKILL.md index c14537a3b5..edb2b347de 100644 --- a/.claude/skills/mendix/json-structures-and-mappings/SKILL.md +++ b/.claude/skills/mendix/json-structures-and-mappings/SKILL.md @@ -2,7 +2,29 @@ name: json-structures-and-mappings description: "Create and manage JSON structures, import mappings and export mappings in MDL, plus the domain-model shapes they map onto. Use when turning a JSON payload into entities, mapping a REST or queue response, or exporting objects as JSON." --- - + +## Find it fast + +`sed -n ',p' SKILL.md` one section rather than reading the file; +`./mxcli syntax ` answers *how do I write this* for a fraction of either. + +| lines | section | +|---|---| +| 32-109 | Key Concepts | +| 110-121 | Export Mappings | +| 122-186 | JSON Structures | +| 187-228 | Message Definitions | +| 229-389 | Import Mappings | +| 390-487 | Export Mappings | +| 488-505 | Starting a Mapping Below the Payload Root | +| 506-575 | Microflow Actions | +| 576-588 | Browse | +| 589-636 | Export Workflow: PE → NPE → JSON | +| 637-643 | Realistic Example: Countries REST API | +| 644-667 | Placing Documents in Folders | +| 668-682 | Common Mistakes | + + # JSON Structures, Import Mappings & Export Mappings This skill covers creating and managing JSON structures, import mappings, and export mappings in Mendix using MDL. diff --git a/.claude/skills/mendix/live-edit-with-studio-pro/SKILL.md b/.claude/skills/mendix/live-edit-with-studio-pro/SKILL.md index 46c77c7491..47fcff31d5 100644 --- a/.claude/skills/mendix/live-edit-with-studio-pro/SKILL.md +++ b/.claude/skills/mendix/live-edit-with-studio-pro/SKILL.md @@ -2,7 +2,23 @@ name: live-edit-with-studio-pro description: "Change the model through mxcli while Studio Pro has the project open, with edits appearing live over MCP. Use when Claude Code and Studio Pro run on the same machine and a save-and-reopen cycle is unwanted." --- - + +## Find it fast + +`sed -n ',p' SKILL.md` one section rather than reading the file; +`./mxcli syntax ` answers *how do I write this* for a fraction of either. + +| lines | section | +|---|---| +| 28-35 | When to use this skill | +| 36-42 | How it works (hybrid: local reads, live writes) | +| 43-61 | Connect | +| 62-91 | What you can change via MCP — check first | +| 92-104 | Two MCP servers — use the built-in one by default | +| 105-114 | Caveats | +| 115-122 | Verify your change | + + # Live-Editing an Open Studio Pro Project with mxcli (MCP) Use mxcli to change the model **while Studio Pro has the project open**, with edits diff --git a/.claude/skills/mendix/manage-navigation/SKILL.md b/.claude/skills/mendix/manage-navigation/SKILL.md index 065fb57993..f3d4d0cb66 100644 --- a/.claude/skills/mendix/manage-navigation/SKILL.md +++ b/.claude/skills/mendix/manage-navigation/SKILL.md @@ -2,7 +2,28 @@ name: manage-navigation description: "Inspect and change navigation profiles in MDL — home pages, menus, login and not-found pages, and role-based routing. Use when asked to change where the app opens, restructure the menu, or route roles to different home pages." --- + +## Find it fast +`sed -n ',p' SKILL.md` one section rather than reading the file; +`./mxcli syntax ` answers *how do I write this* for a fraction of either. + +| lines | section | +|---|---| +| 31-40 | When to Use This Skill | +| 41-59 | Navigation Concepts | +| 60-77 | Show Commands (Read-Only) | +| 78-358 | CREATE OR MODIFY NAVIGATION (Full Replacement) | +| 359-380 | Round-Trip Workflow | +| 381-402 | Catalog Queries | +| 403-452 | Common Patterns | +| 453-504 | Menu Documents (standalone, reusable) | +| 505-555 | Offline Profiles | +| 556-569 | Checklist | +| 570-573 | Offline synchronization (`CREATE NAVIGATION … SYNC (…)`) | +| 574-576 | Menu documents (CREATE OR MODIFY/DESCRIBE/DROP MENU) | + + # Navigation Management Skill This skill covers inspecting and modifying Mendix navigation profiles via MDL: home pages, menu items, login pages, role-based routing, and navigation catalog queries. diff --git a/.claude/skills/mendix/manage-security/SKILL.md b/.claude/skills/mendix/manage-security/SKILL.md index 4a2bf86b27..4225948cc1 100644 --- a/.claude/skills/mendix/manage-security/SKILL.md +++ b/.claude/skills/mendix/manage-security/SKILL.md @@ -2,7 +2,24 @@ name: manage-security description: "Configure Mendix security in MDL — module and user roles, access to microflows, pages and entities, project security level, demo users, and guest access. Use when setting up or changing who can see or do what in an app." --- + +## Find it fast +`sed -n ',p' SKILL.md` one section rather than reading the file; +`./mxcli syntax ` answers *how do I write this* for a fraction of either. + +| lines | section | +|---|---| +| 27-35 | When to Use This Skill | +| 36-43 | Security Concepts | +| 44-110 | The System-module ceiling — decide the role model around this FIRST | +| 111-540 | Syntax Reference | +| 541-555 | Starlark Lint Rule APIs | +| 556-608 | Common Workflow: Setting Up Module Security | +| 609-618 | Common Mistakes | +| 619-628 | Validation | + + # Security Management Skill This skill covers Mendix security configuration via MDL: module roles, user roles, access control (microflows, pages, entities), project security settings, and demo users. diff --git a/.claude/skills/mendix/master-detail-pages/SKILL.md b/.claude/skills/mendix/master-detail-pages/SKILL.md index e0a4ae2ec7..0249409720 100644 --- a/.claude/skills/mendix/master-detail-pages/SKILL.md +++ b/.claude/skills/mendix/master-detail-pages/SKILL.md @@ -2,7 +2,23 @@ name: master-detail-pages description: "The master-detail page pattern in MDL: a selectable list beside a detail form driven by SELECTION. Use when building a selection-based screen rather than a plain list or form." --- - + +## Find it fast + +`sed -n ',p' SKILL.md` one section rather than reading the file; +`./mxcli syntax ` answers *how do I write this* for a fraction of either. + +| lines | section | +|---|---| +| 24-29 | Overview | +| 30-114 | MDL Syntax | +| 115-154 | Complete Example | +| 155-179 | Key Patterns | +| 180-198 | Syntax Summary | +| 199-204 | Related Skills | +| 205-210 | Implementation Notes | + + # Master-Detail Pages ## Overview diff --git a/.claude/skills/mendix/mdl-entities/SKILL.md b/.claude/skills/mendix/mdl-entities/SKILL.md index 4972341140..71d21fc085 100644 --- a/.claude/skills/mendix/mdl-entities/SKILL.md +++ b/.claude/skills/mendix/mdl-entities/SKILL.md @@ -2,7 +2,29 @@ name: mdl-entities description: "Complete syntax reference for entities, attributes and associations — every entity kind, attribute type, and association form. Use when writing or altering a domain model and the exact spelling matters." --- - + +## Find it fast + +`sed -n ',p' SKILL.md` one section rather than reading the file; +`./mxcli syntax ` answers *how do I write this* for a fraction of either. + +| lines | section | +|---|---| +| 32-39 | Entity Types | +| 40-77 | Persistent Entity | +| 78-94 | Non-Persistent Entity | +| 95-109 | Attribute Types | +| 110-127 | Attribute Modifiers | +| 128-173 | Generalization (Inheritance) | +| 174-222 | Associations | +| 223-239 | Enumerations | +| 240-281 | View Entity (OQL) | +| 282-305 | Entity with Index | +| 306-367 | Complete Domain Model Example | +| 368-385 | Changing an Existing Domain Model | +| 386-428 | Quick Reference | + + # MDL Entity Syntax Reference Complete syntax reference for creating entities, attributes, and associations. diff --git a/.claude/skills/mendix/migrate-design-prototype/SKILL.md b/.claude/skills/mendix/migrate-design-prototype/SKILL.md index f9fc503679..5b09bc6970 100644 --- a/.claude/skills/mendix/migrate-design-prototype/SKILL.md +++ b/.claude/skills/mendix/migrate-design-prototype/SKILL.md @@ -2,7 +2,30 @@ name: migrate-design-prototype description: "Reproduce a Claude Design prototype or design handoff (HTML/CSS, .dc.html export, tokens, screenshots) inside a Mendix app: build the palette with `mxcli theme create --from`, then apply classes in pages with MDL. Use when given a design artefact and asked to make the app look like it." --- + +## Find it fast +`sed -n ',p' SKILL.md` one section rather than reading the file; +`./mxcli syntax ` answers *how do I write this* for a fraction of either. + +| lines | section | +|---|---| +| 31-48 | When to Use This Skill | +| 49-97 | The Pipeline at a Glance | +| 98-122 | Where the Theme Lives (read this first — it avoids the main friction) | +| 123-213 | ① Build the Theme — `mxcli theme create --from` | +| 214-281 | ② Rebuild Components — Atlas block/class first, custom class only for identity | +| 282-389 | Component → Mendix widget map | +| 390-485 | The App Shell: Navigation & Layout (built once, not per page) | +| 486-572 | ③ Apply Classes in Pages (MDL) | +| 573-586 | ④ Build & Preview | +| 587-594 | ⑤ Verify Against the Prototype | +| 595-631 | Never put a grid on a Mendix widget's own class | +| 632-652 | Gotchas (learned building this app) | +| 653-704 | MDL gotchas that block a faithful build | +| 705-723 | Checklist | + + # Migrate a Claude Design Prototype into a Mendix App (Theme + Pages) ## When to Use This Skill diff --git a/.claude/skills/mendix/migrate-k2-nintex/SKILL.md b/.claude/skills/mendix/migrate-k2-nintex/SKILL.md index 8c9f2e56a5..8919a7971c 100644 --- a/.claude/skills/mendix/migrate-k2-nintex/SKILL.md +++ b/.claude/skills/mendix/migrate-k2-nintex/SKILL.md @@ -2,7 +2,26 @@ name: migrate-k2-nintex description: "Assess and migrate K2 / Nintex K2 applications to Mendix — SmartObjects to domain models, SmartForms to pages, K2 workflows to microflows and Mendix workflows. Use when analysing or converting a K2 application." --- - + +## Find it fast + +`sed -n ',p' SKILL.md` one section rather than reading the file; +`./mxcli syntax ` answers *how do I write this* for a fraction of either. + +| lines | section | +|---|---| +| 29-37 | When to Use This Skill | +| 38-91 | Understanding K2 Application Architecture | +| 92-128 | Export and Package Options | +| 129-288 | Migration Strategy | +| 289-341 | Assessment Workflow | +| 342-407 | Migration Execution Order | +| 408-456 | Common Challenges and Solutions | +| 457-480 | Pre-Migration Checklist | +| 481-492 | Investigation Tips | +| 493-500 | Related Skills | + + # K2/Nintex to Mendix Migration Skill This skill provides comprehensive guidance for assessing and migrating K2 (now Nintex K2) applications to Mendix using MDL (Mendix Definition Language). diff --git a/.claude/skills/mendix/migrate-oracle-forms/SKILL.md b/.claude/skills/mendix/migrate-oracle-forms/SKILL.md index c31c12eac1..e4a82ed507 100644 --- a/.claude/skills/mendix/migrate-oracle-forms/SKILL.md +++ b/.claude/skills/mendix/migrate-oracle-forms/SKILL.md @@ -2,7 +2,28 @@ name: migrate-oracle-forms description: "Assess and migrate Oracle Forms applications to Mendix — .fmb forms to pages, PL/SQL to microflows, and a staged migration strategy. Use when analysing or converting an Oracle Forms system." --- - + +## Find it fast + +`sed -n ',p' SKILL.md` one section rather than reading the file; +`./mxcli syntax ` answers *how do I write this* for a fraction of either. + +| lines | section | +|---|---| +| 31-38 | When to Use This Skill | +| 39-46 | Migration Overview | +| 47-79 | Reserved Word Conflicts | +| 80-159 | Script Organization | +| 160-220 | Validation Feedback | +| 221-300 | PL/SQL to Microflow Mapping | +| 301-360 | UI Component Mapping | +| 361-376 | Triggers to Microflows | +| 377-398 | Migration Checklist | +| 399-408 | Common Migration Errors | +| 409-418 | Tips for Success | +| 419-424 | Related Skills | + + # Oracle Forms to Mendix Migration Skill This skill provides comprehensive guidance for migrating Oracle Forms applications to Mendix using MDL (Mendix Definition Language). diff --git a/.claude/skills/mendix/mock-rest-apis/SKILL.md b/.claude/skills/mendix/mock-rest-apis/SKILL.md index 28f8547fff..4f24dce1d7 100644 --- a/.claude/skills/mendix/mock-rest-apis/SKILL.md +++ b/.claude/skills/mendix/mock-rest-apis/SKILL.md @@ -2,7 +2,24 @@ name: mock-rest-apis description: "Stand up an HTTP endpoint you control instead of a live third-party API, and point the Mendix app at it — Prism from an OpenAPI contract, a constant swap, or a forward proxy. Use when building or debugging a REST integration without the real API, forcing a 404/500 through an error handler, or running offline or in CI." --- + +## Find it fast +`sed -n ',p' SKILL.md` one section rather than reading the file; +`./mxcli syntax ` answers *how do I write this* for a fraction of either. + +| lines | section | +|---|---| +| 34-41 | When to Use This Skill | +| 42-54 | Two separate problems | +| 55-117 | 1. Point the app at the mock | +| 118-156 | 2. Prism: serve an OpenAPI contract as a mock | +| 157-186 | 3. Forward proxy: when the model cannot change | +| 187-199 | Verify the mock before blaming Mendix | +| 200-211 | Failure modes, symptoms first | +| 212-218 | Related Skills | + + # Mock REST APIs Skill Use this skill when a REST integration needs an endpoint you control instead of a diff --git a/.claude/skills/mendix/odata-data-sharing/SKILL.md b/.claude/skills/mendix/odata-data-sharing/SKILL.md index 51c2d3a39e..bca05e1182 100644 --- a/.claude/skills/mendix/odata-data-sharing/SKILL.md +++ b/.claude/skills/mendix/odata-data-sharing/SKILL.md @@ -2,7 +2,27 @@ name: odata-data-sharing description: "Share data between Mendix apps over OData — published services, view entities as an abstraction layer, consumed clients and external entities. Use when exposing data from one app to another, consuming another app's OData service, or refreshing a cached contract." --- - + +## Find it fast + +`sed -n ',p' SKILL.md` one section rather than reading the file; +`./mxcli syntax ` answers *how do I write this* for a fraction of either. + +| lines | section | +|---|---| +| 30-42 | Reference files | +| 43-52 | When to Use This Skill | +| 53-71 | MetadataUrl Formats | +| 72-123 | Refreshing the Cached Contract | +| 124-152 | ServiceUrl Must Be a Constant | +| 153-191 | Architecture Overview | +| 192-224 | API Versioning | +| 225-270 | Folder Organization | +| 271-282 | Module Organization Conventions | +| 283-303 | Checklist | +| 304-331 | Exploration Commands | + + # OData Data Sharing Between Mendix Apps This skill covers how to use OData services to share data between Mendix applications, with emphasis on using view entities as an abstraction layer to decouple the API contract from the internal domain model. diff --git a/.claude/skills/mendix/organize-project/SKILL.md b/.claude/skills/mendix/organize-project/SKILL.md index 83682fd43d..36e1280689 100644 --- a/.claude/skills/mendix/organize-project/SKILL.md +++ b/.claude/skills/mendix/organize-project/SKILL.md @@ -2,7 +2,27 @@ name: organize-project description: "Organise documents into folders and move them between folders and modules with MOVE. Use when a module has grown unstructured, when restructuring a project, or when a document is in the wrong place." --- - + +## Find it fast + +`sed -n ',p' SKILL.md` one section rather than reading the file; +`./mxcli syntax ` answers *how do I write this* for a fraction of either. + +| lines | section | +|---|---| +| 30-38 | When to Use This Skill | +| 39-73 | Folder Conventions | +| 74-133 | Creating Documents in Folders | +| 134-174 | Reading the Layout Back | +| 175-223 | Moving Documents | +| 224-231 | Folder Rules | +| 232-287 | Supported Document Types | +| 288-318 | Example: Reorganize a Module | +| 319-337 | Moving Folders | +| 338-354 | Deleting Folders | +| 355-362 | Validation Checklist | + + # Project Organization: Folders and Moving Documents This skill covers organizing Mendix project documents (pages, microflows, snippets, nanoflows) into folders and moving them between folders and modules. diff --git a/.claude/skills/mendix/overview-pages/SKILL.md b/.claude/skills/mendix/overview-pages/SKILL.md index 3d162f011a..801c1612b9 100644 --- a/.claude/skills/mendix/overview-pages/SKILL.md +++ b/.claude/skills/mendix/overview-pages/SKILL.md @@ -2,7 +2,32 @@ name: overview-pages description: "The CRUD overview page pattern in MDL — a navigation snippet, a list page and a new/edit page wired together. Use when building the standard list-plus-edit screens for an entity." --- - + +## Find it fast + +`sed -n ',p' SKILL.md` one section rather than reading the file; +`./mxcli syntax ` answers *how do I write this* for a fraction of either. + +| lines | section | +|---|---| +| 33-40 | Overview | +| 41-48 | Pattern Summary | +| 49-89 | Navigation Menu Snippet | +| 90-257 | Overview Page Template | +| 258-315 | NewEdit Page Template | +| 316-408 | Complete Example: Store Entity | +| 409-466 | Complete Example: Car Entity | +| 467-488 | Widget Selection Guide | +| 489-500 | Naming Conventions | +| 501-510 | Button Styles | +| 511-526 | Folder Organization | +| 527-559 | Parameterized Snippets | +| 560-611 | Entity Menu Snippets with NavigationList | +| 612-681 | Handling Circular Dependencies | +| 682-688 | Related Skills | +| 689-699 | Snippet Commands Reference | + + # Overview Pages - CRUD Page Pattern ## Overview diff --git a/.claude/skills/mendix/patterns-crud/SKILL.md b/.claude/skills/mendix/patterns-crud/SKILL.md index ecf5138549..1716433d6b 100644 --- a/.claude/skills/mendix/patterns-crud/SKILL.md +++ b/.claude/skills/mendix/patterns-crud/SKILL.md @@ -2,7 +2,26 @@ name: patterns-crud description: "Standard Create/Read/Update/Delete microflow patterns with the naming conventions that go with them. Use when writing the action microflows behind page buttons rather than looking up individual syntax." --- - + +## Find it fast + +`sed -n ',p' SKILL.md` one section rather than reading the file; +`./mxcli syntax ` answers *how do I write this* for a fraction of either. + +| lines | section | +|---|---| +| 29-37 | Naming Conventions | +| 38-67 | Save Pattern (Create/Update) | +| 68-107 | Validation Pattern | +| 108-131 | Delete Pattern | +| 132-155 | Cancel Pattern | +| 156-179 | Create New Pattern | +| 180-202 | Data Source Pattern | +| 203-224 | Edit Pattern | +| 225-238 | Complete CRUD Set | +| 239-246 | Best Practices | + + # CRUD Action Patterns Standard patterns for Create, Read, Update, Delete operations on entities. diff --git a/.claude/skills/mendix/patterns-data-processing/SKILL.md b/.claude/skills/mendix/patterns-data-processing/SKILL.md index b92f14997a..b890a28c28 100644 --- a/.claude/skills/mendix/patterns-data-processing/SKILL.md +++ b/.claude/skills/mendix/patterns-data-processing/SKILL.md @@ -2,7 +2,24 @@ name: patterns-data-processing description: "Patterns for loops, aggregates, batch processing and list transformation — including which retrieve source is legal where, and why nested loops are the wrong tool. Use when a microflow processes a list, merges data, or does anything in bulk." --- - + +## Find it fast + +`sed -n ',p' SKILL.md` one section rather than reading the file; +`./mxcli syntax ` answers *how do I write this* for a fraction of either. + +| lines | section | +|---|---| +| 27-106 | Loop Patterns | +| 107-191 | Retrieve by Association | +| 192-270 | Aggregate Patterns | +| 271-361 | List Operations | +| 362-405 | Batch Processing | +| 406-462 | Data Transformation | +| 463-496 | Error Handling in Loops | +| 497-504 | Best Practices | + + # Data Processing Patterns Patterns for loops, aggregates, batch processing, and data transformation. diff --git a/.claude/skills/mendix/project-brain/SKILL.md b/.claude/skills/mendix/project-brain/SKILL.md index 3fbc6748da..17f7cda5c1 100644 --- a/.claude/skills/mendix/project-brain/SKILL.md +++ b/.claude/skills/mendix/project-brain/SKILL.md @@ -2,7 +2,30 @@ name: project-brain description: "Project-specific knowledge mxcli cannot compute — the requirements and slices being built from (a spec, a prototype, a conversation), why a pattern was chosen here, what is still undecided, which marketplace version broke what. Use when starting from requirements that live outside git, before designing something that looks like it was decided before, when you have had to correct the same thing twice, and when an mxbuild error is resolved by something non-obvious." --- + +## Find it fast +`sed -n ',p' SKILL.md` one section rather than reading the file; +`./mxcli syntax ` answers *how do I write this* for a fraction of either. + +| lines | section | +|---|---| +| 49-66 | The rule that makes it work | +| 67-103 | Reading it | +| 104-124 | Writing to it | +| 125-204 | Recording requirements and slices | +| 205-219 | When to capture a decision | +| 220-254 | Recording what is NOT decided | +| 255-274 | Write the anchor, not the name | +| 275-296 | Checking it | +| 297-312 | Size | +| 313-329 | Commands | +| 330-344 | Renaming | +| 345-354 | What not to record | +| 355-380 | Handing a slice to another agent | +| 381-383 | Project brain (`mxcli brain init/capture/staged/promote/drop/check/show`) | + + # Project brain The brain holds what mxcli **cannot** compute about this project. Two halves: diff --git a/.claude/skills/mendix/project-settings/SKILL.md b/.claude/skills/mendix/project-settings/SKILL.md index 892ddd7c6c..4689b85da8 100644 --- a/.claude/skills/mendix/project-settings/SKILL.md +++ b/.claude/skills/mendix/project-settings/SKILL.md @@ -2,7 +2,21 @@ name: project-settings description: "Read and change project settings in MDL — database configuration, constant overrides, after-startup and before-shutdown microflows, the enabled languages (add/modify/remove, which decide what translations a build emits), and the rest of Studio Pro's Settings dialog. Use when configuring a deployment, wiring a startup microflow, or enabling a language before translating an app." --- + +## Find it fast +`sed -n ',p' SKILL.md` one section rather than reading the file; +`./mxcli syntax ` answers *how do I write this* for a fraction of either. + +| lines | section | +|---|---| +| 22-31 | When to Use This Skill | +| 32-304 | Commands | +| 305-332 | Common Patterns | +| 333-341 | Checklist | +| 342-346 | AfterStartupMicroflow Must Return Boolean | + + # Project Settings ## When to Use This Skill diff --git a/.claude/skills/mendix/record-narrated-demo/SKILL.md b/.claude/skills/mendix/record-narrated-demo/SKILL.md index b9b74a290f..78ee1885f1 100644 --- a/.claude/skills/mendix/record-narrated-demo/SKILL.md +++ b/.claude/skills/mendix/record-narrated-demo/SKILL.md @@ -2,7 +2,23 @@ name: record-narrated-demo description: "Record a narrated walkthrough video of a working Mendix app — the human-facing half of the journey/demo pair. Use when asked to demo, show off, or produce a walkthrough recording, after the app's end-to-end journey already passes." --- + +## Find it fast +`sed -n ',p' SKILL.md` one section rather than reading the file; +`./mxcli syntax ` answers *how do I write this* for a fraction of either. + +| lines | section | +|---|---| +| 41-66 | Before recording: the journey must exist and pass | +| 67-85 | Seed the data before you record | +| 86-333 | Recording mechanics | +| 334-411 | The take has to be true, not only watchable | +| 412-432 | What to narrate | +| 433-474 | Where this stops | +| 475-500 | Checklist | + + # Record a Narrated Demo Proving a feature works and showing it off are two different jobs, and one script diff --git a/.claude/skills/mendix/regular-expressions/SKILL.md b/.claude/skills/mendix/regular-expressions/SKILL.md index ae30fc31fb..093d3043ec 100644 --- a/.claude/skills/mendix/regular-expressions/SKILL.md +++ b/.claude/skills/mendix/regular-expressions/SKILL.md @@ -2,7 +2,25 @@ name: regular-expressions description: "Create and manage named regular-expression documents and bind them to attribute validation rules. Use when adding an email, phone or identifier pattern, changing a shared pattern, or working out why a regex cannot attach directly to an attribute." --- - + +## Find it fast + +`sed -n ',p' SKILL.md` one section rather than reading the file; +`./mxcli syntax ` answers *how do I write this* for a fraction of either. + +| lines | section | +|---|---| +| 26-32 | When to Use This Skill | +| 33-58 | What a Mendix regular expression is | +| 59-68 | Writing the pattern | +| 69-83 | Go cannot check every legal pattern | +| 84-142 | Binding a regex to an attribute | +| 143-155 | Finding out who uses one | +| 156-167 | Common Mistakes | +| 168-173 | Related | +| 174-176 | Regular expressions (LIST/DESCRIBE/CREATE [OR MODIFY]/DROP REGULAR EXPRESSION) | + + # Regular Expressions ## When to Use This Skill diff --git a/.claude/skills/mendix/resolve-forward-references/SKILL.md b/.claude/skills/mendix/resolve-forward-references/SKILL.md index 10ec9786c3..edded6c05b 100644 --- a/.claude/skills/mendix/resolve-forward-references/SKILL.md +++ b/.claude/skills/mendix/resolve-forward-references/SKILL.md @@ -2,7 +2,25 @@ name: resolve-forward-references description: "Order MDL statements so that every reference resolves — execution is sequential and immediate, so a document must exist before anything points at it. Use when a script fails on a reference to something defined later in the same file." --- + +## Find it fast +`sed -n ',p' SKILL.md` one section rather than reading the file; +`./mxcli syntax ` answers *how do I write this* for a fraction of either. + +| lines | section | +|---|---| +| 26-52 | Why Forward References Fail | +| 53-70 | The Placeholder Pattern | +| 71-137 | Pattern 1 — Shared Navigation Snippet (most common) | +| 138-185 | Pattern 2 — Page References Another Page (new/edit from overview) | +| 186-218 | Pattern 3 — Microflow References a Page Not Yet Created | +| 219-237 | Ordering Rules for Dependency-Free Scripts | +| 238-261 | Choosing Between CREATE OR MODIFY and ALTER SNIPPET | +| 262-336 | Script Template for a Full CRUD Module | +| 337-341 | Related Skills | + + # Resolving Forward References in MDL Scripts ## Why Forward References Fail diff --git a/.claude/skills/mendix/rest-call-from-json/SKILL.md b/.claude/skills/mendix/rest-call-from-json/SKILL.md index 3733a3a125..dc0606d930 100644 --- a/.claude/skills/mendix/rest-call-from-json/SKILL.md +++ b/.claude/skills/mendix/rest-call-from-json/SKILL.md @@ -2,7 +2,26 @@ name: rest-call-from-json description: "Generate the whole integration stack from a JSON payload: JSON structure, non-persistent entities, import mapping, and the REST CALL microflow. Use when starting from an example response and needing everything between it and a working microflow." --- - + +## Find it fast + +`sed -n ',p' SKILL.md` one section rather than reading the file; +`./mxcli syntax ` answers *how do I write this* for a fraction of either. + +| lines | section | +|---|---| +| 35-43 | Overview — Four Steps | +| 44-64 | Step 1 — JSON Structure | +| 65-98 | Step 2 — Non-Persistent Entities | +| 99-133 | Step 3 — Import Mapping | +| 134-173 | Step 4 — REST CALL Microflow | +| 174-213 | Step 5 — Import/Export Mapping in Microflows (Optional) | +| 214-277 | Step 6 — Sending a Request Body (Optional) | +| 278-353 | Complete Example — Bible Verse API | +| 354-369 | Gotchas and Common Errors | +| 370-379 | Naming Conventions (MES) | + + # REST Call from JSON Payload — End-to-End Skill Use this skill to generate the full stack of Mendix integration artifacts from a JSON payload: diff --git a/.claude/skills/mendix/rest-client/SKILL.md b/.claude/skills/mendix/rest-client/SKILL.md index d9684eb1d1..d555e47c4d 100644 --- a/.claude/skills/mendix/rest-client/SKILL.md +++ b/.claude/skills/mendix/rest-client/SKILL.md @@ -2,7 +2,25 @@ name: rest-client description: "Call external REST APIs from Mendix — the three approaches (inline REST CALL, consumed REST client document, generated from OpenAPI) and how to choose. Use when integrating with any external REST API." --- - + +## Find it fast + +`sed -n ',p' SKILL.md` one section rather than reading the file; +`./mxcli syntax ` answers *how do I write this* for a fraction of either. + +| lines | section | +|---|---| +| 28-43 | Three Approaches | +| 44-90 | Approach 0: OpenAPI Import (Fastest) | +| 91-287 | Approach 1: REST Client (Manual) | +| 288-333 | Approach 2: REST CALL (Inline HTTP) | +| 334-366 | Data Transformers (JSLT — Mendix 11.9+) | +| 367-404 | JSON Structures & Mappings | +| 405-476 | Complete Pipeline Example | +| 477-486 | Mendix Validation Rules | +| 487-507 | BSON Types Reference | + + # REST Integration Skill Use this skill when integrating with external REST APIs from Mendix. diff --git a/.claude/skills/mendix/run-app/SKILL.md b/.claude/skills/mendix/run-app/SKILL.md index 03fd567984..e0afd20549 100644 --- a/.claude/skills/mendix/run-app/SKILL.md +++ b/.claude/skills/mendix/run-app/SKILL.md @@ -2,7 +2,26 @@ name: run-app description: "Build and start the Mendix app in Docker, and restart it after changes. Use when asked to run, launch or restart the app in a container. For the fast inner loop prefer run-local." --- - + +## Find it fast + +`sed -n ',p' SKILL.md` one section rather than reading the file; +`./mxcli syntax ` answers *how do I write this* for a fraction of either. + +| lines | section | +|---|---| +| 29-35 | When to Use This Skill | +| 36-59 | Prerequisites Check | +| 60-79 | Quick Start: One Command | +| 80-110 | After Making MDL Changes | +| 111-127 | Step-by-Step Alternative | +| 128-145 | Query Data (OQL) | +| 146-165 | Monitoring | +| 166-177 | Stop | +| 178-203 | Running Multiple Projects | +| 204-211 | Architecture Note | + + # Run App Skill This skill builds and starts the Mendix application in Docker. diff --git a/.claude/skills/mendix/run-local/SKILL.md b/.claude/skills/mendix/run-local/SKILL.md index 684f624e9e..4abf4a06b3 100644 --- a/.claude/skills/mendix/run-local/SKILL.md +++ b/.claude/skills/mendix/run-local/SKILL.md @@ -2,7 +2,39 @@ name: run-local description: "The warm Docker-free dev loop, `mxcli run --local` — model changes live in about a second, with watch, screenshots, metrics, tracing and external preview, run as a background service (`--detach`, `run status|wait|stop|restart`). Use for the fastest edit-to-running-app cycle, especially when driving the model programmatically with MDL, and whenever you would reach for nohup, a sleep/grep poll loop or pkill to manage the app." --- + +## Find it fast +`sed -n ',p' SKILL.md` one section rather than reading the file; +`./mxcli syntax ` answers *how do I write this* for a fraction of either. + +| lines | section | +|---|---| +| 40-47 | Overview | +| 48-60 | When to Use This Skill | +| 61-116 | Usage | +| 117-129 | How apply is chosen | +| 130-195 | Prerequisites | +| 196-213 | The intended loop | +| 214-238 | Editing themes (SCSS) — rebuild, never clear caches | +| 239-265 | `--watch` waits for a write to finish before it builds | +| 266-294 | "My edit didn't show up" — stale process, not stale cache | +| 295-322 | Flags | +| 323-376 | Metrics and OpenTelemetry | +| 377-434 | Pages render in the browser | +| 435-452 | Pixel-perfect page loop | +| 453-473 | Debugging a server-side error | +| 474-494 | "Sign in failed" that is not about the password | +| 495-527 | The same licence also stops the app after a few hours | +| 528-607 | External browser preview (`--hub`) | +| 608-616 | Validation checklist | +| 617-656 | Constant values come from a configuration | +| 657-660 | Warm local dev loop (`mxcli run --local [--watch] [--screenshot]`) | +| 661-664 | External browser preview (`mxcli run --hub ` + `mxcli tunnel-hub`) | +| 665-668 | Tunnel-hub GitHub authentication (opt-in, gated on `--github-oauth-client-id`; absent = today's open hub) | +| 669-671 | Runtime metrics + settings passthrough (`mxcli run --local --metrics` / `--runtime-setting Key=Value`) | + + # Warm Local Dev Loop — `mxcli run --local` ## Overview diff --git a/.claude/skills/mendix/runtime-admin-api/SKILL.md b/.claude/skills/mendix/runtime-admin-api/SKILL.md index c98be90781..999309807f 100644 --- a/.claude/skills/mendix/runtime-admin-api/SKILL.md +++ b/.claude/skills/mendix/runtime-admin-api/SKILL.md @@ -2,7 +2,26 @@ name: runtime-admin-api description: "Call the Mendix M2EE admin API on port 8090 directly — OQL, runtime info, and the rest, with curl examples. Use when querying a running app from a script, or when debugging connectivity to the admin port." --- - + +## Find it fast + +`sed -n ',p' SKILL.md` one section rather than reading the file; +`./mxcli syntax ` answers *how do I write this* for a fraction of either. + +| lines | section | +|---|---| +| 29-36 | When to Use This Skill | +| 37-45 | Admin API Overview | +| 46-57 | Authentication | +| 58-135 | curl Examples | +| 136-152 | Using mxcli oql (Preferred) | +| 153-171 | Devcontainer / DinD Connectivity | +| 172-186 | Prerequisites | +| 187-200 | Configuration Reference | +| 201-222 | Troubleshooting | +| 223-228 | Related Skills | + + # Runtime Admin API Skill This skill documents the Mendix M2EE admin API exposed on port 8090, including how to call it directly with curl. diff --git a/.claude/skills/mendix/scheduled-events-and-queues/SKILL.md b/.claude/skills/mendix/scheduled-events-and-queues/SKILL.md index dd98b5a27b..54c4ab7575 100644 --- a/.claude/skills/mendix/scheduled-events-and-queues/SKILL.md +++ b/.claude/skills/mendix/scheduled-events-and-queues/SKILL.md @@ -2,7 +2,27 @@ name: scheduled-events-and-queues description: "Scheduled events (Mendix's cron) and task queues in MDL — the eight repeat variants and the fields each takes, and what a queue does and does not throttle. Use when running a microflow on a schedule, or bounding how many background tasks run at once." --- - + +## Find it fast + +`sed -n ',p' SKILL.md` one section rather than reading the file; +`./mxcli syntax ` answers *how do I write this* for a fraction of either. + +| lines | section | +|---|---| +| 28-38 | When to Use This Skill | +| 39-143 | Scheduled Events | +| 144-168 | Task Queues | +| 169-178 | Common Mistakes | +| 179-270 | Binding a Call to a Queue — `IN QUEUE` | +| 271-285 | Placing Them in Folders | +| 286-300 | Validation Checklist | +| 301-322 | Querying and Linting | +| 323-328 | Related | +| 329-332 | Scheduled events — Mendix's cron (LIST/DESCRIBE/CREATE [OR MODIFY]/DROP). `Repeat:` names one of the eight `ScheduledEvents$*Schedule` variants and only that variant's fields are accepted; a field from another repeat is refused by `mxcli check` (MDL-SCHED01) and by exec, which call the same function. The document shape is pinned by re-serializing three whole Studio Pro-authored events (Workflow Commons 4.11.0, OIDC SSO 4.6.0, SAML 4.2.1) element by element — `modelsdk/gen` is **wrong** about two properties here | +| 333-335 | Task queues (LIST/DESCRIBE/CREATE [OR MODIFY]/DROP TASK QUEUE). `Config.ParallelismExpression` is a **string** and the sibling int32 `Parallelism` is not written — matching all four Studio Pro queues in Business Events 3.12.1. Binding a *call* to a queue is not yet authorable, so `CREATE OR REPLACE|MODIFY MICROFLOW` is **refused** when the stored microflow has a queued call (guard-don't-drop, ADR-0005) | + + # Scheduled Events and Task Queues ## When to Use This Skill diff --git a/.claude/skills/mendix/system-module/SKILL.md b/.claude/skills/mendix/system-module/SKILL.md index d719170f92..486b02cea2 100644 --- a/.claude/skills/mendix/system-module/SKILL.md +++ b/.claude/skills/mendix/system-module/SKILL.md @@ -2,7 +2,29 @@ name: system-module description: "Reference for the built-in System module — User, FileDocument, Image, workflow and queue entities, and the associations you are allowed to make to them. Use when linking a record to System.User, handling file uploads, or working with workflow entities." --- + +## Find it fast +`sed -n ',p' SKILL.md` one section rather than reading the file; +`./mxcli syntax ` answers *how do I write this* for a fraction of either. + +| lines | section | +|---|---| +| 32-49 | Access to System entities has a ceiling | +| 50-56 | Reference files | +| 57-66 | When to Use This Skill | +| 67-127 | How to Reference System Entities | +| 128-220 | 1. User Management & Authentication | +| 221-274 | 2. File Management | +| 275-332 | 3. HTTP / Web Services | +| 333-359 | 4. Error Handling | +| 360-383 | 7. Utility Entities | +| 384-448 | 8. Enumerations | +| 449-480 | 9. Inheritance Hierarchies | +| 481-536 | 10. Common Patterns | +| 537-539 | Source | + + # Mendix System Module Reference The `System` module is a built-in Mendix module present in every application. It provides core entities for user management, file handling, workflows, task queues, HTTP services, and more. These entities are not defined in the application's domain model but are available for use in microflows, pages, associations, and Java actions. diff --git a/.claude/skills/mendix/teamserver-git/SKILL.md b/.claude/skills/mendix/teamserver-git/SKILL.md index 27354db15f..27fec27f20 100644 --- a/.claude/skills/mendix/teamserver-git/SKILL.md +++ b/.claude/skills/mendix/teamserver-git/SKILL.md @@ -2,7 +2,23 @@ name: teamserver-git description: "Commit, branch and push a Mendix project in git the way Studio Pro and Mendix Team Server expect — branches created on the server (not with git checkout -b), the mx_metadata commit note written with `mxcli git note`, and the notes ref pushed with the branch. Use before any git commit, branch, merge, rebase or push in a Mendix app repository, and when Studio Pro shows conflicts or changes that git does not, fails to open the project, or cannot deploy a revision." --- - + +## Find it fast + +`sed -n ',p' SKILL.md` one section rather than reading the file; +`./mxcli syntax ` answers *how do I write this* for a fraction of either. + +| lines | section | +|---|---| +| 35-41 | The rule: let Studio Pro do version control when it can | +| 42-55 | Branches: create them on the server | +| 56-78 | Commit from the agent | +| 79-93 | Push the branch and the notes together, and soon | +| 94-106 | Never rewrite or merge model history with git | +| 107-117 | Reading what Studio Pro recorded | +| 118-126 | Checklist | + + # Working with Git on a Mendix Team Server Project A Mendix project in git is more than its files. Studio Pro keeps four things in diff --git a/.claude/skills/mendix/test-app/SKILL.md b/.claude/skills/mendix/test-app/SKILL.md index 381e31226d..d299ce6bb5 100644 --- a/.claude/skills/mendix/test-app/SKILL.md +++ b/.claude/skills/mendix/test-app/SKILL.md @@ -2,7 +2,29 @@ name: test-app description: "Verify a running Mendix app in a browser: `mxcli playwright check` for a one-call text verdict per page (renders? error banner? console errors? rows?), playwright-cli scripts for interaction, OQL for data. Use when asked to check a page, see whether it renders, verify in the browser, test the app end to end, or confirm generated pages work — before reaching for a screenshot." --- - + +## Find it fast + +`sed -n ',p' SKILL.md` one section rather than reading the file; +`./mxcli syntax ` answers *how do I write this* for a fraction of either. + +| lines | section | +|---|---| +| 32-42 | When to Use This Skill | +| 43-84 | Start Here: `mxcli playwright check` (one call, text, no screenshot) | +| 85-135 | Prerequisites | +| 136-160 | Quick Start | +| 161-182 | Widget Name Selectors | +| 183-298 | Verification Patterns | +| 299-372 | CI/CD: Test Scripts | +| 373-416 | Session Management | +| 417-444 | Debugging | +| 445-477 | Selector Rules | +| 478-494 | Known Gotchas | +| 495-528 | Feedback Loop Workflow | +| 529-536 | Related Skills | + + # Test App Skill This skill guides you through verifying a running Mendix application using playwright-cli for browser automation and mxcli oql for data assertions. diff --git a/.claude/skills/mendix/test-microflows/SKILL.md b/.claude/skills/mendix/test-microflows/SKILL.md index 12d37a099c..d175e69166 100644 --- a/.claude/skills/mendix/test-microflows/SKILL.md +++ b/.claude/skills/mendix/test-microflows/SKILL.md @@ -2,7 +2,30 @@ name: test-microflows description: "Write and run MDL-based microflow tests with `mxcli test` — annotations, file formats, and the warm local test loop. Use when testing microflow logic rather than the UI: return values, entity changes, control flow." --- + +## Find it fast +`sed -n ',p' SKILL.md` one section rather than reading the file; +`./mxcli syntax ` answers *how do I write this* for a fraction of either. + +| lines | section | +|---|---| +| 33-39 | Reference files | +| 40-50 | When to Use This Skill | +| 51-178 | Prerequisites | +| 179-221 | Test File Formats | +| 222-237 | string Operations | +| 238-261 | Running Tests | +| 262-433 | How It Works | +| 434-460 | Check a test file before you run it | +| 461-567 | Writing Good Tests | +| 568-582 | Test File Organization | +| 583-593 | Interpreting Failures | +| 594-615 | CI Integration | +| 616-622 | Related Skills | +| 623-625 | Warm test loop (`mxcli test --local [--watch]`, `--attach`, `run --local --test-endpoint`) | + + # Test Microflows Skill This skill guides you through writing and running MDL-based microflow tests using `mxcli test`. diff --git a/.claude/skills/mendix/theme-styling/SKILL.md b/.claude/skills/mendix/theme-styling/SKILL.md index f2cff62ac5..8b08c9d286 100644 --- a/.claude/skills/mendix/theme-styling/SKILL.md +++ b/.claude/skills/mendix/theme-styling/SKILL.md @@ -2,7 +2,23 @@ name: theme-styling description: "The SCSS workflow and its traps — where styling actually compiles, custom-variables.scss, themesource directories, hot reload, design-property errors, and the mxcli theme commands (apply, create --from a design, switchable sets, light/dark). Use when writing or debugging SCSS, when applying or building a theme, when giving an app a brand palette or design tokens, or when styling silently fails to appear." --- - + +## Find it fast + +`sed -n ',p' SKILL.md` one section rather than reading the file; +`./mxcli syntax ` answers *how do I write this* for a fraction of either. + +| lines | section | +|---|---| +| 24-43 | When to Use This Skill | +| 44-297 | SCSS Compilation Chain | +| 298-316 | CSS Hot-Reload Workflow | +| 317-416 | Caveats | +| 417-431 | Validation with `mxcli check -p` | +| 432-440 | Checklist | +| 441-443 | Default styling + runtime theme switching (`mxcli theme list/show/create/apply/remove/switcher`, `mxcli new --theme`) | + + # Theme & Styling — SCSS Workflow and Caveats ## When to Use This Skill diff --git a/.claude/skills/mendix/translations/SKILL.md b/.claude/skills/mendix/translations/SKILL.md index 8a49c809b0..90a9314d1d 100644 --- a/.claude/skills/mendix/translations/SKILL.md +++ b/.claude/skills/mendix/translations/SKILL.md @@ -2,7 +2,22 @@ name: translations description: "Translate a Mendix app's user-visible strings with MDL — DESCRIBE TRANSLATIONS to export one file per language, CREATE [OR MODIFY|REPLACE] TRANSLATIONS to write them back. Use when localising an app, filling in a language with an LLM, or auditing which strings are still untranslated. Covers why a translation can be stored and still never appear." --- - + +## Find it fast + +`sed -n ',p' SKILL.md` one section rather than reading the file; +`./mxcli syntax ` answers *how do I write this* for a fraction of either. + +| lines | section | +|---|---| +| 27-42 | The loop | +| 43-74 | Statements | +| 75-94 | The trap: a language that is not enabled | +| 95-113 | Drift: a source string that was edited | +| 114-131 | Notes | +| 132-147 | Common tasks | + + # Translations Every user-visible string of a Mendix app — page titles, widget captions, button diff --git a/.claude/skills/mendix/upgrade-mendix-version/SKILL.md b/.claude/skills/mendix/upgrade-mendix-version/SKILL.md index efbef9bdb6..57a6e4a86a 100644 --- a/.claude/skills/mendix/upgrade-mendix-version/SKILL.md +++ b/.claude/skills/mendix/upgrade-mendix-version/SKILL.md @@ -2,7 +2,24 @@ name: upgrade-mendix-version description: "Move a project to a newer Mendix version headlessly, with no Studio Pro. Use when raising a project's Mendix version, when a build complains the project version does not match MxBuild, or before adopting a newer runtime. Covers the one check that decides whether a converter must run, and the two green false successes that upgrade nothing." --- - + +## Find it fast + +`sed -n ',p' SKILL.md` one section rather than reading the file; +`./mxcli syntax ` answers *how do I write this* for a fraction of either. + +| lines | section | +|---|---| +| 25-37 | Why this needs a skill | +| 38-89 | The check that decides everything | +| 90-119 | Doing it | +| 120-139 | Verifying MPR v2 survived | +| 140-170 | Traps | +| 171-181 | What generalises, and what does not | +| 182-189 | After the upgrade | +| 190-195 | Related | + + # Upgrading a Mendix version without Studio Pro ## Why this needs a skill diff --git a/.claude/skills/mendix/validation-microflows/SKILL.md b/.claude/skills/mendix/validation-microflows/SKILL.md index a831026ada..d76196d3f0 100644 --- a/.claude/skills/mendix/validation-microflows/SKILL.md +++ b/.claude/skills/mendix/validation-microflows/SKILL.md @@ -2,7 +2,24 @@ name: validation-microflows description: "Validation microflows for new/edit pages — attribute checks, feedback messages, and conditional validation chains. Use when a form needs input validation with messages shown against the fields." --- - + +## Find it fast + +`sed -n ',p' SKILL.md` one section rather than reading the file; +`./mxcli syntax ` answers *how do I write this* for a fraction of either. + +| lines | section | +|---|---| +| 27-36 | When to Use This Skill | +| 37-51 | The Validation Pattern | +| 52-75 | MDL Syntax | +| 76-163 | Complete Example | +| 164-273 | Validation Patterns | +| 274-298 | Implementation Checklist | +| 299-309 | Files Modified | +| 310-312 | Validation rules (CREATE VALIDATION RULE) | + + # Validation Microflows Skill This skill provides guidance for creating validation microflows in MDL that validate user input on NewEdit pages and provide feedback to users. diff --git a/.claude/skills/mendix/verify-with-oql/SKILL.md b/.claude/skills/mendix/verify-with-oql/SKILL.md index 97d9c8ae0c..06ea3e79b7 100644 --- a/.claude/skills/mendix/verify-with-oql/SKILL.md +++ b/.claude/skills/mendix/verify-with-oql/SKILL.md @@ -2,7 +2,23 @@ name: verify-with-oql description: "Verify microflow side effects and data changes with OQL against a running app. Use after executing changes to confirm data was created, updated or deleted as expected, or to back a browser test with a data assertion." --- - + +## Find it fast + +`sed -n ',p' SKILL.md` one section rather than reading the file; +`./mxcli syntax ` answers *how do I write this* for a fraction of either. + +| lines | section | +|---|---| +| 26-33 | When to Use This Skill | +| 34-54 | The Pattern | +| 55-103 | OQL Verification Examples | +| 104-163 | Combining with Playwright | +| 164-187 | Combining with Hot Reload | +| 188-204 | Tips | +| 205-210 | Related Skills | + + # Verify with OQL Skill This skill documents how to verify microflow side effects and data changes using OQL queries against a running Mendix app. diff --git a/.claude/skills/mendix/write-layouts/SKILL.md b/.claude/skills/mendix/write-layouts/SKILL.md index cb21761931..904bbd7e4c 100644 --- a/.claude/skills/mendix/write-layouts/SKILL.md +++ b/.claude/skills/mendix/write-layouts/SKILL.md @@ -2,7 +2,25 @@ name: write-layouts description: "CREATE LAYOUT syntax — the frame a page is built on: scroll-container regions, the navigation tree, and the placeholders pages bind to. Use when a topbar, sidebar or page frame has to change, or when copying an Atlas layout into a module you own." --- - + +## Find it fast + +`sed -n ',p' SKILL.md` one section rather than reading the file; +`./mxcli syntax ` answers *how do I write this* for a fraction of either. + +| lines | section | +|---|---| +| 30-135 | Do not edit Atlas_Core | +| 136-165 | Syntax | +| 166-205 | The four things only a layout has | +| 206-217 | Layout type, and why there is no `native:` flag | +| 218-239 | The layout class is load-bearing | +| 240-285 | Gotchas | +| 286-298 | Validate | +| 299-311 | What `mxcli new` scaffolds | +| 312-314 | Layouts (SHOW/DESCRIBE/CREATE [OR REPLACE] LAYOUT) | + + # CREATE LAYOUT — the frame a page is built on A **layout** is the frame every page renders inside: the topbar, the navigation diff --git a/.claude/skills/mendix/write-lint-rules/SKILL.md b/.claude/skills/mendix/write-lint-rules/SKILL.md index e1e97919a8..f0cf7cbebb 100644 --- a/.claude/skills/mendix/write-lint-rules/SKILL.md +++ b/.claude/skills/mendix/write-lint-rules/SKILL.md @@ -2,7 +2,22 @@ name: write-lint-rules description: "Write custom Starlark lint rules in .claude/lint-rules/ that run beside the built-ins under `mxcli lint`. Use when a project convention should be enforced automatically." --- - + +## Find it fast + +`sed -n ',p' SKILL.md` one section rather than reading the file; +`./mxcli syntax ` answers *how do I write this* for a fraction of either. + +| lines | section | +|---|---| +| 25-67 | Rule File Structure | +| 68-161 | Available Query Functions | +| 162-600 | Object Properties | +| 601-613 | Helper Functions | +| 614-700 | Common Patterns | +| 701-710 | Validation | + + # Writing Custom Starlark Lint Rules Custom lint rules are written in Starlark (a Python-like language) and placed in `.claude/lint-rules/` as `.star` files. They run alongside the built-in rules when `mxcli lint -p app.mpr` is executed. diff --git a/.claude/skills/mendix/write-microflows/SKILL.md b/.claude/skills/mendix/write-microflows/SKILL.md index c6c8284dcb..b1807a22b2 100644 --- a/.claude/skills/mendix/write-microflows/SKILL.md +++ b/.claude/skills/mendix/write-microflows/SKILL.md @@ -2,7 +2,33 @@ name: write-microflows description: "Microflow syntax reference in MDL — every activity type, control flow, expressions, and the mistakes that fail `mxcli check`. Use before writing any CREATE MICROFLOW, and when debugging a microflow syntax error." --- + +## Find it fast +`sed -n ',p' SKILL.md` one section rather than reading the file; +`./mxcli syntax ` answers *how do I write this* for a fraction of either. + +| lines | section | +|---|---| +| 34-51 | Reference files | +| 52-61 | When to Use This Skill | +| 62-76 | Changing an Existing Microflow | +| 77-95 | When to Use a Microflow vs a Nanoflow | +| 96-112 | Key Differences from Nanoflows | +| 113-266 | Microflow Structure | +| 267-344 | Variable Declarations | +| 345-402 | Operators | +| 403-419 | Logging | +| 420-445 | Special Values | +| 446-492 | Complete Example | +| 493-549 | Calling Microflows | +| 550-576 | Page Navigation | +| 577-593 | Validation Checklist | +| 594-605 | Tips for Success | +| 606-612 | Related Documentation | +| 613-727 | Quick Reference | + + # Mendix Microflow Skill ## Reference files @@ -43,9 +69,10 @@ Choose the mode by who owns the microflow ([choose-edit-mode](../choose-edit-mod (targets from `describe microflow X with handles`). `create or modify` of `describe` output patches too: unchanged writes nothing; a statement (even one whose handler returns), guard clause, `return` value, `if` condition, header clause, parameter (added/retyped; removed only if unused) or stated `@position`/`@start` change is - patched in place (a move keeps the node's flows). A redrawn `@anchor`/`@curve`, loop body, - error handler or other `return` added/taken away rebuilds under mdl 0 (`MDL-V1-REBUILD`: IDs - renumbered, merges and curves lost) and is refused under `mdl 1;`. **To change a loop body, `alter … replace` the whole loop** — neither mode edits inside one ([pitfalls](reference/pitfalls.md#11-changing-something-inside-a-loop-body)). **To rebuild deliberately, `drop` and `create` in ONE script** — the grants carry only within it ([pitfalls](reference/pitfalls.md#drop--create-is-still-a-new-document)). + patched in place (a move keeps the node's flows), as is a loop-body activity with new values. A redrawn + `@anchor`/`@curve`, other loop-body change, error handler or other `return` added/taken away rebuilds under + mdl 0 (`MDL-V1-REBUILD`: IDs renumbered, merges and curves lost) and is refused under `mdl 1;`. **To add, + remove or reorder in a loop body, `alter … replace` the whole loop** ([pitfalls](reference/pitfalls.md#11-changing-something-inside-a-loop-body)). **To rebuild deliberately, `drop` and `create` in ONE script** — the grants carry only within it ([pitfalls](reference/pitfalls.md#drop--create-is-still-a-new-document)). ## When to Use a Microflow vs a Nanoflow diff --git a/.claude/skills/mendix/write-microflows/reference/pitfalls.md b/.claude/skills/mendix/write-microflows/reference/pitfalls.md index 080d00b4bc..e45c218d5f 100644 --- a/.claude/skills/mendix/write-microflows/reference/pitfalls.md +++ b/.claude/skills/mendix/write-microflows/reference/pitfalls.md @@ -277,16 +277,24 @@ the same CE0117, and mxcli refuses the statement rather than writing it. ### 11. Changing Something Inside a Loop Body -**Refusal**: `create or modify` under `mdl 1;` — "the Loop at (x, y) changes inside -its body; the splice does not edit inside a loop"; `alter` aimed at an activity in -the loop — "… is inside the body of loop $Car in $Cars; alter does not splice inside -a loop body". Both refusals spell out the statement below. - -Neither editing mode splices **inside** a loop, so do not try one after the other. -Replace the **whole loop**, addressed by its handle from `describe microflow X with -handles`, with the body as it should be: - -❌ **REFUSED** (either way): +**What is spliced** (both modes, ako/mxcli#1106): an activity in the body replaced by +one activity **of the same kind**, in place — a new expression in a `set`, a new +message in a `log`, `commit $Car` → `commit $Car without events`. Re-running a +`create or modify` whose loop body has the same statements in the same order, each +as stored or with new values, writes just those activities; the loop and everything +else keep their `$ID`s and positions. The loop's iterator is in scope. + +**Refusal** for anything else inside the body — a statement added, removed, +reordered or moved, one kind swapped for another, an `if` or `return` changed, a +custom error handler, a note or connector redrawn: `create or modify` under +`mdl 1;` — "the Loop at (x, y) changes inside its body; the splice does not edit +inside a loop"; `alter` aimed at an activity in the loop — "… is inside the body of +loop $Car in $Cars; alter does not splice inside a loop body but to replace one +activity by one of the same kind". A body node's coordinates are relative to the +loop box, so those changes would need the body laid out again. Both refusals spell +out the statement below. + +✅ **SPLICED** in place: ```mdl mdl 1; alter microflow MyModule.ACT_SaveAll { @@ -294,6 +302,14 @@ alter microflow MyModule.ACT_SaveAll { }; ``` +❌ **REFUSED** (either way) — the body changes shape: +```mdl +mdl 1; +alter microflow MyModule.ACT_SaveAll { + insert after commit $Car begin log info node 'App' 'saved'; end; +}; +``` + ✅ **CORRECT** — replace the loop: ```mdl mdl 1; @@ -301,7 +317,8 @@ alter microflow MyModule.ACT_SaveAll { replace loop $Car in $Cars with begin loop $Car in $Cars begin - commit $Car without events; + commit $Car; + log info node 'App' 'saved'; end loop; end; }; @@ -310,7 +327,7 @@ alter microflow MyModule.ACT_SaveAll { Everything outside the loop keeps its `$ID`s, positions and flows; the loop and its body are rebuilt (new IDs, drawn by mxcli). A `while` loop is the same: `replace while $N < 3 with begin while $N < 3 begin … end while; end;`. For a loop -nested in another loop, replace the outer one — alter addresses nothing inside a loop. +nested in another loop, replace the outer one. ## Implicit Variable Creation (CE0111 Duplicate Variable) diff --git a/.claude/skills/mendix/write-nanoflows/SKILL.md b/.claude/skills/mendix/write-nanoflows/SKILL.md index e5b9bb28d9..d6996c37a6 100644 --- a/.claude/skills/mendix/write-nanoflows/SKILL.md +++ b/.claude/skills/mendix/write-nanoflows/SKILL.md @@ -2,7 +2,32 @@ name: write-nanoflows description: "Nanoflow syntax in MDL — shared with microflows, but client-side and restricted. Use before writing any CREATE NANOFLOW, when a nanoflow validation error needs explaining, or when deciding between a nanoflow and a microflow." --- + +## Find it fast +`sed -n ',p' SKILL.md` one section rather than reading the file; +`./mxcli syntax ` answers *how do I write this* for a fraction of either. + +| lines | section | +|---|---| +| 35-44 | When to Use This Skill | +| 45-78 | Changing an Existing Nanoflow | +| 79-97 | When to Use a Nanoflow vs a Microflow | +| 98-112 | Key Differences from Microflows | +| 113-136 | Nanoflow Structure | +| 137-145 | Naming Convention | +| 146-214 | Supported Activities | +| 215-481 | Real-World Patterns | +| 482-552 | NanoflowCommons JavaScript Actions Reference | +| 553-591 | Disallowed Activities | +| 592-595 | Return Type Restrictions | +| 596-639 | Error Handling in Nanoflows | +| 640-647 | Security (GRANT/REVOKE) | +| 648-660 | Management Commands | +| 661-671 | Common Mistakes | +| 672-686 | Validation Checklist | + + # Mendix Nanoflow Skill This skill provides guidance for writing Mendix nanoflows in MDL syntax. Nanoflows share syntax with microflows but execute client-side with restricted capabilities. @@ -43,12 +68,13 @@ Choose the mode by who owns the nanoflow ([choose-edit-mode](../choose-edit-mode the same script or REPL session; a drop in one run and a create in the next loses every `grant execute`. `drop` prints the roles it removed and the `grant` to restore them. - **Changing something inside a loop body** (either owner) -: neither `create or modify` - (under `mdl 1;`) nor an `alter` aimed at an activity in the loop can make it — `alter` - does not splice inside a loop. Replace the **whole loop**, addressed by its handle, with - the body as it should be (`replace loop $Item in $Items with begin loop $Item in $Items - begin … end loop; end;`). Everything outside the loop keeps its `$ID`s, positions and - curves; the loop and its body are rebuilt. For a nested loop, replace the outer one. +: an activity replaced by one of the same kind (new values, same statements in the same + order) is spliced in place by both `create or modify` and `alter`. Anything else — a + statement added, removed, reordered or moved — neither can make: replace the **whole + loop**, addressed by its handle, with the body as it should be (`replace loop $Item in + $Items with begin loop $Item in $Items begin … end loop; end;`). Everything outside the + loop keeps its `$ID`s, positions and curves; the loop and its body are rebuilt. For a + nested loop, replace the outer one. ## When to Use a Nanoflow vs a Microflow diff --git a/.claude/skills/mendix/write-oql-queries/SKILL.md b/.claude/skills/mendix/write-oql-queries/SKILL.md index df07b718d3..9c3cb377f8 100644 --- a/.claude/skills/mendix/write-oql-queries/SKILL.md +++ b/.claude/skills/mendix/write-oql-queries/SKILL.md @@ -2,7 +2,26 @@ name: write-oql-queries description: "Write OQL for Mendix VIEW entities — joins, aggregates, calculated fields, and the syntax the runtime actually accepts. Use when creating a VIEW entity or building a report or analytics query." --- - + +## Find it fast + +`sed -n ',p' SKILL.md` one section rather than reading the file; +`./mxcli syntax ` answers *how do I write this* for a fraction of either. + +| lines | section | +|---|---| +| 27-32 | Reference files | +| 33-35 | Purpose | +| 36-42 | When to Use This Skill | +| 43-378 | Critical OQL Syntax Rules | +| 379-515 | Step-by-Step Process | +| 516-611 | Common Mistakes to Avoid | +| 612-643 | Testing OQL Queries | +| 644-662 | Integration with MDL Linter | +| 663-670 | References | +| 671-695 | Summary Checklist | + + # Skill: Write OQL Queries for Mendix VIEW Entities ## Reference files diff --git a/.claude/skills/mendix/write-rules/SKILL.md b/.claude/skills/mendix/write-rules/SKILL.md index 4211b94ce3..919b363e99 100644 --- a/.claude/skills/mendix/write-rules/SKILL.md +++ b/.claude/skills/mendix/write-rules/SKILL.md @@ -2,7 +2,24 @@ name: write-rules description: "Write Mendix rules — a special kind of microflow returning a Boolean or enumeration, callable only from a decision. Use when writing CREATE RULE, or deciding whether logic belongs in a rule or a microflow." --- + +## Find it fast +`sed -n ',p' SKILL.md` one section rather than reading the file; +`./mxcli syntax ` answers *how do I write this* for a fraction of either. + +| lines | section | +|---|---| +| 29-38 | When to Use This Skill | +| 39-52 | When to Use a Rule vs a Microflow | +| 53-67 | Key Differences from Microflows | +| 68-111 | Syntax | +| 112-129 | Reading and managing rules | +| 130-144 | What a rule may not contain | +| 145-155 | Validation checklist | +| 156-158 | Rules (LIST/DESCRIBE/CREATE [OR MODIFY]/DROP/MOVE RULE) | + + # Mendix Rule Skill Guidance for writing Mendix **rules** in MDL. Mendix's own reference calls a rule diff --git a/.claude/skills/mendix/write-workflows/SKILL.md b/.claude/skills/mendix/write-workflows/SKILL.md index 6f191ffc06..83ee2f54eb 100644 --- a/.claude/skills/mendix/write-workflows/SKILL.md +++ b/.claude/skills/mendix/write-workflows/SKILL.md @@ -2,7 +2,29 @@ name: write-workflows description: "Author Mendix workflows in MDL — user tasks, decisions, parallel splits, jumps, waits and boundary events, with CREATE, ALTER and DROP. Use when building a business process with human steps, timers or parallel branches." --- + +## Find it fast +`sed -n ',p' SKILL.md` one section rather than reading the file; +`./mxcli syntax ` answers *how do I write this* for a fraction of either. + +| lines | section | +|---|---| +| 35-46 | When to Use This Skill | +| 47-110 | Syntax — CREATE WORKFLOW | +| 111-282 | Activities | +| 283-289 | DROP WORKFLOW | +| 290-355 | ALTER WORKFLOW | +| 356-376 | DESCRIBE round-trip | +| 377-397 | Activity names, and why `jump to` depends on them | +| 398-418 | Rewriting an existing workflow | +| 419-506 | Microflow statements for workflow tasks | +| 507-527 | System-module enumerations are synthesized, not stored | +| 528-644 | Platform rules | +| 645-675 | Observing a running workflow | +| 676-685 | Validate before presenting | + + # Mendix Workflows Skill Guidance for **authoring** workflows in Mendix projects with MDL — not just diff --git a/.claude/skills/mendix/xpath-constraints/SKILL.md b/.claude/skills/mendix/xpath-constraints/SKILL.md index 634d04def1..46b1b869d7 100644 --- a/.claude/skills/mendix/xpath-constraints/SKILL.md +++ b/.claude/skills/mendix/xpath-constraints/SKILL.md @@ -2,7 +2,25 @@ name: xpath-constraints description: "XPath constraint syntax for MDL — retrieve WHERE clauses, page data sources, and row-level entity access, including association paths and functions. Use when writing or debugging any XPath in a project." --- - + +## Find it fast + +`sed -n ',p' SKILL.md` one section rather than reading the file; +`./mxcli syntax ` answers *how do I write this* for a fraction of either. + +| lines | section | +|---|---| +| 28-34 | When to Use This Skill | +| 35-75 | XPath vs Mendix Expressions | +| 76-119 | Syntax Reference | +| 120-278 | How a Constraint Is Laid Out on Disk | +| 279-329 | Usage Contexts | +| 330-369 | Enumeration Attributes | +| 370-411 | Common Patterns | +| 412-423 | Validation | +| 424-433 | Troubleshooting | + + # XPath Constraints in MDL This skill provides reference for writing XPath constraint expressions in MDL RETRIEVE statements, page data sources, and security rules. diff --git a/.github/workflows/push-test.yml b/.github/workflows/push-test.yml index 9ebc077c95..1b876f965d 100644 --- a/.github/workflows/push-test.yml +++ b/.github/workflows/push-test.yml @@ -167,6 +167,12 @@ jobs: # The table of contents must describe the wiki, both directions. It # drifted for three months before anyone noticed. run: make check-wiki-pages + - name: Check the skill section indexes + # Each SKILL.md opens with a generated line-numbered index so an agent + # can read one section instead of the file — measured on BENCH-001 as + # ~68% of a session's re-read cost. A hand-kept table of contents is + # the drift this repo has paid for twice, so the generator owns it. + run: make check-skill-index - name: Check every go test sets -timeout # Without one Go applies 10m per test binary, and a package that passes # in seconds alone can exceed it under `./...` load (mendixlabs/mxcli#1291). diff --git a/.gitignore b/.gitignore index 27e1738351..d8249d8b7a 100644 --- a/.gitignore +++ b/.gitignore @@ -47,6 +47,7 @@ mdl/grammar/parser/ cmd/mxcli/skills/ cmd/mxcli/skillpacks/ cmd/mxcli/commands/ +cmd/mxcli/mods/ cmd/mxcli/lint-rules/ cmd/mxcli/changelog.md cmd/mxcli/vscode-mdl.vsix diff --git a/Makefile b/Makefile index c6e5572ab2..1acb591ac6 100644 --- a/Makefile +++ b/Makefile @@ -35,7 +35,7 @@ GO_BUILD_FLAGS = -trimpath # Clean version for VS Code extension (must be valid semver: major.minor.patch) VSCE_VERSION = $(shell echo "$(VERSION)" | sed 's/^v//; s/-.*//' | grep -E '^[0-9]+\.[0-9]+\.[0-9]+$$' || echo "0.0.0") -.PHONY: build build-debug size release clean test test-mdl check-mdl check-skill-mdl check-conformance conformance-shrink gen-migration-reference check-migration-reference check-skill-pack-js check-findings check-wiki-pages digest-status check-tunnel-deps check-test-timeouts check-widget-versions test-integration test-integration-executor test-integration-roundtrip test-integration-parity test-integration-upgrade test-integration-other grammar completions sync-skills sync-skill-packs sync-commands sync-lint-rules sync-changelog sync-all docs documentation docs-site docs-serve vscode-ext vscode-install source-tree sbom sbom-report lint lint-go lint-ts fmt fmt-check vet +.PHONY: build build-debug size release clean test test-mdl check-mdl check-skill-mdl check-conformance conformance-shrink gen-migration-reference check-migration-reference check-skill-pack-js check-findings check-wiki-pages digest-status check-tunnel-deps check-test-timeouts check-widget-versions test-integration test-integration-executor test-integration-roundtrip test-integration-parity test-integration-upgrade test-integration-other grammar completions sync-skills sync-skill-packs sync-commands sync-mods sync-lint-rules sync-changelog sync-all docs documentation docs-site docs-serve vscode-ext vscode-install source-tree sbom sbom-report lint lint-go lint-ts fmt fmt-check vet skill-index check-skill-index # Helper: copy file only if content differs (avoids mtime updates that invalidate go build cache) # Usage: $(call copy-if-changed,src,dst) @@ -47,7 +47,18 @@ endef # Skills are directory-shaped (/SKILL.md, Agent Skills standard), so this # mirrors a tree rather than copying a flat list. --delete matters: a renamed or # removed skill must not linger in the embed dir, or the binary keeps shipping it. -sync-skills: +# Refresh the generated line-numbered section index at the top of each SKILL.md. +# Measured on BENCH-001 (2026-10-10): reading the skills was ~68% of a session's +# re-read cost, and the agent had no cheap way to read one section. +skill-index: + @python3 -I scripts/skill-index.py + +# Fails when a SKILL.md moved on without its index. A hand-kept table of +# contents is the drift this repo has already paid for twice. +check-skill-index: + @python3 -I scripts/skill-index.py --check + +sync-skills: skill-index @mkdir -p cmd/mxcli/skills @rsync -a --delete --exclude='.DS_Store' .claude/skills/mendix/ cmd/mxcli/skills/ 2>/dev/null \ || { rm -rf cmd/mxcli/skills && mkdir -p cmd/mxcli/skills && cp -R .claude/skills/mendix/. cmd/mxcli/skills/; } @@ -60,6 +71,14 @@ sync-skill-packs: @rsync -a --delete --exclude='.DS_Store' .claude/skills/packs/ cmd/mxcli/skillpacks/ 2>/dev/null \ || { rm -rf cmd/mxcli/skillpacks && mkdir -p cmd/mxcli/skillpacks && cp -R .claude/skills/packs/. cmd/mxcli/skillpacks/; } +# Sync Claude Code mods from mods/ to cmd/mxcli/mods for embedding. Only what a +# project runs is shipped: a mod's tests and the types the engine lays beside a +# loaded mod stay behind. +sync-mods: + @mkdir -p cmd/mxcli/mods + @rsync -a --delete --exclude='.DS_Store' --exclude='tests/' --exclude='*.test.ts' --exclude='*.test.tsx' \ + --exclude='.claude-plugin/types/' --exclude='tsconfig.json' mods/ cmd/mxcli/mods/ + # Sync commands from .claude/commands/mendix to cmd/mxcli/commands for embedding sync-commands: @mkdir -p cmd/mxcli/commands @@ -100,7 +119,7 @@ sync-changelog: $(call copy-if-changed,CHANGELOG.md,cmd/mxcli/changelog.md) # Sync skills, commands, lint rules, and changelog -sync-all: sync-skills sync-skill-packs sync-commands sync-lint-rules sync-vsix sync-changelog +sync-all: sync-skills sync-skill-packs sync-mods sync-commands sync-lint-rules sync-vsix sync-changelog # Generate LSP completion items from grammar (only rewrites file if content changed) completions: diff --git a/cmd/mxcli/check_v1_policy_test.go b/cmd/mxcli/check_v1_policy_test.go new file mode 100644 index 0000000000..ce82632064 --- /dev/null +++ b/cmd/mxcli/check_v1_policy_test.go @@ -0,0 +1,104 @@ +// SPDX-License-Identifier: Apache-2.0 + +package main + +import ( + "strings" + "testing" + + "github.com/mendixlabs/mxcli/mdl/deprecation" +) + +// --v1 is to the MDL-V1-* warnings what --deprecations is to MDL-DEPRnnn: a +// headerless (mdl 0) script that would not parse under `mdl 1;` warns by +// default, and fails the run under --v1=error (ako/mxcli-sudoku FINDINGS #64). +// Without it a project could pass --deprecations=error and still not be ready +// for the header, with no gate saying so. + +// slashScript is headerless and ends a statement with the SQL*Plus `/` line: +// MDL-V1-SLASH under mdl 0, a syntax error under mdl 1. +const slashScript = "create module V1Gate;\n/\n" + +// headedScript is the same statement written for mdl 1. +const headedScript = "mdl 1;\ncreate module V1Gate;\n" + +func checkWithV1(t *testing.T, value, src string) (int, string) { + t.Helper() + if value != "" { + if err := checkCmd.Flags().Set("v1", value); err != nil { + t.Fatalf("set --v1=%s: %v", value, err) + } + defer func() { + _ = checkCmd.Flags().Set("v1", "warn") + checkCmd.Flags().Lookup("v1").Changed = false + }() + } + file := writeScript(t, t.TempDir(), "s.mdl", src) + var code int + out := captureStd(t, func() { code = runCheckFiles(checkCmd, []string{file}) }) + return code, out +} + +func TestCheckV1WarnsByDefault(t *testing.T) { + code, out := checkWithV1(t, "", slashScript) + if code != 0 { + t.Errorf("a headerless script failed by default (exit %d); MDL-V1-* must only warn:\n%s", code, out) + } + if !strings.Contains(out, "MDL-V1-SLASH") { + t.Errorf("MDL-V1-SLASH not reported:\n%s", out) + } +} + +func TestCheckV1ErrorFailsTheRun(t *testing.T) { + code, out := checkWithV1(t, "error", slashScript) + if code == 0 { + t.Errorf("--v1=error passed a headerless script with a `/` terminator:\n%s", out) + } + if !strings.Contains(out, "MDL-V1-SLASH") { + t.Errorf("the failing run must name the code:\n%s", out) + } + if !strings.Contains(out, "1 error") { + t.Errorf("the summary must count the promoted warning as an error:\n%s", out) + } +} + +// The control: a script already headed `mdl 1;` reports no MDL-V1-* warning, +// so --v1=error does not fail everything. +func TestCheckV1ErrorPassesAHeadedScript(t *testing.T) { + code, out := checkWithV1(t, "error", headedScript) + if code != 0 || strings.Contains(out, "MDL-V1-") { + t.Errorf("an `mdl 1;` script failed --v1=error (exit %d):\n%s", code, out) + } +} + +// --deprecations=error does not gate MDL-V1-*: the two flags gate different +// things, which is why both exist. +func TestCheckDeprecationsErrorDoesNotGateV1(t *testing.T) { + if err := checkCmd.Flags().Set("deprecations", "error"); err != nil { + t.Fatal(err) + } + defer func() { + _ = checkCmd.Flags().Set("deprecations", "warn") + checkCmd.Flags().Lookup("deprecations").Changed = false + }() + code, out := checkWithV1(t, "", slashScript) + if code != 0 { + t.Errorf("--deprecations=error failed on an MDL-V1-* warning (exit %d):\n%s", code, out) + } +} + +func TestParseV1PolicyRejectsUnknownValue(t *testing.T) { + for in, want := range map[string]deprecation.Policy{"": deprecation.Warn, "warn": deprecation.Warn, "ERROR": deprecation.Error} { + got, err := parseV1Policy(in) + if err != nil || got != want { + t.Errorf("parseV1Policy(%q) = %v, %v; want %v", in, got, err, want) + } + } + _, err := parseV1Policy("eror") + if err == nil { + t.Fatal("a typo must be refused, not read as warn: a CI gate meant to fail would pass") + } + if !strings.Contains(err.Error(), "--v1") { + t.Errorf("the error must name the flag: %v", err) + } +} diff --git a/cmd/mxcli/cmd_check.go b/cmd/mxcli/cmd_check.go index 969f3fff4d..ef2a61d095 100644 --- a/cmd/mxcli/cmd_check.go +++ b/cmd/mxcli/cmd_check.go @@ -99,6 +99,15 @@ canonical form, such as "create or replace" for "create or modify" or "show" for "list" — is reported as an MDL-DEPRnnn warning naming the canonical form. Pass --deprecations=error to fail the run on one instead, e.g. in CI over docs. +A headerless script is mdl 0, and every construct in it that means something +else, or is refused, under "mdl 1;" is reported as an MDL-V1-* warning +(MDL-V1-SLASH, MDL-V1-SEMI, MDL-V1-LIMIT1, MDL-V1-QUOTEDEXPR, the MDL-V1-REBUILD +warning form, ...). Pass --v1=error to fail the run on one. The two flags gate +different things: a script can pass --deprecations=error and still not parse +under mdl 1, so --deprecations=error --v1=error together is the full mdl 1 +readiness gate. "mxcli fmt --upgrade --header" is the fix for both. A script +already headed "mdl 1;" reports no MDL-V1-* warning. + Use --post-migration to scan an existing project (independent of the script) for legacy native widgets that have pluggable replacements — Studio Pro does not auto-migrate these on a Mendix major-version upgrade. @@ -118,6 +127,9 @@ Examples: mxcli check script.mdl --format json mxcli check script.mdl -p app.mpr --format sarif > results.sarif + # Gate a headerless script on being ready for "mdl 1;" + mxcli check script.mdl -p app.mpr --deprecations=error --v1=error + # Check a script set, run in this order mxcli check 10-domain.mdl 20-flows.mdl 30-pages.mdl -p app.mpr @@ -157,6 +169,7 @@ func runCheckFileAgainst(cmd *cobra.Command, filePath, against string) int { checkRefs = checkRefs || projectPath != "" postMigration, _ := cmd.Flags().GetBool("post-migration") depPolicy := deprecationPolicy(cmd) + v1 := v1Policy(cmd) format := resolveFormat(cmd, "text") isStructured := format != "" && format != "text" @@ -310,6 +323,7 @@ func runCheckFileAgainst(cmd *cobra.Command, filePath, against string) int { } } violations = executor.ApplyDeprecationPolicy(violations, depPolicy) + violations = executor.ApplyV1Policy(violations, v1) if isStructured { // Always emit structured output (even when clean) @@ -443,6 +457,9 @@ func runCheckFileAgainst(cmd *cobra.Command, filePath, against string) int { // language leaves every required caption written in the old one empty // in the new one (ako/mxcli#944). projectViolations = append(projectViolations, exec.CheckDefaultLanguageCaptions(prog)...) + // --v1=error: the MDL-V1-REBUILD warning form (a headerless script's + // flow change exec rebuilds) fails the run like any other MDL-V1-*. + projectViolations = executor.ApplyV1Policy(projectViolations, v1) if len(projectViolations) > 0 { if isStructured { diff --git a/cmd/mxcli/cmd_describe.go b/cmd/mxcli/cmd_describe.go index d2c8377171..80ab7e6298 100644 --- a/cmd/mxcli/cmd_describe.go +++ b/cmd/mxcli/cmd_describe.go @@ -45,6 +45,7 @@ Types: buildingblock Describe a building block (read-only, also: "building block") layout Describe a layout javaaction Describe a java action + javascriptaction Describe a JavaScript action (also: "javascript action") jsonstructure Describe a JSON structure (also: "json structure") importmapping Describe an import mapping (also: "import mapping") exportmapping Describe an export mapping (also: "export mapping") @@ -128,7 +129,7 @@ Example: mdlCmd, ok := describeMDLCommand(objectType, name) if !ok { fmt.Fprintf(os.Stderr, "Unknown type: %s\n", strings.Join(args[:len(args)-1], " ")) - fmt.Fprintln(os.Stderr, "Valid types: module, entity, association, enumeration, constant, microflow, nanoflow, workflow, page, snippet, layout, javaaction, jsonstructure, importmapping, exportmapping, restclient, odataclient, odataservice, publishedrestservice, imagecollection, menu, queue, scheduledevent, businesseventservice, databaseconnection, agent, aimodel, knowledgebase, consumedmcpservice, datatransformer, modulerole, userrole, projectsecurity, settings, demouser, navigation, systemoverview") + fmt.Fprintln(os.Stderr, "Valid types: module, entity, association, enumeration, constant, microflow, nanoflow, workflow, page, snippet, layout, javaaction, javascriptaction, jsonstructure, importmapping, exportmapping, restclient, odataclient, odataservice, publishedrestservice, imagecollection, menu, queue, scheduledevent, businesseventservice, databaseconnection, agent, aimodel, knowledgebase, consumedmcpservice, datatransformer, modulerole, userrole, projectsecurity, settings, demouser, navigation, systemoverview") fmt.Fprintln(os.Stderr, "Multi-word types also accepted: json structure, import mapping, export mapping, rest client, published rest service, image collection, scheduled event, regular expression, business event service, agent, model, knowledge base, consumed mcp service, data transformer, etc.") os.Exit(1) } @@ -263,6 +264,8 @@ func describeMDLCommand(objectType, name string) (mdlCmd string, ok bool) { return fmt.Sprintf("DESCRIBE DEMO USER '%s'", name), true case "JAVAACTION", "JAVA ACTION": return fmt.Sprintf("DESCRIBE JAVA ACTION %s", name), true + case "JAVASCRIPTACTION", "JAVASCRIPT ACTION": + return fmt.Sprintf("DESCRIBE JAVASCRIPT ACTION %s", name), true case "CONSTANT": return fmt.Sprintf("DESCRIBE CONSTANT %s", name), true case "JSONSTRUCTURE", "JSON STRUCTURE": @@ -296,6 +299,12 @@ func describeMDLCommand(objectType, name string) (mdlCmd string, ok bool) { case "EXTERNALENTITY", "EXTERNAL ENTITY": return fmt.Sprintf("DESCRIBE ENTITY %s", name), true case "NAVIGATION": + // "Navigation" is project-tree's name for the navigation document + // itself, not a profile (profiles are named by kind: Responsive, + // Phone, ...), so it describes every profile. + if strings.EqualFold(name, "Navigation") { + return "DESCRIBE NAVIGATION", true + } return fmt.Sprintf("DESCRIBE NAVIGATION %s", name), true case "NAVPROFILE": return fmt.Sprintf("DESCRIBE NAVIGATION %s", name), true @@ -347,8 +356,7 @@ var objectTypeToDescribe = map[string]string{ "AI_MODEL": "model", "KNOWLEDGE_BASE": "knowledgebase", "CONSUMED_MCP_SERVICE": "consumedmcpservice", - // JAVASCRIPT_ACTION is cataloged but has no `describe` CLI form, so it is - // intentionally omitted from auto-detect. + "JAVASCRIPT_ACTION": "javascriptaction", } // unitTypeToDescribe maps a top-level unit's BSON $Type to the `describe` @@ -366,6 +374,7 @@ var unitTypeToDescribe = map[string]string{ "Enumerations$Enumeration": "enumeration", "Constants$Constant": "constant", "JavaActions$JavaAction": "javaaction", + "JavaScriptActions$JavaScriptAction": "javascriptaction", "JsonStructures$JsonStructure": "jsonstructure", "ImportMappings$ImportMapping": "importmapping", "ExportMappings$ExportMapping": "exportmapping", diff --git a/cmd/mxcli/cmd_eval.go b/cmd/mxcli/cmd_eval.go index 7654ff9676..354511600a 100644 --- a/cmd/mxcli/cmd_eval.go +++ b/cmd/mxcli/cmd_eval.go @@ -49,9 +49,11 @@ var evalCheckCmd = &cobra.Command{ Checks include: - entity_exists: Verify entity with matching name exists - entity_has_attribute: Verify entity has attribute with expected type + - entity_has_type: Verify the Type column reads Persistent, View, Non-Persistent or External - page_exists: Verify page with matching name exists - page_has_widget: Verify page contains expected widget type - microflow_exists: Verify microflow with matching name exists + - workflow_exists: Verify workflow with matching name exists - navigation_has_item: Verify navigation menu has items - association_exists: Verify an association with matching name exists - module_role_exists: Verify a module role with matching name exists diff --git a/cmd/mxcli/cmd_fix_property_sets.go b/cmd/mxcli/cmd_fix_property_sets.go new file mode 100644 index 0000000000..2321be0196 --- /dev/null +++ b/cmd/mxcli/cmd_fix_property_sets.go @@ -0,0 +1,179 @@ +// SPDX-License-Identifier: Apache-2.0 + +package main + +import ( + "errors" + "fmt" + "io" + "os" + "path/filepath" + "sort" + + mpr "github.com/mendixlabs/mxcli/modelsdk/mpr" + "github.com/spf13/cobra" +) + +// cmd_fix_property_sets.go completes the property sets of units already stored +// (ako/mxcli#1097). Like `fix hashes` it needs no mx: the normalization is +// mxcli's own, the same one every write applies. + +var fixPropertySetsCmd = &cobra.Command{ + Use: "property-sets", + Short: "Give every stored unit the property names Studio Pro writes (unblocks mx diff / merge)", + Long: `Normalize the property names of every unit already in the project. + +Mendix's merge and diff engine compares two revisions of an element by its +property NAMES and throws when they differ: + + Objects with ID … of type Microflows$RestOperationCallAction do not have + the same properties. + +The model is valid — mx check passes and Studio Pro opens it — but every diff, +merge and the Version Control panel fail once Studio Pro has saved the +document with its full property set. + +Every mxcli write already gives a unit that set for the project's Mendix +version: missing keys are added with the value Mendix gives an absent one, and +empty keys the version does not declare yet are dropped. A unit written by an +older mxcli and never rewritten since keeps its gaps. This command applies the +same normalization to every stored unit, and writes the units whose property +names change. It adds and removes keys only: no element $ID moves and no +existing GUID changes. The one GUID it adds is on a project user role that has +none (older mxcli wrote them without): the role's own $ID, the value a role +mxcli creates gets — without it the build invents a random GUID each time and +the runtime can only find the role's row by name. A second run writes nothing. + +A unit holding a value in a key the project's version does not declare is +reported and left alone — dropping the value would change the model. Like every +mxcli write, the repair is refused while Studio Pro has the project open. + +Exits 1 when a unit could not be repaired, or, with --dry-run, when one needs it.`, + Example: ` mxcli fix property-sets -p app.mpr --dry-run + mxcli fix property-sets -p app.mpr`, + Args: cobra.NoArgs, + RunE: runFixPropertySets, + SilenceUsage: true, + SilenceErrors: true, +} + +func init() { + fixPropertySetsCmd.Flags().StringP("project", "p", "", "path to the Mendix project (.mpr)") + _ = fixPropertySetsCmd.MarkFlagRequired("project") + fixPropertySetsCmd.Flags().Bool("dry-run", false, "report the units that would change, write nothing") + fixCmd.AddCommand(fixPropertySetsCmd) +} + +// errPropertySetsRemain makes the command exit 1 after it printed the report. +var errPropertySetsRemain = errors.New("property-set gaps remain") + +func runFixPropertySets(cmd *cobra.Command, _ []string) error { + mprPath, _ := cmd.Flags().GetString("project") + dryRun, _ := cmd.Flags().GetBool("dry-run") + if _, err := os.Stat(mprPath); err != nil { + return fmt.Errorf("project not found: %s", mprPath) + } + err := fixPropertySets(mprPath, dryRun, cmd.OutOrStdout()) + if errors.Is(err, errPropertySetsRemain) { + os.Exit(1) + } + return err +} + +// fixPropertySets is the command without its exit: it prints the report and +// returns errPropertySetsRemain when a unit still differs afterwards. +func fixPropertySets(mprPath string, dryRun bool, out io.Writer) error { + var ( + r *mpr.Reader + w *mpr.Writer + ) + if dryRun { + var err error + if r, err = mpr.Open(mprPath); err != nil { + return fmt.Errorf("open project: %w", err) + } + defer r.Close() + } else { + var err error + if w, err = mpr.NewWriter(mprPath); err != nil { + return fmt.Errorf("open project: %w", err) + } + defer w.Close() + r = w.ConcreteReader() + } + + pv := r.ProjectVersion() + if pv == nil || pv.MajorVersion == 0 { + fmt.Fprintf(out, "%s has no readable Mendix version; nothing is normalized without one.\n", filepath.Base(mprPath)) + return nil + } + ids, err := r.ListAllUnitIDs() + if err != nil { + return fmt.Errorf("list units: %w", err) + } + sort.Strings(ids) + + var ( + changed []mpr.PropertySetRepair + failed []string + added, removed int + ) + for _, id := range ids { + var rep mpr.PropertySetRepair + if dryRun { + rep, err = r.StoredPropertySetGaps(id) + } else { + rep, err = w.CompleteStoredPropertySets(id) + } + if err != nil { + failed = append(failed, fmt.Sprintf(" FAILED %s%s: %v", id, repairLabel(rep), err)) + continue + } + if rep.Changed() { + changed = append(changed, rep) + added += rep.Added + removed += rep.Stripped + } + } + + verb := "Completed" + if dryRun { + verb = "Would complete" + } + fmt.Fprintf(out, "Checked %d unit(s) in %s against the Mendix %d.%d.%d property sets.\n", + len(ids), filepath.Base(mprPath), pv.MajorVersion, pv.MinorVersion, pv.PatchVersion) + for _, rep := range changed { + fmt.Fprintf(out, " %-9s %s%s: +%d key(s), -%d key(s)\n", map[bool]string{true: "NEEDS", false: "FIXED"}[dryRun], + rep.UnitID, repairLabel(rep), rep.Added, rep.Stripped) + } + for _, f := range failed { + fmt.Fprintln(out, f) + } + switch { + case len(changed) == 0 && len(failed) == 0: + fmt.Fprintln(out, "Every unit already has the property names Studio Pro writes.") + case len(changed) > 0: + fmt.Fprintf(out, "\n%s %d unit(s): %d key(s) added, %d stripped.\n", verb, len(changed), added, removed) + } + if dryRun && len(changed) > 0 { + fmt.Fprintf(out, "Run 'mxcli fix property-sets -p %s' to write them.\n", mprPath) + } + if len(failed) > 0 { + fmt.Fprintf(out, "%d unit(s) could not be normalized and were left as they are.\n", len(failed)) + } + if len(failed) > 0 || (dryRun && len(changed) > 0) { + return errPropertySetsRemain + } + return nil +} + +// repairLabel names the unit (" [Microflows$Microflow Name]"), or "". +func repairLabel(rep mpr.PropertySetRepair) string { + switch { + case rep.Type == "": + return "" + case rep.Name == "": + return " [" + rep.Type + "]" + } + return " [" + rep.Type + " " + rep.Name + "]" +} diff --git a/cmd/mxcli/cmd_fix_property_sets_test.go b/cmd/mxcli/cmd_fix_property_sets_test.go new file mode 100644 index 0000000000..7f842465af --- /dev/null +++ b/cmd/mxcli/cmd_fix_property_sets_test.go @@ -0,0 +1,128 @@ +// SPDX-License-Identifier: Apache-2.0 + +package main + +import ( + "bytes" + "errors" + "os" + "path/filepath" + "strings" + "testing" + + "go.mongodb.org/mongo-driver/v2/bson" +) + +// ako/mxcli#1097: `mxcli fix property-sets` on a real, Studio Pro-authored +// project (testdata/pedapp, 11.13.0). + +// dropKeyFromUnit removes key from the top level of the first unit of the given +// $Type and Name, as a unit written by an older mxcli would lack it. Returns the +// file and its new bytes. +func dropKeyFromUnit(t *testing.T, mprPath, typ, name, key string) string { + t.Helper() + files, _ := filepath.Glob(filepath.Join(filepath.Dir(mprPath), "mprcontents", "*", "*", "*.mxunit")) + for _, f := range files { + b, err := os.ReadFile(f) + if err != nil { + t.Fatal(err) + } + raw := bson.Raw(b) + if ty, _ := raw.Lookup("$Type").StringValueOK(); ty != typ { + continue + } + if n, _ := raw.Lookup("Name").StringValueOK(); n != name { + continue + } + var d bson.D + if err := bson.Unmarshal(b, &d); err != nil { + t.Fatal(err) + } + out := d[:0] + found := false + for _, e := range d { + if e.Key == key { + found = true + continue + } + out = append(out, e) + } + if !found { + t.Fatalf("subject: %s %s has no %s", typ, name, key) + } + nb, err := bson.Marshal(out) + if err != nil { + t.Fatal(err) + } + if err := os.WriteFile(f, nb, 0o644); err != nil { + t.Fatal(err) + } + return f + } + t.Fatalf("subject: no %s %s in the fixture", typ, name) + return "" +} + +// The control: a project Studio Pro wrote has nothing to repair. A table row +// that added a key Studio Pro does not write would show up here. +func TestFixPropertySets_StudioProProjectIsClean(t *testing.T) { + mprPath := copyPedApp(t) + var out bytes.Buffer + if err := fixPropertySets(mprPath, true, &out); err != nil { + t.Fatalf("dry run on an untouched Studio Pro project: %v\n%s", err, out.String()) + } + if !strings.Contains(out.String(), "Every unit already has") { + t.Fatalf("control not reported clean:\n%s", out.String()) + } +} + +func TestFixPropertySets_CompletesAUnitNothingRewrites(t *testing.T) { + mprPath := copyPedApp(t) + f := dropKeyFromUnit(t, mprPath, "ImportMappings$ImportMapping", "IMM_PostResponse", "MessageDefinition2") + // The stored file now differs from its ContentsHash; rewrite the index so + // the subject is "a unit an older mxcli wrote", not "a reverted file". + var sink bytes.Buffer + if err := fixHashes(mprPath, true, &sink); err != nil { + t.Fatalf("re-index the subject: %v\n%s", err, sink.String()) + } + edited, _ := os.ReadFile(f) + + var out bytes.Buffer + err := fixPropertySets(mprPath, true, &out) + if !errors.Is(err, errPropertySetsRemain) { + t.Fatalf("dry run: want errPropertySetsRemain, got %v\n%s", err, out.String()) + } + if !strings.Contains(out.String(), "IMM_PostResponse]: +1 key(s), -0 key(s)") || !strings.Contains(out.String(), "Would complete 1 unit(s)") { + t.Fatalf("dry run did not report the gap:\n%s", out.String()) + } + if b, _ := os.ReadFile(f); !bytes.Equal(b, edited) { + t.Fatal("the dry run wrote the unit") + } + + out.Reset() + if err := fixPropertySets(mprPath, false, &out); err != nil { + t.Fatalf("repair: %v\n%s", err, out.String()) + } + if !strings.Contains(out.String(), "Completed 1 unit(s): 1 key(s) added, 0 stripped") { + t.Fatalf("repair not reported:\n%s", out.String()) + } + b, _ := os.ReadFile(f) + if v, err := bson.Raw(b).LookupErr("MessageDefinition2"); err != nil || v.StringValue() != "" { + t.Fatalf("MessageDefinition2 after repair: %v %v", v, err) + } + + // A second run writes nothing, and the index still describes the files. + out.Reset() + if err := fixPropertySets(mprPath, false, &out); err != nil { + t.Fatalf("second run: %v\n%s", err, out.String()) + } + if !strings.Contains(out.String(), "Every unit already has") { + t.Fatalf("second run was not a no-op:\n%s", out.String()) + } + if again, _ := os.ReadFile(f); !bytes.Equal(again, b) { + t.Fatal("the second run rewrote the unit") + } + if w := contentsHashDriftWarning(mprPath); w != "" { + t.Fatalf("the repair left the ContentsHash index stale: %s", w) + } +} diff --git a/cmd/mxcli/evalrunner/checks.go b/cmd/mxcli/evalrunner/checks.go index b6280c50ef..c616941f2e 100644 --- a/cmd/mxcli/evalrunner/checks.go +++ b/cmd/mxcli/evalrunner/checks.go @@ -57,6 +57,8 @@ func runCheck(check Check, opts CheckOptions, entityList, pageList, microflowLis return checkEntityExists(check, entityList) case "entity_has_attribute": return checkEntityHasAttribute(check, opts, entityList, describeCache) + case "entity_has_type": + return checkEntityHasType(check, entityList) case "page_exists": return checkPageExists(check, pageList) case "page_has_widget": @@ -76,6 +78,8 @@ func runCheck(check Check, opts CheckOptions, entityList, pageList, microflowLis return checkListed(check, cachedList(opts, describeCache, "list associations"), "association") case "module_role_exists": return checkListed(check, cachedList(opts, describeCache, "list module roles"), "module role") + case "workflow_exists": + return checkListed(check, cachedList(opts, describeCache, "list workflows"), "workflow") case "file_exists": return checkFileExists(check, opts) case "tests_pass": @@ -98,6 +102,51 @@ func checkEntityExists(check Check, entityList string) CheckResult { return CheckResult{Check: check, Passed: false, Detail: "entity not found"} } +// checkEntityHasType verifies that `SHOW ENTITIES` lists the entity under the +// expected Type column: "Persistent", "View", "Non-Persistent" or "External". +// Args format: "Service.OpenRequestsPerFab View". +// +// entity_exists cannot do this. It matches on the qualified name alone, so a +// brief that asks for a view entity passes when the agent builds an ordinary +// persistent entity of the same name — which is the whole of what makes a view +// entity a view entity. The Type column is the only place the distinction is +// visible in a listing, and a name that merely ends in "View" is not evidence. +func checkEntityHasType(check Check, entityList string) CheckResult { + parts := strings.Fields(check.Args) + if len(parts) != 2 { + return CheckResult{Check: check, Passed: false, Detail: `invalid args: expected "EntityPattern Type"`} + } + pattern, want := parts[0], parts[1] + + for _, line := range strings.Split(entityList, "\n") { + matched := "" + for _, name := range extractQualifiedNames(line) { + if matchPattern(name, pattern) { + matched = name + break + } + } + if matched == "" { + continue + } + for _, field := range strings.Fields(line) { + field = strings.Trim(field, "()[]{}:,;|") + // The qualified name itself is never the type, so an entity called + // Mod.View does not answer for a view entity. + if field == matched { + continue + } + if strings.EqualFold(field, want) { + return CheckResult{Check: check, Passed: true, Detail: fmt.Sprintf("found: %s (%s)", matched, field)} + } + } + // Distinguished on purpose from "not found": the entity was built and + // is the wrong kind, which every other check kind reads as a pass. + return CheckResult{Check: check, Passed: false, Detail: fmt.Sprintf("found %s, but not as %s", matched, want)} + } + return CheckResult{Check: check, Passed: false, Detail: "entity not found"} +} + // checkEntityHasAttribute verifies that an entity has an attribute with the expected type. // Args format: "*.Book.Title String" or "*.Book.Title" (type optional) func checkEntityHasAttribute(check Check, opts CheckOptions, entityList string, describeCache map[string]string) CheckResult { diff --git a/cmd/mxcli/evalrunner/checks_types_test.go b/cmd/mxcli/evalrunner/checks_types_test.go new file mode 100644 index 0000000000..9ee5800013 --- /dev/null +++ b/cmd/mxcli/evalrunner/checks_types_test.go @@ -0,0 +1,129 @@ +// SPDX-License-Identifier: Apache-2.0 + +package evalrunner + +import "testing" + +// `SHOW ENTITIES` as the renderer actually emits it — a pipe-delimited markdown +// table (mdl/executor/format.go, writeResultTable). The fixture is in that +// shape on purpose: a space-delimited one passes these tests while telling us +// nothing about the output the check is given in a real run. +// +// The Type column is what distinguishes a view entity from a persistent one, +// and a brief that asks for a view entity is unscored without it — the agent +// can satisfy entity_exists with an ordinary entity of the same name. +const showEntities = `| Entity | Type | Attrs | Assocs | Validations | Indexes | Events | AccessRules | +|----------------------------|----------------|-------|--------|-------------|---------|--------|-------------| +| Service.Fab | Persistent | 3 | 0 | 0 | 0 | 0 | 1 | +| Service.OpenRequestsPerFab | View | 3 | 1 | 0 | 0 | 0 | 1 | +| Service.RequestView | Persistent | 2 | 0 | 0 | 0 | 0 | 0 | +| Service.Scratch | Non-Persistent | 1 | 0 | 0 | 0 | 0 | 0 | + +(4 entities)` + +func TestCheckEntityHasType(t *testing.T) { + cases := []struct { + name string + args string + want bool + why string + }{ + {"a view entity is a View", "Service.OpenRequestsPerFab View", true, ""}, + {"a persistent entity is Persistent", "Service.Fab Persistent", true, ""}, + {"a hyphenated type is one field", "Service.Scratch Non-Persistent", true, ""}, + {"case does not matter", "Service.Fab persistent", true, ""}, + {"wildcards resolve like every other check", "*.OpenRequestsPerFab View", true, ""}, + + // The control the check exists for: without the Type column these two + // would pass, which is precisely the hole in scoring a view entity. + {"a persistent entity is not a View", "Service.Fab View", false, + "a brief asking for a view entity must fail when an ordinary one was built"}, + {"a name ending in View is not a View", "Service.RequestView View", false, + "the Type column decides, not the entity's name"}, + {"a view entity is not Persistent", "Service.OpenRequestsPerFab Persistent", false, ""}, + + {"an absent entity fails", "Service.Nope View", false, ""}, + {"a missing type is rejected", "Service.Fab", false, ""}, + {"a third field is rejected", "Service.Fab View extra", false, ""}, + } + for _, c := range cases { + t.Run(c.name, func(t *testing.T) { + got := runCheck(Check{Type: "entity_has_type", Args: c.args}, + CheckOptions{}, showEntities, "", "", "", map[string]string{}) + if got.Passed != c.want { + t.Errorf("entity_has_type %q = %v (%s), want %v. %s", + c.args, got.Passed, got.Detail, c.want, c.why) + } + }) + } +} + +// A failing entity_has_type must say which of the two things went wrong: the +// entity is missing, or it is there and is the wrong kind. The second is the +// interesting one and reads as a pass in every other check kind. +func TestCheckEntityHasTypeDistinguishesAbsentFromMistyped(t *testing.T) { + absent := runCheck(Check{Type: "entity_has_type", Args: "Service.Nope View"}, + CheckOptions{}, showEntities, "", "", "", map[string]string{}) + mistyped := runCheck(Check{Type: "entity_has_type", Args: "Service.Fab View"}, + CheckOptions{}, showEntities, "", "", "", map[string]string{}) + if absent.Detail == mistyped.Detail { + t.Fatalf("both failures read %q; the mistyped case must be distinguishable", absent.Detail) + } + if !contains(mistyped.Detail, "Service.Fab") { + t.Errorf("the mistyped detail %q should name the entity it did find", mistyped.Detail) + } +} + +// workflow_exists reads `list workflows`. The cache is the seam: a seeded entry +// means the check is graded without running the binary, so this asserts the +// wiring (right listing, right matcher) rather than the backend. +func TestCheckWorkflowExists(t *testing.T) { + cache := map[string]string{ + "list:list workflows": `| Qualified Name | Activities | User Tasks | Decisions | Parameter Entity | +|---------------------------|------------|------------|-----------|------------------------| +| Service.WF_ServiceRequest | 4 | 1 | 1 | Service.ServiceRequest | +| Service.WF_Onboarding | 2 | 1 | 0 | Service.Engineer | + +(2 workflows)`, + } + for _, c := range []struct { + args string + want bool + }{ + {"Service.WF_ServiceRequest", true}, + {"*.WF_Onboarding", true}, + {"Service.WF_Missing", false}, + } { + got := runCheck(Check{Type: "workflow_exists", Args: c.args}, + CheckOptions{}, "", "", "", "", cache) + if got.Passed != c.want { + t.Errorf("workflow_exists %q = %v (%s), want %v", c.args, got.Passed, got.Detail, c.want) + } + } +} + +// A workflow is not a microflow: before workflow_exists existed, a brief whose +// centrepiece was a workflow could only be scored by mx_check_passes, and +// microflow_exists over `SHOW MICROFLOWS` does not see one. +func TestWorkflowIsNotFoundByMicroflowExists(t *testing.T) { + microflows := `| Qualified Name | Parameters | Activities | +|---------------------------------|------------|------------| +| Service.ACT_Request_Assign | 1 | 4 | +| Service.SUB_WorkOrder_TotalCost | 1 | 3 | + +(2 microflows)` + got := runCheck(Check{Type: "microflow_exists", Args: "Service.WF_ServiceRequest"}, + CheckOptions{}, "", "", microflows, "", map[string]string{}) + if got.Passed { + t.Fatal("microflow_exists matched a workflow; the two listings must stay distinct") + } +} + +func contains(s, sub string) bool { + for i := 0; i+len(sub) <= len(s); i++ { + if s[i:i+len(sub)] == sub { + return true + } + } + return false +} diff --git a/cmd/mxcli/exec_preflight_verdict_test.go b/cmd/mxcli/exec_preflight_verdict_test.go index 59208c7fe6..167049d9be 100644 --- a/cmd/mxcli/exec_preflight_verdict_test.go +++ b/cmd/mxcli/exec_preflight_verdict_test.go @@ -62,8 +62,10 @@ end;` t.Fatalf("setup: %v", err) } + // A statement added inside the loop body: a change of the body's shape, + // which the splice refuses (an in-place edit is spliced, ako/mxcli#1106). changed := strings.Replace(strings.Replace(flow, "create microflow", "create or modify microflow", 1), - "commit $It;", "commit $It without events;", 1) + "commit $It;", "commit $It;\n log info node 'X' 'committed';", 1) script := "mdl 1;\ncreate microflow MyFirstModule.Before () begin end;\n" + strings.TrimPrefix(changed, "mdl 1;\n") + "\ncreate microflow MyFirstModule.After () begin end;\n" prog, errs := visitor.Build(script) diff --git a/cmd/mxcli/init.go b/cmd/mxcli/init.go index c26a9ecb26..538b3247a5 100644 --- a/cmd/mxcli/init.go +++ b/cmd/mxcli/init.go @@ -290,6 +290,15 @@ Container Runtime: fmt.Printf(" Created %d skills in %s/\n", skillRes.Total, filepath.ToSlash(rel)) } + // The companion mod (a Claude Code pane), beside the skills. Same + // code path as the SessionStart refresh, and only for Claude projects. + if written, err := syncCompanion(absDir); err != nil { + fmt.Fprintf(os.Stderr, "Error writing the companion mod: %v\n", err) + os.Exit(1) + } else if len(written) > 0 { + fmt.Printf(" Created the mxcli companion in .claude/skills/%s/ (/mxcli-project opens the Project pane)\n", companionFolder) + } + // Write tool-specific configurations for _, toolName := range tools { toolConfig, ok := SupportedTools[toolName] diff --git a/cmd/mxcli/init_claudemd.go b/cmd/mxcli/init_claudemd.go index 137c3fd704..03af98515d 100644 --- a/cmd/mxcli/init_claudemd.go +++ b/cmd/mxcli/init_claudemd.go @@ -182,8 +182,9 @@ func generateClaudeMD(projectName, mprFile string) string { // mean something else. Mixing the two in one file is the failure to // prevent, so the upgrade comes before the edit. w("**Each starts with " + bt + "mdl 1;" + bt + ".** A headerless one is " + bt + "mdl 0" + bt + ": upgrade it first,\n") - w(bt + "./mxcli fmt --upgrade --header -p " + mprPath + " -w " + bt + ", then " + bt + "check" + bt + " it; never mix the two\n") - w("in a file. Exec a change twice: the second run must write nothing (unchanged).\n\n") + w(bt + "./mxcli fmt --upgrade --header -p " + mprPath + " -w " + bt + ", then\n") + w(bt + "check --deprecations=error --v1=error" + bt + " it; never mix them in a file.\n") + w("Exec a change twice: the second run must write nothing (unchanged).\n\n") // ── The gates ─────────────────────────────────────────────────── // Ordered cheapest-first on purpose: each one is only worth paying for @@ -215,7 +216,7 @@ func generateClaudeMD(projectName, mprFile string) string { w("These change every release, and a copy that disagrees with the tool is worse than none.\n\n") w("| To find out | Run |\n") w("|---|---|\n") - w("| What MDL can say, and how | " + bt + "./mxcli syntax" + bt + " → " + bt + "./mxcli syntax [sub]" + bt + " (" + bt + "--json" + bt + " for bulk) |\n") + w("| How to write any MDL — **before any skill** | " + bt + "./mxcli syntax" + bt + " → " + bt + "./mxcli syntax [sub]" + bt + " |\n") w("| Which lint rules exist | " + bt + "./mxcli lint -p " + mprPath + " --list-rules" + bt + " |\n") w("| What a command takes | " + bt + "./mxcli help " + bt + " |\n") w("| What this project contains | " + bt + "./mxcli -p " + mprPath + " -c \"DESCRIBE STRUCTURE\"" + bt + " |\n") @@ -223,8 +224,19 @@ func generateClaudeMD(projectName, mprFile string) string { w("\n") w("**Skills** are in " + bt + ".ai-context/skills//SKILL.md" + bt + " (and " + bt + ".claude/skills/" + bt + ", which is\n") w("the path Claude Code scans). Each one's frontmatter " + bt + "description" + bt + " says when to reach for\n") - w("it — that IS the index, so list the directory rather than looking for a table. Read the\n") - w("matching skill **before** writing microflows, pages, security, or anything touching data.\n\n") + w("it — that IS the index, so list the directory rather than looking for a table.\n\n") + // Measured on the BENCH-001 baseline (2026-10-10): reading the skills was + // ~68% of the session's re-read cost against ~31% of its calls, and one + // `cat write-microflows/SKILL.md` (7.1k tokens, re-read 43 times) cost more + // than every mxcli invocation in the costliest ten put together. The same + // answers cost 0.1-0.6k through `syntax`. Both paths existed; the line here + // said only "read the matching skill", so it read whole files. + // + // The routing itself rides the table row above rather than a paragraph of + // its own: this file is re-read every session, so prose about saving tokens + // that costs 500 bytes is self-defeating. What is left is the one thing the + // table cannot say. + w("**Read a skill's section, not the file** — each opens with a line-numbered index.\n\n") // ── The non-derivable conventions ─────────────────────────────── // Short, and each one is here precisely because no command reports it. diff --git a/cmd/mxcli/init_companion.go b/cmd/mxcli/init_companion.go new file mode 100644 index 0000000000..8e970a434c --- /dev/null +++ b/cmd/mxcli/init_companion.go @@ -0,0 +1,108 @@ +// SPDX-License-Identifier: Apache-2.0 + +package main + +import ( + "bytes" + "fmt" + "io/fs" + "os" + "path/filepath" + "sort" + "strings" +) + +// init_companion.go ships the mxcli companion — a Claude Code mod that shows +// the project tree and MDL source in a pane — into a project +// (docs/11-proposals/proposal-mxcli-companion-mod.md, slice 1). +// +// Claude Code loads a plugin folder under the project's .claude/skills// +// in place, once the workspace is trusted, with no install step. So the mod is +// written there, beside the skills, and refreshed with them on every session +// start. Its source is mods/companion/ in the mxcli repo, embedded through +// cmd/mxcli/mods/ by `make sync-mods`; it is kept out of .claude/skills/mendix/ +// so the skills sync neither ships it to other tools' folders nor deletes it. + +// companionFolder is the mod's folder under .claude/skills/. It must not be the +// name of a skill. +const companionFolder = "mxcli-companion" + +// companionKeep are paths inside the mod folder the sync never removes: the +// engine lays the API types and a tsconfig beside a mod it has loaded. +var companionKeep = []string{".claude-plugin/types/", "tsconfig.json"} + +// syncCompanion writes the embedded mod into .claude/skills/mxcli-companion/ +// when the project is set up for Claude Code (.claude/ exists, the same signal +// skillDests uses), returning the paths it wrote or removed, relative to the +// mod folder. It writes only what differs, so an unchanged project reports +// nothing — this runs on every session start. +func syncCompanion(projectDir string) ([]string, error) { + if st, err := os.Stat(filepath.Join(projectDir, ".claude")); err != nil || !st.IsDir() { + return nil, nil + } + dest := filepath.Join(projectDir, ".claude", "skills", companionFolder) + const root = "mods/companion" + + var changed []string + shipped := map[string]bool{} + err := fs.WalkDir(modsFS, root, func(p string, d fs.DirEntry, err error) error { + if err != nil { + return err + } + if d.IsDir() { + return nil + } + rel := strings.TrimPrefix(p, root+"/") + shipped[rel] = true + want, err := modsFS.ReadFile(p) + if err != nil { + return fmt.Errorf("reading embedded %s: %w", p, err) + } + target := filepath.Join(dest, filepath.FromSlash(rel)) + if have, readErr := os.ReadFile(target); readErr == nil && bytes.Equal(have, want) { + return nil + } + if err := os.MkdirAll(filepath.Dir(target), 0o755); err != nil { + return err + } + if err := os.WriteFile(target, want, 0o644); err != nil { + return fmt.Errorf("writing %s: %w", target, err) + } + changed = append(changed, rel) + return nil + }) + if err != nil { + return changed, err + } + if len(shipped) == 0 { + return nil, fmt.Errorf("no embedded companion mod: cmd/mxcli/mods is empty or stale; " + + "it is generated from mods/ — run `make sync-mods` (or `make build`) first") + } + + // Retire files an older mod shipped and this one does not. + err = filepath.WalkDir(dest, func(p string, d fs.DirEntry, err error) error { + if err != nil || d.IsDir() { + return err + } + rel, err := filepath.Rel(dest, p) + if err != nil { + return err + } + rel = filepath.ToSlash(rel) + if shipped[rel] { + return nil + } + for _, keep := range companionKeep { + if rel == keep || (strings.HasSuffix(keep, "/") && strings.HasPrefix(rel, keep)) { + return nil + } + } + if err := os.Remove(p); err != nil { + return err + } + changed = append(changed, rel) + return nil + }) + sort.Strings(changed) + return changed, err +} diff --git a/cmd/mxcli/init_companion_test.go b/cmd/mxcli/init_companion_test.go new file mode 100644 index 0000000000..ffbebd6484 --- /dev/null +++ b/cmd/mxcli/init_companion_test.go @@ -0,0 +1,99 @@ +// SPDX-License-Identifier: Apache-2.0 + +package main + +import ( + "os" + "path/filepath" + "slices" + "testing" +) + +func companionDir(projectDir string) string { + return filepath.Join(projectDir, ".claude", "skills", companionFolder) +} + +// The companion is a plugin Claude Code loads from .claude/skills// — +// a manifest, the hooks list, and the module. Without all three it is not a mod. +func TestSyncCompanion_WritesTheModForClaudeProjects(t *testing.T) { + dir := t.TempDir() + if err := os.Mkdir(filepath.Join(dir, ".claude"), 0o755); err != nil { + t.Fatal(err) + } + changed, err := syncCompanion(dir) + if err != nil { + t.Fatal(err) + } + for _, f := range []string{".claude-plugin/plugin.json", "hooks/hooks.json", "hooks/register.tsx", "hooks/tree.ts", "types/index.d.ts"} { + if _, err := os.Stat(filepath.Join(companionDir(dir), filepath.FromSlash(f))); err != nil { + t.Errorf("%s not written: %v", f, err) + } + if !slices.Contains(changed, f) { + t.Errorf("%s written but not reported: %v", f, changed) + } + } + // The mod's own tests are development files; a user project has no use for them. + if _, err := os.Stat(filepath.Join(companionDir(dir), "tests")); err == nil { + t.Error("the mod's tests were shipped into the project") + } + // A second run finds nothing to do: this runs on every session start. + again, err := syncCompanion(dir) + if err != nil { + t.Fatal(err) + } + if len(again) != 0 { + t.Errorf("second sync changed %v, want nothing", again) + } +} + +// Presence of .claude/ is the signal, as for the .claude/skills mirror: a +// project set up only for another tool gets no Claude Code plugin. +func TestSyncCompanion_SkipsProjectsWithoutClaude(t *testing.T) { + dir := t.TempDir() + changed, err := syncCompanion(dir) + if err != nil { + t.Fatal(err) + } + if len(changed) != 0 { + t.Errorf("changed %v in a project without .claude/", changed) + } + if _, err := os.Stat(filepath.Join(dir, ".claude")); err == nil { + t.Error(".claude/ created in a project without it") + } +} + +// A refresh replaces a stale copy and retires files a newer mod no longer +// ships — inside the mod's own folder only. The types the engine lays beside a +// loaded mod, and everything else in .claude/skills/, are left alone. +func TestSyncCompanion_RefreshesOnlyItsOwnFolder(t *testing.T) { + dir := t.TempDir() + if err := os.Mkdir(filepath.Join(dir, ".claude"), 0o755); err != nil { + t.Fatal(err) + } + mod := companionDir(dir) + mustWrite(t, filepath.Join(mod, "hooks", "register.tsx"), "// an older mod\n") + mustWrite(t, filepath.Join(mod, "hooks", "retired.ts"), "// gone in this version\n") + mustWrite(t, filepath.Join(mod, ".claude-plugin", "types", "claude-code", "index.d.ts"), "// laid by the engine\n") + userSkill := filepath.Join(dir, ".claude", "skills", "my-skill", "SKILL.md") + mustWrite(t, userSkill, "---\nname: my-skill\n---\n") + + changed, err := syncCompanion(dir) + if err != nil { + t.Fatal(err) + } + if !slices.Contains(changed, "hooks/register.tsx") { + t.Errorf("stale register.tsx not refreshed: %v", changed) + } + if got, _ := os.ReadFile(filepath.Join(mod, "hooks", "register.tsx")); string(got) == "// an older mod\n" { + t.Error("register.tsx still holds the old content") + } + if _, err := os.Stat(filepath.Join(mod, "hooks", "retired.ts")); err == nil { + t.Error("a file the mod no longer ships was kept") + } + if _, err := os.Stat(filepath.Join(mod, ".claude-plugin", "types", "claude-code", "index.d.ts")); err != nil { + t.Errorf("the engine's types were removed: %v", err) + } + if _, err := os.Stat(userSkill); err != nil { + t.Errorf("a user's skill beside the mod was removed: %v", err) + } +} diff --git a/cmd/mxcli/init_skills_references_test.go b/cmd/mxcli/init_skills_references_test.go index 412f516e6a..02436c2fa0 100644 --- a/cmd/mxcli/init_skills_references_test.go +++ b/cmd/mxcli/init_skills_references_test.go @@ -67,9 +67,33 @@ func TestLargeSkillsWereSplit(t *testing.T) { t.Errorf("%s: %v", skill, err) continue } - if n := strings.Count(string(body), "\n"); n > bound { - t.Errorf("%s/SKILL.md is %d lines (> %d) and has no supporting files to move detail into; "+ - "a body this long is loaded whole every time the skill is used", skill, n, bound) + if n := strings.Count(skillBodyWithoutIndex(string(body)), "\n"); n > bound { + t.Errorf("%s/SKILL.md is %d lines of body (> %d) and has no supporting files to move "+ + "detail into; a body this long is loaded whole every time the skill is used", + skill, n, bound) } } } + +// skillBodyWithoutIndex drops the generated "Find it fast" block before the +// body is measured. +// +// This is not a loosening of the bound above. The block is NAVIGATION — it +// exists so a reader can `sed -n` one section instead of loading the body, +// which is the very cost this test is defending. Counting it as body would +// penalise the mechanism for the problem it addresses, and it did: adding the +// index pushed five skills from 694-699 lines of body to 710-726 of file, with +// no content added at all. The bound still applies to every line an author +// writes (scripts/skill-index.py owns the rest). +func skillBodyWithoutIndex(text string) string { + const begin, end = "" + i := strings.Index(text, begin) + if i == -1 { + return text + } + j := strings.Index(text[i:], end) + if j == -1 { + return text + } + return text[:i] + strings.TrimLeft(text[i+j+len(end):], "\n") +} diff --git a/cmd/mxcli/init_tooling_sync.go b/cmd/mxcli/init_tooling_sync.go index 802da90852..9f91019483 100644 --- a/cmd/mxcli/init_tooling_sync.go +++ b/cmd/mxcli/init_tooling_sync.go @@ -205,6 +205,7 @@ func syncBundledLintRules(lintRulesDir string) ([]string, error) { // toolingSyncResult reports a full tooling refresh. type toolingSyncResult struct { Skills skillSyncResult + Companion []string // companion mod files written or retired LintRules []string // bundled rule files rewritten Docs []string // owned docs whose mxcli section was rewritten NoMarkers []string // owned docs left alone: no markers to refresh between @@ -235,6 +236,10 @@ func syncProjectTooling(projectDir string) (toolingSyncResult, error) { } res.Skills = skills + if res.Companion, err = syncCompanion(projectDir); err != nil { + return res, fmt.Errorf("syncing the companion mod: %w", err) + } + if res.LintRules, err = syncBundledLintRules(filepath.Join(projectDir, ".claude", "lint-rules")); err != nil { return res, fmt.Errorf("syncing lint rules: %w", err) } @@ -277,6 +282,10 @@ func reportToolingSync(w, errw io.Writer, res toolingSyncResult) { return } reportSkillSync(w, res.Skills) + if len(res.Companion) > 0 { + fmt.Fprintf(w, "Refreshed %d file(s) of the mxcli companion in .claude/skills/%s/: %s\n", + len(res.Companion), companionFolder, abridge(res.Companion)) + } if len(res.LintRules) > 0 { fmt.Fprintf(w, "Refreshed %d bundled lint rule(s) in .claude/lint-rules/ to match this mxcli: %s\n", len(res.LintRules), abridge(res.LintRules)) diff --git a/cmd/mxcli/project_tree.go b/cmd/mxcli/project_tree.go index b6a6bdf1ce..e6c97501ea 100644 --- a/cmd/mxcli/project_tree.go +++ b/cmd/mxcli/project_tree.go @@ -7,6 +7,7 @@ import ( "fmt" "os" "sort" + "strings" modelsdkbackend "github.com/mendixlabs/mxcli/mdl/backend/modelsdk" "github.com/mendixlabs/mxcli/mdl/executor" @@ -18,10 +19,14 @@ import ( // TreeNode represents a node in the project tree JSON output. type TreeNode struct { - Label string `json:"label"` - Type string `json:"type"` - QualifiedName string `json:"qualifiedName,omitempty"` - Children []*TreeNode `json:"children,omitempty"` + Label string `json:"label"` + Type string `json:"type"` + QualifiedName string `json:"qualifiedName,omitempty"` + // Describable says `mxcli describe ` answers this + // node, so a client can offer its MDL source without keeping its own list + // of which types have one (see treeTypesWithoutSource). + Describable bool `json:"describable,omitempty"` + Children []*TreeNode `json:"children,omitempty"` } // treeElement holds a name, type, and container ID for building the tree hierarchy. @@ -58,6 +63,7 @@ Example: fmt.Fprintf(os.Stderr, "Error: %v\n", err) os.Exit(1) } + markDescribable(tree) enc := json.NewEncoder(os.Stdout) enc.SetIndent("", " ") @@ -68,6 +74,56 @@ Example: }, } +// treeTypesWithoutSource lists the node types project-tree emits that have no +// MDL of their own, with the reason. Every other type it emits must be one +// `describe` answers; TestProjectTree_EveryNodeTypeIsDescribableOrDeclaredWithoutSource +// holds the two together. +var treeTypesWithoutSource = map[string]string{ + "root": "the tree's own root", + "folder": "a container", + "category": "a container (User Roles, Demo Users)", + "security": "a container of module roles", + "module": "a container here: describe module prints the whole module", + "domainmodel": "a container of entities, associations and enumerations", + "settingscategory": "a group inside Settings, which describes as a whole", + "settingsitem": "a line of Settings", + "navhome": "a line of a navigation profile", + "navlogin": "a line of a navigation profile", + "navmenu": "part of a navigation profile", + "navmenuitem": "part of a navigation profile", + "restresource": "part of a published REST service", + "restoperation": "part of a REST service or client", + "odataentityset": "part of an OData service or client", + "odatamember": "part of an OData entity set", + "bechannel": "part of a business event service", + "bemessage": "part of a business event service", + "beattribute": "part of a business event message", + "dbquery": "part of a database connection", + "dbqueryparam": "part of a database connection", + "dbtablemapping": "part of a database connection", + "dbcolumnmapping": "part of a database connection", + "pagetemplate": "no describe form exists for page templates", + "systemoverview": "a diagram only (describe --format elk)", +} + +// treeNodeDescribable reports whether describe answers a tree node of this type. +func treeNodeDescribable(nodeType string) bool { + if _, none := treeTypesWithoutSource[nodeType]; none { + return false + } + stmt, ok := describeMDLCommand(strings.ToUpper(nodeType), "x") + return ok && stmt != "" +} + +// markDescribable sets Describable on every node that has a qualified name and +// a type describe answers. +func markDescribable(nodes []*TreeNode) { + for _, n := range nodes { + n.Describable = n.QualifiedName != "" && treeNodeDescribable(n.Type) + markDescribable(n.Children) + } +} + func buildProjectTree(projectPath string) ([]*TreeNode, error) { // Through the backend, not a concrete reader: every call below is a semantic // read (ListModules, ListMicroflows, ListWorkflows, …) already on diff --git a/cmd/mxcli/project_tree_describable_test.go b/cmd/mxcli/project_tree_describable_test.go new file mode 100644 index 0000000000..fb094412cc --- /dev/null +++ b/cmd/mxcli/project_tree_describable_test.go @@ -0,0 +1,113 @@ +// SPDX-License-Identifier: Apache-2.0 + +package main + +import ( + "os" + "regexp" + "slices" + "sort" + "strings" + "testing" + + "github.com/mendixlabs/mxcli/mdl/visitor" +) + +// treeNodeTypesInSource reads every node type project_tree.go can emit from its +// source: each `Type: "x"` literal, plus `externalentity`, which is assigned +// through a variable. Reading the source rather than a hand-kept list is the +// point — a list is what drifted (the VS Code tree offered javascriptaction and +// pagetemplate clicks that no describe could answer). +func treeNodeTypesInSource(t *testing.T) []string { + t.Helper() + src, err := os.ReadFile("project_tree.go") + if err != nil { + t.Fatal(err) + } + seen := map[string]bool{"externalentity": true} + for _, m := range regexp.MustCompile(`Type:\s*"([a-z]+)"`).FindAllStringSubmatch(string(src), -1) { + seen[m[1]] = true + } + var out []string + for k := range seen { + out = append(out, k) + } + sort.Strings(out) + return out +} + +// TestProjectTree_EveryNodeTypeIsDescribableOrDeclaredWithoutSource is the +// drift guard between `project-tree` and `describe`: a node type the tree emits +// is either one `describe` answers (so a client may offer its MDL) or on the +// explicit no-source list. A new type in the tree fails here until someone +// decides which. +func TestProjectTree_EveryNodeTypeIsDescribableOrDeclaredWithoutSource(t *testing.T) { + for _, typ := range treeNodeTypesInSource(t) { + _, noSource := treeTypesWithoutSource[typ] + describable := treeNodeDescribable(typ) + switch { + case describable && noSource: + t.Errorf("%q is describable but also listed in treeTypesWithoutSource; remove it from the list", typ) + case !describable && !noSource: + t.Errorf("project-tree emits %q, which describe cannot answer and which is not in treeTypesWithoutSource; "+ + "add a describe keyword or list it with the reason", typ) + } + } + for typ := range treeTypesWithoutSource { + if !slices.Contains(treeNodeTypesInSource(t), typ) { + t.Errorf("treeTypesWithoutSource lists %q, which project-tree never emits", typ) + } + } +} + +// TestProjectTree_DescribableTypesParse pins that each type the tree marks +// describable produces an MDL statement that parses — a keyword mapped to a +// statement the grammar rejects would mark a node the client then fails on. +func TestProjectTree_DescribableTypesParse(t *testing.T) { + for _, typ := range treeNodeTypesInSource(t) { + if !treeNodeDescribable(typ) { + continue + } + stmt, _ := describeMDLCommand(strings.ToUpper(typ), "MyModule.Thing") + if _, errs := visitor.Build(stmt); len(errs) > 0 { + t.Errorf("%q maps to %q, which does not parse: %v", typ, stmt, errs) + } + } +} + +func TestProjectTree_MarkDescribable(t *testing.T) { + tree := []*TreeNode{{ + Label: "Mod", Type: "module", QualifiedName: "Mod", + Children: []*TreeNode{ + {Label: "Folder", Type: "folder"}, + {Label: "ACT_X", Type: "microflow", QualifiedName: "Mod.ACT_X"}, + {Label: "JS_X", Type: "javascriptaction", QualifiedName: "Mod.JS_X"}, + {Label: "Tmpl", Type: "pagetemplate", QualifiedName: "Mod.Tmpl"}, + }, + }, {Label: "Navigation", Type: "navigation", QualifiedName: "Navigation"}} + markDescribable(tree) + + want := map[string]bool{"Mod": false, "Folder": false, "ACT_X": true, "JS_X": true, "Tmpl": false, "Navigation": true} + var walk func([]*TreeNode) + walk = func(nodes []*TreeNode) { + for _, n := range nodes { + if n.Describable != want[n.Label] { + t.Errorf("%s (%s): describable = %v, want %v", n.Label, n.Type, n.Describable, want[n.Label]) + } + walk(n.Children) + } + } + walk(tree) +} + +// TestDescribeMDLCommand_NavigationRoot pins that the tree's Navigation node +// (qualified name "Navigation", not a profile) describes every profile rather +// than looking for a profile called "Navigation". +func TestDescribeMDLCommand_NavigationRoot(t *testing.T) { + if got, _ := describeMDLCommand("NAVIGATION", "Navigation"); got != "DESCRIBE NAVIGATION" { + t.Errorf("navigation root = %q, want DESCRIBE NAVIGATION", got) + } + if got, _ := describeMDLCommand("NAVIGATION", "Responsive"); got != "DESCRIBE NAVIGATION Responsive" { + t.Errorf("navigation profile = %q, want DESCRIBE NAVIGATION Responsive", got) + } +} diff --git a/cmd/mxcli/skills_content.go b/cmd/mxcli/skills_content.go index f1ac90347d..f3c25d8f56 100644 --- a/cmd/mxcli/skills_content.go +++ b/cmd/mxcli/skills_content.go @@ -38,6 +38,14 @@ var skillsFS embed.FS //go:embed all:skillpacks var skillPacksFS embed.FS +// Embed the Claude Code mods from the synced directory (mods/ in the repo): +// plugin folders written into a project's .claude/skills/ by init_companion.go. +// `all:` because a plugin's manifest lives in `.claude-plugin/`, which a plain +// go:embed of a directory skips. +// +//go:embed all:mods +var modsFS embed.FS + // Embed all command files from the synced directory // //go:embed commands/*.md diff --git a/cmd/mxcli/syntax/features_page.go b/cmd/mxcli/syntax/features_page.go index 5e4fdfac14..975effdc1c 100644 --- a/cmd/mxcli/syntax/features_page.go +++ b/cmd/mxcli/syntax/features_page.go @@ -154,7 +154,12 @@ LIST IMPACT OF htmlelement; "DATEPICKER name (Attribute: Start, DateFormat: DateTime)\n" + "DATEPICKER name (Attribute: Start, DateFormat: Custom, CustomDateFormat: 'dd-MM-yyyy HH:mm')\n" + "-- A text box's numeric formatting:\n" + - "TEXTBOX name (Attribute: Amount, DecimalPrecision: 2, GroupDigits: true)\n\n" + + "TEXTBOX name (Attribute: Amount, DecimalPrecision: 2, GroupDigits: true)\n" + + "-- Submit while typing (TEXTBOX, TEXTAREA): OnEndEditing (default) | WhileEditing,\n" + + "-- with the delay in milliseconds (default 300):\n" + + "TEXTBOX name (Attribute: Query, SubmitBehaviour: WhileEditing, SubmitOnInputDelay: 500)\n" + + "-- A layout grid's width: FullWidth (default) | FixedWidth:\n" + + "LAYOUTGRID name (Width: FixedWidth) { ROW { COLUMN (DesktopWidth: 12) { ... } } }\n\n" + "-- Conditional visibility / editability: a bare client expression, stored as\n" + "-- written (name attributes as $currentObject/Attr). A plain value is static.\n" + "TEXTBOX name (Attribute: Attr, Visible: $currentObject/IsActive, Editable: $currentObject/Status != 'Closed')\n" + diff --git a/cmd/mxcli/v1_flag.go b/cmd/mxcli/v1_flag.go new file mode 100644 index 0000000000..56ff83f7eb --- /dev/null +++ b/cmd/mxcli/v1_flag.go @@ -0,0 +1,46 @@ +// SPDX-License-Identifier: Apache-2.0 + +package main + +import ( + "fmt" + "os" + "strings" + + "github.com/mendixlabs/mxcli/mdl/deprecation" + "github.com/spf13/cobra" +) + +// The --v1 flag decides what `check` does with an MDL-V1-* warning: a construct +// a headerless (mdl 0) script keeps at its old meaning, or that `mdl 1;` would +// refuse. It mirrors --deprecations, which gates a different set of codes +// (MDL-DEPRnnn): a script can pass --deprecations=error and still not parse +// under mdl 1. Both at error is the full mdl 1 readiness gate. +const v1FlagUsage = "What to do with an MDL-V1-* warning (a headerless script not ready for `mdl 1;`): " + + "warn, or error to fail the run (with --deprecations=error, the full mdl 1 readiness gate)" + +func init() { + checkCmd.Flags().String("v1", "warn", v1FlagUsage) +} + +// parseV1Policy reads a --v1 value. Same values as --deprecations. +func parseV1Policy(raw string) (deprecation.Policy, error) { + policy, err := deprecation.ParsePolicy(raw) + if err != nil { + return deprecation.Warn, fmt.Errorf("invalid --v1 value %q: want warn or error", strings.TrimSpace(raw)) + } + return policy, nil +} + +// v1Policy reads --v1, exiting with a usage error on a value it does not know: +// silently treating a typo as `warn` would let a CI gate pass that was meant +// to fail. +func v1Policy(cmd *cobra.Command) deprecation.Policy { + raw, _ := cmd.Flags().GetString("v1") + policy, err := parseV1Policy(raw) + if err != nil { + fmt.Fprintf(os.Stderr, "Error: %v\n", err) + os.Exit(2) + } + return policy +} diff --git a/docs-site/src/SUMMARY.md b/docs-site/src/SUMMARY.md index dd798c7ea9..4b7a26f27f 100644 --- a/docs-site/src/SUMMARY.md +++ b/docs-site/src/SUMMARY.md @@ -142,6 +142,7 @@ - [Writing Custom Rules](tools/custom-rules.md) - [mxcli lint](tools/mxcli-lint.md) - [mxcli report](tools/mxcli-report.md) +- [Claude Code Companion](tools/companion.md) - [Terminal UI (TUI)](tools/tui.md) - [Navigation and Layout](tools/tui-navigation.md) - [Command Palette](tools/tui-commands.md) diff --git a/docs-site/src/appendixes/error-messages.md b/docs-site/src/appendixes/error-messages.md index 989307cd7f..4df80423a7 100644 --- a/docs-site/src/appendixes/error-messages.md +++ b/docs-site/src/appendixes/error-messages.md @@ -238,6 +238,8 @@ line 1: `create or replace …` (enumeration) is deprecated; write `create or mo `check` and `exec` report these as **warnings**. To fail the run on one, for example in CI over documentation and examples, pass `--deprecations=error`. +`--deprecations=error` does not gate the `MDL-V1-*` warnings — what a headerless (`mdl 0`) script still has to change before it can carry `mdl 1;` — so a script can pass it and still not parse under `mdl 1`. `mxcli check --v1=error` fails the run on those; `--deprecations=error --v1=error` together is the full `mdl 1` readiness gate, and `mxcli fmt --upgrade --header` is the fix. + `mxcli help ` prints the entry for any `MDL-DEPRnnn` or `MDL-V1-*` code, `mxcli syntax --deprecated` lists a topic's old spellings, and [Language Versions and Migration](../language/versions.md) tabulates them all. The registry of deprecated spellings is `mdl/deprecation/deprecation.go`. It holds the code, old form, canonical form, rewrite and removal version of each entry. A spelling is only registered where it means exactly the same as its canonical form. `create or replace view entity`, for example, drops and recreates the view entity, so it is not reported. Likewise `show` is reported as `MDL-DEPR002` only where its canonical form is `list` (plurals, relationship queries and the summary tables), and as `MDL-DEPR090` where it is `describe` with the same output. `show entity X` and `show version` are neither: the first has no mdl 1 statement (`MDL-V1-SHOWSUMMARY`), the second is a session command (`MDL-V1-SESSION`). diff --git a/docs-site/src/language/basics.md b/docs-site/src/language/basics.md index efc416aedd..e9bf42fe2b 100644 --- a/docs-site/src/language/basics.md +++ b/docs-site/src/language/basics.md @@ -122,7 +122,7 @@ What `mdl 1` makes strict: - A session command — `connect`, `disconnect`, `use`, `set format = …`, `status`, `show version`, `show status`, `show connections`, `show catalog status`, `check`, `build`, `lint`, `debug`, `execute script`, `execute runtime`, `help`, `introspect api` — is an error in a script. Type it at the REPL, or use the command-line flag (`mxcli exec script.mdl -p app.mpr --json`). - `DynamicClasses`, `DynamicCellClass` and an OData client's credential or header value take the expression bare; a quoted value is a Mendix string. -The manual, the skills, `mxcli syntax` and the example scripts are written in the canonical form, and CI holds them to it: `make check-conformance` parses every MDL block in them as `mdl 1` and fails on a deprecated spelling (`mxcli check --deprecations=error` does the same for a script of your own). The design is in [ADR-0011](https://github.com/mendixlabs/mxcli/blob/main/docs/13-decisions/0011-mdl-language-versioning.md); `mxcli syntax language-header` has the details. +The manual, the skills, `mxcli syntax` and the example scripts are written in the canonical form, and CI holds them to it: `make check-conformance` parses every MDL block in them as `mdl 1` and fails on a deprecated spelling (`mxcli check --deprecations=error` does the same for a script of your own; add `--v1=error` to also fail on what a headerless script must change before it can carry `mdl 1;`). The design is in [ADR-0011](https://github.com/mendixlabs/mxcli/blob/main/docs/13-decisions/0011-mdl-language-versioning.md); `mxcli syntax language-header` has the details. ## Re-runnable Creates: `or modify` and `if not exists` diff --git a/docs-site/src/language/versions.md b/docs-site/src/language/versions.md index dee5837034..c523cd1d2e 100644 --- a/docs-site/src/language/versions.md +++ b/docs-site/src/language/versions.md @@ -114,6 +114,11 @@ Without the header a template literal that spans lines and has no parameters is by the same code exec uses, so a refusal shows up here and not halfway through a run. + To gate a script that is meant to be ready for the header — in CI, say — + pass `--deprecations=error --v1=error`: the first fails on a deprecated + spelling (`MDL-DEPRnnn`), the second on an `MDL-V1-*` warning, which only a + headerless script reports. Either alone lets the other through. + 5. **Run it twice.** The first `exec` applies the script; the second must write nothing, and report every document unchanged. A second run that writes is a script that does not say what the project holds — the upgrade's job is to @@ -191,7 +196,7 @@ A construct with no mechanical rewrite is reported with the reason, and `fmt` re `-p app.mpr` also settles a bare `commit $X;` in a `create or modify microflow|nanoflow`, with or without the header. Since mendixlabs/mxcli#895 a bare commit means *with* events, Studio Pro's default; an older mxcli stored the same statement *without* events. Where the stored flow commits the variable without events, `fmt --upgrade -p app.mpr` writes `commit $X without events;`, so re-running the script keeps what is stored instead of turning the event handlers on (or, inside a loop under `mdl 1`, being refused). A new flow, a new commit, or a stored commit with events is left as written; where the stored flow commits the variable both ways, the statement is left and reported. Without `-p`, `fmt` prints a note (`MDL067`) for each flow with a bare commit. -`-p app.mpr` also tells the upgrade which statements `exec` would refuse once the header is there. Under `mdl 1` a `create or modify microflow|nanoflow` of a stored flow is applied as a patch, and a change the patch cannot make (inside a loop body or an error handler, a redrawn connector, a `return` added) is refused with nothing written; without the header the same statement rebuilds the whole flow (`MDL-V1-REBUILD`). No rewrite keeps that meaning, so `fmt` names each such statement and leaves the header off that file, applying the rest of the upgrade. Change those flows with `alter`, or drop and create them, and upgrade again; `--force-header` adds the header regardless. `mxcli check script.mdl -p app.mpr` reports the same statements (an `MDL-V1-REBUILD` error under `mdl 1`, the warning without the header), computed by the same code as `exec` and `diff`. +`-p app.mpr` also tells the upgrade which statements `exec` would refuse once the header is there. Under `mdl 1` a `create or modify microflow|nanoflow` of a stored flow is applied as a patch, and a change the patch cannot make (inside a loop body anything but an activity replaced by one of the same kind, an error handler, a redrawn connector, a `return` added) is refused with nothing written; without the header the same statement rebuilds the whole flow (`MDL-V1-REBUILD`). No rewrite keeps that meaning, so `fmt` names each such statement and leaves the header off that file, applying the rest of the upgrade. Change those flows with `alter`, or drop and create them, and upgrade again; `--force-header` adds the header regardless. `mxcli check script.mdl -p app.mpr` reports the same statements (an `MDL-V1-REBUILD` error under `mdl 1`, the warning without the header), computed by the same code as `exec` and `diff`. ## Behaviour changes to know about diff --git a/docs-site/src/reference/page/create-page.md b/docs-site/src/reference/page/create-page.md index b4a30d5b74..9c7bfd0832 100644 --- a/docs-site/src/reference/page/create-page.md +++ b/docs-site/src/reference/page/create-page.md @@ -46,7 +46,7 @@ The widget tree inside `{ ... }` defines the page content. Widgets are nested hi | Widget | Description | Key Properties | |--------|-------------|----------------| -| `LAYOUTGRID` | Responsive grid container | `Class`, `Style` | +| `LAYOUTGRID` | Responsive grid container | `Class`, `Style`, `Width` (`FullWidth` default, `FixedWidth`) | | `ROW` | Grid row | `Class` | | `COLUMN` | Grid column | `Class` | | `CONTAINER` | Generic div container | `Class`, `Style`, `DesignProperties` | @@ -65,8 +65,8 @@ The widget tree inside `{ ... }` defines the page content. Widgets are nested hi | Widget | Description | Key Properties | |--------|-------------|----------------| -| `TEXTBOX` | Single-line text input | `Label`, `Attribute` | -| `TEXTAREA` | Multi-line text input | `Label`, `Attribute` | +| `TEXTBOX` | Single-line text input | `Label`, `Attribute`, `Password`, `SubmitBehaviour` (`OnEndEditing` default, `WhileEditing`), `SubmitOnInputDelay` (ms, default 300) | +| `TEXTAREA` | Multi-line text input | `Label`, `Attribute`, `SubmitBehaviour`, `SubmitOnInputDelay` | | `CHECKBOX` | Boolean toggle | `Label`, `Attribute` | | `RADIOBUTTONS` | Radio button group | `Label`, `Attribute` | | `DATEPICKER` | Date/time selector | `Label`, `Attribute` | diff --git a/docs-site/src/tools/bootstrap-prompt.md b/docs-site/src/tools/bootstrap-prompt.md index f3ba257f48..e691a8ff4b 100644 --- a/docs-site/src/tools/bootstrap-prompt.md +++ b/docs-site/src/tools/bootstrap-prompt.md @@ -80,7 +80,11 @@ a longer prompt. "what the template ships with"; afterwards every figure is yours plus the template's, with nothing to subtract. See [mxcli report](mxcli-report.md) for how to read the six category scores. -7. **Proposes the model in MDL and waits** — module, entities, roles, pages — before +7. **Hands over and asks what next** — points you at the project explorer (run + `/reload-plugins`, then `/mxcli-project`; see [Claude Code Companion](companion.md)) + and offers a choice: propose the domain model, import requirements from a + document first, open the explorer, or something else. +8. **Proposes the model in MDL and waits** — module, entities, roles, pages — before building anything. Every script it writes, from the first, starts with `mdl 1;`, so the project never holds a headerless (`mdl 0`) script to upgrade. Two of those choices it makes deliberately rather than by default: a process with steps someone has to act on becomes a **workflow** (state diff --git a/docs-site/src/tools/companion.md b/docs-site/src/tools/companion.md new file mode 100644 index 0000000000..8e49bf7681 --- /dev/null +++ b/docs-site/src/tools/companion.md @@ -0,0 +1,65 @@ +# Claude Code Companion + +The mxcli companion is a [Claude Code](https://code.claude.com) mod: a pane inside +your Claude Code session that shows the Mendix app you are building while the +agent builds it. + +> **Early access.** Mods are an early-access Claude Code feature, and the companion +> is at its first slice. Its plan is in +> `docs/11-proposals/proposal-mxcli-companion-mod.md`. + +## What it shows + +- **Project** — the app's structure: modules, their domain model, folders and + documents, project security, settings and navigation, as `mxcli project-tree` + reports it. +- **MDL source** — select a document to see its MDL, as `mxcli describe` prints + it. Folders and other containers list what they hold instead. + +In a wide pane (90 columns or more) the tree sits on the left and the source on +the right, like a file explorer; each column pages on its own with its ▲ / ▼ +rows. A narrower pane stacks them. + +The tree reloads by itself when the agent changes the model (`mxcli exec`, a +writing `-c` one-liner, `new`, `rename`, `marketplace`). Press **Refresh** after +changing the app elsewhere, for example in Studio Pro. + +## Getting it + +`mxcli init` writes the companion to `.claude/skills/mxcli-companion/` in a project +set up for Claude Code, and `mxcli init --sync-skills` (run by the session hook +on every start) keeps it current with your mxcli. Claude Code loads it from there +once you trust the folder; there is nothing to install. + +Start Claude Code from the project folder, then type: + +``` +/mxcli-project +``` + +The first time the companion loads in a project, a band above the prompt says it +is installed and offers **Open Project pane**. It does not come back once you +open the pane or press **Not now**. + +A companion written during the current session — as the bootstrap prompt does — +loads after `/reload-plugins` or in the next session; Claude Code reads plugins +at session start. + +## Options + +Both are set in Claude Code's plugin configuration for `mxcli-companion`: + +| Option | Default | +|---|---| +| `mprPath` | the single `.mpr` in the session folder, or one level down; with several, the pane asks | +| `mxcliPath` | `./mxcli` when present, else `mxcli` on `PATH` | + +## Notes + +- Every click runs mxcli, which reads the model. On a large app a document takes a + few seconds the first time; going back to it is instant until the model changes. +- The companion only reads. It never changes the project. +- The pane draws on the surface that runs the session. A session started in a + terminal and continued from the desktop app over remote control shows the pane + in the terminal only. +- It is for Claude Code only; other tools get what `mxcli init` gives them today. diff --git a/docs-site/src/tools/outside-studio-pro.md b/docs-site/src/tools/outside-studio-pro.md index 2f6bb34cf2..35cac97dfd 100644 --- a/docs-site/src/tools/outside-studio-pro.md +++ b/docs-site/src/tools/outside-studio-pro.md @@ -1,8 +1,8 @@ # Working Outside Studio Pro A project edited with mxcli, git and an editor is sometimes opened in Studio Pro -afterwards. Two kinds of state that Studio Pro depends on are invisible to -`mx check`, and mxcli checks both. +afterwards. Several kinds of state that Studio Pro depends on are invisible to +`mx check`, and mxcli checks them. ## The ContentsHash index: `mxcli fix hashes` @@ -35,6 +35,44 @@ and the command says so. warning when it finds a mismatch or a missing file. It never fails them; measured on a 900-unit project, the whole verify takes about 0.2 s. +## Property names Studio Pro's merge compares: `mxcli fix property-sets` + +Mendix's diff and merge engine compares two revisions of an element by its +property **names** before any value, and throws when they differ: + +``` +Objects with ID … of type Microflows$RestOperationCallAction do not have the same properties. +``` + +The model is valid — `mx check`, the build and the runtime fill an absent +property with its default — but `mx diff`, merging and Studio Pro's Version +Control panel fail on it once Studio Pro has saved the document with its full +property set. Every mxcli write gives a unit the property set Studio Pro writes +for the project's Mendix version. A unit written by an older mxcli and not +rewritten since keeps its gaps; this command reaches those. + +```bash +mxcli fix property-sets -p app.mpr --dry-run # list the units that would change; exits 1 when one would +mxcli fix property-sets -p app.mpr # write them +``` + +Each unit's report counts the keys added (with the value Mendix gives an absent +one) and the empty keys stripped (properties the project's version does not +declare yet). A project user role with no `GUID` — older mxcli wrote them +without one — gets its own `$ID` as its GUID, the value a role mxcli creates +now gets; every mxcli write of the security settings does the same. Without +it the build invents a random GUID each time, so the runtime can only match +the role's database row by name. A role that already has a GUID keeps it. +Nothing else changes: no element `$ID` moves, and no existing `GUID` is +touched. A second run writes nothing. + +The filled GUID protects a later **rename** of the role only in a database +first synchronised after the fill. A database synchronised before it keeps +the old random GUID and still finds the role by name, so renaming the role +there still deletes its row and the user assignments that are not demo users. A unit holding a *value* in a +key its version does not declare is reported and left alone, and, like every +mxcli write, the command is refused while Studio Pro has the project open. + ## Git states that crash Studio Pro Studio Pro 11.13 fails to open a project with *"Unable to find 'system' diff --git a/docs/01-project/MDL_QUICK_REFERENCE.md b/docs/01-project/MDL_QUICK_REFERENCE.md index ab651bf5da..aa893d1acb 100644 --- a/docs/01-project/MDL_QUICK_REFERENCE.md +++ b/docs/01-project/MDL_QUICK_REFERENCE.md @@ -1487,6 +1487,8 @@ MDL uses explicit property declarations for pages: | Attribute binding | `attribute: AttrName` | `textbox txt (label: 'Name', attribute: Name)` | | Attribute over an association | `attribute: Assoc/Attr` (bare association name, multi-hop OK) | `textbox txt (label: 'Rule', attribute: RuleAction_BusinessRule/Name)` — works on textbox, textarea, datepicker, dropdown, checkbox and radiobuttons, the same as on a data grid column | | Password field | `Password: true` on a textbox | `textbox tbPw (attribute: Secret, Password: true)` — omitted when false. Without it a describe → exec round trip turns a password field into a plaintext one | +| Submit while typing | `SubmitBehaviour: OnEndEditing \| WhileEditing`, `SubmitOnInputDelay: ` on a textbox or textarea | `textbox tbSearch (attribute: Query, SubmitBehaviour: WhileEditing, SubmitOnInputDelay: 500)` — defaults `OnEndEditing` / `300`, omitted by describe when default. An invalid value is MDL-WIDGET18; on any other widget the keys draw MDL-WIDGET07 | +| Layout grid width | `Width: FullWidth \| FixedWidth` on a layoutgrid | `layoutgrid lg (Width: FixedWidth) { … }` — default `FullWidth`, omitted by describe when default | | Widget validation | `Validation: ''`, `ValidationMessage: ''` | `Validation: 'length(toString($value)) > 0'` — a Mendix expression over `$value`, QUOTED not bracketed (`[...]` is the XPath spelling and parses as an array) | | Variable binding | `datasource: $Var` | `dataview dv (datasource: $Product) { ... }` | | Action binding | `action: type` | `actionbutton btn (caption: 'Save', action: save changes)` — the forms are a closed set (`mxcli syntax page.action`); anything else is **MDL-WIDGET28** | diff --git a/docs/03-development/PAGE_BSON_SERIALIZATION.md b/docs/03-development/PAGE_BSON_SERIALIZATION.md index ed05775962..e6e0ef94e7 100644 --- a/docs/03-development/PAGE_BSON_SERIALIZATION.md +++ b/docs/03-development/PAGE_BSON_SERIALIZATION.md @@ -73,6 +73,9 @@ Each widget type requires specific default properties to be serialized. Studio P } ``` +`Width` is a `ContainerWidth`: `FullWidth` (default) or `FixedWidth`, written from +MDL `Width:` on the `layoutgrid`. + ### LayoutGridRow ```json @@ -241,6 +244,10 @@ Input widgets require several non-null properties for proper serialization: - `PlaceholderTemplate` - Required `Forms$ClientTemplate` object - `validation` - Required `Forms$WidgetValidation` object +`SubmitBehaviour` (`OnEndEditing` default, `WhileEditing`) and `SubmitOnInputDelay` +(ms, default 300) are on TextBox and TextArea, written from the MDL properties of +the same names. + ### Attribute Path Resolution The `AttributeRef.Attribute` field requires a **fully qualified path** in the format `Module.Entity.AttributeName`. diff --git a/docs/11-proposals/PROPOSAL_agent_loop_efficiency.md b/docs/11-proposals/PROPOSAL_agent_loop_efficiency.md index 2c1bd5bd26..09352d5c36 100644 --- a/docs/11-proposals/PROPOSAL_agent_loop_efficiency.md +++ b/docs/11-proposals/PROPOSAL_agent_loop_efficiency.md @@ -16,7 +16,16 @@ related: **Status:** Draft **Date:** 2026-09-22 (initial), revised 2026-09-23 — three complete builds measured with `diag loop-report`, overturning the `check`-dominant assumption and -promoting the app restart to the wall-time lever (§"Three projects, measured"). +promoting the app restart to the wall-time lever (§"Three projects, measured"); +revised 2026-10-10 — the BENCH-001 baseline recorded, putting skill and doc +reads at ~68% of the re-read cost against mxcli's ~15%, which promotes a new +item 3b, demotes item 4 and closes `mxcli apply` (§"The baseline, measured"). +Revised again 2026-10-10 — a second BENCH-001 run shows the first pass at item 3b +was adopted (whole-file skill reads became `sed` ranges) but did not reduce the +bill, with one adverse shape it appears to have induced; the comparison is n = 1 +per side and is recorded as unsettled (§"Run 2, measured"); a third run +makes the post-change side n = 2 and tight, and shows the skills' share of the +bill halved with `syntax` sweeps taking the freed share (§"Run 3"). ## The report @@ -126,6 +135,312 @@ the 7 boots that closed; the 27 it excludes add ~41 min at the measured median, so the real figure is around 70 min — more than double, and enough to change which levers are worth pulling. +## The baseline, measured (BENCH-001, 2026-10-10) + +The first `eval run` of the fixed brief, on Mendix 11.15.0, scoring 20 of 21 — +the one failure environmental (`postgresql: unrecognized service`, so the +`tests --local` tier never ran). + +| | | +|---|---:| +| model calls | 47 | +| tool calls | 48 (Bash only) | +| wall | 7m00s | +| avg context | 103 k | +| cache read | 4.7 M | +| output | 27 k | +| tool results | 48 k, re-read cost **1.7 M (37% of cache read)** | +| mxcli invocations | 70 across 38 Bash calls, **17 chained >1**: `exec` 20, `check` 15, `syntax` 6, `-c describe` 5, `lint` 4, `test` 4 | +| categories | orientation 31%, validate 23%, apply 17%, verify 13%, retry 15%, write 2% | + +`47 × 103 k ≈ 4.8 M` — the run behaves exactly as `N × S` predicts, and output +is a rounding error again. + +**It is NOT comparable to the 523-call report.** Different brief, far smaller +scope, and the levers in this document had already shipped. Nothing here says +they worked; the pre-intervention "before" is not recoverable. This is a +reference point for *future* changes, which is all item 1 ever claimed. + +**It is also n = 1.** Every share below is one sample of a stochastic process +with unmeasured variance. Treat the ordering as the finding and the percentages +as approximate. + +### The dominant cost is reading the skills, not running mxcli + +The top ten results carry ~1.34 M of the 1.7 M re-read cost, and they split: + +| what | re-read cost | share of 1.7 M | +|---|---:|---:| +| skill and doc reads (three `cat …/SKILL.md`, a `wc -l` over three more, a `sed` range, a `grep` over a skill) | ~1.16 M | **68%** | +| mxcli's own output (`DESCRIBE STRUCTURE`, `syntax page datasource`, `syntax entity`) | ~252 k | 15% | +| `ls -la` | 35 k | 2% | + +`cat write-microflows/SKILL.md` alone is 7.1 k tokens re-read 43 times — 307 k, +more than every mxcli invocation in the top ten put together. With +`DESCRIBE STRUCTURE` and `ls -la`, **orientation is ~77% of the re-read bill +against 31% of the calls.** + +The cheap path exists and was used: `mxcli syntax page datasource` cost 2.6 k +and `syntax entity` 1.1 k, against 7.1 k for the skill that covers the same +ground. The agent also tried to read a skill in part (`sed -n 314,420p`), so the +intent is there and the structure is not. + +**This is a lever this document did not have**, and it dominates the two it did: +make consuming the skills cheap. Cheapest first — point the skills at `mxcli +syntax` instead of restating syntax in prose; give them sections an agent can +read one of; and only then spend on item 4, which addresses the 15%. + +### First pass at the fix, and what still has to be measured + +Two changes, deliberately only two, so a second benchmark run stays +interpretable: + +- **Routing.** The generated CLAUDE.md said "read the matching skill before + writing microflows, pages, security…", which is the instruction that produced + the `cat`. Its lookup table now reads *How to write any MDL — **before any + skill*** against `./mxcli syntax`, and one line says to read a skill's section + rather than the file. It had to be folded into the existing table row rather + than added as a paragraph: that file is re-read every session and + `TestGeneratedClaudeMDStaysWithinItsContextBudget` had 75 bytes of headroom, + so prose about saving tokens would have cost more than it saved. +- **Storage.** Every `SKILL.md` now opens with a generated line-numbered + section index (`scripts/skill-index.py`, `make check-skill-index` in CI), so + `sed -n ',p'` is reachable without first paying for the file or a + `grep`. Generated, never hand-written — a hand-kept table of contents is the + drift this repo has paid for twice — and the ranges are computed to a fixed + point, because the block's own height shifts every range below it. All 745 + rows were verified to land on their heading. + +Both have a cost, and it is honest to state it: the index is ~0.17k tokens on +every skill read, so if the agent keeps reading whole files the change is a +small *loss*. That is the hypothesis a second run tests, and the measurement to +watch is not the total but whether `cat …/SKILL.md` leaves the costliest-results +list. + +Measured consequence worth recording: **the naive version of this lever is +smaller than it looked.** Code blocks are 28% of all skill bytes (32-43% in the +two most-read), so "move the syntax out of the skills into `syntax`" could never +have reclaimed most of the 68% — the skills are mostly prose, which is the part +CLAUDE.md says belongs there. The reachable win is in *how much of a file gets +read*, not in what the file contains. + +### Run 2, measured — the mechanism was adopted and the bill went up + +`BENCH-001-2026-10-10T18-47-02`, session `03e4df6c`, same brief, same 20/21 +(same environmental `postgresql` failure), run against a binary carrying both +changes above. + +| | run 1 | run 2 | +|---|---:|---:| +| model calls | 47 | **61** | +| wall | 7m00 | 7m25 | +| avg context | 103 k | 104 k | +| cache read | 4.7 M | **6.2 M** | +| re-read cost | 1.7 M (37%) | **2.1 M (34%)** | +| mxcli invocations | 70 / 17 chained | 85 / 22 chained | +| `syntax` calls | 6 | 8 | +| doc lookups | 7 | 13 | +| orientation share of calls | 31% | **43%** | + +**The mechanism was adopted.** Three of the four whole-file skill reads became +ranges — `sed -n 47,264p create-page` 3.7 k, `sed -n 112,265p write-microflows` +2.4 k, `sed -n 1,200p test-microflows` 2.1 k, against run 1's 7.1 k and 6.7 k +`cat`s — and `syntax` use rose. The index is being read and used for what it was +built for. + +**The totals got worse anyway**, and one `cat +.ai-context/…/overview-pages/SKILL.md` (6.1 k × 55 = 338 k) is now the single +costliest result in the run. So the hypothesis as written — "watch whether `cat +…/SKILL.md` leaves the costliest-results list" — came back **mixed**, which is +not a pass. + +**One adverse mechanism is visible and plausibly induced by the routing +change.** Run 2 contains three `for t in "…" "…" …; do ./mxcli syntax $t; done` +loops, at slots 3, 4 and 6 of the costliest results (3.4 k + 2.6 k + 1.9 k, +≈ 686 k combined). Run 1 has no such shape. Telling the agent to reach for +`syntax` before a skill appears to have produced *speculative sweeps across +topics*: per-topic `syntax` is still cheap, but a batched sweep is a single +result larger than the whole-file read it replaced, and it is paid on every +subsequent call. The cheap path was made more attractive without being made +narrower. + +**n = 1 per side, so none of the deltas above are attributable.** Two independent +reasons to distrust the comparison, beyond the sample size: + +- The two runs read **different skills** (run 1 write-microflows and + test-microflows; run 2 overview-pages and create-page). Skill *selection* is + itself stochastic, so the per-read costs are not measuring the same text. +- The 47 → 61 call delta is larger than anything the change plausibly touches + directly, which is what a variance-dominated pair looks like. + +So the honest position is: **the first pass at item 3b is not validated, and not +refuted.** What settles it is an A/B with repetition — two more runs of the +current binary and two of `9ce90f32` (the pre-change build), reported as four +points, not two. Until that exists, neither defending the change nor reverting +it has evidence behind it, and the pre-registered revert condition ("if the agent +keeps reading whole files, revert") did not fire cleanly: it kept reading *one*. + +The one unambiguous read of run 2 is about **where the remaining cost is**: with +ranges in place, orientation went from 77% of the re-read bill to still being its +largest block, now spread across more and cheaper results rather than +concentrated in a few expensive ones. Making each read cheaper has not reduced +the number of reads, and the number of reads is the term that enters +quadratically. That points the next attempt at item 3 (state the chain, so fewer +orientation round-trips are needed) rather than at further compression of what +each read returns. + +Separately found while reading the transcript, and not part of this measurement: +the run spent a call on a `brain` subcommand reporting that no store exists +before `brain init` has run. The generated CLAUDE.md routes to `brain plan` for +"planning, or picking work up" without naming that prerequisite. Verified +against the current binary, `brain plan` itself degrades gracefully (`No slices +yet`, exit 0) and `brain brief` is the one that stops; which of the two the run +hit is not recoverable from the pasted output, so this is logged as a routing +question to confirm, not a diagnosed defect. + +### Run 3 — the post-change side is reproducible, and the composition is the finding + +`BENCH-001-2026-10-10T19-40-28`, session `77de22fb`, same binary as run 2, with +**postgresql installed in the container**, so the `tests --local` tier ran and +the run scored **21/21**. + +| | run 1 (pre) | run 2 (post) | run 3 (post) | +|---|---:|---:|---:| +| model calls | 47 | 61 | **59** | +| wall | 7m00 | 7m25 | 6m57 | +| avg context | 103 k | 104 k | 108 k | +| cache read | 4.7 M | 6.2 M | 6.2 M | +| re-read cost | 1.7 M (37%) | 2.1 M (34%) | **2.2 M (35%)** | +| mxcli invocations | 70 / 17 chained | 85 / 22 | **93 / 28** | +| `syntax` invocations | 6 | 8 | **16** | +| doc lookups | 7 | 13 | **21** | +| orientation share of calls | 31% | 43% | 37% | +| score | 20/21 (env) | 20/21 (env) | **21/21** | + +Two things this adds that run 2 could not. + +**The post-change side is now n = 2 and tight** — 59 and 61 calls, 2.1 M and +2.2 M — against a single pre-change point at 47 and 1.7 M. Run 3 is also the +*stronger* of the two post-change points, because it did strictly more work: +postgres present meant the test tier, `docker check` and `playwright check` all +ran (`test` 5, `docker check` 5, `playwright check` 3) where runs 1 and 2 stopped +short. Doing more work for the same call count is the better outcome, and it is +still 26% above the pre-change point. + +**The lever hit its target and the total did not move, because a substitute +appeared.** Composition of the top ten, which carry 1.66 M of the 2.2 M (75%): + +| | re-read | share of 2.2 M | run 1 | +|---|---:|---:|---:| +| skill reads (`cat` ×2, `sed` ×2) | 814 k | **37%** | 68% | +| `syntax` (2 sweep loops, `--json`, 1 topic) | 502 k | **23%** | ~4% | +| `DESCRIBE STRUCTURE` + `DESCRIBE NAVIGATION` | 345 k | 16% | ~11% | + +The skills' share of the bill **halved**, which is exactly what item 3b set out +to do. The freed share was not returned — `syntax` took it, going from a cheap +sixth of a lever to 23% of the re-read bill on 16 invocations, two of them +`for t in …; do ./mxcli syntax $t; done` sweeps (195 k + 105 k) and one a whole +`syntax --json` index dump (90 k). Run 2 showed the same shape with three sweeps; +it reproduces. + +**And the index's realised saving is smaller than the design assumed**, measured +against the files actually read: + +- `overview-pages` (699 lines) was read with a **full `cat`** — 331 k, the single + costliest result in the run. This is the exact case the index exists for and it + did not fire. +- `test-microflows` (625 lines) was read as `40,400p` + `461,570p` — 470 of 625 + lines across two calls, 311 k against ~345 k for a `cat`. A ~10% saving for two + round-trips instead of one. +- `validation-microflows` (312 lines, ~3.2 k) was `cat`-ed, and that is *correct* + — a file that small is cheaper whole than in two ranges. + +So of three skill reads, one was rational, one saved ~10%, and the one that +mattered ignored the index. **A section index makes a partial read possible; it +does not make the agent want one.** + +The read that is now unambiguous across all three runs: compressing what each +orientation read *returns* has not reduced how many orientation reads *happen* +(31% → 43% → 37% of calls; 7 → 13 → 21 doc lookups). Read count is the term that +enters quadratically, and nothing shipped so far touches it. That is item 3's +territory — publish the chain and the facts the agent is round-tripping for — +not further compression. + +### Item 4 is no longer the junior partner — runs 2-3 reverse the demotion + +Run 1 put mxcli's own output at ~15% of the re-read cost against the skills' +68%, and item 4 (terse/delta output) was demoted on that basis. Run 3's +composition inverts it: `syntax` 23% + `DESCRIBE STRUCTURE`/`DESCRIBE +NAVIGATION` 16% = **39% from mxcli's own stdout**, against 37% from skill +reads. Both post-change runs show it, so the demotion was made on the one run +where it happened to be true. + +The correction is not "item 4 was right all along" — it is that the shares move +with the routing, because the agent substitutes between the two sources. A lever +aimed at a share measured before the routing changed is aimed at a number that +no longer exists. **Re-read the composition, not the ordering, before picking.** + +### Three things testable without spending the 3b control + +Each is orthogonal to the `9ce90f32` A/B — none touches the routing line or the +section index — so measuring them does not forfeit the control run. + +1. **Terse by default, verbose on request** (item 4, now co-equal with 3b). + `syntax ` is 1.9-3.9 k per call and was invoked 16 times in run 3; + `DESCRIBE STRUCTURE` is 3.2 k and `DESCRIBE NAVIGATION` 3.3 k. A + signature-only default with `--full` for the prose attacks 39% of the bill + without telling the agent to behave differently, which is the part that + backfired. + +2. **A cheap substitute for the sweep, not a prohibition.** Runs 2 and 3 show + the agent batching topics (`for t in …; do ./mxcli syntax $t; done`, twice in + run 3 at 195 k + 105 k, plus a 90 k `syntax --json` dump). It is batching + because it wants one answer covering a doctype and has to assemble it from + per-topic calls. `syntax --digest ` — every signature for that + doctype, no prose — turns a loop of full topics into one small result. Telling + it not to sweep leaves the want unmet; giving it a cheaper sweep does not. + +3. **Retry elimination as a call-count lever.** Run 3: 11 failures, 8 retry + chains, 8 retry calls — **14% of all model calls**, and each failure's text + is then re-read for the rest of the session. This is the only lever on the + list that removes *calls* rather than bytes, which is the term that enters + quadratically. The chains it recorded are specific and most are mxcli's to + fix, not the agent's: + + | error | whose | + |---|---| + | `create or modify microflow …: this change cannot be spliced into the stored microflow` (agent then dropped and recreated) | mxcli | + | `CE1613 "The selected attribute 'Sales.Order."Order_Customer/Name"' no longer exists"` | mxcli, probably | + | `no store yet — run 'mxcli brain init' first` | routing | + | `ERROR: Action: OPEN LINK without a URL or SHOW PAGE without a page` | authoring, but the message is reachable from `syntax` | + | `injecting test microflows: exit status N` | mxcli | + + All 8 chains resolved in one retry, so none is a loop — but a resolved retry + still costs a call, a result, and that result's re-read for every call after + it. + +A fourth, unmeasured: **collapse the orientation round-trip**. Doc lookups went +7 → 13 → 21 across the three runs while each read got cheaper, so the agent is +making more trips for the same understanding. One call that returns the +project's shape and the canonical chain together is item 3's job, and nothing +yet measures whether it would replace trips or just add one. + +### What that settles about `mxcli apply` + +The criteria recorded above are answered. The second — chained output being a +material share of `tool results` — is **no**: mxcli's output is a minority of +the re-read cost. On this evidence the staged pipeline would be optimising 15% +while 68% sits untouched, so it is not the thing to build, and the entry is +closed in the sequencing table rather than left open. + +One datum is still outstanding on the first criterion. `check` 15 against `exec` +20 looks like the redundant check-before-exec pattern, since `exec` already runs +the full check pass — but plain `check` without a project is a legitimate call, +so the verb counts cannot settle it. `diag loop-report`'s +`check_then_exec_pairs`, run in the same container, is what decides it; if it is +high, the answer is to **state the chain better**, which is item 3, not to wrap +it in a command. + ## What is actually asymmetric — and how little of it is irreducible An earlier draft said: "Vercel's agent writes a `.tsx` file and the work is @@ -691,7 +1006,7 @@ baseline is the first thing to record wherever it does. | | Lever | Effort | Expected effect | |---|---|---|---| -| 1 | `diag loop-report`, `diag session-report` + benchmark harness (`eval run`, `BENCH-001`) (lever 6) — shipped; baseline still to record | S | none directly — makes the rest falsifiable | +| 1 | `diag loop-report`, `diag session-report` + benchmark harness (`eval run`, `BENCH-001`) (lever 6) — **shipped, baseline recorded 2026-10-10** (§"The baseline, measured") | S | none directly — makes the rest falsifiable, and it did: the baseline overturned both candidate levers in favour of orientation cost | | 1b | Measure the check↔build gap rate: how many builds in a real session caught something `check` did not | S | sizes the batching prize, and feeds the parity programme's queue | | 2 | Fix `projectGates` to teach `exec`, not `check`+`exec` (lever 1) | XS | ~1 call per change, every project, immediately | | 2b | Measure `test --attach` on 11.14; pin the bootstrap default off 11.14 | XS | removes a forced 35 s/change from new projects | @@ -699,7 +1014,10 @@ baseline is the first thing to record wherever it does. | 2d | **Count a killed `run` in `diag loop-report` rather than dropping it** — **shipped** | S | was: the restart bill is invisible, `run`'s reported wall time a floor built from 4 of 30 invocations. Now a 30 s `session_alive` heartbeat bounds every unclosed run from below, whatever killed it, and the report separates the measured total, the measured floor and an explicitly-labelled estimate | | 2e | **App lifecycle as commands** — `run --local --detach`, `run status`, `run wait`, `run stop`, `run restart`, taught in the run-local/run-app skills and the generated gate list — **shipped** | M | ~25–28% of all tool calls in measured sessions were hand-rolled `nohup`/poll/`pkill` loops; each becomes one call, and `exec … && run wait` is the per-change chain | | 3 | Publish the canonical `&&` chain in `projectGates` + skills (lever 1) | XS | the 5–8 → 1–2 collapse, with nothing built | -| 4 | Terse/delta output for `exec` and the noisy listings (lever 2) | M | the token half of the chain win; helps every call | +| 3b | **Make consuming the skills cheap** — **first pass shipped 2026-10-10**: the generated CLAUDE.md now routes `syntax` before any skill, and `scripts/skill-index.py` puts a verified line-numbered section index at the head of all 74 (`make check-skill-index` in CI). Not yet done: moving the syntax that skills restate into `syntax` itself — code blocks are only 28% of skill bytes, so that is a smaller prize than it looked | M | the largest measured token item, and it displaces item 4. **Measured, leaning negative**: across runs 2 and 3 the skills' share of the re-read bill halved (68% -> 37%) and `syntax` sweeps took the freed share (23%), leaving the total 26% worse at a reproducible 59-61 calls against one pre-change point at 47. Missing datum is the control: two runs of `9ce90f32`. The identified surgical fix is to narrow the routing line against speculative sweeps, keeping the index (§"Run 3") | +| 4 | Terse/delta output for `exec`, `syntax` and the noisy listings (lever 2) | M | **promoted back to co-equal with 3b**: run 1's ~15% was the one run where that was true. Runs 2-3 put mxcli's own stdout at ~39% of the re-read cost (`syntax` 23%, the two `DESCRIBE`s 16%) against skill reads' 37%, because the agent substitutes between the two sources (§"Item 4 is no longer the junior partner") | +| 4b | **`syntax --digest `** — every signature for a doctype, no prose, one call | S | both post-change runs show the agent assembling exactly this by hand (`for t in …; do mxcli syntax $t; done`, 195 k + 105 k in run 3, plus a 90 k `syntax --json` dump). A cheaper sweep, rather than a rule against sweeping | +| 4c | **Remove the recurring retry chains** — the unspliceable `create or modify`, the CE1613 association-attribute page reference, `injecting test microflows`, the `brain init` routing gap | M | 14% of run 3's model calls were retries, all resolved in one, most of them mxcli's fault. The only listed lever that removes *calls* rather than bytes | | 5 | Tiered verification rule in the skills (lever 3) | S | stops the default path at the cheapest sufficient gate | | 6 | Subagent trigger in the skills (lever 4) | XS | caps the worst tail | | 7 | Workarounds → diagnostics and skills (lever 5) | M, ongoing | compounds across all future sessions | @@ -708,6 +1026,13 @@ Item 1 first is deliberate. Items 2, 3, 5 and 6 are all XS-to-S and can ship immediately after it — item 3 is now the one that changes the shape of the loop, and it is a documentation change. +**Item 1 has now run, and it reordered this table rather than confirming it.** +The baseline put the skills, not mxcli, at the top of the token bill, which +promotes the new item 3b above item 4 and closes `mxcli apply`. That is the +instrument working as intended: the two levers this document argued hardest for +were both aimed at the smaller share. Re-read §"The baseline, measured" before +picking the next item, and remember it is one sample. + **Items 2c and 2d were added after three projects were measured, and 2c is the largest single item in this table by wall time.** It is also the cheapest: a default, in prose, in files that already exist. Note that 2c and 2d attack @@ -716,8 +1041,12 @@ that 2d has to land for 2c to be claimable, because a lever that removes uncounted time cannot be shown to have worked. Item 4 is the only substantial build, and it is what makes item 3 pay in tokens rather than only in call count. -`mxcli apply` is deliberately **not** in this table. It is contingent on item 1 -showing that the published chain is still being composed wrong. +`mxcli apply` is deliberately **not** in this table, and as of the 2026-10-10 +baseline it is **closed rather than pending**: the measurement answered its +criterion no — mxcli's output is ~15% of the re-read cost where the skills are +~68% — so the command would optimise the smaller share. Re-open it only if +`check_then_exec_pairs` shows the published chain still being composed wrong, +and even then prefer item 3 (state the chain) over a wrapper. ## What this does not fix diff --git a/docs/11-proposals/README.md b/docs/11-proposals/README.md index ce98eec667..00a9d7a288 100644 --- a/docs/11-proposals/README.md +++ b/docs/11-proposals/README.md @@ -26,10 +26,10 @@ for display in this README): -## Active Proposals (103) +## Active Proposals (113) -### In Progress (partial) (12) +### In Progress (partial) (15) | Proposal | Status | Summary | |----------|--------|---------| @@ -37,6 +37,7 @@ for display in this README): | [Atlas Design System — a skill for visually appealing Mendix apps](PROPOSAL_atlas_design_system.md) | Partial | mxcli can generate a fully-functional Mendix app, but the default output looks bland — it leans | | [Default styling — generated apps that look designed on first boot](PROPOSAL_default_styling.md) | Partial | mxcli new produces a blank Atlas app. | | [Eval Framework for mxcli + Claude Code](proposal-eval-framework.md) | Partial | An evaluation framework that systematically tests how well Claude Code + mxcli handles real-world Mendix app generation tasks. | +| [First-class expressions for expression-typed MDL properties](PROPOSAL_first_class_expressions.md) | Partial | A property that holds a Mendix expression but is declared as a quoted string | | [Implement `mxcli structure` Command](mxcli-structure-proposal.md) | Partial | mxcli is a Go CLI tool for working with Mendix projects. | | [Multi-Version Support: Consolidated Architecture & Status](MULTI_VERSION_SUPPORT.md) | Partial | Mendix projects vary along three versioning axes, and mxcli must handle all of them correctly: | | [mxcli Linter - Implementation Proposal (Revised)](PROPOSAL_mxcli_linter.md) | Partial | Add an extensible linting framework to mxcli that leverages the existing SQLite-based catalog system. | @@ -45,12 +46,15 @@ for display in this README): | [Podman Support as Docker Alternative](PROPOSAL_podman_support.md) | Partial | Docker Desktop requires a paid subscription for larger organizations. | | [SHOW/DESCRIBE/USE Building Blocks](show-describe-building-blocks.md) | Partial | Document type: Pages$BuildingBlock (NOT Forms$BuildingBlock — the reader now | | [Skill packs — shipping a skill that carries more than prose](PROPOSAL_skill_packs.md) | Partial | mxcli ships 65 skills today and every one of them is a single Markdown file. | +| [Structured description of irreducible microflow graphs](PROPOSAL_structured_microflow_description.md) | Partial | Phase 0 (detector, MDL-FLOW01), Phase E (merge/join for error-path | +| [Translations — preserve, describe, author, and auto-translate](PROPOSAL_translations.md) | Partial | A Mendix app ships its user-visible strings in every language it supports. | -### Proposed (39) +### Proposed (40) | Proposal | Status | Summary | |----------|--------|---------| | [Bootstrap hooks that fetch the mxcli the project was built with](PROPOSAL_bootstrap_source.md) | Proposed | mxcli init writes .claude/bootstrap-mxcli.sh with a hard-coded download of | +| [Built-in file database for run --local (HSQLDB)](PROPOSAL_run_local_builtin_database.md) | Proposed | Depends on: the Windows process-helper fix (PR #1147) — this builds on that branch. | | [Comprehensive MDL Syntax and Grammar Improvements (v2)](PROPOSAL_mdl_syntax_improvements_v2.md) | Proposed | This document consolidates and expands upon previous proposals, offering a unified set of syntax and grammar improvements for all major Mend | | [Concurrent Access Safety for mxcli](PROPOSAL_concurrent_access.md) | Proposed | When Claude Code spawns multiple subagents, each may execute mxcli commands against the same Mendix project simultaneously. | | [Consumed REST Services (SHOW / DESCRIBE / CREATE)](show-describe-consumed-rest-services.md) | Proposed | Document type: rest$ConsumedRestService | @@ -90,16 +94,15 @@ for display in this README): | [VS Code Search — Quick Pick + Workspace Symbol](PROPOSAL_vscode_search.md) | Proposed | Full-text search exists in mxcli (mxcli search) but is only accessible via the terminal. | | [Workflow Improvements: ALTER WORKFLOW + Cross-References](PROPOSAL_workflow_improvements.md) | Proposed | Workflow support in mxcli has full CREATE/DESCRIBE/DROP/SHOW coverage with 13 activity types and BSON round-trip fidelity. | -### Draft (45) +### Draft (46) | Proposal | Status | Summary | |----------|--------|---------| | [Agent Document Type Support in MDL](PROPOSAL_agent_document_support.md) | Draft | Mendix 11.9 introduces Agents as a first-class concept for building agentic AI applications. | +| [Agent loop efficiency — making an mxcli session cost what the work costs](PROPOSAL_agent_loop_efficiency.md) | Draft | measured with diag loop-report, overturning the check-dominant assumption and | | [Architecture Graph Visualization (communities, layers, god-nodes)](PROPOSAL_architecture_graph_visualization.md) | Draft | Issue: TBD (file before implementation) | | [Association Mapping in IMPORT](PROPOSAL_import_associations.md) | Draft | Parent: PROPOSAL_mxcli_sql.md (Phase 3 extension) | | [Authorable layouts, page templates and building blocks — CREATE, not COPY](PROPOSAL_authorable_layouts.md) | Draft | An app built entirely through MDL cannot put anything in its own topbar. | -| [Authorable message definitions — the last mapping source MDL cannot write](PROPOSAL_authorable_message_definitions.md) | Draft | A mapping is bound to one of four schema sources. | -| [Backend Strategy — adopt engalar's modelsdk base + multi-backend (MCP first)](PROPOSAL_backend_strategy.md) | Draft | - Adopt engalar's modelsdk foundation as the base rather than merging 1109 | | [Bulk Change Custom Widget Properties](PROPOSAL_bulk_widget_property_updates.md) | Draft | Custom widgets (pluggable widgets) in Mendix have complex nested property structures. | | [Bulk External Action Support from OData Contracts](PROPOSAL_external_actions_bulk_create.md) | Draft | Issue #143 requests importing all entities and actions from a consumed OData service. | | [Claude Code Prompt: Sprotty-based Domain Model Visualization PoC](proposal_sprotty_visualization.md) | Draft | I'm building a VS Code extension that visualizes Mendix Domain Models using Sprotty. | @@ -126,19 +129,20 @@ for display in this README): | [mxcli auth — Mendix Platform Authentication](PROPOSAL_platform_auth.md) | Draft | A growing set of mxcli features need to talk to Mendix platform APIs on behalf of the user: | | [mxcli catalog — Mendix Catalog Integration](PROPOSAL_catalog_integration.md) | Draft | ⚠️ TERMINOLOGY NOTE: This proposal covers the external Mendix Catalog service at catalog.mendix.com (CLI: mxcli catalog search), which is se | | [mxcli check — mine Mendix's diagnostics catalog to close the check↔mxbuild gap](PROPOSAL_check_diagnostics_catalog.md) | Draft | Relates to: PROPOSAL_check_mxbuild_gap_heuristics.md (one tactical slice of this — | +| [mxcli companion — a Claude Code mod that shows the project, its brain, and how they connect](proposal-mxcli-companion-mod.md) | Draft | This is an umbrella proposal. | | [mxcli marketplace diff — detect local modification and plan a GUID-preserving module upgrade](PROPOSAL_marketplace_module_upgrade.md) | Draft | 2026-08-11 (DESCRIBE coverage measured — §7; GUID = database identity measured — §8) | | [mxcli Playground](mxcli-playground.md) | Draft | A public GitHub repository (mendixlabs/mxcli-playground) containing a ready-to-use Mendix project pre-configured with mxcli, Claude Code ski | +| [Offline synchronization configuration — the one thing an offline profile needs that MDL cannot say](PROPOSAL_offline_sync_configuration.md) | Draft | mxcli can author an offline navigation profile — PhoneOffline, | | [Owning the modelsdk/gen codegen — why the vendored generator cannot be adopted as-is](PROPOSAL_codegen_ownership.md) | Draft | Constraint set by the maintainer (2026-08-13): the reflection data is an | | [Playwright Session Reuse and Lifecycle Control](PROPOSAL_playwright_session_reuse.md) | Draft | Builds on proposal-playwright-cli.md, which | | [Project brain — an opt-in store, in a user's Mendix project, for what mxcli cannot compute](PROPOSAL_project_brain.md) | Draft | Audience: users of mxcli building Mendix projects — not mxcli's own | | [RENAME with Reference Refactoring](PROPOSAL_rename_refactoring.md) | Draft | Renaming entities, microflows, pages, and modules is one of the most common refactoring operations. | | [Replace Generated Playwright Tests with playwright-cli](proposal-playwright-cli.md) | Draft | The current approach (documented in proposal-playwright-testing.md) has Claude Code generate TypeScript test files (.spec.ts), then run them | | [Self-Describing Syntax Feature Registry](syntax-feature-registry.md) | Draft | Branch: research/recursive-help-discovery | -| [Structured description of irreducible microflow graphs](PROPOSAL_structured_microflow_description.md) | Draft | DESCRIBE MICROFLOW renders a microflow's control flow as nested if/then/else. | -| [Widgets as first-class MDL, not a second dialect](PROPOSAL_def_driven_widget_bodies.md) | Draft | A widget is named, described and tracked differently from every other MDL extension point, and resolves against hardcoded grammar lists rather than the project. | -| [Translations — preserve, describe, author, and auto-translate](PROPOSAL_translations.md) | Partial | A Mendix app ships its user-visible strings in every language it supports. | +| [The SOAP request body — the half of a web service call MDL cannot say](PROPOSAL_soap_request_body.md) | Draft | CALL WEB SERVICE can say who to call and what to do with the answer. | | [Version-Aware Agent Support](PROPOSAL_version_aware_agent_support.md) | Draft | Three use cases require mxcli to be version-aware at the MDL level: | | [warm dev loop — Docker-free run and iPad split-screen preview](PROPOSAL_mxcli_dev_warm_loop.md) | Draft | Relates to: PROPOSAL_check_mxbuild_gap_heuristics.md (the static-check gate that | +| [Widgets as first-class MDL, not a second dialect](PROPOSAL_def_driven_widget_bodies.md) | Draft | Using a widget should feel like calling a Java action. | | [Workflow / Microflow Syntax Alignment](PROPOSAL_workflow_microflow_syntax_alignment.md) | Draft | MDL spells the same concept differently depending on which document type you are | diff --git a/docs/11-proposals/proposal-mxcli-companion-mod.md b/docs/11-proposals/proposal-mxcli-companion-mod.md new file mode 100644 index 0000000000..5e93f2638d --- /dev/null +++ b/docs/11-proposals/proposal-mxcli-companion-mod.md @@ -0,0 +1,595 @@ +--- +title: mxcli companion — a Claude Code mod that shows the project, its brain, and how they connect +status: draft +date: 2026-10-11 +related: + - PROPOSAL_project_brain.md + - journey-architecture-viz.md + - PROPOSAL_multi_project_tree.md + - proposal-eval-framework.md + - PROPOSAL_vscode_visualizations.md +--- + +# mxcli companion — a Claude Code mod that shows the project, its brain, and how they connect + +This is an **umbrella proposal**. It describes the whole idea, fixes the +principles every part must follow, and splits the work into slices that can each +get their own proposal. Only the first slice — the **project tree with MDL +source** — is specified in enough detail to build. It is chosen first because it +needs no new mxcli capability and so tests the delivery mechanism (a mod shipped +by `mxcli init`) on its own. + +## 1. Problem + +A user building a Mendix app with Claude Code and mxcli works through a +transcript. Everything the agent builds — modules, entities, microflows, pages — +and everything that explains it — intent, specs, decisions, tests — is invisible +unless the user asks for it in words, and reads the answer as text that scrolls +away. Concretely: + +- **There is no persistent picture of the app.** "What did it build?" is a + question, not something the user can glance at. Studio Pro has an App Explorer; + a Claude Code session has none. +- **Starting is a blank prompt.** A fresh repo after `mxcli init` gives the user a + cursor. The procedures that make a good start exist (`bootstrap-app`, the + brain's slices) but the user has to know to ask for them; today that is done by + pasting an "empty-repo seed prompt". +- **The reasoning is not connected to the result.** The brain records + requirements anchored to model elements, and `mxcli test` runs tests, but + nothing connects a spec to the tests that prove it or to the documents that + implement it. Nothing shows the user whether what they asked for is built, + tested, and passing. +- **Explaining an app to a newcomer** is done with hand-made decks (sketch-deck + journey slides: persona → screens → logic → data → external per step). They are + good documentation but are made by hand, and nothing keeps them in step with + the model. + +Claude Code can now load **mods**: plugins of function hooks that draw panes, +bands above the prompt, and status entries, run host commands, and submit +prompts. That makes a persistent, interactive view next to the agent possible +without leaving the session. + +## 2. What exists today (verified) + +The UI is new; nearly all the data it needs already exists. + +| Need | Existing piece | Notes | +|---|---|---| +| Project tree | `mxcli project-tree -p app.mpr` (`cmd/mxcli/project_tree.go`) | JSON `[{label, type, qualifiedName?, children?}]`; ~60 node types; modules, folders, domain model, security, navigation, settings. Built for IDE integrations; used by `vscode-mdl` and the TUI | +| MDL of a document | `mxcli describe -p app.mpr [] --json` | `{name, type, mdl, executable}`. Type is optional: it is resolved from the catalog, then the live reader | +| Prior art for tree → source | `vscode-mdl/src/projectTreeProvider.ts`, `mdlContentProvider.ts` | Same flow: spawn `project-tree`, click → `describe`, with a type fallback list | +| Change detection | `cmd/mxcli/tui/watcher.go` | fsnotify on `.mpr` and `mprcontents/` subdirs, 500 ms debounce | +| Diagrams | `describe --format elk` / `mermaid` | **Not SVG** — layout and rendering happen in a browser (elkjs / mermaid from a CDN). Microflow objects and entities do carry stored canvas positions | +| Brain | `mxcli brain …` (`cmd/mxcli/brain/`), `docs/brain/` | Markdown, committed. Decisions, requirements (slices, forward anchors), open questions. `--json` on every verb. Progress derived from anchors resolving | +| Tests | `mxcli test` (`.test.mdl` / `.test.md`), `mxcli playwright verify` | Console + JUnit with `--junit`. **No link from a test to a requirement; no default results file** | +| Acceptance checks | `mxcli eval` check kinds (`entity_exists`, `page_has_widget`, `tests_pass`, …) | Per-check pass/fail in `score.json` | +| Shipping into a project | `mxcli init` writes `.claude/skills/`, commands, `settings.json`, a SessionStart bootstrap hook | Embedded via go:embed; refreshed with `init --sync-skills` | + +And on the Claude Code side (checked against this build's mod API and the +plugin-loading docs): + +| Capability | How | +|---|---| +| Ship with the project, no marketplace | A plugin folder under the project's **`.claude/skills//`** containing `.claude-plugin/plugin.json` is loaded in place (origin `@skills-dir`), enabled by default, **after the workspace trust dialog**. It loads only when the session is started from the repo root, and desktop worktrees load the main checkout's copy | +| Pane / band / status / toast | `$.ui.open` + `ui.render` on `Pane`; `AbovePrompt`; `$.ui.status`; `$.ui.toast` | +| Elements | `Box`, `Text`, `Button`, `Select`, `Input`, `Code` (syntax-highlighted, line numbers), `Markdown`; `Svg` on desktop/VS Code/mobile only, **max 131,072 characters**, no clicks inside | +| Run mxcli | `$.process.run(argv)` (one-shot, 30 s default, 4 MiB stdout); `$.process.spawn` (streaming, long-lived) | +| Start a workflow from a button | `$.prompt.submit({ text })` starts a turn as if typed; `$.prompt.fill` drafts it | +| React to the agent's work | `tool.call` hook sees every Bash/Edit call and its result | +| Greeting | **Not** `initialUserMessage` (that works in `-p` mode only). A `session.start` hook draws an `AbovePrompt` band, or opens a pane — but a pane opened unasked only takes its place from 144 terminal columns | + +## 3. The idea + +One mod, shipped by `mxcli init`, called the **mxcli companion**. It has six +views, which the slices in §5 build one at a time: + +1. **Start** — a band above the prompt in a fresh or idle session, offering + prepared workflows: *New app from a PRD*, *Change the app via a proposal*, + *Explore this app*. A button starts the workflow; the workflow's skill does the + interviewing. +2. **Project** — the app's structure as a tree. Select a document to see its MDL + source; for microflows and domain models, a diagram as well. +3. **Brain** — the project's documentation as the brain holds it: intent, + specs with their requirements, plans, decisions, open questions, reviews — per + slice, laid out by stage, with links into the project view. +4. **Verification** — which tests prove which spec, and whether they passed in + the last run; which requirements are built (anchors resolve) but untested, and + which are tested but not built. +5. **Inbox** — records an extractor proposed from the conversation + (requirements, test intents, tasks, issues, open questions), waiting for the + user to accept, edit or drop them (§5.7). +6. **Journeys** — user journeys held in the brain, drawn as journey maps + (persona → screens → logic → data → external per step) whose cards link to + the documents they name. They serve as a map of the app. The same data can be + exported as a sketch-deck for documentation (§5.6). + +## 4. Principles + +These hold for every slice. Each is a decision the slice proposals inherit, not +reopen. + +**P1 — The mod draws; mxcli knows.** Every view reads a `--json` command. The mod +never parses an MPR, a brain Markdown file, or a deck. This keeps all the logic +in Go, where it is tested and where `vscode-mdl` and the TUI can use the same +outputs. A view that needs data mxcli cannot provide gets a new mxcli command, +not mod code. + +**P2 — The brain holds all project documentation.** Intent, PRDs, specs, plans, +decisions, open questions, reviews and journeys all live in the brain, so there +is one store to read, one place to check, and one format the agent and the mod +process. A second documentation home (`docs/specs/`, a deck folder) would be a +second thing to keep in step. See §6 for what this changes in the brain's design. + +**P3 — Links are anchors, and anchors are checked.** Every link from +documentation to the app is a brain anchor (`@Module.Doc[.Member]`), checked +against the catalog. A spec that names a renamed microflow, a journey card +pointing at a deleted page, or a test naming an unknown requirement is reported +by `brain check`, not discovered by a reader. + +**P4 — State is derived, never written.** "Built" means the anchors resolve; +"tested" means a test names the requirement; "passing" means the last results +file says so. No status column, no checkbox, no figure in prose. This is the +brain proposal's A6, applied to the whole companion. + +*The one stated exception:* a **task's or issue's status** (open, done) cannot +be computed from the model, so it is written — on the record itself, and nowhere +else (§6.1). Requirements keep their derived status; a requirement never gains a +status field because tasks have one. + +**P5 — It works on every surface, degrading where it must.** The terminal has no +`Svg`, mobile no `Select`. Each view has a text form that works everywhere; a +picture is added where the surface can draw one. + +## 5. Slices + +Each slice is usable on its own. The order is by dependency and by what each one +proves. + +| # | Slice | New mxcli capability | Depends on | +|---|---|---|---| +| 1 | Project tree + MDL source, shipped by `init` | none (one small fix: describe keywords) | — | +| 2 | Start band + workflow launcher | workflow frontmatter on skills, `mxcli workflow list --json`, new user workflow skills | 1 (delivery) | +| 3 | Brain as the documentation store + Brain view | new brain document kinds and links (§6) | 1 | +| 4 | Test ↔ spec traceability + Verification view | `@requirement` on tests, JSON results file, coverage in `brain check` | 3 | +| 5 | Diagrams in the Project view | `describe --format svg` for microflows, nanoflows, domain models | 1 | +| 6 | Journeys + sketch-deck export | journey documents in the brain, `mxcli journey …` | 3, 5 (renderer) | +| 7 | Conversation → structure: an extractor that turns the conversation into staged brain records | provenance on `brain capture`, new record kinds (test intent, task, issue) | 3, 4 | + +Slice 1 is specified in §7. The rest are sketched below with what is known and +what each needs to decide. + +### 5.2 Start band and workflows + +**Where workflows live:** as skills in `.claude/skills/mendix/`, shipped by +`init` like every other skill, so one file is both the agent's procedure and the +thing the UI offers. A skill becomes a workflow by declaring it in frontmatter: + +```yaml +workflow: + title: New app from a PRD + stages: [intent, spec, plan, build, test, review] + offer: fresh # fresh | always | never — when the Start band shows it +``` + +`mxcli workflow list --json` returns the declared workflows (P1: the mod does not +read skill files). A button calls `$.prompt.submit` with a prompt that invokes the +skill. While a workflow runs, the band shows its current stage, derived from the +brain (P4), not from the conversation. + +**What exists:** `bootstrap-app` covers most of *new app* (interview, `mxcli new`, +requirements as slices, propose the model before building). **What does not:** +any user-facing *PRD*, *change via proposal*, or *plan → review* workflow; those +exist only as contributor commands (`mxcli-dev:proposal`, `adr-new`, `review`) +and are not shipped. Writing those skills is the bulk of this slice. + +**Greeting:** a fresh project (no `docs/brain/`, or an empty model) shows the +Start band on `session.start`. It replaces pasting the empty-repo seed prompt. + +### 5.3 Brain as the documentation store + +Specified in §6, because it amends an existing proposal. The Brain view is a +per-slice board across stages (intent → spec → plan → built → tested → +reviewed), read from `brain plan --json` / `brain show --json`, with each +anchor linking into the Project view. + +### 5.4 Test ↔ spec traceability + +- **Link:** a `@requirement ` annotation in the `.test.mdl` javadoc block + (`testrunner/parser.go`), and the same in a header comment of + `tests/verify-*.test.sh`. A test may name several. +- **Results:** `mxcli test` and `mxcli playwright verify` always write + `.mxcli/test-results/latest.json` (gitignored) — per test: id, file, kind, + requirement ids, outcome, duration, message. JUnit stays as an extra. +- **Coverage:** `brain check` reports requirements with no test, and tests naming + unknown requirements. `brain plan --json` gains `tested` / `passing` per + requirement, derived from the annotation and the results file (P4). +- **Acceptance checks:** a spec's acceptance criteria may be written as the eval + framework's typed checks (`entity_exists`, `page_has_widget`, …), so part of + "is this spec met" is machine-checked without writing a test. Whether to reuse + `evalrunner/checks.go` directly is for the slice proposal. +- **View:** spec → requirements → tests → last outcome, and requirement → + anchors → built. A table, not a picture: it needs to scroll and be read. +- **Refresh:** a `tool.call` hook that sees the agent run `mxcli test` or + `playwright verify` reloads the view when the call returns. + +### 5.5 Diagrams (`describe --format svg`) + +The mod cannot run elkjs or mermaid (no DOM), so the SVG must come finished from +mxcli. For **microflows, nanoflows and domain models** the model stores canvas +positions, so a Go renderer can draw at Studio Pro's own coordinates with no +layout engine — the result looks like Studio Pro's canvas. Page wireframes and +the system overview have no positions and need a simple layout; they are later +work. The 131,072-character cap means compact output (no per-element styles, short +paths). VS Code, the TUI and `mxcli serve` could use the same renderer and drop +their CDN dependency, which is why this is its own slice and not mod code. + +### 5.6 Journeys and sketch-deck export + +**Journeys are brain documents** (P2): a persona (a user role), steps, and per +step the cards in each layer, each card carrying a label, a note, a status +(AI / plain logic / scripted / not in the demo / offline — the deck's legend) and +**anchors** to the documents it names (P3). `brain check` reports stale cards. +`mxcli journey suggest` can propose cards for a step from the catalog's refs +(page → microflows it calls → entities they touch → REST clients), for the agent +or user to accept; detecting whole journeys automatically was tried and rejected +(`docs/plans/architecture-diagram-plan.md`: cyclic navigation, per-role paths and +popups make it noise), so journeys are authored. + +**In the mod**, a journey is drawn from the brain data (P1: `mxcli journey show +--json`), not cut out of a deck. Cards cannot be clicked inside an SVG, so the +steps and cards are also listed beside the picture, and selecting a card opens +its documents in the Project view; a selected card is highlighted by injecting a +CSS rule into the SVG. + +**Sketch-deck export** turns the brain's journeys (plus a title slide and an +overview) into a sketch-deck HTML file as user documentation. The format was +read from an existing deck (`mxcli-agent-dx-presentation.html`): one HTML file, +one large SVG, one `` layer per slide with `data-sd-slide=""` and +`data-sd-frame="x y w h"` giving the camera view, steps revealed with +`data-sd-step`, fonts from Google Fonts, and a script for the camera. Each +journey slide in that deck is 90–145 KB, which is one more reason the mod draws +from data rather than from the deck: a slide is near or over the `Svg` cap. +Whether export is an mxcli command or the sketch-deck skill reading +`mxcli journey show --json` is for the slice proposal; P1 says either way the +data comes from mxcli. + +### 5.7 Conversation → structure + +Requirements, tests, tasks and issues are mostly *said* before they are written +down: "inspectors log a defect on a tablet", "from severity 2 up, after +maintenance approves", "photos are a must, five per defect is enough", "the SAP +sync sometimes fails at night". Today they stay in the transcript unless someone +remembers to capture them — the brain proposal's A11 names exactly this gap ("a +curation step with no trigger stops"). This slice adds the trigger: an extractor +that follows the conversation and proposes structured records. + +**It reuses the brain's existing pipeline unchanged in shape.** The extractor +*captures*, the human *promotes* — the same split the brain already has between +`staged.jsonl` and the committed files, and the reason promotion was made a +human step in the first place. Nothing the extractor produces becomes project +documentation without a person accepting it. + +**Mechanism (all in the mod's reach today):** + +1. On `turn.complete`, the mod runs `$.model.fork` — a tool-less completion over + the session's **own** transcript, served from the prompt cache — with an + extraction prompt: *list the requirements, test cases, tasks, issues and open + questions stated since the last extraction, as JSON, each with the turn it + came from.* Cheaper alternatives for quiet turns: a `$.model.classify` gate + ("did this turn state anything new?") before the fork, or extraction only on + idle / on a button. +2. Each record goes to `mxcli brain capture … --json` (P1: the mod does not write + brain files), with its kind, slice guess, anchors the model proposed, and + **provenance** (session id and message id). mxcli de-duplicates against what + is already staged or promoted and validates the anchors. +3. The Brain view gains an **Inbox**: staged records grouped by kind, each with + *Accept* (`brain promote`), *Edit* (fills the prompt box via `$.prompt.fill` + with a correction request), and *Drop*. Accepting places the record in the + structure — slice → feature → record — which is the third column of the + "conversation to structure" picture. + +**"And back."** Provenance makes the link two-way: a promoted record shows the +turn it came from, and a record can be sent back into the conversation as work +("implement REQ *Up to 5 photos per defect*", "write the test for *Severity 1 +creates no order*") with `$.prompt.submit`. + +**Record kinds.** The picture uses REQ, TEST, TASK, ISSUE and TODO, grouped into +phases (MVP, Phase 2, Backlog) and features (Defect capture, Work orders). +Against the brain: + +| Picture | Brain | Status | +|---|---|---| +| Phase | slice | exists | +| Feature | spec (a document, §6) | slice 3 | +| REQ | requirement entry | exists | +| Open question (implied: "should a defect create a work order?") | open question | exists | +| TEST | **test intent**: a planned test naming a requirement; satisfied when a real test carries `@requirement` for it (§5.4) | new | +| TASK / TODO | **task**: work that is not a requirement (check an integration, sketch a form) | new — a brain record, synced out (§6.1) | +| ISSUE | **issue**: a known defect or risk, anchored to what it affects | new — a brain record, synced out (§6.1) | + +A test intent is the useful new kind: it is a TEST card before any test file +exists, and §5.4's coverage check turns it into "tested" once a test names the +requirement — derived, not ticked (P4). + +**Risks the slice proposal must answer:** cost per turn (measure fork cost on a +real session before choosing per-turn vs on-idle); duplicate and contradictory +extractions as a conversation revises itself ("five photos" then "ten"); the +extractor inventing requirements nobody stated (provenance makes every record +checkable against its source turn, and the eval framework can score extraction +against a scripted conversation); and whether extraction also runs on a +**pasted** source — a PRD, meeting notes — which is the same extraction over a +document instead of the transcript, and the natural first step of the *New app +from a PRD* workflow (§5.2). + +## 6. Amendment to the project brain + +P2 changes the brain's scope. This section states the change so the brain +proposal can be amended rather than contradicted. + +**What it was:** a small, capped, always-loaded store of the "negative space" — +only what cannot be derived from the model. Its §3 explains why the caps are +"the single most important thing in it": a store loaded every session that grows +without bound stops being read. + +**What survives unchanged:** +- **Negative space.** Specs, intent, plans, journeys and reviews are all things + the model cannot tell you. None of them may transcribe the model; they + reference it by anchors. A spec that lists an entity's attributes is wrong in + the same way a decision that lists them is. +- **Caps on what is always loaded.** `project.md` and the module shards keep + their caps and stay the session-start reading. + +**What changes:** the brain gains a second tier, **documents**, alongside the +existing **entries**: + +| | Entries (today) | Documents (new) | +|---|---|---| +| Kinds | decision, requirement, open question | intent / PRD, spec, plan, journey, review | +| Size | one paragraph | pages | +| Loaded | every session (capped) | on demand: `brain brief`, the mod, a workflow step | +| Cap | per shard, enforced by `promote` | none on size; `brain show` reports size | +| Anchors | yes, checked per kind | yes; a document's requirements are entries | +| Location | `project.md`, `modules/`, `plan/.md` | `docs/brain//.md`, with frontmatter (id, kind, slice, links) | + +Requirements stay entries, and a spec **contains** them: the spec document is the +prose and reasoning, its requirements are the checkable, anchored lines that +`brain plan` counts and that tests name (§5.4). That keeps the existing +"built = anchors resolve" mechanism intact and gives tests something small and +stable to point at. + +**Open for the slice-3 proposal:** whether documents go through `capture` / +`promote` (human promotion was deliberate for entries; a 3-page spec written in a +workflow may need a different gate), how a review links to the spec and change +it reviews, and whether a PRD and an intent are one kind or two. + +### 6.1 Tasks, issues and test intents: one JSON file per record + +**The brain is the source of truth for tasks and issues, not a tracker.** Not +every user has GitHub; some use Jira, Azure DevOps, Linear, or nothing. So the +record lives in the repo, and a sync adapter mirrors it to whatever the user +uses — the reverse of making one tracker the home and the brain a cache. + +**Format — decided: option (a).** The three new kinds are stored as **one JSON +file per record**; the existing kinds stay as they are. + +| Kind | Stored as | Why | +|---|---|---| +| decision, requirement, open question | Markdown shards, unchanged | always loaded, capped, reviewed as prose in a PR | +| task, issue, test intent | `docs/brain/{tasks,issues,tests}/.json`, one file each | queryable across repos, conflict-free on parallel branches | +| documents (§6) | `docs/brain//.md` with frontmatter | pages of prose | + +One file per record, not one shared `.jsonl`, because this repo has measured +the alternative: the bug findings began as a single appended file, parallel fix +branches conflicted on it constantly, and `merge=union` let 256 duplicates through +unnoticed. They were split into one file per record +(`.claude/skills/fix-issue/findings//-.json`) and are queried +with DuckDB. Each branch now adds its own files and nothing conflicts. The +same holds for a project where the agent captures tasks on several branches. + +Whether decisions and requirements should later move to the same format — with +`mxcli brain show` rendering the Markdown view — is left until the document tier +settles its format. Moving them costs the prose review the brain proposal values; +not moving them leaves those kinds out of cross-repo queries. + +**The record** (fields, not final): + +```json +{ + "schema_version": 1, + "id": "T-20261011-check-sap-assets", + "kind": "task", + "title": "Check SAP asset integration", + "body": "The asset list lives in SAP; confirm the consumed service covers it.", + "slice": "02-sap-integration", + "status": "open", + "anchors": ["@SapIntegration.RC_Assets"], + "source": { "session": "…", "message": "…" }, + "external": { "system": "jira", "id": "QE-142", "url": "https://…" }, + "created": "2026-10-11" +} +``` + +`status` is the P4 exception, written here only. `source` is the provenance from +§5.7. `external` is set by sync and never by hand. The schema is a contract +checked by `brain check` (and a `make`-style check in mxcli's own CI), with +`schema_version` so a cross-repo query survives projects on different mxcli +versions. + +**Sync.** `mxcli brain sync --to ` with one adapter per system +(GitHub, GitLab, Jira, Azure DevOps, Linear to start; the adapter interface is +for the slice proposal). `external` makes re-sync update the same ticket instead +of creating a duplicate. **Ownership, to avoid two-way conflicts:** the brain +owns content (title, body, anchors, slice); the external system owns workflow +state (status, assignee) once a record is synced, and sync pulls it back into +`status`. A record with no `external` is owned entirely by the brain. Sync is +opt-in and never runs without being asked. + +**Overview across projects.** Because the records are plain files in each repo, +an overview of every open task is a DuckDB query over local checkouts — no +server, no tracker: + +```sql +select regexp_extract(filename, 'repos/([^/]+)/', 1) as project, + kind, title, status, slice +from read_json_auto('~/repos/*/docs/brain/{tasks,issues}/*.json', filename = true) +where status <> 'done' +order by project, slice; +``` + +Caveat: it reads checkouts, so it is as fresh as each repo's last pull. A thin +`mxcli brain overview ` can `git fetch` first and read the default branch +with `git show`, giving the same result without touching working trees; it can +also feed a cross-project view in the companion later. + +## 7. Slice 1 — project tree and MDL source + +### 7.1 Scope + +- A pane, **Project**, opened by the `/mxcli` command (and by a button on the + status entry). It shows the tree from `mxcli project-tree`, expandable, and + below or beside it the MDL source of the selected document. +- Selecting a document runs `mxcli describe --json` and shows the `mdl` field in a + `Code` element. +- The tree reloads when the agent changes the model, and on a Refresh button. +- Shipped by `mxcli init` and refreshed by `init --sync-skills`. + +Out of scope for slice 1: editing MDL in the pane (the agent edits), diagrams, +the brain, the Start band, a long-lived mxcli process. + +### 7.2 Design + +**Locating the project.** In order: a mod option (`userConfig.mprPath`), else the +single `*.mpr` in the session's directory, else one level down (the same rule +`init` uses for solution repos). Several candidates → a `Select` in the pane. + +**Data.** `$.process.run(["mxcli", "project-tree", "-p", mpr])` and +`$.process.run(["mxcli", "describe", "-p", mpr, type, qn, "--json"])`, with +`MXCLI_QUIET=1`. The mxcli binary is found as `./mxcli` (where the bootstrap +script puts it) or on `PATH`. One process per request is acceptable for slice 1; +§7.4 measures whether it stays acceptable. + +**Drawing the tree.** A row per visible node: indent, a disclosure `Button` +(`plain`) for nodes with children, the label, and the type dimmed. Expanded +nodes and the selection live in `$.state` atoms so a reload keeps them. Only +the rows that fit `e.props.bodyColumns` × the viewport are drawn — a tree is held +to 100,000 characters of text per drawing, which a large project's full tree +would exceed. Folders and category nodes expand; document nodes select. + +**Drawing the source.** `Code` with `startLine: 1`. MDL has no highlighter +grammar yet; slice 1 draws it with `language: "sql"` as the closest fit, and a +contributed MDL grammar is an open question (§10). Documents above the drawing +limit are cut by the engine with a note; acceptable for slice 1. + +**Node types with no MDL.** `project-tree` emits nodes that `describe` cannot +answer: container nodes (folder, domainmodel as a whole, category, security) and +a few document types. Containers show their children as a list instead of +source. For document types, the mod sends the tree's `type` and mxcli resolves it +(P1) — which needs the fix in §7.3. + +**Refresh.** A `tool.call` hook, after the call returns, reloads the tree when the +call was a Bash command running `mxcli` with `exec`, `-c`, `new`, `rename`, or +`marketplace`, or an Edit/Write under `mprcontents/` or of an `.mpr`; debounced, +so a burst of agent calls reloads once. Changes made outside the session (Studio +Pro) are picked up by the Refresh button. A watcher is slice-later work (§10). + +**Surfaces.** Terminal (fullscreen docks the pane beside the transcript; the main +screen opens it inline) and desktop. Both draw from the same tree; the test kit +runs the UI tests on both. + +### 7.3 Changes to mxcli + +| File | Change | +|---|---| +| `mods/companion/` (new) | The mod source: `.claude-plugin/plugin.json`, `hooks/hooks.json`, `hooks/register.tsx`, `types/index.d.ts`, `*.test.ts`. Kept out of `.claude/skills/mendix/` so the skills sync (`rsync --delete`, mirrored into `.ai-context/`) neither deletes it nor ships it to non-Claude tools | +| `Makefile` | `sync-mods` target copying `mods/companion/` into an embed dir; part of `sync-all` | +| `cmd/mxcli/skills_content.go` | Embed the mod directory | +| `cmd/mxcli/init.go`, `init_tooling_sync.go` | Write the mod to `.claude/skills/mxcli-companion/` for the `claude` tool only; refresh it in `--sync-skills`, including stale-file removal inside that folder only | +| `cmd/mxcli/cmd_describe.go` | Accept every type `project-tree` emits that the executor can describe (today JavaScript actions and others have no CLI keyword), and return a clear "no MDL for this type" error for the rest, so the mod needs no fallback list | +| `cmd/mxcli/project_tree_test.go` | A test that every node type `project-tree` emits is either describable or on an explicit no-MDL list, so the two cannot drift again | + +No BSON or MDL syntax changes. No change to the project file is ever made by the +mod. + +### 7.4 Hypotheses slice 1 tests + +Slice 1 exists as much to answer these as to deliver the tree. Each has the +measurement that settles it. + +1. **A mod shipped in `.claude/skills/` loads in a fresh session after trust**, + in the terminal and in the desktop Code tab, with no install step. Measure on + a fresh `mxcli init` repo, both surfaces; record what the user sees. +2. **One process per request is fast enough.** Time `project-tree` and `describe` + on a small app and on a large one (Evora-sized). If `project-tree` exceeds a + couple of seconds or its output nears `$.process.run`'s 4 MiB cap, slice 2 + adds a long-lived mxcli process (`$.process.spawn`) before going further. +3. **Refresh from `tool.call` catches the agent's changes.** Build a small module + through the agent and count missed reloads. +4. **The tree is usable in a docked pane** at typical widths, with keyboard and + mouse. + +### 7.5 Test plan + +- **Mod:** `claude plugin validate mods/companion`; `claude plugin test` with + `$.process` mocked: tree renders on `terminal` and `desktop`; expanding, + selecting and refreshing; a `describe` error shows a message, not a blank + pane; a `tool.call` running `mxcli exec` triggers one reload, a `git status` + none. Each test reverted against a stubbed hook to show it can fail. +- **mxcli:** the drift test in §7.3; `describe` keyword tests for the newly + accepted types; `init` and `init --sync-skills` integration tests that the mod + lands in `.claude/skills/mxcli-companion/` for `--tool claude` and not for other + tools, and that a sync leaves user files in `.claude/skills/` alone. +- **By hand, recorded in the PR:** hypotheses 1–4 on a fresh repo, terminal and + desktop. + +## 8. Version compatibility + +- **Mendix:** none of slice 1 depends on the Mendix version beyond what + `project-tree` and `describe` already support. +- **Claude Code:** mods need a build that supports function-hook plugins and + `@skills-dir` loading. What an older Claude Code does with the folder is + **unmeasured** — slice 1 must check it does no harm (hypothesis 1), and `init` + should say which version the companion needs. +- **Other tools:** the companion is Claude Code only. Every other tool keeps + what `init` gives it today; the `--json` commands the mod uses are available to + any of them. + +## 9. Alternatives considered + +- **Extend `mxcli tui` instead.** It already browses the project and previews + MDL, and has an agent socket. But it is a separate terminal program; the point + here is a view *inside* the agent session that reacts to what the agent does. + The TUI and the mod should share mxcli's `--json` outputs (P1), not code. +- **Ship via a marketplace** (`extraKnownMarketplaces` + `enabledPlugins` in + project settings). Needs a GitHub or git source and an install step per user; + `@skills-dir` needs neither. A marketplace remains an option for users who want + the companion in repos not made by `init`. +- **Specs outside the brain** (`docs/specs/`). Rejected by P2: two stores to keep + in step, two places for the agent to look. +- **Cut journey slides out of an existing deck** for the mod. Rejected: the deck + is output, not source; slides are near the `Svg` cap; and the cards would need + anchors added by hand to a drawing. + +## 10. Open questions + +1. **Name and command.** "companion", `/mxcli`? The folder name in + `.claude/skills/` must not collide with a skill. +2. **MDL highlighting.** Contribute an MDL grammar to the `Code` element (the API + allows plugin-contributed grammars), or draw as SQL? The VS Code extension has + a TextMate grammar that may be reusable. +3. **When does a long-lived mxcli process become necessary** — slice 2, or only if + hypothesis 2 fails? Candidates: a `serve --stdio` JSON-RPC mode, custom requests + on `mxcli lsp`, or a headless TUI agent socket. Any of them would serve VS Code + too. +4. **Changes from outside the session** (Studio Pro open alongside): add a watcher + (a streaming `mxcli watch` the mod spawns) or keep the Refresh button? +5. **Settings merge in `init`.** `init` overwrites `.claude/settings.json`. If any + slice needs a setting (e.g. `enabledPlugins` to let users turn the companion + off), `init` must merge instead. +6. **Brain documents and promotion** (§6): does a workflow-written spec go through + `promote`? +7. **Multiple apps in one repo** (`PROPOSAL_multi_project_tree.md`): slice 1 picks + one `.mpr`; does the Project view need a switcher from the start? +8. **Decisions and requirements as JSON too** (§6.1): revisit once the document + tier's format is settled. +9. **Sync adapters** (§6.1): which systems first, how credentials are held, and + whether sync pulls new tickets created outside the brain back in as records. diff --git a/docs/14-eval/eval-bench-002.md b/docs/14-eval/eval-bench-002.md new file mode 100644 index 0000000000..2ce4b25827 --- /dev/null +++ b/docs/14-eval/eval-bench-002.md @@ -0,0 +1,156 @@ +--- +id: BENCH-002 +category: Benchmark/Loop +tags: [benchmark, loop-efficiency, entity, association, enumeration, view-entity, pages, microflow, workflow, security, navigation, test] +timeout: 75m +--- + +# BENCH-002: Field service desk (larger agent-loop benchmark) + +The second fixed brief for measuring an agent session end to end +(docs/11-proposals/PROPOSAL_agent_loop_efficiency.md, lever 6). BENCH-001 is a +seven-minute run; this one is roughly 2× its model surface — six entities, +one view entity, five associations, three enumerations, nine pages, five +microflows, a workflow and three module roles — so a change whose effect is +below BENCH-001's run-to-run noise has somewhere larger to show up. + +It deliberately reaches the two doctypes BENCH-001 never touches. A +request-to-completion flow is what a Mendix **workflow** is for, and +reporting is what a **view entity** is for; modelling both with a status +enumeration and a microflow, as the first draft of this brief did, kept the +benchmark on the well-trodden half of the loop. Those are the doctypes whose +skills an agent has to go and read, which is the orientation surface the +benchmark exists to measure. + +Same contract as BENCH-001: **keep the prompt unchanged**, because the only +thing that makes two runs comparable is that they asked for the same work. And +**run it from a terminal, not from inside an agent session** — `eval run` spawns +`claude -p --dangerously-skip-permissions` (hardcoded, `evalrunner/run.go`) and a +host that sandboxes its children refuses to create a permission-bypassed agent. + +Why it is specified to this level of detail, when the brief it came from was a +paragraph: an *open* brief lets the agent choose its own scope, and scope is the +thing the cost is proportional to. Two runs of an open brief can differ by a +whole entity cluster, which swamps the effect of any loop change being tested — +the same confound that already made a 47-vs-61-call comparison unreadable. The +open version of this brief is worth running, but as a **design-quality** eval, +not as a cost A/B. Keep the two apart. + +## Prompt +Build a field service and maintenance app in this Mendix project, in a new module called Service. The company builds and services semiconductor lithography systems installed at customer fabs; the app lets fab staff report system issues and lets service staff run the request-to-completion flow. Keep it simple: one happy path per process, no edge cases or exception handling. + +Domain model: +- Fab: Name (required), City, Country. +- System: SerialNumber (required), Model, InstalledOn (date and time), Status (enumeration SystemStatus: Running, Degraded, Down; default Running). +- Engineer: Name (required), Email, Certified (boolean, default false). +- ServiceRequest: RequestNumber (autonumber), ReportedOn (date and time, defaults to now), Title (required), Description, Priority (enumeration Priority: Low, Medium, High; default Medium), Status (enumeration RequestStatus: New, Assigned, Completed; default New). +- WorkOrder: OrderNumber (autonumber), ScheduledOn (date and time), HoursSpent (decimal), TotalCost (decimal), Summary. +- PartUsed: Name (required), Quantity (integer), UnitCost (decimal). +- A system is installed at one fab, a service request is about one system, a work order belongs to one service request and is assigned to one engineer, and a part used belongs to one work order. Name associations the Mendix way: System_Fab, ServiceRequest_System, WorkOrder_ServiceRequest, WorkOrder_Engineer, PartUsed_WorkOrder. + +Reporting: +- A view entity OpenRequestsPerFab with FabName (String(200)) and RequestCount (Integer): one row per fab, counting that fab's service requests. The rows come from the database, so this is a view entity, not a persistent entity that something fills. + +Pages: +- An overview page and an edit page for Fab, System, Engineer, ServiceRequest and WorkOrder (Fab_Overview, Fab_NewEdit, and so on). +- The service request edit page shows the request's work orders. +- The work order edit page shows the parts used and lets the user add, edit and remove them. +- A read-only overview page OpenRequestsPerFab_Overview listing the view entity's rows. + +Logic: +- ACT_Request_Assign: it creates a work order for the service request, sets the request's Status to Assigned and commits. +- ACT_Request_Complete: it sets the request's Status to Completed and commits. +- ACT_WorkOrder_Complete, called from a Complete button on the work order edit page: it sets the work order's TotalCost from its parts and commits. +- Put the parts cost calculation in its own sub-microflow, SUB_WorkOrder_TotalCost, so it can be tested. It returns the sum of quantity × unit cost over the work order's parts. + +Workflow: +- A workflow WF_ServiceRequest whose parameter entity is ServiceRequest, driving the request from report to completion: a user task "Assign engineer" targeted at the ServiceEngineer module role, then a call to ACT_Request_Assign, then a user task "Carry out the service work" targeted at the same role, then a call to ACT_Request_Complete. +- A Start button on the service request edit page starts the workflow for that request. +- If a user task needs its own task page, create one. + +Security: +- Three module roles in Service: Administrator (full access), ServiceEngineer (can read fabs and systems, can create and edit work orders and parts, can edit service requests), FabUser (can create and edit service requests, read-only on everything else). Map Administrator to the Administrator project role and the other two to the User project role. + +Navigation: +- Menu items for the System, ServiceRequest and WorkOrder overview pages. + +Tests: +- A microflow test file in tests/ (a .test.mdl) that checks SUB_WorkOrder_TotalCost for a work order with two parts. + +When you are done, the project must pass `mx check` with no errors, and the tests must pass. + +## Expected Outcome +A Service module with six persistent entities, one view entity, five +associations and three enumerations; overview and edit pages for five entities +plus a read-only page over the view, with work orders nested in the request page +and parts nested in the work order page; three action microflows and one tested +sub-microflow; a workflow over ServiceRequest with two role-targeted user tasks +driving the two action microflows; three module roles mapped to project roles; +three menu items; and a passing microflow test. + +## Checks +- entity_exists: "Service.Fab" +- entity_exists: "Service.System" +- entity_exists: "Service.Engineer" +- entity_exists: "Service.ServiceRequest" +- entity_exists: "Service.WorkOrder" +- entity_exists: "Service.PartUsed" +- entity_has_attribute: "Service.System.SerialNumber" +- entity_has_attribute: "Service.System.Status" +- entity_has_attribute: "Service.ServiceRequest.Status" +- entity_has_attribute: "Service.ServiceRequest.Priority" +- entity_has_attribute: "Service.WorkOrder.HoursSpent Decimal" +- entity_has_attribute: "Service.WorkOrder.TotalCost Decimal" +- entity_has_attribute: "Service.PartUsed.Quantity Integer" +- entity_has_attribute: "Service.PartUsed.UnitCost Decimal" +- entity_exists: "Service.OpenRequestsPerFab" +- entity_has_type: "Service.OpenRequestsPerFab View" +- entity_has_attribute: "Service.OpenRequestsPerFab.RequestCount Integer" +- entity_has_attribute: "Service.OpenRequestsPerFab.FabName" +- association_exists: "Service.System_Fab" +- association_exists: "Service.ServiceRequest_System" +- association_exists: "Service.WorkOrder_ServiceRequest" +- association_exists: "Service.WorkOrder_Engineer" +- association_exists: "Service.PartUsed_WorkOrder" +- page_exists: "Service.Fab_Overview" +- page_exists: "Service.System_Overview" +- page_exists: "Service.Engineer_Overview" +- page_exists: "Service.ServiceRequest_Overview" +- page_exists: "Service.ServiceRequest_NewEdit" +- page_exists: "Service.WorkOrder_Overview" +- page_exists: "Service.WorkOrder_NewEdit" +- page_exists: "Service.OpenRequestsPerFab_Overview" +- microflow_exists: "Service.ACT_Request_Assign" +- microflow_exists: "Service.ACT_Request_Complete" +- microflow_exists: "Service.ACT_WorkOrder_Complete" +- microflow_exists: "Service.SUB_WorkOrder_TotalCost" +- workflow_exists: "Service.WF_ServiceRequest" +- module_role_exists: "Service.Administrator" +- module_role_exists: "Service.ServiceEngineer" +- module_role_exists: "Service.FabUser" +- navigation_has_item: true +- file_exists: "tests/*.test.mdl" +- tests_pass: "tests --local" +- mx_check_passes: true + +## Acceptance Criteria +- Starting the workflow on a request produces an "Assign engineer" task for a ServiceEngineer +- Completing that task runs ACT_Request_Assign, so a work order exists and the request is Assigned +- Completing a work order sets TotalCost from its parts +- OpenRequestsPerFab returns one row per fab with that fab's request count +- FabUser can create a service request but cannot edit a work order +- The microflow test exercises SUB_WorkOrder_TotalCost, not the action microflow + +## Note on the checks +`workflow_exists` and `entity_has_type` were added for this brief +(`evalrunner/checks.go`). Both close a hole rather than add convenience: +`microflow_exists` reads `SHOW MICROFLOWS` and never sees a workflow, so before +`workflow_exists` a workflow was scored only by `mx_check_passes` — "it +compiled", not "it is there"; and `entity_exists` matches on the qualified name +alone, so `entity_has_type: "… View"` is the only thing that fails a run which +builds an ordinary persistent entity where a view entity was asked for. + +`enumeration_exists` is still not a check kind, so the three enumerations are +covered indirectly by `entity_has_attribute` on the attributes that use them. If +a run passes those while building plain strings instead, that is a gap in the +instrument, not a pass — add the check kind rather than loosening the brief. diff --git a/mdl-examples/bug-tests/1106-create-or-modify-edit-inside-loop-body.mdl b/mdl-examples/bug-tests/1106-create-or-modify-edit-inside-loop-body.mdl new file mode 100644 index 0000000000..c3df1a7a6f --- /dev/null +++ b/mdl-examples/bug-tests/1106-create-or-modify-edit-inside-loop-body.mdl @@ -0,0 +1,44 @@ +-- ============================================================================ +-- ako/mxcli#1106 — create or modify cannot change anything inside a loop body +-- ============================================================================ +-- +-- Under `mdl 1;`, re-running a `create or modify microflow` whose only change +-- was one expression inside a loop body was refused: +-- +-- the Loop at (1295, 200) changes inside its body; the splice does not edit +-- inside a loop, and replacing the whole loop would rebuild every node it +-- holds. Nothing was written. [MDL-V1-REBUILD] +-- +-- so no single script could both provision and update a flow that loops. +-- +-- To reproduce: exec this file against a project, then change +-- `formatDecimal($It/Amount, '0.#')` to `toString(round($It/Amount, 1))` and +-- exec it again. The second run now reports +-- Modified microflow: BugTest1106.BuildJson (spliced: 1 replaced) +-- and a third run writes nothing. The loop and the other activities keep their +-- $IDs and positions; only the edited `set` is replaced, in place. +-- +-- Still refused (needs the body laid out again): a statement added, removed, +-- reordered or moved inside the body, or one kind of statement swapped for +-- another. For those, `alter … replace loop $It in $Items with begin … end;`. + +mdl 1; + +create module BugTest1106; + +create or modify entity BugTest1106.Item ( + Amount: Decimal +); + +create or modify microflow BugTest1106.BuildJson ($Items: List of BugTest1106.Item) +returns String as $Json +begin + declare $Json string = '['; + loop $It in $Items + begin + set $Json = $Json + formatDecimal($It/Amount, '0.#') + ','; + log info node 'BugTest1106' 'step'; + end loop; + set $Json = $Json + ']'; + return $Json; +end; diff --git a/mdl-examples/bug-tests/pages-ledger170-submit-behaviour-grid-width.mdl b/mdl-examples/bug-tests/pages-ledger170-submit-behaviour-grid-width.mdl new file mode 100644 index 0000000000..ef25ec2829 --- /dev/null +++ b/mdl-examples/bug-tests/pages-ledger170-submit-behaviour-grid-width.mdl @@ -0,0 +1,42 @@ +mdl 1; +-- ============================================================================ +-- ako/mxcli-ledger FINDINGS #170 — `describe page` → `exec` reset two stored +-- values MDL could not express, `mx check` clean on both sides: +-- +-- Forms$TextBox / Forms$TextArea .SubmitBehaviour WhileEditing → OnEndEditing +-- Forms$TextBox / Forms$TextArea .SubmitOnInputDelay → 300 +-- Forms$LayoutGrid.Width FixedWidth → FullWidth +-- +-- And `check` accepted `SubmitBehaviour: WhileEditing` (MDL-WIDGET07 only +-- warns) while exec dropped it; `Width: FixedWidth` on a layout grid did not +-- even warn. +-- +-- EXPECTED: exec stores WhileEditing / 500 / FixedWidth, `describe page` prints +-- them back, and re-executing the description writes nothing. +-- +-- As with the #550 bug-test, this file is a reproduction scaffold: mxcli +-- round-tripping its own output agrees with itself. The detector is +-- TestSubmitBehaviourAndGridWidth_StudioProPageRoundTrips over PedApp's Studio +-- Pro-authored FeedbackModule.ShareFeedback and MyFirstModule.Home_Web. +-- ============================================================================ + +create module Ledger170; + +create non-persistent entity Ledger170.Search (Query: string(200), Notes: string(2000)); + +create page Ledger170.Search_Live ( + title: 'Search', + layout: Atlas_Core.Atlas_Default, + params: ( $Search: Ledger170.Search ) +) { + layoutgrid gridFixed (Width: FixedWidth) { + row { + column (DesktopWidth: 12) { + dataview dvSearch (DataSource: $Search) { + textbox tbQuery (Label: 'Query', Attribute: Query, SubmitBehaviour: WhileEditing, SubmitOnInputDelay: 500) + textarea taNotes (Label: 'Notes', Attribute: Notes, SubmitBehaviour: WhileEditing) + } + } + } + } +}; diff --git a/mdl-examples/doctype-tests/03-page-examples.mdl b/mdl-examples/doctype-tests/03-page-examples.mdl index d3404cc1c4..ba51fd4eea 100644 --- a/mdl-examples/doctype-tests/03-page-examples.mdl +++ b/mdl-examples/doctype-tests/03-page-examples.mdl @@ -2766,3 +2766,31 @@ create page PgTest.P014_InertControls ( dynamictext txtCard (Content: 'A card that is not clickable') } }; + +-- ============================================================================ +-- MARK: Submit while typing, and a fixed-width layout grid +-- ============================================================================ +-- SubmitBehaviour (OnEndEditing default | WhileEditing) and SubmitOnInputDelay +-- (milliseconds, default 300) are a text box's and a text area's; Width +-- (FullWidth default | FixedWidth) is a layout grid's. describe prints each only +-- when it is not the default. Before ako/mxcli-ledger FINDINGS #170 the writer +-- hard-coded the defaults, so describe -> exec reset a Studio Pro page's +-- WhileEditing and FixedWidth, and check accepted the property and dropped it. +create page PgTest.P043_SubmitWhileTyping ( + Title: 'Search customers', + Layout: Atlas_Core.Atlas_Default, + Params: ( $Customer: PgTest.Customer ) +) +{ + layoutgrid gridFixed (Width: FixedWidth) { + row { + column (DesktopWidth: 12) { + dataview dvCustomer (DataSource: $Customer) { + textbox tbName (Label: 'Name', Attribute: Name, SubmitBehaviour: WhileEditing, SubmitOnInputDelay: 500) + textarea taAddress (Label: 'Address', Attribute: Address, SubmitBehaviour: WhileEditing) + textbox tbEmail (Label: 'Email', Attribute: Email) + } + } + } + } +}; diff --git a/mdl/backend/mcp/page_submit_width_test.go b/mdl/backend/mcp/page_submit_width_test.go new file mode 100644 index 0000000000..ce33049924 --- /dev/null +++ b/mdl/backend/mcp/page_submit_width_test.go @@ -0,0 +1,40 @@ +// SPDX-License-Identifier: Apache-2.0 + +package mcp + +import ( + "testing" + + "github.com/mendixlabs/mxcli/sdk/pages" +) + +// ako/mxcli-ledger FINDINGS #170 on the MCP path: an authored submit behaviour +// and grid width are sent; unset sends what it sent before. +func TestMapPageWidget_SubmitBehaviourAndGridWidth(t *testing.T) { + b := &Backend{} + delay := 500 + tb := &pages.TextBox{AttributePath: "M.E.Name", SubmitBehaviour: "WhileEditing", SubmitOnInputDelay: &delay} + tb.Name = "tb" + m, err := b.mapPageWidget(tb) + if err != nil || m["submitBehaviour"] != "WhileEditing" || m["submitOnInputDelay"] != 500 { + t.Fatalf("textbox: %+v / %v", m, err) + } + ta := &pages.TextArea{AttributePath: "M.E.Name", SubmitBehaviour: "WhileEditing"} + ta.Name = "ta" + if m, err = b.mapPageWidget(ta); err != nil || m["submitBehaviour"] != "WhileEditing" { + t.Fatalf("textarea: %+v / %v", m, err) + } + if _, sent := m["submitOnInputDelay"]; sent { + t.Errorf("textarea: an unauthored delay was sent: %+v", m) + } + plain := &pages.TextBox{AttributePath: "M.E.Name"} + plain.Name = "plain" + if m, _ = b.mapPageWidget(plain); m["submitBehaviour"] != nil || m["submitOnInputDelay"] != nil { + t.Errorf("plain textbox gained submit settings: %+v", m) + } + grid := &pages.LayoutGrid{Width: "FixedWidth"} + grid.Name = "lg" + if m, err = b.mapPageWidget(grid); err != nil || m["width"] != "FixedWidth" { + t.Fatalf("layout grid: %+v / %v", m, err) + } +} diff --git a/mdl/backend/mcp/page_widgets.go b/mdl/backend/mcp/page_widgets.go index 03c7e26160..c8a032324a 100644 --- a/mdl/backend/mcp/page_widgets.go +++ b/mdl/backend/mcp/page_widgets.go @@ -181,7 +181,7 @@ func (b *Backend) mapPageWidgetBody(w pages.Widget) (map[string]any, error) { "name": wd.Name, "appearance": pageAppearance(wd.Class, wd.Style), "tabIndex": wd.TabIndex, - "width": "FullWidth", + "width": layoutGridWidth(wd.Width), "rows": rows, }, nil case *pages.ListView: @@ -236,13 +236,15 @@ func (b *Backend) mapPageWidgetBody(w pages.Widget) (map[string]any, error) { case *pages.DataGrid: return nil, fmt.Errorf("legacy DataGrid is not supported by the MCP backend — pg_patch_page has no Pages$DataGrid type (use a ListView, or DataGrid 2 which is a pluggable widget)") case *pages.TextBox: - return withFormattingInfo(inputWidget("Pages$TextBox", wd.Name, wd.Label, wd.AttributePath, wd.Class, wd.Style, wd.SourceVariable), wd.FormattingInfo), nil + w := withFormattingInfo(inputWidget("Pages$TextBox", wd.Name, wd.Label, wd.AttributePath, wd.Class, wd.Style, wd.SourceVariable), wd.FormattingInfo) + return withSubmitBehaviour(w, wd.SubmitBehaviour, wd.SubmitOnInputDelay), nil case *pages.CheckBox: return inputWidget("Pages$CheckBox", wd.Name, wd.Label, wd.AttributePath, wd.Class, wd.Style, wd.SourceVariable), nil case *pages.DatePicker: return withFormattingInfo(inputWidget("Pages$DatePicker", wd.Name, wd.Label, wd.AttributePath, wd.Class, wd.Style, wd.SourceVariable), wd.FormattingInfo), nil case *pages.TextArea: - return inputWidget("Pages$TextArea", wd.Name, wd.Label, wd.AttributePath, wd.Class, wd.Style, wd.SourceVariable), nil + w := inputWidget("Pages$TextArea", wd.Name, wd.Label, wd.AttributePath, wd.Class, wd.Style, wd.SourceVariable) + return withSubmitBehaviour(w, wd.SubmitBehaviour, wd.SubmitOnInputDelay), nil case *pages.RadioButtons: return inputWidget("Pages$RadioButtonGroup", wd.Name, wd.Label, wd.AttributePath, wd.Class, wd.Style, wd.SourceVariable), nil default: @@ -293,6 +295,27 @@ func withFormattingInfo(w map[string]any, fi *pages.FormattingInfo) map[string]a return w } +// withSubmitBehaviour adds a text box's or text area's authored submit +// behaviour and input delay. Unset leaves both to Studio Pro's defaults +// (OnEndEditing, 300), as before (ako/mxcli-ledger FINDINGS #170). +func withSubmitBehaviour(w map[string]any, behaviour string, delay *int) map[string]any { + if behaviour != "" { + w["submitBehaviour"] = behaviour + } + if delay != nil { + w["submitOnInputDelay"] = *delay + } + return w +} + +// layoutGridWidth is a layout grid's authored Width, or Studio Pro's default. +func layoutGridWidth(width string) string { + if width == "" { + return pages.DefaultLayoutGridWidth + } + return width +} + // pageVariable builds a Pages$PageVariable naming a data view and, beside it, // the data view's own variable. Studio Pro 11.14 fills that variable in itself // when given only the widget, so sending it is the same document either way. diff --git a/mdl/backend/mfmutator/splice.go b/mdl/backend/mfmutator/splice.go index f3897240d8..cc3481f019 100644 --- a/mdl/backend/mfmutator/splice.go +++ b/mdl/backend/mfmutator/splice.go @@ -25,8 +25,9 @@ package mfmutator // What the splice cannot do safely it refuses, with the reason: an insert // after a decision (which branch?), before an activity several flows enter // (which path?), a drop or replace of an activity with an error handler (the -// handler would be orphaned), and — for now — anything inside a loop body, -// whose coordinates are relative to the loop box. +// handler would be orphaned), and — for now — anything inside a loop body +// but an activity replaced by one of the same kind, in place, since a body +// node's coordinates are relative to the loop box (ako/mxcli#1106). import ( "bytes" @@ -393,6 +394,9 @@ func fragmentNode(frag *backend.MicroflowFragment, id model.ID) *node { // annotations attached to target are attached to the fragment's entry. func (m *Mutator) Replace(target model.ID, frag *backend.MicroflowFragment) error { g := m.graph() + if n, ok := g.nodes[string(target)]; ok && n.loop != "" { + return m.replaceInLoop(g, n, frag) + } x, err := g.node(target) if err != nil { return err @@ -453,6 +457,81 @@ func (m *Mutator) Replace(target model.ID, frag *backend.MicroflowFragment) erro return m.removeObject(x) } +// replaceInLoop replaces an activity inside a loop body by one activity of +// the same kind, in place (ako/mxcli#1106): the new activity takes the stored +// one's place in the loop's collection, is drawn where it was at the size it +// was, and the flows that entered and left it now enter and leave the new one. +// Nothing else in the loop moves, so no layout is needed — which is why this +// is the one edit inside a loop body the splice makes. A body node's +// coordinates are relative to the loop box, so anything that adds, removes or +// redraws a node there would have to re-lay the body out; that is refused. +func (m *Mutator) replaceInLoop(g *graph, x *node, frag *backend.MicroflowFragment) error { + if frag == nil || len(frag.Objects) != 1 || len(frag.Flows) > 0 || len(frag.AnnotationFlows) > 0 || + frag.ExitCase != nil || frag.Entry != frag.Objects[0].GetID() || frag.Exit != frag.Entry { + return fmt.Errorf("%s is inside a loop body, where the splice replaces an activity by one activity "+ + "only (nothing in the body is added, removed or redrawn); replace the whole loop instead", describeNode(x)) + } + if x.typ != "Microflows$ActionActivity" { + return fmt.Errorf("cannot replace %s in place: only an activity can be", describeNode(x)) + } + // The body's last activity ends it, with no flow out. + normal, errs := g.outgoing(x.id) + if len(errs) > 0 { + return fmt.Errorf("cannot replace %s: it has an error handler, which would be left with no activity; "+ + "replace the whole loop instead", describeNode(x)) + } + if len(normal) > 1 { + return fmt.Errorf("cannot replace %s: it has %d outgoing flows, not one", describeNode(x), len(normal)) + } + loop := g.nodes[x.loop] + if loop == nil { + return fmt.Errorf("%s is in a loop that is not in the flow", describeNode(x)) + } + oc := dDoc(loop.doc, "ObjectCollection") + obj := frag.Objects[0] + obj.SetPosition(model.Point{X: x.pos.X, Y: x.pos.Y}) + d, err := m.deps.SerializeObject(obj) + if err != nil { + return fmt.Errorf("serialize %T: %w", obj, err) + } + if d == nil { + return fmt.Errorf("cannot write a %T into a flow", obj) + } + if t := dString(d, "$Type"); t != x.typ { + return fmt.Errorf("%s is inside a loop body, where the splice replaces an activity by one of the same "+ + "kind only, not by a %s; replace the whole loop instead", describeNode(x), t) + } + // Drawn as the stored one is: a Studio Pro activity may have been resized. + dSet(d, "RelativeMiddlePoint", x.pos.String()) + dSet(d, "Size", x.size.String()) + objs := arrayElements(dGet(oc, "Objects")) + found := false + for i, el := range objs { + if od, ok := el.(bson.D); ok && binaryID(dGet(od, "$ID")) == x.id { + objs[i], found = d, true + break + } + } + if !found || !setArray(oc, "Objects", objs) { + return fmt.Errorf("%s is not in its loop's collection", describeNode(x)) + } + m.markRemoved(x.doc) + for _, in := range g.incoming(x.id) { + setPointer(in.doc, "DestinationPointer", frag.Entry) + } + for _, out := range normal { + setPointer(out.doc, "OriginPointer", frag.Exit) + } + for _, af := range g.annotationFlows(x.id) { + key := "DestinationPointer" + if af.origin == x.id { + key = "OriginPointer" + } + setPointer(af.doc, key, frag.Entry) + } + return nil +} + // setCase gives a stored flow the case value cv, in the form the engine // writes it: the case properties of a flow serialized with cv replace the // stored ones, and the flow keeps its $ID and everything else. The stored diff --git a/mdl/backend/mfmutator/splice_test.go b/mdl/backend/mfmutator/splice_test.go index 49cabcb28a..4a8a07c39d 100644 --- a/mdl/backend/mfmutator/splice_test.go +++ b/mdl/backend/mfmutator/splice_test.go @@ -510,3 +510,84 @@ func TestSplice_RemoveNotesBeforeAReplace(t *testing.T) { } }) } + +// An activity inside a loop body is replaced by one activity in place +// (ako/mxcli#1106): the new one takes the stored one's slot in the loop's +// collection, its position and its size (a Studio Pro activity may have been +// resized), and the body flow through it now runs through the new one. The +// last body activity has no flow out. Nothing else in the unit changes. +func TestSplice_ReplaceInsideALoopIsInPlace(t *testing.T) { + build := func() ([]bson.D, []bson.D) { + inner := obj("inner", "Microflows$ActionActivity", 100, 60) + dSet(inner, "Size", "200;60") + loop := obj("loop", "Microflows$LoopedActivity", 420, 200) + loop = append(loop, bson.E{Key: "ObjectCollection", Value: bson.D{ + {Key: "$ID", Value: bin("loopoc")}, + {Key: "$Type", Value: "Microflows$MicroflowObjectCollection"}, + {Key: "Objects", Value: bson.A{int32(3), inner, obj("inner2", "Microflows$ActionActivity", 260, 60)}}, + }}) + return []bson.D{obj("start", "Microflows$StartEvent", 100, 200), obj("a", "Microflows$ActionActivity", 250, 200), loop}, + []bson.D{flow("f1", "start", "a", 1, 3, false), flow("f2", "a", "loop", 1, 3, false), flow("fi", "inner", "inner2", 1, 3, false)} + } + for _, tc := range []struct{ target, pos, size string }{ + {"inner", "100;60", "200;60"}, + {"inner2", "260;60", "120;60"}, + } { + t.Run(tc.target, func(t *testing.T) { + objs, flows := build() + m, deps := newMutator(t, unit(objs, flows)) + frag := oneActivity() + if err := m.Replace(model.ID(uid(tc.target)), frag); err != nil { + t.Fatal(err) + } + if err := m.Save(); err != nil { + t.Fatalf("save: %v", err) + } + if bytes.Contains(deps.saved, types.UUIDToBlob(uid(tc.target))) { + t.Errorf("%s is still in the unit", tc.target) + } + // What the replace should leave: the stored unit with the + // target's document swapped, and the body flow repointed. + newID := primitive.Binary{Data: types.UUIDToBlob(string(frag.Entry))} + wantObjs, wantFlows := build() + oc := dDoc(wantObjs[2], "ObjectCollection") + body := arrayElements(dGet(oc, "Objects")) + for i, el := range body { + if d := el.(bson.D); binaryID(dGet(d, "$ID")) == uid(tc.target) { + body[i] = bson.D{ + {Key: "$ID", Value: newID}, + {Key: "$Type", Value: "Microflows$ActionActivity"}, + {Key: "RelativeMiddlePoint", Value: tc.pos}, + {Key: "Size", Value: tc.size}, + } + } + } + setArray(oc, "Objects", body) + key := "OriginPointer" + if tc.target == "inner2" { + key = "DestinationPointer" + } + dSet(wantFlows[2], key, newID) + raw, _ := bson.Marshal(unit(wantObjs, wantFlows)) + if !bytes.Equal(deps.saved, raw) { + var got bson.D + _ = bson.Unmarshal(deps.saved, &got) + t.Errorf("the replace changed more than the activity's document and the flow through it:\n got %v\nwant %v", got, unit(wantObjs, wantFlows)) + } + }) + } + + // Two activities for one would change the body's shape: refused. + t.Run("two for one refused", func(t *testing.T) { + objs, flows := build() + m, _ := newMutator(t, unit(objs, flows)) + frag := oneActivity() + second := oneActivity() + frag.Objects = append(frag.Objects, second.Objects...) + frag.Exit = second.Exit + err := m.Replace(model.ID(uid("inner")), frag) + if err == nil || !strings.Contains(err.Error(), "replace the whole loop") { + t.Fatalf("want the in-loop refusal, got %v", err) + } + }) +} diff --git a/mdl/backend/modelsdk/consumed_rest_isdefaulttype_test.go b/mdl/backend/modelsdk/consumed_rest_isdefaulttype_test.go new file mode 100644 index 0000000000..fef20c662a --- /dev/null +++ b/mdl/backend/modelsdk/consumed_rest_isdefaulttype_test.go @@ -0,0 +1,85 @@ +// SPDX-License-Identifier: Apache-2.0 + +package modelsdkbackend + +import ( + "testing" + + "github.com/mendixlabs/mxcli/model" + "github.com/mendixlabs/mxcli/modelsdk/codec" + "github.com/mendixlabs/mxcli/modelsdk/element" + "go.mongodb.org/mongo-driver/v2/bson" +) + +// ako/mxcli#1097. The consumed-REST inline mapping writer (Body: MAPPING and +// Response: MAPPING) still wrote IsDefaultType on every +// Import/ExportMappings$ValueMappingElement after #882 fixed the standalone +// import/export-mapping writer — see TestMappingValueElementsOmitIsDefaultType +// for the authority. Mendix declares isDefaultType on the OBJECT mapping element +// only, so `mx diff`/merge throws on the undeclared key. +func TestRestInlineMappingValueElementsOmitIsDefaultType(t *testing.T) { + mappings := []*model.RestResponseMapping{ + {Attribute: "Name", ExposedName: "name"}, + { + Entity: "M.Child", Association: "M.Child_Root", ExposedName: "child", + Children: []*model.RestResponseMapping{ + {Attribute: "Code", ExposedName: "code"}, + }, + }, + } + + for _, tc := range []struct { + name string + g element.Element + }{ + {"response", restImplicitMappingResponseToGen("M.Root", mappings)}, + {"body", restImplicitMappingBodyToGen("M.Root", mappings)}, + } { + raw, err := (&codec.Encoder{}).Encode(tc.g) + if err != nil { + t.Fatalf("%s: encode: %v", tc.name, err) + } + root, ok := bson.Raw(raw).Lookup("RootMappingElement").DocumentOK() + if !ok { + t.Fatalf("%s: no RootMappingElement", tc.name) + } + var objects, values int + var walk func(doc bson.Raw) + walk = func(doc bson.Raw) { + _, present := doc.Lookup("IsDefaultType").BooleanOK() + switch rawStr(doc, "ElementType") { + case "Object": + objects++ + if !present { + t.Errorf("%s: OBJECT element %q lost IsDefaultType — it owns the property", + tc.name, rawStr(doc, "JsonPath")) + } + case "Value": + values++ + if present { + t.Errorf("%s: VALUE element %q carries IsDefaultType, which "+ + "ValueMappingElement does not declare — mx diff/merge throws on it", + tc.name, rawStr(doc, "JsonPath")) + } + } + children, ok := doc.Lookup("Children").ArrayOK() + if !ok { + return + } + vals, err := children.Values() + if err != nil { + t.Fatalf("%s: read Children: %v", tc.name, err) + } + for _, v := range vals { + if child, ok := v.DocumentOK(); ok { + walk(child) + } + } + } + walk(root) + if objects != 2 || values != 2 { + t.Errorf("%s: walked %d object / %d value elements, want 2 / 2 — the assertion proved nothing", + tc.name, objects, values) + } + } +} diff --git a/mdl/backend/modelsdk/consumed_rest_write.go b/mdl/backend/modelsdk/consumed_rest_write.go index 3a1f903d1e..79f97b3c90 100644 --- a/mdl/backend/modelsdk/consumed_rest_write.go +++ b/mdl/backend/modelsdk/consumed_rest_write.go @@ -369,7 +369,8 @@ func restInlineMappingElementToGen(entity, association, exposedName, jsonPath st addInt32(v, "MinOccurs", 0) addInt32(v, "MaxOccurs", 1) addBool(v, "Nillable", true) - addBool(v, "IsDefaultType", false) + // No IsDefaultType: Mendix declares it on the object mapping element + // only, and mx diff/merge throws on it here (#1097; see #882). addStr(v, "ElementType", "Value") addStr(v, "Documentation", "") addStr(v, "Converter", "") diff --git a/mdl/backend/modelsdk/page_write.go b/mdl/backend/modelsdk/page_write.go index 1f6f3d7a83..2660a5146a 100644 --- a/mdl/backend/modelsdk/page_write.go +++ b/mdl/backend/modelsdk/page_write.go @@ -79,6 +79,7 @@ func (b *Backend) UpdatePage(page *pages.Page) error { contents, err := encodePage(page, pv, func(g *genPg.Page) { b.carryStoredPageHeader(page.ID, g, pv) b.carryNoActionExecution(page.ID, g) + b.carryLayoutGridPreviewWidth(page.ID, g) }) if err != nil { return fmt.Errorf("UpdatePage: encode: %w", err) diff --git a/mdl/backend/modelsdk/snippet_write.go b/mdl/backend/modelsdk/snippet_write.go index 3a1c02f8aa..a378b01d58 100644 --- a/mdl/backend/modelsdk/snippet_write.go +++ b/mdl/backend/modelsdk/snippet_write.go @@ -98,6 +98,7 @@ func (b *Backend) UpdateSnippet(snippet *pages.Snippet) error { contents, err := encodeSnippet(snippet, b.ProjectVersion(), func(g *genPg.Snippet) { b.carryStoredSnippetHeader(snippet.ID, g) b.carryNoActionExecution(snippet.ID, g) + b.carryLayoutGridPreviewWidth(snippet.ID, g) }) if err != nil { return fmt.Errorf("UpdateSnippet: encode: %w", err) diff --git a/mdl/backend/modelsdk/widget_layoutgrid_carry.go b/mdl/backend/modelsdk/widget_layoutgrid_carry.go new file mode 100644 index 0000000000..efd65f0590 --- /dev/null +++ b/mdl/backend/modelsdk/widget_layoutgrid_carry.go @@ -0,0 +1,108 @@ +// SPDX-License-Identifier: Apache-2.0 + +package modelsdkbackend + +import ( + "go.mongodb.org/mongo-driver/bson" + + "github.com/mendixlabs/mxcli/mdl/backend/bsonnav" + "github.com/mendixlabs/mxcli/model" + "github.com/mendixlabs/mxcli/modelsdk/element" + genPg "github.com/mendixlabs/mxcli/modelsdk/gen/pages" +) + +// carryLayoutGridPreviewWidth copies Forms$LayoutGridColumn.PreviewWidth from +// the stored document onto the rebuilt columns. +// +// PreviewWidth is a Studio Pro canvas value — the width a column is drawn at in +// the editor — with no MDL spelling and no runtime effect. layoutGridColumnToGen +// writes -1, which is what Studio Pro stores on 731 of 732 columns across +// ako/TestApp and PedApp, so the hardcode looked safe; the one column that +// stored 1 (ako/mxcli-ledger FINDINGS #170) was rewritten by every +// describe → exec. Carried for the reason carryStoredPageHeader gives: a value +// nobody asked to change should not change. +// +// A column has no name, so it is paired by position inside the stored grid of +// the same name: row i, column j. A row whose column count differs from the +// stored row's is not carried — its columns cannot be paired — and a column of +// a grid with no stored counterpart keeps -1. An ambiguous grid name is skipped. +// +// A missing or unreadable stored unit leaves the rebuilt values in place. +func (b *Backend) carryLayoutGridPreviewWidth(id model.ID, root element.Element) { + if b.reader == nil || id == "" || root == nil { + return + } + raw, err := b.reader.GetRawUnitBytes(string(id)) + if err != nil { + return + } + var stored bson.D + if err := bson.Unmarshal(raw, &stored); err != nil { + return + } + carryLayoutGridPreviewWidthFrom(stored, root) +} + +// carryLayoutGridPreviewWidthFrom is carryLayoutGridPreviewWidth against an +// already decoded stored document. +func carryLayoutGridPreviewWidthFrom(stored bson.D, root element.Element) { + storedWidgets := map[string]bson.D{} + ambiguous := map[string]bool{} + indexNamedDocs(stored, storedWidgets, ambiguous) + + element.Walk(root, func(e element.Element) bool { + grid, ok := e.(*genPg.LayoutGrid) + if !ok || grid.Name() == "" || ambiguous[grid.Name()] { + return true + } + sg, ok := storedWidgets[grid.Name()] + if !ok || bsonnav.DGetString(sg, "$Type") != "Forms$LayoutGrid" { + return true + } + storedRows := bsonnav.DGetArrayElements(bsonnav.DGet(sg, "Rows")) + for i, r := range grid.RowsItems() { + row, ok := r.(*genPg.LayoutGridRow) + if !ok || i >= len(storedRows) { + continue + } + sr, ok := storedRows[i].(bson.D) + if !ok { + continue + } + storedCols := bsonnav.DGetArrayElements(bsonnav.DGet(sr, "Columns")) + cols := row.ColumnsItems() + if len(cols) != len(storedCols) { + continue + } + for j, c := range cols { + col, ok := c.(*genPg.LayoutGridColumn) + if !ok { + continue + } + sc, ok := storedCols[j].(bson.D) + if !ok { + continue + } + // Only a differing value is set: setting an equal one marks the + // column dirty and re-encodes an element that was kept (#721 L4). + if v, ok := storedInt32(bsonnav.DGet(sc, "PreviewWidth")); ok && col.PreviewWidth() != v { + col.SetPreviewWidth(v) + } + } + } + return true + }) +} + +// storedInt32 reads a BSON integer as int32, whichever width it was stored at. +func storedInt32(v any) (int32, bool) { + switch x := v.(type) { + case int32: + return x, true + case int64: + return int32(x), true + case int: + return int32(x), true + } + return 0, false +} diff --git a/mdl/backend/modelsdk/widget_layoutgrid_carry_test.go b/mdl/backend/modelsdk/widget_layoutgrid_carry_test.go new file mode 100644 index 0000000000..670ed63aa8 --- /dev/null +++ b/mdl/backend/modelsdk/widget_layoutgrid_carry_test.go @@ -0,0 +1,138 @@ +// SPDX-License-Identifier: Apache-2.0 + +package modelsdkbackend + +import ( + "testing" + + "go.mongodb.org/mongo-driver/bson" + + "github.com/mendixlabs/mxcli/modelsdk/codec" + "github.com/mendixlabs/mxcli/modelsdk/element" + genPg "github.com/mendixlabs/mxcli/modelsdk/gen/pages" + "github.com/mendixlabs/mxcli/sdk/pages" +) + +func storedGridColumn(previewWidth int32) bson.D { + return bson.D{ + {Key: "$Type", Value: "Forms$LayoutGridColumn"}, + {Key: "PreviewWidth", Value: previewWidth}, + {Key: "Widgets", Value: bson.A{int32(2)}}, + } +} + +func storedGrid(name string, rows ...bson.A) bson.D { + rowDocs := bson.A{int32(2)} + for _, cols := range rows { + rowDocs = append(rowDocs, bson.D{ + {Key: "$Type", Value: "Forms$LayoutGridRow"}, + {Key: "Columns", Value: append(bson.A{int32(2)}, cols...)}, + }) + } + return bson.D{ + {Key: "$Type", Value: "Forms$LayoutGrid"}, + {Key: "Name", Value: name}, + {Key: "Rows", Value: rowDocs}, + } +} + +func builtGrid(t *testing.T, name string, colsPerRow ...int) *genPg.LayoutGrid { + t.Helper() + lg := &pages.LayoutGrid{BaseWidget: pages.BaseWidget{Name: name}} + for _, n := range colsPerRow { + row := &pages.LayoutGridRow{} + for i := 0; i < n; i++ { + row.Columns = append(row.Columns, &pages.LayoutGridColumn{}) + } + lg.Rows = append(lg.Rows, row) + } + el, err := widgetToGen(lg) + if err != nil { + t.Fatal(err) + } + return el.(*genPg.LayoutGrid) +} + +func previewWidths(g *genPg.LayoutGrid) [][]int32 { + var out [][]int32 + for _, r := range g.RowsItems() { + var row []int32 + for _, c := range r.(*genPg.LayoutGridRow).ColumnsItems() { + row = append(row, c.(*genPg.LayoutGridColumn).PreviewWidth()) + } + out = append(out, row) + } + return out +} + +// Forms$LayoutGridColumn.PreviewWidth is a Studio Pro canvas value with no MDL +// spelling and no runtime effect. The writer hardcoded -1, so a describe → exec +// of a page whose column stored 1 rewrote it (ako/mxcli-ledger FINDINGS #170). +// The stored value is carried onto the rebuilt column at the same position of +// the same-named grid. +func TestCarryLayoutGridPreviewWidth(t *testing.T) { + stored := bson.D{ + {Key: "$Type", Value: "Forms$Page"}, + {Key: "Widgets", Value: bson.A{int32(2), + storedGrid("lg1", bson.A{storedGridColumn(1), storedGridColumn(-1)}), + }}, + } + g := builtGrid(t, "lg1", 2) + other := builtGrid(t, "lgNew", 2) + root := genPg.NewDataView() + root.SetName("dv") + root.AddWidgets(g) + root.AddWidgets(other) + + carryLayoutGridPreviewWidthFrom(stored, root) + + if got := previewWidths(g); got[0][0] != 1 || got[0][1] != -1 { + t.Errorf("lg1 preview widths = %v, want [[1 -1]] carried from the stored grid", got) + } + // Control: a grid with no stored counterpart keeps the writer's -1. + if got := previewWidths(other); got[0][0] != -1 || got[0][1] != -1 { + t.Errorf("lgNew preview widths = %v; a new grid took a value from nowhere", got) + } +} + +// A row whose column count no longer matches the stored one is not carried: +// the columns cannot be paired, and a new column keeps -1. +func TestCarryLayoutGridPreviewWidth_ReshapedRowKeepsDefault(t *testing.T) { + stored := bson.D{ + {Key: "$Type", Value: "Forms$Page"}, + {Key: "Widgets", Value: bson.A{int32(2), + storedGrid("lg1", bson.A{storedGridColumn(1)}), + }}, + } + g := builtGrid(t, "lg1", 2) + carryLayoutGridPreviewWidthFrom(stored, g) + if got := previewWidths(g); got[0][0] != -1 || got[0][1] != -1 { + t.Errorf("preview widths = %v, want [[-1 -1]] for a reshaped row", got) + } +} + +// Carrying the value a decoded column already holds must leave it clean, so a +// kept element is not re-encoded (the #721 L4 rule carryNoActionExecution keeps). +func TestCarryLayoutGridPreviewWidth_EqualValueLeavesClean(t *testing.T) { + stored := bson.D{ + {Key: "$Type", Value: "Forms$Page"}, + {Key: "Widgets", Value: bson.A{int32(2), + storedGrid("lg1", bson.A{storedGridColumn(-1)}), + }}, + } + raw, err := bson.Marshal(storedGrid("lg1", bson.A{storedGridColumn(-1)})) + if err != nil { + t.Fatal(err) + } + kept, err := codec.NewDecoder(codec.DefaultRegistry).Decode(raw) + if err != nil { + t.Fatal(err) + } + carryLayoutGridPreviewWidthFrom(stored, kept) + element.Walk(kept, func(e element.Element) bool { + if e.IsDirty() { + t.Errorf("%s dirtied by carrying the value it already holds", e.TypeName()) + } + return true + }) +} diff --git a/mdl/backend/modelsdk/widget_list_marker_test.go b/mdl/backend/modelsdk/widget_list_marker_test.go new file mode 100644 index 0000000000..41423a5b44 --- /dev/null +++ b/mdl/backend/modelsdk/widget_list_marker_test.go @@ -0,0 +1,278 @@ +// SPDX-License-Identifier: Apache-2.0 + +package modelsdkbackend + +import ( + "testing" + + bsonv1 "go.mongodb.org/mongo-driver/bson" + + "github.com/mendixlabs/mxcli/modelsdk/codec" + genPg "github.com/mendixlabs/mxcli/modelsdk/gen/pages" + "github.com/mendixlabs/mxcli/modelsdk/mpr" + "github.com/mendixlabs/mxcli/sdk/pages" +) + +func arrayMarker(t *testing.T, doc bsonv1.D, key string) (int32, int) { + t.Helper() + for _, e := range doc { + if e.Key != key { + continue + } + arr, ok := e.Value.(bsonv1.A) + if !ok || len(arr) == 0 { + t.Fatalf("%s is %T %v, want a typed array", key, e.Value, e.Value) + } + m, _ := arr[0].(int32) + return m, len(arr) - 1 + } + t.Fatalf("%s missing from %v", key, docKeys(doc)) + return 0, 0 +} + +func subDocs(t *testing.T, doc bsonv1.D, key string) []bsonv1.D { + t.Helper() + for _, e := range doc { + if e.Key == key { + var out []bsonv1.D + for _, v := range e.Value.(bsonv1.A)[1:] { + out = append(out, v.(bsonv1.D)) + } + return out + } + } + t.Fatalf("%s missing", key) + return nil +} + +// Studio Pro writes the typed-array marker 2 on every `Widgets` list, whatever +// its first child is and whether or not it is empty: 22,000+ Widgets lists +// across ako/TestApp and PedApp, every owner type, not one exception — 8 of +// them a LayoutGridColumn whose first child is a Forms$TabControl. The encoder +// took the marker from the first child's $Type, and Forms$TabControl had no +// registration, so such a column was written with 3 and a describe → exec of +// an unchanged page rewrote it (ako/mxcli-ledger FINDINGS #170). +// +// The tab control's own TabPages list is the control: Studio Pro stores 3 +// there (32 of 32), and it must stay 3. +func TestWidgetsListMarker_TabControlFirstChildIsTwo(t *testing.T) { + lg := &pages.LayoutGrid{BaseWidget: pages.BaseWidget{Name: "lg"}} + lg.Rows = []*pages.LayoutGridRow{{Columns: []*pages.LayoutGridColumn{{ + Widgets: []pages.Widget{&pages.TabContainer{ + BaseWidget: pages.BaseWidget{Name: "tabs"}, + TabPages: []*pages.TabPage{ + {Name: "tp1", Widgets: []pages.Widget{&pages.TextBox{BaseWidget: pages.BaseWidget{Name: "tb"}}}}, + {Name: "tp2"}, + }, + }}, + }}}} + el, err := widgetToGen(lg) + if err != nil { + t.Fatal(err) + } + grid := encodeElement(t, el) + row := subDocs(t, grid, "Rows")[0] + col := subDocs(t, row, "Columns")[0] + if m, n := arrayMarker(t, col, "Widgets"); m != 2 || n != 1 { + t.Errorf("LayoutGridColumn.Widgets (first child Forms$TabControl) marker = %d (len %d), want 2", m, n) + } + tabs := subDocs(t, col, "Widgets")[0] + if m, _ := arrayMarker(t, tabs, "TabPages"); m != 3 { + t.Errorf("TabControl.TabPages marker = %d, want 3 as Studio Pro stores it", m) + } + tps := subDocs(t, tabs, "TabPages") + if m, _ := arrayMarker(t, tps[0], "Widgets"); m != 2 { + t.Errorf("TabPage.Widgets marker = %d, want 2", m) + } +} + +// An empty Widgets list is marker 2 as well (TabPage 20 of 20, LayoutGridColumn +// 13 of 13, DivContainer 8 of 8 empty lists in the same sweep), where the +// encoder's empty-list default is 3. Emptying a stored list rebuilds it whole, +// which is where the empty marker is written. +func TestWidgetsListMarker_EmptiedListIsTwo(t *testing.T) { + box := decodeStoredBox(t) + box.RemoveWidgets(1) + box.RemoveWidgets(0) + doc := encodeElement(t, box) + if m, n := arrayMarker(t, doc, "Widgets"); m != 2 || n != 0 { + t.Errorf("emptied Widgets = marker %d len %d, want [2]", m, n) + } +} + +func decodeStoredBox(t *testing.T) *genPg.DivContainer { + t.Helper() + stored := bsonv1.D{ + {Key: "$ID", Value: idBinaryForTest(t)}, + {Key: "$Type", Value: "Forms$DivContainer"}, + {Key: "Name", Value: "box"}, + {Key: "Widgets", Value: bsonv1.A{int32(2), + bsonv1.D{{Key: "$ID", Value: idBinaryForTest(t)}, {Key: "$Type", Value: "Forms$DivContainer"}, {Key: "Name", Value: "inner1"}, {Key: "Widgets", Value: bsonv1.A{int32(2)}}}, + bsonv1.D{{Key: "$ID", Value: idBinaryForTest(t)}, {Key: "$Type", Value: "Forms$DivContainer"}, {Key: "Name", Value: "inner2"}, {Key: "Widgets", Value: bsonv1.A{int32(2)}}}, + }}, + } + raw, err := bsonv1.Marshal(stored) + if err != nil { + t.Fatal(err) + } + el, err := codec.NewDecoder(codec.DefaultRegistry).Decode(raw) + if err != nil { + t.Fatal(err) + } + return el.(*genPg.DivContainer) +} + +// A stored container whose child changed is re-encoded selectively: clean +// children pass through, and the encoder writes the list's marker itself. That +// path used 3 for any list without a property registration, so editing one +// widget inside a stored Widgets list rewrote the list's marker from 2 to 3. +func TestWidgetsListMarker_SelectiveRebuildKeepsTwo(t *testing.T) { + box := decodeStoredBox(t) + box.WidgetsItems()[1].(*genPg.DivContainer).SetName("renamed") + doc := encodeElement(t, box) + if m, n := arrayMarker(t, doc, "Widgets"); m != 2 || n != 2 { + t.Errorf("selectively rebuilt Widgets = marker %d len %d, want marker 2 with both children", m, n) + } +} + +func idBinaryForTest(t *testing.T) any { + t.Helper() + return mpr.IDToBsonBinary(mpr.GenerateID()) +} + +// findDoc returns the first sub-document of $Type typeName, depth-first. +func findDoc(doc bsonv1.D, typeName string) bsonv1.D { + for _, e := range doc { + if e.Key == "$Type" && e.Value == typeName { + return doc + } + } + for _, e := range doc { + switch v := e.Value.(type) { + case bsonv1.D: + if d := findDoc(v, typeName); d != nil { + return d + } + case bsonv1.A: + for _, x := range v { + if d, ok := x.(bsonv1.D); ok { + if found := findDoc(d, typeName); found != nil { + return found + } + } + } + } + } + return nil +} + +// Studio Pro never omits a `Widgets` list: every owner in ako/TestApp and +// PedApp carries the key, empty or not (TabPage 89 of 89, 20 of them empty; +// LayoutGridColumn 732 of 732, 13 empty; DivContainer 838 of 838, 8 empty; +// CustomWidgets$WidgetValue 20,775 of 20,775). A freshly built container with +// no children wrote no Widgets key at all, so an empty tab page differed from +// the same page saved by Studio Pro. +func TestWidgetsListMarker_EmptyNewOwnerWritesTwo(t *testing.T) { + cases := []struct { + owner string + widget pages.Widget + }{ + {"Forms$TabPage", &pages.TabContainer{BaseWidget: pages.BaseWidget{Name: "tabs"}, TabPages: []*pages.TabPage{{Name: "tp"}}}}, + {"Forms$DivContainer", &pages.Container{BaseWidget: pages.BaseWidget{Name: "c"}}}, + {"Forms$GroupBox", &pages.GroupBox{BaseWidget: pages.BaseWidget{Name: "g"}}}, + {"Forms$LayoutGridColumn", &pages.LayoutGrid{BaseWidget: pages.BaseWidget{Name: "lg"}, Rows: []*pages.LayoutGridRow{{Columns: []*pages.LayoutGridColumn{{}}}}}}, + {"Forms$ScrollContainerRegion", &pages.ScrollContainer{BaseWidget: pages.BaseWidget{Name: "sc"}, Regions: []*pages.ScrollContainerRegion{{Slot: pages.ScrollSlotTop}}}}, + {"Forms$NavigationListItem", &pages.NavigationList{BaseWidget: pages.BaseWidget{Name: "nl"}, Items: []*pages.NavigationListItem{{}}}}, + {"Forms$DataView", &pages.DataView{BaseWidget: pages.BaseWidget{Name: "dv"}}}, + } + for _, c := range cases { + t.Run(c.owner, func(t *testing.T) { + el, err := widgetToGen(c.widget) + if err != nil { + t.Fatal(err) + } + owner := findDoc(encodeElement(t, el), c.owner) + if owner == nil { + t.Fatalf("no %s in the encoded widget", c.owner) + } + if m, n := arrayMarker(t, owner, "Widgets"); m != 2 || n != 0 { + t.Errorf("empty %s.Widgets = marker %d len %d, want [2]", c.owner, m, n) + } + }) + } +} + +// The control: the mandatory list is keyed on the owner, so a type Studio Pro +// stores without a Widgets key does not gain one. Neither Forms$TabControl nor +// Forms$ScrollContainer carries Widgets in any TestApp or PedApp document. +func TestWidgetsListMarker_NoWidgetsKeyOnNonOwners(t *testing.T) { + for _, w := range []pages.Widget{ + &pages.TabContainer{BaseWidget: pages.BaseWidget{Name: "tabs"}, TabPages: []*pages.TabPage{{Name: "tp"}}}, + &pages.ScrollContainer{BaseWidget: pages.BaseWidget{Name: "sc"}, Regions: []*pages.ScrollContainerRegion{{Slot: pages.ScrollSlotTop}}}, + } { + el, err := widgetToGen(w) + if err != nil { + t.Fatal(err) + } + doc := encodeElement(t, el) + if hasKey(doc, "Widgets") { + t.Errorf("%T wrote a Widgets key Studio Pro never stores on it", w) + } + } +} + +// Forms$TemplateGridContents holds a Widgets list too, but neither TestApp nor +// PedApp has a template grid, so #1104 left it on the child-type path. Measured +// with mx convert (Mendix 11.15.0), which re-serializes every unit: a template +// grid injected with its contents' Widgets written as [3] — empty, and with a +// Forms$TabControl first child — came back [2] both times, and a bare grid was +// given an empty Widgets [2] it did not have. Before the registration the +// encoder wrote 3 for the tab-control case and the emptied list, and no key at +// all for a new empty one. +func TestWidgetsListMarker_TemplateGridContents(t *testing.T) { + const owner = "Forms$TemplateGridContents" + + empty := genPg.NewTemplateGridContents() + if m, n := arrayMarker(t, encodeElement(t, empty), "Widgets"); m != 2 || n != 0 { + t.Errorf("new empty %s.Widgets = marker %d len %d, want [2]", owner, m, n) + } + + withTabs := genPg.NewTemplateGridContents() + tabs := genPg.NewTabContainer() + tabs.SetName("tabs") + withTabs.AddWidgets(tabs) + if m, n := arrayMarker(t, encodeElement(t, withTabs), "Widgets"); m != 2 || n != 1 { + t.Errorf("%s.Widgets (first child Forms$TabControl) = marker %d len %d, want 2", owner, m, n) + } + + stored := bsonv1.D{ + {Key: "$ID", Value: idBinaryForTest(t)}, + {Key: "$Type", Value: owner}, + {Key: "Widgets", Value: bsonv1.A{int32(2), + bsonv1.D{{Key: "$ID", Value: idBinaryForTest(t)}, {Key: "$Type", Value: "Forms$DivContainer"}, {Key: "Name", Value: "a"}, {Key: "Widgets", Value: bsonv1.A{int32(2)}}}, + bsonv1.D{{Key: "$ID", Value: idBinaryForTest(t)}, {Key: "$Type", Value: "Forms$DivContainer"}, {Key: "Name", Value: "b"}, {Key: "Widgets", Value: bsonv1.A{int32(2)}}}, + }}, + } + raw, err := bsonv1.Marshal(stored) + if err != nil { + t.Fatal(err) + } + decode := func() *genPg.TemplateGridContents { + el, err := codec.NewDecoder(codec.DefaultRegistry).Decode(raw) + if err != nil { + t.Fatal(err) + } + return el.(*genPg.TemplateGridContents) + } + edited := decode() + edited.WidgetsItems()[1].(*genPg.DivContainer).SetName("renamed") + if m, n := arrayMarker(t, encodeElement(t, edited), "Widgets"); m != 2 || n != 2 { + t.Errorf("selectively rebuilt %s.Widgets = marker %d len %d, want 2", owner, m, n) + } + emptied := decode() + emptied.RemoveWidgets(1) + emptied.RemoveWidgets(0) + if m, n := arrayMarker(t, encodeElement(t, emptied), "Widgets"); m != 2 || n != 0 { + t.Errorf("emptied %s.Widgets = marker %d len %d, want [2]", owner, m, n) + } +} diff --git a/mdl/backend/modelsdk/widget_list_markers.go b/mdl/backend/modelsdk/widget_list_markers.go new file mode 100644 index 0000000000..7e39f1a9ab --- /dev/null +++ b/mdl/backend/modelsdk/widget_list_markers.go @@ -0,0 +1,69 @@ +// SPDX-License-Identifier: Apache-2.0 + +package modelsdkbackend + +import "github.com/mendixlabs/mxcli/modelsdk/codec" + +// widgetsListOwners are the $Types that hold a `Widgets` PartList in stored +// Mendix 11 documents. Each is the owner's storage name, as the encoder sees it +// (Forms$FormCallArgument, not gen's Forms$LayoutCallArgument). +var widgetsListOwners = []string{ + "CustomWidgets$WidgetValue", + "Forms$BuildingBlock", + "Forms$DataView", + "Forms$DivContainer", + "Forms$FormCallArgument", + "Forms$GroupBox", + "Forms$LayoutGridColumn", + "Forms$ListView", + "Forms$ListViewTemplate", + "Forms$NativeLayoutContent", + "Forms$NavigationListItem", + "Forms$ScrollContainerRegion", + "Forms$Snippet", + "Forms$TabPage", + "Forms$TemplateGridContents", + "Forms$WebLayoutContent", +} + +// Every `Widgets` list is typed-array marker 2, keyed on the property rather +// than on the first child. Measured across ako/TestApp and PedApp: 22,000+ +// Widgets lists on every owner above but TemplateGridContents, empty and populated, with no +// exception — including 14 whose first child is a Forms$TabControl, whose own +// TabPages list is 3 (32 of 32) and stays on the child-type default. +// +// The child-type registrations (RegisterListMarker per widget type) gave the +// right answer only for a first child someone had registered: a column opening +// with a tab control was written as 3 (ako/mxcli-ledger FINDINGS #170). They +// also never reach two other paths, which the property registration does: an +// emptied list (the encoder's empty default is 3) and a selectively rebuilt one +// (a stored list with one dirty child, written with 3 unless the property is +// registered). +// +// Forms$TemplateGridContents has no reference document; it was measured with +// mx convert (Mendix 11.15.0), which re-serializes every unit. Injected with +// its Widgets written as [3] — empty, and with a TabControl first child — it +// came back [2] both times, and a bare grid gained an empty Widgets [2]. The +// same run's control, a DivContainer written as [3], also came back [2]. +// +// The other gen Widgets owners cannot occur in a Mendix 11 project, so no +// marker is registered for them: convert refuses Forms$TableCell, +// Forms$VerticalFlow, NativePages$NativeLayoutCallArgument and +// NativePages$NativePage with "the type cache does not contain a type". A +// Mendix 11 table cell is Forms$DbTableCell (Widgets [2] on convert, empty or +// with a TabControl first), which gen does not model: the decoder keeps it as +// opaque raw bytes, so the encoder never writes its list. Forms$Layout and +// Forms$LayoutCall hold no Widgets list themselves — their widgets sit in +// WebLayoutContent / NativeLayoutContent and FormCallArgument, all above. +// +// The list is also mandatory: no owner above is ever stored without its Widgets +// key (TestApp + PedApp: TabPage 89/89 with 20 empty, LayoutGridColumn 732/732 +// with 13 empty, DivContainer 838/838 with 8 empty, WidgetValue 20,775/20,775, +// every other owner likewise). A freshly built TabPage, DivContainer, GroupBox, +// LayoutGridColumn, ScrollContainerRegion or NavigationListItem with no children +// wrote no key at all; RegisterMandatoryPropertyList writes it as [2]. +func init() { + for _, owner := range widgetsListOwners { + codec.RegisterMandatoryPropertyList(owner, "Widgets", 2) + } +} diff --git a/mdl/backend/modelsdk/widget_write.go b/mdl/backend/modelsdk/widget_write.go index c1e861e3bb..c3d3abd237 100644 --- a/mdl/backend/modelsdk/widget_write.go +++ b/mdl/backend/modelsdk/widget_write.go @@ -427,7 +427,9 @@ func widgetToGen(w pages.Widget) (element.Element, error) { case *pages.LayoutGrid: g := genPg.NewLayoutGrid() applyWidgetBase(g, &x.BaseWidget) - g.SetWidth("FullWidth") + // FixedWidth was rewritten to the default on every describe → exec + // (ako/mxcli-ledger FINDINGS #170). Empty is the default. + g.SetWidth(orDefaultStr(x.Width, pages.DefaultLayoutGridWidth)) for _, row := range x.Rows { rg, err := layoutGridRowToGen(row) if err != nil { @@ -535,8 +537,8 @@ func widgetToGen(w pages.Widget) (element.Element, error) { g.SetOnLeaveAction(noActionGen()) g.SetPlaceholderTemplate(textAsClientTemplate(x.Placeholder)) g.SetReadOnlyStyle("Inherit") - g.SetSubmitBehaviour("OnEndEditing") - g.SetSubmitOnInputDelay(300) + g.SetSubmitBehaviour(orDefaultStr(x.SubmitBehaviour, pages.DefaultSubmitBehaviour)) + g.SetSubmitOnInputDelay(submitOnInputDelay(x.SubmitOnInputDelay)) g.SetValidation(widgetValidationToGenWith(x.ValidationExpression, x.ValidationMessage)) return g, nil @@ -629,8 +631,8 @@ func widgetToGen(w pages.Widget) (element.Element, error) { g.SetOnLeaveAction(noActionGen()) g.SetPlaceholderTemplate(textAsClientTemplate(x.Placeholder)) g.SetReadOnlyStyle("Inherit") - g.SetSubmitBehaviour("OnEndEditing") - g.SetSubmitOnInputDelay(300) + g.SetSubmitBehaviour(orDefaultStr(x.SubmitBehaviour, pages.DefaultSubmitBehaviour)) + g.SetSubmitOnInputDelay(submitOnInputDelay(x.SubmitOnInputDelay)) // Studio Pro stores one on every textarea; without the key a rewrite // had nowhere to carry the stored translations and deleted the message // (ako/mxcli#705). Empty, like CounterMessage above. @@ -1021,7 +1023,8 @@ func layoutGridRowToGen(row *pages.LayoutGridRow) (element.Element, error) { } // layoutGridColumnToGen converts a LayoutGridColumn. Weights default to -1 (auto, -// via columnWeight); PreviewWidth is always -1 — matching the legacy serializer. +// via columnWeight); PreviewWidth is -1, Studio Pro's value for a new column — +// a rewrite carries the stored one (carryLayoutGridPreviewWidth). func layoutGridColumnToGen(col *pages.LayoutGridColumn) (element.Element, error) { g := genPg.NewLayoutGridColumn() if col.ID != "" { @@ -1890,19 +1893,18 @@ func formSettingsToGen(pageName string) element.Element { return ps } -// staticAddressToGen builds the Forms$StaticOrDynamicString an open-link action -// nests as its Address. -// -// Only the STATIC form is authored: MDL spells `OPEN_LINK 'https://…'` and has -// no syntax for the dynamic one. 6 of the 31 Studio Pro references are dynamic -// (IsDynamic true, an AttributeRef, an empty Value) — DESCRIBE flags those -// rather than rendering them as a literal, because a dynamic address printed as -// a static one round-trips into a different link. -// -// AttributeRef is deliberately left unset. gen declares a fourth property here, -// `Attribute`, that not one of the 31 documents carries; writing a key Mendix -// does not store is what makes a document mxbuild accepts and Studio Pro cannot -// open (CLAUDE.md, "Overlay Writes: Never Invent a Key"). +// A static address stores AttributeRef as null: 21 of 21 static +// Forms$StaticOrDynamicString across ako/TestApp and PedApp carry the key. +// staticAddressToGen never sets it, so without this default the key was +// dropped and every describe → exec of a page with a link button rewrote the +// address (ako/mxcli-ledger FINDINGS #170). dynamicAddressToGen sets a real +// ref, which the default (applied only to an unset key) does not touch. +func init() { + codec.RegisterTypeDefaults("Forms$StaticOrDynamicString", codec.TypeDefaults{ + NullFields: []string{"AttributeRef"}, + }) +} + // dynamicAddressToGen builds a link address read from an attribute at runtime. // Pinned against FeedbackModule.PopupSuccess (Feedback v4.0.2): IsDynamic true, // Value "", and an AttributeRef with a null EntityRef naming the attribute. @@ -1915,6 +1917,22 @@ func dynamicAddressToGen(attrQN string) element.Element { return s } +// staticAddressToGen builds the Forms$StaticOrDynamicString an open-link action +// nests as its Address. +// +// Only the STATIC form is authored: MDL spells `OPEN_LINK 'https://…'` and has +// no syntax for the dynamic one. 6 of the 31 Studio Pro references are dynamic +// (IsDynamic true, an AttributeRef, an empty Value) — DESCRIBE flags those +// rather than rendering them as a literal, because a dynamic address printed as +// a static one round-trips into a different link. +// +// gen declares a fourth property here, `Attribute` (deleted in 7.11.0), that not +// one of the 31 documents carries; it is never set, because writing a key Mendix +// does not store is what makes a document mxbuild accepts and Studio Pro cannot +// open. AttributeRef is a different key and IS stored on a static address — as +// null, on 21 of 21 static addresses across ako/TestApp and PedApp. The init +// above registers that null; dynamicAddressToGen sets a real ref, which the +// default does not touch. func staticAddressToGen(address string) element.Element { s := genPg.NewStaticOrDynamicString() assignID(s) diff --git a/mdl/backend/modelsdk/widget_write_open_link_address_test.go b/mdl/backend/modelsdk/widget_write_open_link_address_test.go new file mode 100644 index 0000000000..7c86ff6164 --- /dev/null +++ b/mdl/backend/modelsdk/widget_write_open_link_address_test.go @@ -0,0 +1,61 @@ +// SPDX-License-Identifier: Apache-2.0 + +package modelsdkbackend + +import ( + "reflect" + "testing" + + "github.com/mendixlabs/mxcli/model" + genPg "github.com/mendixlabs/mxcli/modelsdk/gen/pages" + "github.com/mendixlabs/mxcli/sdk/pages" +) + +// A static OPEN_LINK address is written with Studio Pro's exact key set: +// $ID, $Type, AttributeRef (null), IsDynamic, Value. Measured 21 of 21 static +// Forms$StaticOrDynamicString across ako/TestApp and PedApp, every one storing +// AttributeRef as null. Dropping the key made a describe → exec of any page with +// a link button rewrite the address (ako/mxcli-ledger FINDINGS #170). +func TestStaticAddress_WritesNullAttributeRef(t *testing.T) { + el, err := clientActionToGen(&pages.LinkClientAction{ + BaseElement: model.BaseElement{ID: "link-id"}, + LinkType: pages.LinkTypeWeb, + Address: "https://example.com", + }) + if err != nil { + t.Fatalf("unexpected error: %v", err) + } + doc := encodeElement(t, el.(*genPg.OpenLinkClientAction).Address()) + want := []string{"$ID", "$Type", "AttributeRef", "IsDynamic", "Value"} + if got := docKeys(doc); !reflect.DeepEqual(got, want) { + t.Fatalf("static address keys = %v, want %v", got, want) + } + for _, e := range doc { + if e.Key == "AttributeRef" && e.Value != nil { + t.Errorf("AttributeRef = %v, want null on a static address", e.Value) + } + } +} + +// CONTROL: the dynamic address keeps its own AttributeRef document — the +// registered null only fills the key when nothing set it. +func TestDynamicAddress_KeepsAttributeRef(t *testing.T) { + el, err := clientActionToGen(&pages.LinkClientAction{ + BaseElement: model.BaseElement{ID: "link-id"}, + LinkType: pages.LinkTypeWeb, + AddressAttribute: "FeedbackModule.ResponseHelper.URL", + }) + if err != nil { + t.Fatalf("unexpected error: %v", err) + } + doc := encodeElement(t, el.(*genPg.OpenLinkClientAction).Address()) + want := []string{"$ID", "$Type", "AttributeRef", "IsDynamic", "Value"} + if got := docKeys(doc); !reflect.DeepEqual(got, want) { + t.Fatalf("dynamic address keys = %v, want %v", got, want) + } + for _, e := range doc { + if e.Key == "AttributeRef" && e.Value == nil { + t.Error("AttributeRef is null on a dynamic address — the registered default flattened it") + } + } +} diff --git a/mdl/backend/modelsdk/widget_write_submit.go b/mdl/backend/modelsdk/widget_write_submit.go new file mode 100644 index 0000000000..257f43cabd --- /dev/null +++ b/mdl/backend/modelsdk/widget_write_submit.go @@ -0,0 +1,26 @@ +// SPDX-License-Identifier: Apache-2.0 + +package modelsdkbackend + +import ( + "math" + + "github.com/mendixlabs/mxcli/sdk/pages" +) + +// submitOnInputDelay is the stored Forms$TextBox/TextArea.SubmitOnInputDelay: +// the authored value, or Studio Pro's default 300 when none was authored. A +// value outside int32 cannot come from a stored widget or pass the builder's +// check; it is clamped rather than wrapped. +func submitOnInputDelay(v *int) int32 { + if v == nil { + return pages.DefaultSubmitOnInputDelay + } + switch { + case *v > math.MaxInt32: + return math.MaxInt32 + case *v < 0: + return 0 + } + return int32(*v) +} diff --git a/mdl/backend/modelsdk/widget_write_submit_width_test.go b/mdl/backend/modelsdk/widget_write_submit_width_test.go new file mode 100644 index 0000000000..8d98154fd6 --- /dev/null +++ b/mdl/backend/modelsdk/widget_write_submit_width_test.go @@ -0,0 +1,76 @@ +// SPDX-License-Identifier: Apache-2.0 + +// ako/mxcli-ledger FINDINGS #170, the writer half: SubmitBehaviour, +// SubmitOnInputDelay and Forms$LayoutGrid.Width were hard-coded, so a stored +// WhileEditing / FixedWidth was rewritten to the default on every exec. +package modelsdkbackend + +import ( + "testing" + + "github.com/mendixlabs/mxcli/mdl/backend/bsonnav" + "github.com/mendixlabs/mxcli/model" + "github.com/mendixlabs/mxcli/sdk/pages" +) + +func intRef(n int) *int { return &n } + +func TestWriteTextInput_SubmitBehaviour(t *testing.T) { + for _, tc := range []struct { + name string + behaviour string + delay *int + wantB string + wantD int64 + }{ + // Unset writes Studio Pro's defaults, byte-for-byte as before. + {"unset", "", nil, "OnEndEditing", 300}, + {"WhileEditing", "WhileEditing", nil, "WhileEditing", 300}, + {"WhileEditing with delay", "WhileEditing", intRef(750), "WhileEditing", 750}, + {"delay zero", "", intRef(0), "OnEndEditing", 0}, + } { + t.Run(tc.name, func(t *testing.T) { + base := pages.BaseWidget{BaseElement: model.BaseElement{ID: "w"}, Name: "w"} + for kind, w := range map[string]pages.Widget{ + "textbox": &pages.TextBox{BaseWidget: base, SubmitBehaviour: tc.behaviour, SubmitOnInputDelay: tc.delay}, + "textarea": &pages.TextArea{BaseWidget: base, SubmitBehaviour: tc.behaviour, SubmitOnInputDelay: tc.delay}, + } { + d := encodeWidget(t, w) + if got := bsonnav.DGetString(d, "SubmitBehaviour"); got != tc.wantB { + t.Errorf("%s: SubmitBehaviour = %q, want %q", kind, got, tc.wantB) + } + if got, ok := asInt64(bsonnav.DGet(d, "SubmitOnInputDelay")); !ok || got != tc.wantD { + t.Errorf("%s: SubmitOnInputDelay = %v, want %d", kind, bsonnav.DGet(d, "SubmitOnInputDelay"), tc.wantD) + } + } + }) + } +} + +func TestWriteLayoutGrid_Width(t *testing.T) { + for _, tc := range []struct{ authored, want string }{ + {"", "FullWidth"}, + {"FullWidth", "FullWidth"}, + {"FixedWidth", "FixedWidth"}, + } { + d := encodeWidget(t, &pages.LayoutGrid{ + BaseWidget: pages.BaseWidget{BaseElement: model.BaseElement{ID: "lg"}, Name: "lg"}, + Width: tc.authored, + }) + if got := bsonnav.DGetString(d, "Width"); got != tc.want { + t.Errorf("authored %q: Width = %q, want %q", tc.authored, got, tc.want) + } + } +} + +func asInt64(v any) (int64, bool) { + switch n := v.(type) { + case int32: + return int64(n), true + case int64: + return n, true + case int: + return int64(n), true + } + return 0, false +} diff --git a/mdl/backend/pagemutator/mutator.go b/mdl/backend/pagemutator/mutator.go index e8f194f390..bbc9288608 100644 --- a/mdl/backend/pagemutator/mutator.go +++ b/mdl/backend/pagemutator/mutator.go @@ -3010,6 +3010,21 @@ func setRawWidgetPropertyMut(widget bson.D, propName string, value any) error { return setWidgetAttributeRefMut(widget, value) case "dateformat", "customdateformat", "decimalprecision", "groupdigits": return setWidgetFormattingMut(widget, propName, value) + case "submitbehaviour", "submitoninputdelay": + return setWidgetSubmitMut(widget, propName, value) + case "width": + // A layout grid's Width is its ContainerWidth enum. On any other + // widget the key keeps its old route, the pluggable setter. + if typ := bsonnav.DGetString(widget, "$Type"); typ == "Forms$LayoutGrid" || typ == "Pages$LayoutGrid" { + s, isString := value.(string) + width, ok := pages.CanonicalLayoutGridWidth(s) + if !isString || !ok { + return fmt.Errorf("a layout grid's Width is FullWidth or FixedWidth, not %v", value) + } + bsonnav.DSet(widget, "Width", width) + return nil + } + return setPluggableWidgetPropertyMut(widget, propName, value) default: // Try as pluggable widget property return setPluggableWidgetPropertyMut(widget, propName, value) @@ -3081,6 +3096,37 @@ func setWidgetFormattingMut(widget bson.D, propName string, value any) error { return nil } +// setWidgetSubmitMut sets a text box's or text area's SubmitBehaviour or +// SubmitOnInputDelay (ako/mxcli-ledger FINDINGS #170) with the members CREATE +// PAGE accepts. Any other widget refuses the key. +func setWidgetSubmitMut(widget bson.D, propName string, value any) error { + typ := bsonnav.DGetString(widget, "$Type") + if !pages.HasSubmitBehaviour(typ) { + return fmt.Errorf("a %s has no %s property — SubmitBehaviour and SubmitOnInputDelay are a text box's and a text area's", + widgetTypeLabel(typ), propName) + } + if strings.EqualFold(propName, "SubmitBehaviour") { + s, isString := value.(string) + canon, ok := pages.CanonicalSubmitBehaviour(s) + if !isString || !ok { + return fmt.Errorf("SubmitBehaviour is OnEndEditing or WhileEditing, not %v", value) + } + bsonnav.DSet(widget, "SubmitBehaviour", canon) + return nil + } + n, ok := formattingInt(value) + if !ok || n < 0 || n > math.MaxInt32 { + return fmt.Errorf("SubmitOnInputDelay is a non-negative number of milliseconds, not %v", value) + } + // Keep the stored integer width, as for DecimalPrecision. + if _, isInt64 := bsonnav.DGet(widget, "SubmitOnInputDelay").(int64); isInt64 { + bsonnav.DSet(widget, "SubmitOnInputDelay", int64(n)) + } else { + bsonnav.DSet(widget, "SubmitOnInputDelay", int32(n)) + } + return nil +} + // formattingInt reads an integer property value in any of the forms the // visitor produces. func formattingInt(v any) (int, bool) { diff --git a/mdl/executor/cmd_alter_flow.go b/mdl/executor/cmd_alter_flow.go index f95d2d97c9..1fff277ba8 100644 --- a/mdl/executor/cmd_alter_flow.go +++ b/mdl/executor/cmd_alter_flow.go @@ -85,14 +85,21 @@ func (a *alterFlowContext) applyTo(ctx *ExecContext, mut backend.MicroflowMutato fail := func(err error) error { return mdlerrors.NewValidation(fmt.Sprintf("alter %s %s: %s %s: %v", s.Kind(), s.Name, op.Op, op.Target, err)) } - // An activity inside a loop body is refused by the splice whatever - // the operation. Say so first: the checks below see only the flow + // An activity inside a loop body is refused by the splice for any + // change of the body's shape. Say so first: the checks below see only the flow // around the loop, so a fragment reading the iterator would otherwise // be refused as reading an undeclared variable, which is not why. - if loop := a.enclosingLoop(target.ID); loop != nil { - return fail(fmt.Errorf("%s is inside the body of %s; alter does not splice inside a loop body. %s", + // The one exception is an activity replaced by one activity, in place + // (ako/mxcli#1106): nothing in the body moves, so it needs no layout. + loop := a.enclosingLoop(target.ID) + inLoop := func() error { + return fail(fmt.Errorf("%s is inside the body of %s; alter does not splice inside a loop body "+ + "but to replace one activity by one of the same kind. %s", target.Statement, loop.Statement, replaceLoopAdvice(s.Kind(), s.Name.String(), loop.Statement))) } + if loop != nil && (op.Op != ast.AlterFlowReplace || op.SetCondition || op.ReplaceNotes || len(op.Body) != 1) { + return inLoop() + } if op.ReplaceNotes { // The statement states the activity's notes, so the stored ones // go with it rather than stay (ako/mxcli#859). @@ -138,6 +145,9 @@ func (a *alterFlowContext) applyTo(ctx *ExecContext, mut backend.MicroflowMutato if err != nil { return fail(err) } + if loop != nil && (len(frag.Objects) != 1 || len(frag.Flows) > 0 || len(frag.AnnotationFlows) > 0) { + return inLoop() + } // A stated @position on the fragment is where it goes (ako/mxcli#818). frag.Placed = firstStatementPlaced(op.Body) if !frag.Placed && laterStatementPlaced(op.Body) { @@ -219,6 +229,44 @@ func (a *alterFlowContext) enclosingLoop(id model.ID) *mfmutator.Candidate { return nil } +// loopsAround returns the loops whose body holds the stored object id, the +// innermost first; none when id is at top level. +func (a *alterFlowContext) loopsAround(id model.ID) []*microflows.LoopedActivity { + var path []*microflows.LoopedActivity + var find func(oc *microflows.MicroflowObjectCollection) bool + find = func(oc *microflows.MicroflowObjectCollection) bool { + if oc == nil { + return false + } + for _, obj := range oc.Objects { + if obj.GetID() == id { + return true + } + if l, ok := obj.(*microflows.LoopedActivity); ok { + if find(l.ObjectCollection) { + path = append(path, l) + return true + } + } + } + return false + } + find(a.mf.ObjectCollection) + return path +} + +// iteratorsAround returns the iterator variables of the loops around the +// stored object id: they exist wherever it runs. +func (a *alterFlowContext) iteratorsAround(id model.ID) []string { + var out []string + for _, l := range a.loopsAround(id) { + if src, ok := l.LoopSource.(*microflows.IterableList); ok && src.VariableName != "" { + out = append(out, src.VariableName) + } + } + return out +} + func loopHolds(loop *microflows.LoopedActivity, id model.ID) bool { if loop.ObjectCollection == nil { return false @@ -688,6 +736,9 @@ func (a *alterFlowContext) checkFragmentScope(ctx *ExecContext, op *ast.AlterFlo for _, p := range a.mf.Parameters { inScope[p.Name] = true } + for _, v := range a.iteratorsAround(target.ID) { + inScope[v] = true + } for id := range a.upstreamOf(target.ID, op.Op == ast.AlterFlowInsertAfter) { for _, c := range a.cands { if c.ID == id && c.OutputVariable != "" { @@ -727,6 +778,9 @@ func (a *alterFlowContext) unscopedRefs(expr string, id model.ID) []string { for _, p := range a.mf.Parameters { inScope[p.Name] = true } + for _, v := range a.iteratorsAround(id) { + inScope[v] = true + } for up := range a.upstreamOf(id, false) { for _, c := range a.cands { if c.ID == up && c.OutputVariable != "" { @@ -752,13 +806,31 @@ func (a *alterFlowContext) unscopedRefs(expr string, id model.ID) []string { // stored flows — the activities whose outputs exist when the flow gets there. // id itself is included only when including says so (an insert after it runs // once it has). +// +// Inside a loop body, what runs before the loop runs before id too, at every +// level of nesting (ako/mxcli#1106). func (a *alterFlowContext) upstreamOf(id model.ID, including bool) map[model.ID]bool { preds := map[model.ID][]model.ID{} - for _, f := range a.mf.ObjectCollection.Flows { - preds[f.DestinationID] = append(preds[f.DestinationID], f.OriginID) + var collect func(oc *microflows.MicroflowObjectCollection) + collect = func(oc *microflows.MicroflowObjectCollection) { + if oc == nil { + return + } + for _, f := range oc.Flows { + preds[f.DestinationID] = append(preds[f.DestinationID], f.OriginID) + } + for _, obj := range oc.Objects { + if l, ok := obj.(*microflows.LoopedActivity); ok { + collect(l.ObjectCollection) + } + } } + collect(a.mf.ObjectCollection) out := map[model.ID]bool{} queue := append([]model.ID(nil), preds[id]...) + for _, l := range a.loopsAround(id) { + queue = append(queue, preds[l.ID]...) + } for len(queue) > 0 { n := queue[0] queue = queue[1:] diff --git a/mdl/executor/cmd_entities.go b/mdl/executor/cmd_entities.go index da71b7dcdb..8b95c9b836 100644 --- a/mdl/executor/cmd_entities.go +++ b/mdl/executor/cmd_entities.go @@ -433,6 +433,9 @@ func execCreateEntity(ctx *ExecContext, s *ast.CreateEntityStmt) error { // own defect report; access rules would have been the third, and the // external-source and OData fields the next. Starting from what is stored // and overwriting only what the statement declares has no such tail. + // Recorded before the merge, which may reuse the stored attributes: + // the views over a changed attribute are resynced after the write. + storedSignatures := attributeSignatures(existingEntity) entity = mergeDeclaredOntoStoredEntity(existingEntity, entity, s) if droppedIdx := reconcileDroppedIndexes(entity, existingEntity); droppedIdx > 0 { fmt.Fprintf(ctx.Output, @@ -451,6 +454,7 @@ func execCreateEntity(ctx *ExecContext, s *ast.CreateEntityStmt) error { invalidateHierarchy(ctx) invalidateDomainModelsCache(ctx) ctx.ReportMutation("Modified", "entity: %s", s.Name) + resyncViewsOverSource(ctx, s.Name.String(), changedAttributes(storedSignatures, entity)) } else { // Create new entity if err := ctx.Backend.CreateEntity(dm.ID, entity); err != nil { @@ -1435,6 +1439,7 @@ func execAlterEntity(ctx *ExecContext, s *ast.AlterEntityStmt) error { return err } found := false + storedSignatures := attributeSignatures(entity) for _, attr := range entity.Attributes { if attr.Name == s.AttributeName { storedType := attr.Type @@ -1483,6 +1488,7 @@ func execAlterEntity(ctx *ExecContext, s *ast.AlterEntityStmt) error { invalidateHierarchy(ctx) invalidateDomainModelsCache(ctx) ctx.ReportMutation("Modified", "attribute '%s' on entity %s", s.AttributeName, s.Name) + resyncViewsOverSource(ctx, s.Name.String(), changedAttributes(storedSignatures, entity)) case ast.AlterEntityDropAttribute: if isViewEntity(entity) { diff --git a/mdl/executor/cmd_flow_loop_advice_pedapp_test.go b/mdl/executor/cmd_flow_loop_advice_pedapp_test.go index 9224ee721f..6236e8b5df 100644 --- a/mdl/executor/cmd_flow_loop_advice_pedapp_test.go +++ b/mdl/executor/cmd_flow_loop_advice_pedapp_test.go @@ -11,7 +11,7 @@ import ( "github.com/mendixlabs/mxcli/mdl/visitor" ) -// A change inside a loop body is refused by both editing modes: `create or +// A change of a loop body's shape is refused by both editing modes: `create or // modify` under mdl 1 (the splice does not edit inside a loop) and `alter` // (the mutator does not splice inside a loop). Each refusal used to send the // reader to the other — create or modify said "change activities with alter", @@ -86,7 +86,9 @@ func TestFlowLoopRefusals_PointAtReplaceLoop(t *testing.T) { t.Run("create or modify names alter replace loop", func(t *testing.T) { src := strings.Replace(loopAdviceFlow, "create microflow", "create or modify microflow", 1) - src = strings.Replace(src, "commit $It;", "commit $It without events;", 1) + // An insert inside the body: a change of the body's shape, which + // needs a layout (an in-place edit is spliced, ako/mxcli#1106). + src = strings.Replace(src, "commit $It;", "commit $It;\n log info node 'X' 'committed';", 1) // check predicts exec's refusal, with the same advice. checkSays(t, exec, src, loopAdviceReplace) err := agreeExec(t, exec, src) @@ -109,7 +111,8 @@ func TestFlowLoopRefusals_PointAtReplaceLoop(t *testing.T) { t.Run("alter of an activity in the loop names replace loop, not scope", func(t *testing.T) { for _, op := range []string{ - "replace commit $It with begin commit $It without events; end;", + // Replaced by two activities: the body changes shape. + "replace commit $It with begin commit $It; log info node 'X' 'committed'; end;", "insert after commit $It begin log info node 'X' 'committed'; end;", "drop commit $It;", } { @@ -168,6 +171,25 @@ end;`); err != nil { } }) + // One activity replaced by one of the same kind is made in place, with + // the loop's iterator in scope (ako/mxcli#1106); run last but one, as it + // writes. + t.Run("alter replaces one activity in the loop in place", func(t *testing.T) { + err := agreeExec(t, exec, "mdl 1;\nalter microflow MyFirstModule.LoopAdvice {\n"+ + " replace commit $It with begin commit $It without events; end;\n};") + if err != nil { + t.Fatalf("an in-place replace inside the loop: %v", err) + } + var buf bytes.Buffer + exec.output = &buf + if err := afRun(t, exec, "describe microflow MyFirstModule.LoopAdvice;"); err != nil { + t.Fatal(err) + } + if !strings.Contains(buf.String(), "commit $It without events;") { + t.Errorf("the alter did not write the change:\n%s", buf.String()) + } + }) + // Control: the advice works — the replace-loop alter is accepted and writes // the change. t.Run("control: the advised alter applies", func(t *testing.T) { diff --git a/mdl/executor/cmd_flow_loop_body_edit_pedapp_test.go b/mdl/executor/cmd_flow_loop_body_edit_pedapp_test.go new file mode 100644 index 0000000000..ef76b5aa36 --- /dev/null +++ b/mdl/executor/cmd_flow_loop_body_edit_pedapp_test.go @@ -0,0 +1,245 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "bytes" + "context" + "strings" + "testing" + + "github.com/mendixlabs/mxcli/mdl/visitor" + "github.com/mendixlabs/mxcli/model" + "github.com/mendixlabs/mxcli/sdk/microflows" +) + +// ako/mxcli#1106: under `mdl 1`, `create or modify` of a flow whose only +// change is one expression inside a loop body was refused — +// +// the Loop at (…) changes inside its body; the splice does not edit inside +// a loop, and replacing the whole loop would rebuild every node it holds. +// Nothing was written. [MDL-V1-REBUILD] +// +// so one script could not both provision and update a flow that loops. A body +// of the same shape — same statements in the same order, each either as stored +// or the same kind of activity with a new value — needs no layout: each +// changed activity is replaced in place, inside the loop, where it is drawn. + +const loopEditFlow = `mdl 1; +create or modify microflow MyFirstModule.LoopEdit ($Items: List of FeedbackModule.Feedback) +begin + declare $S string = ''; + loop $It in $Items + begin + set $S = $S + 'a'; + log info node 'X' 'in loop'; + commit $It; + end loop; + log info node 'X' $S; +end;` + +// loopEditFlowOf returns the stored LoopEdit microflow, or nil. +func loopEditFlowOf(t *testing.T, exec *Executor, name string) *microflows.Microflow { + t.Helper() + ctx := exec.newExecContext(context.Background()) + h, err := getHierarchy(ctx) + if err != nil { + t.Fatal(err) + } + all, err := ctx.Backend.ListMicroflows() + if err != nil { + t.Fatal(err) + } + for _, m := range all { + if m.Name == name && h.GetModuleName(h.FindModuleID(m.ContainerID)) == "MyFirstModule" { + mf, err := ctx.Backend.GetMicroflow(m.ID) + if err != nil { + t.Fatal(err) + } + return mf + } + } + t.Fatalf("MyFirstModule.%s not found", name) + return nil +} + +// flowObjectIDs returns every object's $ID in the flow, loop bodies included, +// with where it is drawn. +func flowObjectIDs(oc *microflows.MicroflowObjectCollection) map[model.ID]model.Point { + out := map[model.ID]model.Point{} + var walk func(*microflows.MicroflowObjectCollection) + walk = func(oc *microflows.MicroflowObjectCollection) { + if oc == nil { + return + } + for _, obj := range oc.Objects { + out[obj.GetID()] = obj.GetPosition() + if l, ok := obj.(*microflows.LoopedActivity); ok { + walk(l.ObjectCollection) + } + } + } + walk(oc) + return out +} + +func describeFlow(t *testing.T, exec *Executor, name string) string { + t.Helper() + var buf bytes.Buffer + saved := exec.output + exec.output = &buf + defer func() { exec.output = saved }() + if err := afRun(t, exec, "describe microflow MyFirstModule."+name+";"); err != nil { + t.Fatal(err) + } + return buf.String() +} + +func TestCreateOrModify_EditInsideLoopBody(t *testing.T) { + exec, out, _ := openPedAppCopy(t) + if err := agreeExec(t, exec, loopEditFlow); err != nil { + t.Fatalf("setup: %v\n%s", err, out.String()) + } + before := flowObjectIDs(loopEditFlowOf(t, exec, "LoopEdit").ObjectCollection) + + edited := strings.Replace(loopEditFlow, "set $S = $S + 'a';", "set $S = $S + toString(length($S));", 1) + if prog, errs := visitor.Build(edited); len(errs) > 0 { + t.Fatal(errs[0]) + } else { + for _, v := range exec.CheckFlowVerdicts(prog) { + t.Errorf("check -p predicts a refusal for an in-loop expression edit: %s", v.Message) + } + } + out.Reset() + if err := agreeExec(t, exec, edited); err != nil { + t.Fatalf("an expression edit inside a loop body is refused: %v", err) + } + if !strings.Contains(out.String(), "Modified microflow: MyFirstModule.LoopEdit (spliced: 1 replaced)") { + t.Errorf("want the edit reported as one splice, got:\n%s", out.String()) + } + desc := describeFlow(t, exec, "LoopEdit") + if !strings.Contains(desc, "toString(length($S))") || strings.Contains(desc, "+ 'a'") { + t.Errorf("the edit was not written:\n%s", desc) + } + + // Everything but the replaced activity keeps its $ID and its place: the + // loop, its other activity, and the flow around it. + after := flowObjectIDs(loopEditFlowOf(t, exec, "LoopEdit").ObjectCollection) + var gone, added []model.ID + for id := range before { + if _, ok := after[id]; !ok { + gone = append(gone, id) + } + } + for id, p := range after { + if q, ok := before[id]; !ok { + added = append(added, id) + } else if p != q { + t.Errorf("object %s moved from %v to %v", id, q, p) + } + } + if len(gone) != 1 || len(added) != 1 { + t.Fatalf("want exactly the edited activity replaced, got %d gone, %d added", len(gone), len(added)) + } + if before[gone[0]] != after[added[0]] { + t.Errorf("the replacement is drawn at %v, the activity it replaces at %v", after[added[0]], before[gone[0]]) + } + + // The re-run is a fixed point. + out.Reset() + if err := agreeExec(t, exec, edited); err != nil { + t.Fatal(err) + } + if !strings.Contains(out.String(), "Unchanged") { + t.Errorf("re-running the edited script wrote again:\n%s", out.String()) + } + + // An edit that reads the loop's iterator: in scope inside the body + // (the scope check walked only the flow around the loop). + t.Run("reads the iterator", func(t *testing.T) { + src := strings.Replace(edited, "commit $It;", "commit $It without events;", 1) + if err := agreeExec(t, exec, src); err != nil { + t.Fatalf("refused: %v", err) + } + if desc := describeFlow(t, exec, "LoopEdit"); !strings.Contains(desc, "commit $It without events;") { + t.Errorf("the edit was not written:\n%s", desc) + } + edited = src + }) + + // Controls: a change to the body's shape still refuses, writing nothing. + stored := loopEditBytes(t, exec) + for name, src := range map[string]string{ + "insert": strings.Replace(edited, "log info node 'X' 'in loop';", "log info node 'X' 'in loop';\n log info node 'X' 'more';", 1), + "drop": strings.Replace(edited, "log info node 'X' 'in loop';", "", 1), + "kind": strings.Replace(edited, "log info node 'X' 'in loop';", "declare $T string = 'x';", 1), + "swap": strings.Replace(edited, "set $S = $S + toString(length($S));\n log info node 'X' 'in loop';", + "log info node 'X' 'in loop';\n set $S = $S + toString(length($S));", 1), + } { + t.Run("control: "+name+" refused", func(t *testing.T) { + if src == edited { + t.Fatal("control edit did not apply") + } + err := agreeExec(t, exec, src) + if err == nil || !strings.Contains(err.Error(), "changes inside its body") { + t.Fatalf("want the in-loop refusal, got %v", err) + } + if !bytes.Equal(stored, loopEditBytes(t, exec)) { + t.Error("the refused statement wrote") + } + }) + } + + // A change next to the loop and one inside it in the same run. + t.Run("edit next to the loop and inside it", func(t *testing.T) { + src := strings.Replace(edited, "declare $S string = '';", "declare $S string = 'start';", 1) + src = strings.Replace(src, "'in loop'", "'still in loop'", 1) + out.Reset() + if err := agreeExec(t, exec, src); err != nil { + t.Fatalf("refused: %v", err) + } + if !strings.Contains(out.String(), "2 replaced") { + t.Errorf("want two in-place replaces, got:\n%s", out.String()) + } + desc := describeFlow(t, exec, "LoopEdit") + if !strings.Contains(desc, "'still in loop'") || !strings.Contains(desc, "'start'") { + t.Errorf("the edits were not written:\n%s", desc) + } + }) +} + +// A nested loop's body is edited the same way, at any depth. +func TestCreateOrModify_EditInsideNestedLoopBody(t *testing.T) { + exec, out, _ := openPedAppCopy(t) + src := `mdl 1; +create or modify microflow MyFirstModule.LoopEditNested ($Items: List of FeedbackModule.Feedback, $Others: List of FeedbackModule.Feedback) +begin + loop $It in $Items + begin + loop $Other in $Others + begin + log info node 'X' 'inner'; + end loop; + end loop; +end;` + if err := agreeExec(t, exec, src); err != nil { + t.Fatalf("setup: %v\n%s", err, out.String()) + } + edited := strings.Replace(src, "'inner'", "'inner, edited'", 1) + if err := agreeExec(t, exec, edited); err != nil { + t.Fatalf("an edit in a nested loop body is refused: %v", err) + } + if desc := describeFlow(t, exec, "LoopEditNested"); !strings.Contains(desc, "'inner, edited'") { + t.Errorf("the edit was not written:\n%s", desc) + } +} + +func loopEditBytes(t *testing.T, exec *Executor) []byte { + t.Helper() + ctx := exec.newExecContext(context.Background()) + raw, err := ctx.Backend.GetRawUnitBytes(loopEditFlowOf(t, exec, "LoopEdit").ID) + if err != nil { + t.Fatal(err) + } + return append([]byte(nil), raw...) +} diff --git a/mdl/executor/cmd_flow_modify.go b/mdl/executor/cmd_flow_modify.go index 1ef8eb18cb..756b66ddcd 100644 --- a/mdl/executor/cmd_flow_modify.go +++ b/mdl/executor/cmd_flow_modify.go @@ -842,7 +842,7 @@ func (pd *patchDiff) gap(ins []ast.MicroflowStatement, stored []ast.MicroflowSta return pd.statements(d.ElseBody, s.ElseBody) } if sameLoopShell(ins[0], del[0]) { - return pd.loopBodyChanged(del[0]) + return pd.loopBody(ins[0], del[0]) } if sameIgnoringLayout(ins[0], del[0]) { return redrawn(del[0]) @@ -1021,9 +1021,9 @@ func (pd *patchDiff) conditionEdit(declared, stored *ast.IfStmt) error { // diffed branch by branch, rather than left to merge with the changes around // it into one run: a change before a guard and one inside its then-branch were // one replace whose fragment held the guard's return, which the splice cannot -// insert (#859). Loops are not paired this way: a change inside a loop is -// refused either way, and pairing one would turn a replace that spans it into -// that refusal. +// insert (#859). A loop is paired this way only when its body can be edited in +// place (editableLoopBody, ako/mxcli#1106); any other change inside a loop is +// refused either way. // // An `if` whose condition changes is paired the same way (ako/mxcli#888): the // condition is set on the stored decision, and a change around it must not @@ -1061,6 +1061,151 @@ func sameLoopShell(declared, stored ast.MicroflowStatement) bool { return false } +// loopBody diffs a stored loop declared again with a change inside its body +// (ako/mxcli#1106). A body of the same shape — the same statements in the same +// order, each as stored or the same kind of activity with new values, drawn +// and annotated as stored — is spliced: each changed activity is replaced in +// place, inside the loop, and needs no layout. Anything else would add, +// remove, move or redraw a node whose coordinates are relative to the loop +// box, and is refused (loopBodyChanged), naming the top-level loop. +func (pd *patchDiff) loopBody(declared, stored ast.MicroflowStatement) error { + if !editableLoopBody(declared, stored) { + return pd.loopBodyChanged(stored) + } + c, err := pd.loc.locate(stored) + if err != nil { + return err + } + loop, ok := c.Object.(*microflows.LoopedActivity) + if !ok { + return cannotSplice("the stored %s is not drawn as a loop", describeAt(stored)) + } + if err := pd.movedNode(declared, stored); err != nil { + return err + } + return pd.inLoopBody(loop, loopStatements(declared), loopStatements(stored)) +} + +// inLoopBody adds the in-place replaces of one loop body, already shown to be +// of the same shape (editableLoopBody), recursing into a nested loop. +func (pd *patchDiff) inLoopBody(loop *microflows.LoopedActivity, declared, stored []ast.MicroflowStatement) error { + loc := pd.loc.within(loop) + for i := range declared { + d, s := declared[i], stored[i] + if declaredMatches(d, s) { + continue + } + if sameLoopShell(d, s) { + c, err := loc.locate(s) + if err != nil { + return err + } + inner, ok := c.Object.(*microflows.LoopedActivity) + if !ok { + return cannotSplice("the stored %s is not drawn as a loop", describeAt(s)) + } + if err := pd.inLoopBody(inner, loopStatements(d), loopStatements(s)); err != nil { + return err + } + continue + } + c, err := loc.locate(s) + if err != nil { + return err + } + // The stored notes stay on the replacement (editableLoopBody showed + // the declared ones are the same), so the builder does not draw them. + body, _, err := keepStoredNotes(s, []ast.MicroflowStatement{d}) + if err != nil { + return err + } + pd.add(ast.AlterFlowReplace, c, body) + } + return nil +} + +// editableLoopBody reports whether the declared loop differs from the stored +// one only by activities that can be replaced in place: the same number of +// statements, each matching its stored one, or a nested loop of which the +// same holds, or the same kind of plain activity — not a decision, a return, +// a break or continue, nor one with a custom error handler — whose +// annotations (where it is drawn, its connectors, caption, colour and notes) +// state nothing the stored one does not have. +func editableLoopBody(declared, stored ast.MicroflowStatement) bool { + if !sameLoopShell(declared, stored) { + return false + } + dBody, sBody := loopStatements(declared), loopStatements(stored) + if len(dBody) != len(sBody) { + return false + } + for i := range dBody { + d, s := dBody[i], sBody[i] + if declaredMatches(d, s) { + continue + } + if sameLoopShell(d, s) { + if !editableLoopBody(d, s) { + return false + } + continue + } + if !replaceableInPlace(d, s) { + return false + } + } + return true +} + +func replaceableInPlace(declared, stored ast.MicroflowStatement) bool { + if statementKind(declared) != statementKind(stored) || sameExceptPositions(declared, stored) { + return false + } + switch declared.(type) { + case *ast.IfStmt, *ast.ReturnStmt, *ast.LoopStmt, *ast.WhileStmt, *ast.BreakStmt, *ast.ContinueStmt: + return false + } + if customErrorHandler(declared) || customErrorHandler(stored) { + return false + } + if da := statementAnnotations(declared); da != nil { + sa := statementAnnotations(stored) + if sa == nil { + sa = &ast.ActivityAnnotations{} + } + if !declaredMatches(da, sa) { + return false + } + } + return true +} + +// customErrorHandler reports whether a statement has an `on error` with a +// body: a handler path of its own, drawn in the body beside it. +func customErrorHandler(st ast.MicroflowStatement) bool { + v := reflectElem(st) + if !v.IsValid() { + return false + } + f := v.FieldByName("ErrorHandling") + if !f.IsValid() { + return false + } + eh, _ := f.Interface().(*ast.ErrorHandlingClause) + return eh != nil && len(eh.Body) > 0 +} + +// loopStatements is the body of a loop or while statement. +func loopStatements(st ast.MicroflowStatement) []ast.MicroflowStatement { + switch l := st.(type) { + case *ast.LoopStmt: + return l.Body + case *ast.WhileStmt: + return l.Body + } + return nil +} + // loopBodyChanged refuses a change inside the body of the stored loop. The // splice does not edit inside a loop, and the engine would take it as a replace // of the whole loop: every node in it rebuilt, renumbered and redrawn — the @@ -1212,6 +1357,10 @@ func lcsStatements(declared, stored []ast.MicroflowStatement) [][2]int { w[i][j] = 1 case sameBlockShell(declared[i], stored[j]): w[i][j] = 1 + case editableLoopBody(declared[i], stored[j]): + // A loop edited in place inside is that loop: a change beside + // it must not merge with it into a replace (ako/mxcli#1106). + w[i][j] = 1 } } } @@ -1251,15 +1400,29 @@ func lcsStatements(declared, stored []ast.MicroflowStatement) [][2]int { // flow drawn so that nodes do not sit on top of each other. type storedLocator struct { byPos map[model.Point][]mfmutator.Candidate + cands []mfmutator.Candidate } func newStoredLocator(a *alterFlowContext) *storedLocator { + return locatorOver(a.cands, a.mf.ObjectCollection) +} + +// within is the locator of a stored loop's body: describe prints a body +// statement's @position relative to the loop box, unique among the body's +// own objects but not across the flow. +func (l *storedLocator) within(loop *microflows.LoopedActivity) *storedLocator { + return locatorOver(l.cands, loop.ObjectCollection) +} + +func locatorOver(cands []mfmutator.Candidate, oc *microflows.MicroflowObjectCollection) *storedLocator { top := map[model.ID]bool{} - for _, obj := range a.mf.ObjectCollection.Objects { - top[obj.GetID()] = true + if oc != nil { + for _, obj := range oc.Objects { + top[obj.GetID()] = true + } } - loc := &storedLocator{byPos: map[model.Point][]mfmutator.Candidate{}} - for _, c := range a.cands { + loc := &storedLocator{byPos: map[model.Point][]mfmutator.Candidate{}, cands: cands} + for _, c := range cands { if top[c.ID] && c.Object != nil { p := c.Object.GetPosition() loc.byPos[p] = append(loc.byPos[p], c) diff --git a/mdl/executor/cmd_pages_builder_v3_layout.go b/mdl/executor/cmd_pages_builder_v3_layout.go index cb15a1b8a7..cf39ef1e3f 100644 --- a/mdl/executor/cmd_pages_builder_v3_layout.go +++ b/mdl/executor/cmd_pages_builder_v3_layout.go @@ -23,6 +23,12 @@ func (pb *pageBuilder) buildLayoutGridV3(w *ast.WidgetV3) (*pages.LayoutGrid, er Name: w.Name, }, } + // Width: FullWidth | FixedWidth (ako/mxcli-ledger FINDINGS #170). + width, err := layoutGridWidth(w) + if err != nil { + return nil, mdlerrors.NewValidation(err.Error()) + } + lg.Width = width // Build rows from children for _, child := range w.Children { diff --git a/mdl/executor/cmd_pages_builder_v3_widgets.go b/mdl/executor/cmd_pages_builder_v3_widgets.go index 8eaf48d0d7..ee00277616 100644 --- a/mdl/executor/cmd_pages_builder_v3_widgets.go +++ b/mdl/executor/cmd_pages_builder_v3_widgets.go @@ -520,14 +520,18 @@ func (pb *pageBuilder) buildTextBoxV3(w *ast.WidgetV3) (*pages.TextBox, error) { tb.ValidationExpression = w.GetStringProp("Validation") tb.ValidationMessage = w.GetStringProp("ValidationMessage") + // SubmitBehaviour / SubmitOnInputDelay (ako/mxcli-ledger FINDINGS #170). + var err error + if tb.SubmitBehaviour, tb.SubmitOnInputDelay, err = inputSubmitSettings(w); err != nil { + return nil, mdlerrors.NewValidation(err.Error()) + } + // Forms$FormattingInfo: date format and numeric precision/grouping. The // writer hard-coded the defaults, so a Studio Pro text box with a decimal // precision lost it on describe → exec (ako/mxcli#968). - fi, err := inputFormattingInfo(w) - if err != nil { + if tb.FormattingInfo, err = inputFormattingInfo(w); err != nil { return nil, err } - tb.FormattingInfo = fi // Handle Label if label := inputLabel(w); label != "" { @@ -588,6 +592,12 @@ func (pb *pageBuilder) buildTextAreaV3(w *ast.WidgetV3) (*pages.TextArea, error) ta.Label = label } + // SubmitBehaviour / SubmitOnInputDelay, as for a textbox. + var err error + if ta.SubmitBehaviour, ta.SubmitOnInputDelay, err = inputSubmitSettings(w); err != nil { + return nil, mdlerrors.NewValidation(err.Error()) + } + // Placeholder, as for a textbox (ako/mxcli#705). if ph := w.GetPlaceholder(); ph != "" { ta.Placeholder = &model.Text{ diff --git a/mdl/executor/cmd_pages_describe.go b/mdl/executor/cmd_pages_describe.go index e7c728ab9a..faad5af2da 100644 --- a/mdl/executor/cmd_pages_describe.go +++ b/mdl/executor/cmd_pages_describe.go @@ -697,6 +697,13 @@ type rawWidget struct { // input widgets. ValidationExpression string ValidationMessage string + // SubmitBehaviour / SubmitOnInputDelay are a text box's or text area's + // stored submit settings; empty / nil when absent. LayoutGridWidth is a + // layout grid's Width. Printed only when not Studio Pro's default + // (ako/mxcli-ledger FINDINGS #170). + SubmitBehaviour string + SubmitOnInputDelay *int + LayoutGridWidth string // Formatting is the widget's Forms$FormattingInfo as MDL properties // (DateFormat, CustomDateFormat, DecimalPrecision, GroupDigits), only the // non-default ones (ako/mxcli#968). diff --git a/mdl/executor/cmd_pages_describe_output.go b/mdl/executor/cmd_pages_describe_output.go index c4d8e8c0f1..83cf2a5df0 100644 --- a/mdl/executor/cmd_pages_describe_output.go +++ b/mdl/executor/cmd_pages_describe_output.go @@ -447,7 +447,8 @@ func outputWidgetMDLV3(ctx *ExecContext, w rawWidget, indent int) { if w.Name != "" { header += " " + mdlIdent(w.Name) } - props := appendAppearanceProps(ctx, nil, w) + props := describeLayoutGridWidth(nil, w) + props = appendAppearanceProps(ctx, props, w) formatWidgetProps(ctx.Output, prefix, header, props, " {\n") for _, row := range w.Rows { // Mendix stores no name on a row or a column (R12, #749), so @@ -606,6 +607,7 @@ func outputWidgetMDLV3(ctx *ExecContext, w rawWidget, indent int) { } props = append(props, w.Formatting...) props = appendInputValidationProps(ctx, props, w) + props = describeSubmitBehaviour(props, w) props = appendAppearanceProps(ctx, props, w) formatWidgetProps(ctx.Output, prefix, header, props, "\n") @@ -624,6 +626,7 @@ func outputWidgetMDLV3(ctx *ExecContext, w rawWidget, indent int) { if w.OnChange != "" { props = append(props, actionProp("OnChange", w.OnChange)) } + props = describeSubmitBehaviour(props, w) props = appendAppearanceProps(ctx, props, w) formatWidgetProps(ctx.Output, prefix, header, props, "\n") diff --git a/mdl/executor/cmd_pages_describe_parse.go b/mdl/executor/cmd_pages_describe_parse.go index 994ed63d4d..ec07fde58c 100644 --- a/mdl/executor/cmd_pages_describe_parse.go +++ b/mdl/executor/cmd_pages_describe_parse.go @@ -281,6 +281,7 @@ func parseRawWidget(ctx *ExecContext, w map[string]any, parentEntityContext ...s switch typeName { case "Forms$LayoutGrid", "Pages$LayoutGrid": + widget.LayoutGridWidth = extractString(w["Width"]) widget.Rows = parseLayoutGridRows(ctx, w, inheritedCtx) return []rawWidget{widget} @@ -358,6 +359,7 @@ func parseRawWidget(ctx *ExecContext, w map[string]any, parentEntityContext ...s widget.Editable = extractEditable(ctx, w) widget.IsPassword, _ = w["IsPasswordBox"].(bool) widget.ValidationExpression, widget.ValidationMessage = extractWidgetValidation(ctx, w) + widget.SubmitBehaviour, widget.SubmitOnInputDelay = extractSubmitBehaviour(w) widget.OnChange = extractOnChangeAction(ctx, w) widget.Formatting = describeInputFormatting(ctx, "textbox", w) return []rawWidget{widget} @@ -369,6 +371,7 @@ func parseRawWidget(ctx *ExecContext, w map[string]any, parentEntityContext ...s // textarea's placeholder with all its translations (ako/mxcli#705). widget.Placeholder = extractPlaceholderText(ctx, w) widget.Editable = extractEditable(ctx, w) + widget.SubmitBehaviour, widget.SubmitOnInputDelay = extractSubmitBehaviour(w) widget.OnChange = extractOnChangeAction(ctx, w) return []rawWidget{widget} diff --git a/mdl/executor/cmd_pages_submit_width_test.go b/mdl/executor/cmd_pages_submit_width_test.go new file mode 100644 index 0000000000..048a85f87d --- /dev/null +++ b/mdl/executor/cmd_pages_submit_width_test.go @@ -0,0 +1,221 @@ +// SPDX-License-Identifier: Apache-2.0 + +// ako/mxcli-ledger FINDINGS #170 — `describe page` → `exec` rewrote two stored +// values MDL could not express: +// +// Forms$TextBox / Forms$TextArea .SubmitBehaviour WhileEditing → OnEndEditing +// Forms$TextBox / Forms$TextArea .SubmitOnInputDelay → 300 +// Forms$LayoutGrid.Width FixedWidth → FullWidth +// +// The writer hard-coded all three; the model had no field; describe read and +// printed nothing; the builder consumed nothing. Worse, `check` accepted +// `SubmitBehaviour: WhileEditing` (MDL-WIDGET07 is a warning) and exec dropped +// it, and `Width: FixedWidth` on a layout grid drew not even the warning, +// because "Width" is in the cross-widget known-property union. +// +// These tests cover the executor's three layers: describe parse, describe +// output and the builder, plus the check-time refusal of an invalid value. +// The writer is covered in mdl/backend/modelsdk, and the end-to-end round trip +// over a stored page in widget_submit_width_pedapp_test.go. +package executor + +import ( + "strings" + "testing" + + "github.com/mendixlabs/mxcli/mdl/ast" +) + +func TestParseRawWidget_TextBoxSubmitBehaviour(t *testing.T) { + ctx, _ := newMockCtx(t) + for _, typ := range []string{"Forms$TextBox", "Forms$TextArea"} { + got := parseRawWidget(ctx, map[string]any{ + "$Type": typ, "Name": "tb", + "SubmitBehaviour": "WhileEditing", "SubmitOnInputDelay": int32(500), + }) + if len(got) != 1 { + t.Fatalf("%s: parsed %d widgets", typ, len(got)) + } + if got[0].SubmitBehaviour != "WhileEditing" { + t.Errorf("%s: SubmitBehaviour = %q, want WhileEditing", typ, got[0].SubmitBehaviour) + } + if got[0].SubmitOnInputDelay == nil || *got[0].SubmitOnInputDelay != 500 { + t.Errorf("%s: SubmitOnInputDelay = %v, want 500", typ, got[0].SubmitOnInputDelay) + } + } +} + +func TestParseRawWidget_LayoutGridWidth(t *testing.T) { + ctx, _ := newMockCtx(t) + got := parseRawWidget(ctx, map[string]any{"$Type": "Forms$LayoutGrid", "Name": "lg", "Width": "FixedWidth"}) + if got[0].LayoutGridWidth != "FixedWidth" { + t.Errorf("LayoutGridWidth = %q, want FixedWidth", got[0].LayoutGridWidth) + } +} + +func TestDescribeTextInput_EmitsSubmitBehaviour(t *testing.T) { + for _, typ := range []string{"Forms$TextBox", "Forms$TextArea"} { + got := describeOneWidget(t, rawWidget{ + Type: typ, Name: "tb", Content: "Name", + SubmitBehaviour: "WhileEditing", SubmitOnInputDelay: intPtr(500), + }) + for _, want := range []string{"SubmitBehaviour: WhileEditing", "SubmitOnInputDelay: 500"} { + if !strings.Contains(got, want) { + t.Errorf("%s: describe should print %q, got:\n\t%s", typ, want, got) + } + } + } +} + +// Control: the defaults are what every ordinary input stores, so printing them +// would put a clause in every described page that its author never wrote. +func TestDescribeTextInput_OmitsDefaultSubmitBehaviour(t *testing.T) { + for _, typ := range []string{"Forms$TextBox", "Forms$TextArea"} { + for _, w := range []rawWidget{ + {Type: typ, Name: "tb", Content: "Name"}, + {Type: typ, Name: "tb", Content: "Name", SubmitBehaviour: "OnEndEditing", SubmitOnInputDelay: intPtr(300)}, + } { + if got := describeOneWidget(t, w); strings.Contains(got, "Submit") { + t.Errorf("%s: describe printed a default submit setting: %s", typ, got) + } + } + } +} + +func TestDescribeLayoutGrid_EmitsFixedWidth(t *testing.T) { + got := describeOneWidget(t, rawWidget{Type: "Forms$LayoutGrid", Name: "lg", LayoutGridWidth: "FixedWidth"}) + if !strings.Contains(got, "layoutgrid lg (Width: FixedWidth)") { + t.Errorf("describe dropped a layout grid's FixedWidth, got:\n%s", got) + } + for _, def := range []string{"", "FullWidth"} { + got := describeOneWidget(t, rawWidget{Type: "Forms$LayoutGrid", Name: "lg", LayoutGridWidth: def}) + if strings.Contains(got, "Width") { + t.Errorf("describe printed the default width for %q:\n%s", def, got) + } + } +} + +func TestBuildTextInput_SubmitBehaviour(t *testing.T) { + for _, tc := range []struct { + name string + props map[string]any + behaviour string + delay *int + }{ + {"unset keeps the defaults", map[string]any{}, "", nil}, + {"WhileEditing", map[string]any{"SubmitBehaviour": "WhileEditing"}, "WhileEditing", nil}, + {"case-insensitive, canonicalised", map[string]any{"SubmitBehaviour": "whileediting"}, "WhileEditing", nil}, + {"explicit OnEndEditing", map[string]any{"SubmitBehaviour": "OnEndEditing"}, "OnEndEditing", nil}, + {"delay", map[string]any{"SubmitBehaviour": "WhileEditing", "SubmitOnInputDelay": 500}, "WhileEditing", intPtr(500)}, + {"delay zero is a value, not unset", map[string]any{"SubmitOnInputDelay": 0}, "", intPtr(0)}, + } { + t.Run(tc.name, func(t *testing.T) { + tb, err := newPropsBuilder().buildTextBoxV3(&ast.WidgetV3{Name: "tb", Type: "textbox", Properties: tc.props}) + if err != nil { + t.Fatalf("buildTextBoxV3: %v", err) + } + ta, err := newPropsBuilder().buildTextAreaV3(&ast.WidgetV3{Name: "ta", Type: "textarea", Properties: tc.props}) + if err != nil { + t.Fatalf("buildTextAreaV3: %v", err) + } + for kind, got := range map[string][2]any{ + "textbox": {tb.SubmitBehaviour, tb.SubmitOnInputDelay}, + "textarea": {ta.SubmitBehaviour, ta.SubmitOnInputDelay}, + } { + if got[0] != tc.behaviour { + t.Errorf("%s: SubmitBehaviour = %q, want %q", kind, got[0], tc.behaviour) + } + d, _ := got[1].(*int) + if (d == nil) != (tc.delay == nil) || (d != nil && *d != *tc.delay) { + t.Errorf("%s: SubmitOnInputDelay = %v, want %v", kind, d, tc.delay) + } + } + }) + } +} + +func TestBuildTextInput_SubmitBehaviourRejectsInvalid(t *testing.T) { + for _, props := range []map[string]any{ + {"SubmitBehaviour": "Immediately"}, + {"SubmitOnInputDelay": -1}, + {"SubmitOnInputDelay": "fast"}, + } { + if _, err := newPropsBuilder().buildTextBoxV3(&ast.WidgetV3{Name: "tb", Type: "textbox", Properties: props}); err == nil { + t.Errorf("textbox %v was accepted", props) + } + if _, err := newPropsBuilder().buildTextAreaV3(&ast.WidgetV3{Name: "ta", Type: "textarea", Properties: props}); err == nil { + t.Errorf("textarea %v was accepted", props) + } + } +} + +func TestBuildLayoutGrid_Width(t *testing.T) { + for _, tc := range []struct { + value any + want string + }{ + {nil, ""}, + {"FixedWidth", "FixedWidth"}, + {"fixedwidth", "FixedWidth"}, + {"FullWidth", "FullWidth"}, + } { + w := &ast.WidgetV3{Name: "lg", Type: "layoutgrid", Properties: map[string]any{}} + if tc.value != nil { + w.Properties["Width"] = tc.value + } + lg, err := newPropsBuilder().buildLayoutGridV3(w) + if err != nil { + t.Fatalf("%v: %v", tc.value, err) + } + if lg.Width != tc.want { + t.Errorf("%v: Width = %q, want %q", tc.value, lg.Width, tc.want) + } + } + if _, err := newPropsBuilder().buildLayoutGridV3(&ast.WidgetV3{ + Name: "lg", Type: "layoutgrid", Properties: map[string]any{"Width": "Wide"}, + }); err == nil { + t.Error("layoutgrid Width: Wide was accepted") + } +} + +// check must refuse what exec refuses — the same function decides both. +func TestValidateWidgets_SubmitAndWidthValues(t *testing.T) { + for _, tc := range []struct { + name string + w *ast.WidgetV3 + wantErr string // "" = no MDL-WIDGET18 / MDL-WIDGET07 + }{ + {"valid textbox", &ast.WidgetV3{Name: "tb", Type: "textbox", Properties: map[string]any{"SubmitBehaviour": "WhileEditing", "SubmitOnInputDelay": 500}}, ""}, + {"valid textarea", &ast.WidgetV3{Name: "ta", Type: "textarea", Properties: map[string]any{"SubmitBehaviour": "WhileEditing"}}, ""}, + {"valid grid", &ast.WidgetV3{Name: "lg", Type: "layoutgrid", Properties: map[string]any{"Width": "FixedWidth"}}, ""}, + {"bad behaviour", &ast.WidgetV3{Name: "tb", Type: "textbox", Properties: map[string]any{"SubmitBehaviour": "Immediately"}}, "SubmitBehaviour"}, + {"bad delay", &ast.WidgetV3{Name: "ta", Type: "textarea", Properties: map[string]any{"SubmitOnInputDelay": -5}}, "SubmitOnInputDelay"}, + {"bad grid width", &ast.WidgetV3{Name: "lg", Type: "layoutgrid", Properties: map[string]any{"Width": "Wide"}}, "Width"}, + } { + t.Run(tc.name, func(t *testing.T) { + var errs []string + for _, v := range validateInputSubmitAndGridWidth(tc.w, "page M.P") { + errs = append(errs, v.Message) + } + for _, v := range validateStaticWidgetUnknownProps(tc.w, "page M.P") { + errs = append(errs, v.Message) + } + if tc.wantErr == "" && len(errs) > 0 { + t.Errorf("unexpected diagnostics: %v", errs) + } + if tc.wantErr != "" && (len(errs) == 0 || !strings.Contains(strings.Join(errs, "\n"), tc.wantErr)) { + t.Errorf("want a diagnostic naming %s, got %v", tc.wantErr, errs) + } + }) + } +} + +// The submit keys are a text box's and a text area's. On any other widget no +// builder reads them, so they must keep drawing MDL-WIDGET07 rather than join +// the cross-widget union and pass in silence. +func TestValidateWidgets_SubmitBehaviourOnOtherWidgetStillWarns(t *testing.T) { + w := &ast.WidgetV3{Name: "dp", Type: "datepicker", Properties: map[string]any{"SubmitBehaviour": "WhileEditing"}} + if len(validateStaticWidgetUnknownProps(w, "page M.P")) == 0 { + t.Error("SubmitBehaviour on a date picker is dropped on write and must warn MDL-WIDGET07") + } +} diff --git a/mdl/executor/input_submit_width.go b/mdl/executor/input_submit_width.go new file mode 100644 index 0000000000..47b882f6a5 --- /dev/null +++ b/mdl/executor/input_submit_width.go @@ -0,0 +1,186 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "fmt" + "strings" + + "github.com/mendixlabs/mxcli/mdl/ast" + "github.com/mendixlabs/mxcli/mdl/linter" + "github.com/mendixlabs/mxcli/sdk/pages" +) + +// A text box's and text area's submit behaviour, and a layout grid's width +// (ako/mxcli-ledger FINDINGS #170). +// +// The writer hard-coded SubmitBehaviour "OnEndEditing", SubmitOnInputDelay 300 +// and Forms$LayoutGrid.Width "FullWidth"; the model had no field for them, +// describe printed nothing and the builder read nothing. So describe → exec +// over a Studio Pro page turned a "submit while typing" field into an ordinary +// one and a fixed-width grid into a full-width one, mx check clean. And +// `SubmitBehaviour: WhileEditing` passed check (MDL-WIDGET07 only warns) and +// was dropped, while `Width: FixedWidth` on a grid did not even warn, "Width" +// being in the cross-widget known-property union. +// +// The MDL spelling is the stored property name and the metamodel's enum member: +// +// textbox t (Attribute: Name, SubmitBehaviour: WhileEditing, SubmitOnInputDelay: 500) +// layoutgrid g (Width: FixedWidth) { … } +// +// The submit keys are a text box's and a text area's only. They are kept out +// of staticWidgetKnownProps (the union every widget type shares) so a date +// picker carrying one still draws MDL-WIDGET07 — the per-type exemption is +// isInputSubmitProp, the same arrangement as the #968 formatting keys. For the +// same reason this file, not a cmd_pages_builder* or describe-output file, +// holds the reads and the emits. + +// inputSubmitProps are the submit properties, in canonical spelling and +// describe order. +var inputSubmitProps = []string{"SubmitBehaviour", "SubmitOnInputDelay"} + +// isInputSubmitProp reports whether key is a submit property of the MDL widget +// type widgetType. +func isInputSubmitProp(widgetType, key string) bool { + switch strings.ToLower(widgetType) { + case "textbox", "textarea": + default: + return false + } + for _, p := range inputSubmitProps { + if strings.EqualFold(p, key) { + return true + } + } + return false +} + +// inputSubmitProblems validates a text box's or text area's submit properties. +// Each problem is a sentence fragment naming the property; nil means valid. +// The builder refuses with the same messages, so check and exec agree. +func inputSubmitProblems(w *ast.WidgetV3) []string { + if !isInputSubmitProp(w.Type, "SubmitBehaviour") { + return nil + } + var out []string + if v, ok := lookupPropCI(w, "SubmitBehaviour"); ok { + s, isStr := v.(string) + if _, valid := pages.CanonicalSubmitBehaviour(s); !isStr || !valid { + out = append(out, fmt.Sprintf("SubmitBehaviour must be OnEndEditing or WhileEditing, got %s", submitValueText(v))) + } + } + if v, ok := lookupPropCI(w, "SubmitOnInputDelay"); ok { + if n, isInt := toFormatInt(v); !isInt || n < 0 { + out = append(out, fmt.Sprintf("SubmitOnInputDelay must be a non-negative number of milliseconds, got %s", submitValueText(v))) + } + } + return out +} + +// inputSubmitSettings reads a text box's or text area's submit properties. +// Unset returns "" / nil, which the writer turns into Studio Pro's defaults. +func inputSubmitSettings(w *ast.WidgetV3) (string, *int, error) { + if problems := inputSubmitProblems(w); len(problems) > 0 { + return "", nil, fmt.Errorf("%s %s: %s", strings.ToLower(w.Type), w.Name, strings.Join(problems, "; ")) + } + behaviour := "" + if v, ok := lookupPropCI(w, "SubmitBehaviour"); ok { + s, _ := v.(string) + behaviour, _ = pages.CanonicalSubmitBehaviour(s) + } + var delay *int + if v, ok := lookupPropCI(w, "SubmitOnInputDelay"); ok { + n, _ := toFormatInt(v) + delay = &n + } + return behaviour, delay, nil +} + +// submitValueText renders a refused value for a message: a plain value as +// written, anything else (a parsed expression such as `-1`) as what it is. +func submitValueText(v any) string { + switch v.(type) { + case string, bool, int, int32, int64, float64: + return fmt.Sprintf("`%v`", v) + } + return "an expression" +} + +// layoutGridWidthProblem validates a layout grid's Width; "" means valid. +func layoutGridWidthProblem(w *ast.WidgetV3) string { + if !strings.EqualFold(w.Type, "layoutgrid") { + return "" + } + v, ok := lookupPropCI(w, "Width") + if !ok { + return "" + } + s, isStr := v.(string) + if _, valid := pages.CanonicalLayoutGridWidth(s); !isStr || !valid { + return fmt.Sprintf("a layout grid's Width must be FullWidth or FixedWidth, got %s", submitValueText(v)) + } + return "" +} + +// layoutGridWidth reads a layout grid's Width; "" when unset. +func layoutGridWidth(w *ast.WidgetV3) (string, error) { + if p := layoutGridWidthProblem(w); p != "" { + return "", fmt.Errorf("layoutgrid %s: %s", w.Name, p) + } + v, _ := lookupPropCI(w, "Width") + s, _ := v.(string) + width, _ := pages.CanonicalLayoutGridWidth(s) + return width, nil +} + +// validateInputSubmitAndGridWidth (MDL-WIDGET18) reports at check time what +// the builder refuses at exec time, from the same functions. +func validateInputSubmitAndGridWidth(w *ast.WidgetV3, locationPrefix string) []linter.Violation { + if w == nil { + return nil + } + var out []linter.Violation + for _, msg := range inputSubmitProblems(w) { + out = append(out, violation18(locationPrefix, w, msg)) + } + if msg := layoutGridWidthProblem(w); msg != "" { + out = append(out, violation18(locationPrefix, w, msg)) + } + return out +} + +// extractSubmitBehaviour reads a stored text box's or text area's submit +// settings: "" / nil when the key is absent. +func extractSubmitBehaviour(w map[string]any) (string, *int) { + behaviour := extractString(w["SubmitBehaviour"]) + var delay *int + if v, ok := w["SubmitOnInputDelay"]; ok && v != nil { + n := extractInt(v) + delay = &n + } + return behaviour, delay +} + +// describeSubmitBehaviour appends a text input's submit settings when they +// differ from Studio Pro's defaults, so an ordinary input describes as before. +// The delay is printed on its own when it alone differs: Studio Pro keeps it +// when the behaviour is switched back to OnEndEditing, and dropping it would +// make the round trip rewrite it. +func describeSubmitBehaviour(props []string, w rawWidget) []string { + if w.SubmitBehaviour != "" && w.SubmitBehaviour != pages.DefaultSubmitBehaviour { + props = append(props, "SubmitBehaviour: "+w.SubmitBehaviour) + } + if w.SubmitOnInputDelay != nil && *w.SubmitOnInputDelay != pages.DefaultSubmitOnInputDelay { + props = append(props, fmt.Sprintf("SubmitOnInputDelay: %d", *w.SubmitOnInputDelay)) + } + return props +} + +// describeLayoutGridWidth appends a layout grid's Width when it is not the +// default FullWidth. +func describeLayoutGridWidth(props []string, w rawWidget) []string { + if w.LayoutGridWidth != "" && w.LayoutGridWidth != pages.DefaultLayoutGridWidth { + props = append(props, "Width: "+w.LayoutGridWidth) + } + return props +} diff --git a/mdl/executor/oql_view_resync.go b/mdl/executor/oql_view_resync.go new file mode 100644 index 0000000000..6384d56c07 --- /dev/null +++ b/mdl/executor/oql_view_resync.go @@ -0,0 +1,246 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "fmt" + "sort" + "strings" + + "github.com/mendixlabs/mxcli/mdl/ast" + "github.com/mendixlabs/mxcli/model" + "github.com/mendixlabs/mxcli/sdk/domainmodel" +) + +// attributeTypeSignature is what a view column over the attribute depends on: +// the kind, a String's length, an enumeration's reference and a DateTime's +// LocalizeDate. Two attributes with the same signature give a view column the +// same type; a change in it is what can put a view out of sync (CE6770). +func attributeTypeSignature(t domainmodel.AttributeType) string { + switch v := t.(type) { + case nil: + return "" + case *domainmodel.StringAttributeType: + return fmt.Sprintf("String(%d)", v.Length) + case *domainmodel.DateTimeAttributeType: + return fmt.Sprintf("DateTime(localize=%v)", v.LocalizeDate) + case *domainmodel.EnumerationAttributeType: + return "Enumeration(" + v.EnumerationRef + ")" + default: + return t.GetTypeName() + } +} + +// attributeSignatures maps each attribute name to its type signature. +func attributeSignatures(e *domainmodel.Entity) map[string]string { + if e == nil { + return nil + } + out := make(map[string]string, len(e.Attributes)) + for _, a := range e.Attributes { + if a != nil { + out[a.Name] = attributeTypeSignature(a.Type) + } + } + return out +} + +// changedAttributes names the attributes of after whose type signature differs +// from the one recorded in before. An attribute new to the entity is not a +// change: no view can read it yet. +func changedAttributes(before map[string]string, after *domainmodel.Entity) map[string]bool { + changed := map[string]bool{} + for _, a := range after.Attributes { + if a == nil { + continue + } + if old, ok := before[a.Name]; ok && old != attributeTypeSignature(a.Type) { + changed[a.Name] = true + } + } + return changed +} + +// resyncViewsOverSource brings the view entities that read a changed source +// attribute back in line with it, after the source entity has been written. +// +// A view's DateTime LocalizeDate is derived from the source attribute only +// when the view statement runs (viewDateTimeLocalize, #1297), so flipping the +// source's Localize afterwards — `alter entity … modify attribute … DateTime +// not localized`, or the same through `create or modify entity` — left every +// view over it out of sync: check and exec were silent and mx check reported +// CE6770 "View Entity is out of sync with the OQL Query" once per view +// (ako/mxcli-ledger FINDINGS #169). Mendix requires the view attribute to +// match the query whatever the view statement said, so the flag is re-derived +// here and a view is written only when a stored value differs — a repeat run +// writes nothing. +// +// Every other column property (String length, kind, enumeration) is declared +// in the view text and cannot be re-derived without rewriting what the user +// wrote, so a source change that breaks one is reported as a warning naming +// the view and column; the view has to be re-declared. +func resyncViewsOverSource(ctx *ExecContext, sourceQN string, changed map[string]bool) { + if len(changed) == 0 { + return + } + dms, err := ctx.Backend.ListDomainModels() + if err != nil { + return + } + for _, dm := range dms { + for _, view := range dm.Entities { + if !isViewEntity(view) || view.OqlQuery == "" { + continue + } + readCols := viewColumnsReading(view, sourceQN, changed) + if len(readCols) == 0 { + continue + } + viewQN := moduleNameOfDomainModel(ctx, dm) + "." + view.Name + flipped := resyncViewLocalizeDate(ctx, dm.ID, viewQN, view, readCols) + warnStaleViewColumns(ctx, viewQN, view, readCols) + // A view over this view reads the columns just changed. Views + // cannot form a cycle, and a column is only reported when its + // stored value moved, so this terminates. + resyncViewsOverSource(ctx, viewQN, flipped) + } + } +} + +// moduleNameOfDomainModel resolves a domain model's module name the way +// findEntity does — from its container, which also covers System. +func moduleNameOfDomainModel(ctx *ExecContext, dm *domainmodel.DomainModel) string { + h, err := getHierarchy(ctx) + if err != nil || h == nil { + return "" + } + return h.GetModuleName(dm.ContainerID) +} + +// viewAttributesAsAST describes a stored view entity's attributes the way a +// view statement would declare them. +func viewAttributesAsAST(view *domainmodel.Entity) []ast.ViewAttribute { + attrs := make([]ast.ViewAttribute, 0, len(view.Attributes)) + for _, a := range view.Attributes { + attrs = append(attrs, ast.ViewAttribute{Name: a.Name, Type: convertDomainModelTypeToAST(a.Type)}) + } + return attrs +} + +// viewColumnsReading returns, by column position, the view attributes whose +// select expression references one of the changed attributes of sourceQN — +// through any alias the FROM clause binds to it. +func viewColumnsReading(view *domainmodel.Entity, sourceQN string, changed map[string]bool) map[int]bool { + oql := stripOQLComments(view.OqlQuery) + clause := extractSelectClause(oql) + if clause == "" { + return nil + } + aliasMap := extractAliasMap(oql) + reads := map[int]bool{} + for i, expr := range attributeSelectColumns(oql, parseSelectColumns(clause)) { + if i >= len(view.Attributes) { + break + } + if m := oqlAliasSuffixRe.FindStringSubmatch(expr); m != nil { + expr = strings.TrimSuffix(expr, m[0]) + } + for _, ref := range oqlSourceAttrRefRe.FindAllStringSubmatch(oqlStringLiteralRe.ReplaceAllString(expr, "''"), -1) { + if entity, ok := aliasMap[unquoteOQLIdent(ref[1])]; ok && entity == sourceQN && changed[unquoteOQLIdent(ref[2])] { + reads[i] = true + break + } + } + } + return reads +} + +// resyncViewLocalizeDate re-derives the LocalizeDate of the view's DateTime +// columns that read a changed attribute, and writes the view only when one +// differs from what is stored. It returns the columns it changed. +func resyncViewLocalizeDate(ctx *ExecContext, dmID model.ID, viewQN string, view *domainmodel.Entity, readCols map[int]bool) map[string]bool { + derived := viewDateTimeLocalize(ctx, view.OqlQuery, viewAttributesAsAST(view)) + var flipped []string + changed := map[string]bool{} + for i, a := range view.Attributes { + if !readCols[i] { + continue + } + dt, ok := a.Type.(*domainmodel.DateTimeAttributeType) + if !ok { + continue + } + want, ok := derived[a.Name] + if !ok || dt.LocalizeDate == want { + continue + } + dt.LocalizeDate = want + state := "localized" + if !want { + state = "not localized" + } + flipped = append(flipped, a.Name+": "+state) + changed[a.Name] = true + } + if len(flipped) == 0 { + return nil + } + if err := ctx.Backend.UpdateEntity(dmID, view); err != nil { + fmt.Fprintf(ctx.Output, "⚠ could not resync view %s: %v\n", viewQN, err) + return nil + } + invalidateHierarchy(ctx) + invalidateDomainModelsCache(ctx) + ctx.ReportMutation("Resynced", "view %s (%s)", viewQN, strings.Join(flipped, ", ")) + return changed +} + +// warnStaleViewColumns reports the columns over a changed attribute whose +// declared type the source no longer gives them. They cannot be repaired +// without rewriting the view text, so the view must be re-declared. +func warnStaleViewColumns(ctx *ExecContext, viewQN string, view *domainmodel.Entity, readCols map[int]bool) { + attrs := viewAttributesAsAST(view) + columns, _ := inferOQLTypes(ctx, view.OqlQuery, attrs) + var stale []string + idx := make([]int, 0, len(readCols)) + for i := range readCols { + idx = append(idx, i) + } + sort.Ints(idx) + for _, i := range idx { + if i >= len(columns) || i >= len(attrs) { + continue + } + col, attr := columns[i], attrs[i] + if col.InferredType.Kind == ast.TypeUnknown { + continue + } + mismatch := passthroughStringLengthMismatch(attr.Type, col.InferredType, col.SourceAttr) || + !typesCompatible(attr.Type, col.InferredType) || + enumerationRefMismatch(attr.Type, col.InferredType) + if mismatch { + stale = append(stale, fmt.Sprintf("%s declared %s, the query now returns %s", + attr.Name, formatDataTypeForError(attr.Type), formatDataTypeForError(viewColumnType(col.InferredType)))) + } + } + if len(stale) == 0 { + return + } + fmt.Fprintf(ctx.Output, + "⚠ view entity %s is out of sync with its source: %s — mxbuild reports CE6770 "+ + "\"View Entity is out of sync with the OQL Query\". Re-declare the view "+ + "(create or modify view entity %s …) with the new type.\n", + viewQN, strings.Join(stale, "; "), viewQN) +} + +// enumerationRefMismatch reports an enumeration column declared over a +// different enumeration than the one the query returns. +func enumerationRefMismatch(declared, inferred ast.DataType) bool { + if declared.Kind != ast.TypeEnumeration || inferred.Kind != ast.TypeEnumeration { + return false + } + if declared.EnumRef == nil || inferred.EnumRef == nil { + return false + } + return declared.EnumRef.String() != inferred.EnumRef.String() +} diff --git a/mdl/executor/validate_v1_policy.go b/mdl/executor/validate_v1_policy.go new file mode 100644 index 0000000000..890e3aba7c --- /dev/null +++ b/mdl/executor/validate_v1_policy.go @@ -0,0 +1,29 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "github.com/mendixlabs/mxcli/mdl/deprecation" + "github.com/mendixlabs/mxcli/mdl/langver" + "github.com/mendixlabs/mxcli/mdl/linter" +) + +// ApplyV1Policy returns violations with every MDL-V1-* warning promoted to an +// error under deprecation.Error, and unchanged otherwise. It is +// ApplyDeprecationPolicy for the language-version codes: --deprecations gates +// deprecated spellings, --v1 gates what a headerless (mdl 0) script still has +// to change before it can carry `mdl 1;`. Both at error is the full mdl 1 +// readiness gate. +func ApplyV1Policy(violations []linter.Violation, policy deprecation.Policy) []linter.Violation { + if policy != deprecation.Error { + return violations + } + out := make([]linter.Violation, len(violations)) + for i, v := range violations { + if langver.IsV1Code(v.RuleID) { + v.Severity = linter.SeverityError + } + out[i] = v + } + return out +} diff --git a/mdl/executor/validate_v1_policy_test.go b/mdl/executor/validate_v1_policy_test.go new file mode 100644 index 0000000000..354d35fa68 --- /dev/null +++ b/mdl/executor/validate_v1_policy_test.go @@ -0,0 +1,50 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "testing" + + "github.com/mendixlabs/mxcli/mdl/deprecation" + "github.com/mendixlabs/mxcli/mdl/linter" +) + +func TestV1PolicyPromotesOnlyV1Codes(t *testing.T) { + in := []linter.Violation{ + {RuleID: "MDL-V1-SLASH", Severity: linter.SeverityWarning}, + {RuleID: FlowRebuildRule, Severity: linter.SeverityWarning}, + {RuleID: "MDL-DEPR001", Severity: linter.SeverityWarning}, + {RuleID: "MDL065", Severity: linter.SeverityWarning}, + } + if got := ApplyV1Policy(in, deprecation.Warn); got[0].Severity != linter.SeverityWarning { + t.Errorf("--v1=warn promoted %s", got[0].RuleID) + } + got := ApplyV1Policy(in, deprecation.Error) + for i, want := range []linter.Severity{linter.SeverityError, linter.SeverityError, linter.SeverityWarning, linter.SeverityWarning} { + if got[i].Severity != want { + t.Errorf("%s under --v1=error: severity %v, want %v", got[i].RuleID, got[i].Severity, want) + } + } + if in[0].Severity != linter.SeverityWarning { + t.Error("ApplyV1Policy modified its input") + } +} + +// The real path: a headerless script's `/` line reaches ApplyV1Policy as +// MDL-V1-SLASH from ValidateProgram. +func TestV1PolicyThroughValidateProgram(t *testing.T) { + prog := mustProgram(t, "create module V1Gate;\n/\n") + got := ApplyV1Policy(ValidateProgram(prog, ""), deprecation.Error) + found := false + for _, v := range got { + if v.RuleID == "MDL-V1-SLASH" { + found = true + if v.Severity != linter.SeverityError { + t.Errorf("MDL-V1-SLASH severity %v under --v1=error", v.Severity) + } + } + } + if !found { + t.Fatalf("no MDL-V1-SLASH in %+v", got) + } +} diff --git a/mdl/executor/validate_widgets.go b/mdl/executor/validate_widgets.go index 0be0242695..88f989b630 100644 --- a/mdl/executor/validate_widgets.go +++ b/mdl/executor/validate_widgets.go @@ -255,6 +255,8 @@ func validateWidgetTreeIn(widgets []*ast.WidgetV3, registry *WidgetRegistry, loc out = append(out, validateStaticWidget(w, locationPrefix)...) out = append(out, validateDynamicTextFormatting(w, locationPrefix)...) out = append(out, validateInputFormatting(w, locationPrefix)...) + // A text input's SubmitBehaviour and a layout grid's Width (ako/mxcli-ledger FINDINGS #170). + out = append(out, validateInputSubmitAndGridWidth(w, locationPrefix)...) // A quoted template parameter that reads like an expression is stored // as its text (ako/mxcli#969). out = append(out, validateQuotedTemplateParams(w, locationPrefix)...) @@ -864,6 +866,11 @@ func validateStaticWidgetUnknownProps(w *ast.WidgetV3, locationPrefix string) [] if _, ok := inputFormatProp(w.Type, key); ok { continue } + // A text box's or text area's SubmitBehaviour / SubmitOnInputDelay, + // per type for the same reason (ako/mxcli-ledger FINDINGS #170). + if isInputSubmitProp(w.Type, key) { + continue + } hint := "" if suggestion := nearestKey(key, staticWidgetKnownPropList); suggestion != "" { hint = fmt.Sprintf(" — did you mean `%s`?", suggestion) diff --git a/mdl/executor/view_entity_localize_date_test.go b/mdl/executor/view_entity_localize_date_test.go index d331c4c019..db3e95a150 100644 --- a/mdl/executor/view_entity_localize_date_test.go +++ b/mdl/executor/view_entity_localize_date_test.go @@ -3,6 +3,8 @@ package executor import ( + "bytes" + "strings" "testing" "github.com/mendixlabs/mxcli/mdl/ast" @@ -113,3 +115,173 @@ func TestViewDateTimeLocalize_NoSourceKeepsDefault(t *testing.T) { t.Errorf("sources that disagree were resolved to %v; want no derivation", got) } } + +// ledgerResyncFixture is module Ledger with Transaction (TxDate localized, +// CreatedAt localized, Memo String(100)) and three views over it: +// - VDaySpend reads t.TxDate, stored LocalizeDate true (in sync today); +// - VCreated reads t.CreatedAt, stored LocalizeDate false — out of sync on +// purpose, so a resync that touched views not reading the changed attribute +// would rewrite it (the control); +// - VMemo reads t.Memo as String(100). +// +// UpdateEntity replaces the entity in the domain model, as the real backend's +// next read would, and records every write by entity name. +type ledgerResyncFixture struct { + ctx *ExecContext + out *bytes.Buffer + dm *domainmodel.DomainModel + written []string +} + +func newLedgerResyncFixture(t *testing.T) *ledgerResyncFixture { + t.Helper() + mod := mkModule("Ledger") + tx := mkEntity(mod.ID, "Transaction") + tx.Attributes = []*domainmodel.Attribute{ + {BaseElement: model.BaseElement{ID: "attr-txdate"}, Name: "TxDate", Type: &domainmodel.DateTimeAttributeType{LocalizeDate: true}}, + {BaseElement: model.BaseElement{ID: "attr-created"}, Name: "CreatedAt", Type: &domainmodel.DateTimeAttributeType{LocalizeDate: true}}, + {BaseElement: model.BaseElement{ID: "attr-memo"}, Name: "Memo", Type: &domainmodel.StringAttributeType{Length: 100}}, + } + view := func(name, oql string, attrs ...*domainmodel.Attribute) *domainmodel.Entity { + v := mkEntity(mod.ID, name) + v.Source = "DomainModels$OqlViewEntitySource" + v.SourceDocumentRef = "Ledger." + name + v.OqlQuery = oql + v.Attributes = attrs + return v + } + daySpend := view("VDaySpend", "select t.TxDate as TxDate from Ledger.Transaction as t", + &domainmodel.Attribute{Name: "TxDate", Type: &domainmodel.DateTimeAttributeType{LocalizeDate: true}}) + created := view("VCreated", "select t.CreatedAt as CreatedAt from Ledger.Transaction as t", + &domainmodel.Attribute{Name: "CreatedAt", Type: &domainmodel.DateTimeAttributeType{LocalizeDate: false}}) + memo := view("VMemo", "select t.Memo as Memo from Ledger.Transaction as t", + &domainmodel.Attribute{Name: "Memo", Type: &domainmodel.StringAttributeType{Length: 100}}) + f := &ledgerResyncFixture{dm: mkDomainModel(mod.ID, tx, daySpend, created, memo)} + mb := &mock.MockBackend{ + IsConnectedFunc: func() bool { return true }, + ListModulesFunc: func() ([]*model.Module, error) { return []*model.Module{mod}, nil }, + ListDomainModelsFunc: func() ([]*domainmodel.DomainModel, error) { return []*domainmodel.DomainModel{f.dm}, nil }, + GetDomainModelFunc: func(model.ID) (*domainmodel.DomainModel, error) { return f.dm, nil }, + UpdateEntityFunc: func(_ model.ID, e *domainmodel.Entity) error { + f.written = append(f.written, e.Name) + for i, old := range f.dm.Entities { + if old.ID == e.ID { + f.dm.Entities[i] = e + } + } + return nil + }, + } + f.ctx, f.out = newMockCtx(t, withBackend(mb), withHierarchy(mkHierarchy(mod))) + return f +} + +func (f *ledgerResyncFixture) exec(t *testing.T, src string) { + t.Helper() + prog, errs := visitor.Build(src) + if len(errs) > 0 { + t.Fatalf("parse %q: %v", src, errs) + } + for _, stmt := range prog.Statements { + var err error + switch s := stmt.(type) { + case *ast.CreateEntityStmt: + err = execCreateEntity(f.ctx, s) + case *ast.AlterEntityStmt: + err = execAlterEntity(f.ctx, s) + default: + t.Fatalf("unexpected statement %T", stmt) + } + if err != nil { + t.Fatalf("exec %q: %v", src, err) + } + } +} + +func (f *ledgerResyncFixture) entity(name string) *domainmodel.Entity { + for _, e := range f.dm.Entities { + if e.Name == name { + return e + } + } + return nil +} + +func (f *ledgerResyncFixture) wrote(name string) int { + n := 0 + for _, w := range f.written { + if w == name { + n++ + } + } + return n +} + +// ako/mxcli-ledger FINDINGS #169: flipping a source DateTime to `not +// localized` left every view selecting it with LocalizeDate true, and mx check +// reported CE6770 "View Entity is out of sync with the OQL Query" per view — +// check and exec said nothing. The view column is re-derived from the source +// after the source is written; a second run writes no view; a view that does +// not read the changed attribute is never written. +func TestSourceLocalizeChange_ResyncsViews(t *testing.T) { + stmts := map[string]string{ + "alter modify attribute": `alter entity Ledger.Transaction modify attribute TxDate: DateTime not localized;`, + "create or modify": `create or modify persistent entity Ledger.Transaction ( + TxDate: DateTime not localized, + CreatedAt: DateTime, + Memo: String(100) +);`, + } + for name, src := range stmts { + t.Run(name, func(t *testing.T) { + f := newLedgerResyncFixture(t) + f.exec(t, src) + + dt := f.entity("VDaySpend").Attributes[0].Type.(*domainmodel.DateTimeAttributeType) + if dt.LocalizeDate { + t.Errorf("VDaySpend.TxDate LocalizeDate = true over a non-localized source — "+ + "mxbuild reports CE6770 \"View Entity is out of sync with the OQL Query\"\noutput:\n%s", f.out) + } + if f.wrote("VDaySpend") != 1 { + t.Errorf("VDaySpend written %d times, want 1", f.wrote("VDaySpend")) + } + if !strings.Contains(f.out.String(), "Ledger.VDaySpend (TxDate: not localized)") { + t.Errorf("the resync was not reported:\n%s", f.out) + } + // Control: VCreated is out of sync too, but reads nothing that changed. + if n := f.wrote("VCreated"); n != 0 { + t.Errorf("VCreated, which does not read TxDate, was written %d time(s)", n) + } + if n := f.wrote("VMemo"); n != 0 { + t.Errorf("VMemo, which does not read TxDate, was written %d time(s)", n) + } + + // A second run: the source no longer changes, so no view is written. + f.written = nil + f.out.Reset() + f.exec(t, src) + if n := f.wrote("VDaySpend"); n != 0 { + t.Errorf("repeat run rewrote VDaySpend %d time(s); want none\noutput:\n%s", n, f.out) + } + }) + } +} + +// A view column type that is declared in the view text (here a String length) +// cannot be re-derived, so a source change that breaks it is a warning naming +// the view and column; the view is not written. Exec-only: `check` does not +// apply the source change, so it has no changed attribute to look from. +func TestSourceLengthChange_WarnsStaleView(t *testing.T) { + f := newLedgerResyncFixture(t) + f.exec(t, `alter entity Ledger.Transaction modify attribute Memo: String(200);`) + out := f.out.String() + if !strings.Contains(out, "Ledger.VMemo") || !strings.Contains(out, "Memo declared String(100)") { + t.Errorf("no stale-view warning naming Ledger.VMemo and its Memo column:\n%s", out) + } + if strings.Contains(out, "Ledger.VDaySpend") || strings.Contains(out, "Ledger.VCreated") { + t.Errorf("a view not reading Memo was reported:\n%s", out) + } + if n := f.wrote("VMemo"); n != 0 { + t.Errorf("VMemo was written %d time(s); a declared type is the view's to change", n) + } +} diff --git a/mdl/executor/widget_submit_width_pedapp_test.go b/mdl/executor/widget_submit_width_pedapp_test.go new file mode 100644 index 0000000000..e5eef91981 --- /dev/null +++ b/mdl/executor/widget_submit_width_pedapp_test.go @@ -0,0 +1,233 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "context" + "strings" + "testing" + + "go.mongodb.org/mongo-driver/bson" +) + +// ako/mxcli-ledger FINDINGS #170, end to end over Studio Pro-authored pages. +// PedApp stores SubmitBehaviour WhileEditing on three inputs of +// FeedbackModule.ShareFeedback and Width FixedWidth on two layout grids of +// MyFirstModule.Home_Web. describe → exec rewrote both to the defaults with +// `mx check` clean, because MDL had no spelling for either. + +// storedWidgetProps returns, per named widget of the stored page, the values +// of the given keys. +func storedWidgetProps(t *testing.T, exec *Executor, page string, keys ...string) map[string]map[string]any { + t.Helper() + ctx := exec.newExecContext(context.Background()) + unit, err := ctx.Backend.GetRawUnitByName("page", page) + if err != nil || unit == nil { + t.Fatalf("raw unit of %s: %v", page, err) + } + var doc bson.D + if err := bson.Unmarshal(unit.Contents, &doc); err != nil { + t.Fatalf("unmarshal %s: %v", page, err) + } + out := map[string]map[string]any{} + var walk func(v any) + walk = func(v any) { + switch x := v.(type) { + case bson.D: + m := map[string]any{} + for _, e := range x { + m[e.Key] = e.Value + } + if name, _ := m["Name"].(string); name != "" { + for _, k := range keys { + if v, ok := m[k]; ok { + if out[name] == nil { + out[name] = map[string]any{} + } + out[name][k] = v + } + } + } + for _, e := range x { + walk(e.Value) + } + case bson.A: + for _, e := range x { + walk(e) + } + } + } + walk(doc) + return out +} + +func TestSubmitBehaviourAndGridWidth_StudioProPageRoundTrips(t *testing.T) { + for _, tc := range []struct { + page string + key string + widgets []string + value string + mdl string + }{ + {"FeedbackModule.ShareFeedback", "SubmitBehaviour", + []string{"feedback_subject", "textBox1", "textArea2"}, "WhileEditing", "SubmitBehaviour: WhileEditing"}, + {"MyFirstModule.Home_Web", "Width", + []string{"layoutGrid2", "layoutGrid3"}, "FixedWidth", "Width: FixedWidth"}, + } { + t.Run(tc.page, func(t *testing.T) { + exec, out := openPedAppWithWidgets(t) + before := storedWidgetProps(t, exec, tc.page, tc.key) + for _, w := range tc.widgets { + if before[w][tc.key] != tc.value { + t.Fatalf("fixture: %s.%s = %v, want the Studio Pro value %s", w, tc.key, before[w][tc.key], tc.value) + } + } + + described := describeOn(t, exec, out, tc.page) + if n := strings.Count(described, tc.mdl); n != len(tc.widgets) { + t.Errorf("describe printed %q %d times, want %d:\n%s", tc.mdl, n, len(tc.widgets), described) + } + + out.Reset() + if err := afRun(t, exec, described); err != nil { + t.Fatalf("re-exec describe output: %v\n%s", err, out.String()) + } + after := storedWidgetProps(t, exec, tc.page, tc.key) + for _, w := range tc.widgets { + if after[w][tc.key] != tc.value { + t.Errorf("describe → exec rewrote %s.%s %v → %v", w, tc.key, tc.value, after[w][tc.key]) + } + } + + // Control: the stored value is what the description says, not + // something the writer cannot change — an edited description lands. + edited := strings.Replace(described, tc.mdl, map[string]string{ + "SubmitBehaviour": "SubmitBehaviour: OnEndEditing", + "Width": "Width: FullWidth", + }[tc.key], 1) + if edited == described { + t.Fatal("control edit did not apply") + } + if err := afRun(t, exec, edited); err != nil { + t.Fatalf("exec edited description: %v", err) + } + changed := 0 + stored := storedWidgetProps(t, exec, tc.page, tc.key) + for _, w := range tc.widgets { + if stored[w][tc.key] != tc.value { + changed++ + } + } + if changed != 1 { + t.Errorf("the control edit changed %d widgets, want 1", changed) + } + }) + } +} + +// Authored in MDL, then described: the values reach storage and come back. +func TestSubmitBehaviourAndGridWidth_AuthoredInMDL(t *testing.T) { + exec, out, dir := openPedAppCopy(t) + const page = "MyFirstModule.Submit170_Edit" + script := `create non-persistent entity MyFirstModule.Submit170 (Note: String(200)); +create page ` + page + ` ( + Title: 'Submit', Layout: Atlas_Core.Atlas_Default, + Params: ( $Item: MyFirstModule.Submit170 ) +) { + layoutgrid lgMain (Width: FixedWidth) { + row { + column (DesktopWidth: 12) { + dataview dv (DataSource: $Item) { + textbox tbLive (Label: 'Live', Attribute: Note, SubmitBehaviour: WhileEditing, SubmitOnInputDelay: 500) + textarea taLive (Label: 'Live', Attribute: Note, SubmitBehaviour: WhileEditing) + textbox tbPlain (Label: 'Plain', Attribute: Note) + } + } + } + } +};` + if err := afRun(t, exec, script); err != nil { + t.Fatalf("exec: %v\n%s", err, out.String()) + } + got := storedWidgetProps(t, exec, page, "SubmitBehaviour", "SubmitOnInputDelay", "Width") + for name, want := range map[string]map[string]any{ + "lgMain": {"Width": "FixedWidth"}, + "tbLive": {"SubmitBehaviour": "WhileEditing"}, + "taLive": {"SubmitBehaviour": "WhileEditing"}, + "tbPlain": {"SubmitBehaviour": "OnEndEditing"}, + } { + for k, v := range want { + if got[name][k] != v { + t.Errorf("%s.%s = %v, want %v", name, k, got[name][k], v) + } + } + } + if d, _ := toFormatInt(asGoInt(got["tbLive"]["SubmitOnInputDelay"])); d != 500 { + t.Errorf("tbLive.SubmitOnInputDelay = %v, want 500", got["tbLive"]["SubmitOnInputDelay"]) + } + + described := describeOn(t, exec, out, page) + for _, want := range []string{"Width: FixedWidth", "SubmitBehaviour: WhileEditing", "SubmitOnInputDelay: 500"} { + if !strings.Contains(described, want) { + t.Errorf("describe should print %q:\n%s", want, described) + } + } + // GetPut: executing the description writes nothing. + before := projectFiles(t, dir) + out.Reset() + if err := afRun(t, exec, described); err != nil { + t.Fatalf("re-exec describe output: %v\n%s", err, out.String()) + } + if changed := diffProjectFiles(before, projectFiles(t, dir)); len(changed) != 0 { + t.Errorf("executing the describe output wrote %v", changed) + } +} + +func TestSubmitBehaviourAndGridWidth_AlterPageSet(t *testing.T) { + exec, out := openPedAppWithWidgets(t) + const page = "FeedbackModule.ShareFeedback" + if err := afRun(t, exec, `alter page `+page+` { set (SubmitBehaviour: OnEndEditing, SubmitOnInputDelay: 800) on textBox1; };`); err != nil { + t.Fatalf("alter page set: %v\n%s", err, out.String()) + } + got := storedWidgetProps(t, exec, page, "SubmitBehaviour", "SubmitOnInputDelay") + if got["textBox1"]["SubmitBehaviour"] != "OnEndEditing" { + t.Errorf("textBox1.SubmitBehaviour = %v, want OnEndEditing", got["textBox1"]["SubmitBehaviour"]) + } + if d, _ := toFormatInt(asGoInt(got["textBox1"]["SubmitOnInputDelay"])); d != 800 { + t.Errorf("textBox1.SubmitOnInputDelay = %v, want 800", got["textBox1"]["SubmitOnInputDelay"]) + } + // The neighbour is untouched. + if got["textArea2"]["SubmitBehaviour"] != "WhileEditing" { + t.Errorf("textArea2.SubmitBehaviour = %v, want the stored WhileEditing", got["textArea2"]["SubmitBehaviour"]) + } + + const home = "MyFirstModule.Home_Web" + if err := afRun(t, exec, `alter page `+home+` { set Width = FullWidth on layoutGrid2; };`); err != nil { + t.Fatalf("alter page set Width: %v", err) + } + if w := storedWidgetProps(t, exec, home, "Width")["layoutGrid2"]["Width"]; w != "FullWidth" { + t.Errorf("layoutGrid2.Width = %v, want FullWidth", w) + } + + for _, c := range []struct{ page, stmt, want string }{ + {page, `set SubmitBehaviour = Immediately on textBox1`, "OnEndEditing or WhileEditing"}, + {page, `set SubmitOnInputDelay = fast on textBox1`, "non-negative"}, + {home, `set Width = Wide on layoutGrid3`, "FullWidth or FixedWidth"}, + } { + err := afRun(t, exec, `alter page `+c.page+` { `+c.stmt+`; };`) + if err == nil || !strings.Contains(err.Error(), c.want) { + t.Errorf("%s: want an error containing %q, got %v", c.stmt, c.want, err) + } + } + if w := storedWidgetProps(t, exec, home, "Width")["layoutGrid3"]["Width"]; w != "FixedWidth" { + t.Errorf("a refused set changed layoutGrid3.Width to %v", w) + } +} + +// asGoInt normalises a stored BSON integer for toFormatInt. +func asGoInt(v any) any { + if n, ok := v.(int32); ok { + return int(n) + } + return v +} diff --git a/mdl/langver/langver.go b/mdl/langver/langver.go index f2862d0c42..d1eb09bce8 100644 --- a/mdl/langver/langver.go +++ b/mdl/langver/langver.go @@ -250,3 +250,10 @@ func skipTrivia(src string, i int) int { func isIdentByte(c byte) bool { return c == '_' || c >= 'a' && c <= 'z' || c >= 'A' && c <= 'Z' || c >= '0' && c <= '9' || c >= 0x80 } + +// IsV1Code reports whether a violation rule ID is an mdl 1 change code +// (MDL-V1-*): a construct a headerless script keeps at its mdl 0 meaning, or +// that mdl 1 refuses. `check --v1=error` fails the run on one. +func IsV1Code(ruleID string) bool { + return strings.HasPrefix(ruleID, "MDL-V1-") +} diff --git a/mdl/linter/rules/page_navigation_security.go b/mdl/linter/rules/page_navigation_security.go index 9684229b2c..4c118126a7 100644 --- a/mdl/linter/rules/page_navigation_security.go +++ b/mdl/linter/rules/page_navigation_security.go @@ -18,7 +18,7 @@ import ( // It also checks, per navigation profile, that every user role can open the // home page it lands on: its role-based home page if it has one, otherwise the // profile's default. A user role none of whose module roles is allowed on that -// page passes `check --references` and reaches mxbuild, which then reports +// page passes `check --references`; at Production security mxbuild then reports // CE2729 for every widget on the page ("No read access to attribute … for user // role 'X' (with no roles defined in module …)") — ten of them for one leftover // template role in ChipCoV6, and nothing that names the actual cause. @@ -35,7 +35,7 @@ func (r *PageNavigationSecurityRule) Category() string { return func (r *PageNavigationSecurityRule) DefaultSeverity() linter.Severity { return linter.SeverityWarning } func (r *PageNavigationSecurityRule) Description() string { - return "Checks that pages used in navigation have at least one allowed role (CE0557), and that every user role can open its home page (else CE2729 per widget)" + return "Checks that pages used in navigation have at least one allowed role (CE0557), and that every user role can open its home page (at Production security, else CE2729 per widget)" } // navUsage describes how a page is used in navigation. @@ -168,7 +168,7 @@ func (r *PageNavigationSecurityRule) checkHomePagePerUserRole(ctx *linter.LintCo if sharesRole(ur.ModuleRoles, allowed) { continue } - violations = append(violations, homePageViolation(r, ur, pName, target, isMicroflow, viaRole)) + violations = append(violations, homePageViolation(r, ur, pName, target, isMicroflow, viaRole, ps.SecurityLevel)) } } return violations @@ -197,7 +197,7 @@ func effectiveHomePage(profile *types.NavigationProfile, userRole string) (targe return profile.HomePage.Microflow, profile.HomePage.Microflow != "", false } -func homePageViolation(r *PageNavigationSecurityRule, ur *security.UserRole, profile, target string, isMicroflow, viaRole bool) linter.Violation { +func homePageViolation(r *PageNavigationSecurityRule, ur *security.UserRole, profile, target string, isMicroflow, viaRole bool, level string) linter.Violation { kind, docType, verb, grant := "page", "page", "open", "GRANT VIEW ON PAGE" if isMicroflow { kind, docType, verb, grant = "microflow", "microflow", "run", "GRANT EXECUTE ON MICROFLOW" @@ -214,8 +214,8 @@ func homePageViolation(r *PageNavigationSecurityRule, ur *security.UserRole, pro return linter.Violation{ RuleID: r.ID(), Severity: r.DefaultSeverity(), - Message: fmt.Sprintf("User role '%s' cannot %s %s '%s' in %s navigation: none of its module roles is allowed (mxbuild reports CE2729 for each widget on it)", - ur.Name, verb, which, target, profile), + Message: fmt.Sprintf("User role '%s' is not allowed to %s its %s '%s' in %s navigation: none of its module roles is granted it%s", + ur.Name, verb, which, target, profile, homePageConsequence(level, isMicroflow)), Location: linter.Location{ Module: moduleName, DocumentType: docType, @@ -225,6 +225,22 @@ func homePageViolation(r *PageNavigationSecurityRule, ur *security.UserRole, pro } } +// homePageConsequence names what the gap costs at the project's security +// level. Only Production checks entity access, so only there does mxbuild +// report CE2729 per widget (ChipCoV6); at Prototype the page still renders for +// the role, with the data and actions its module roles do not reach withheld +// (ako/mxcli-rest #68: HTTP 200, no error banner, empty grids). +func homePageConsequence(level string, isMicroflow bool) string { + switch { + case level == security.SecurityLevelProduction && !isMicroflow: + return " (mxbuild reports CE2729 for each widget on it)" + case isMicroflow: + return "" + default: + return " (at Prototype security the page still opens, but without the data and actions those module roles would reach)" + } +} + // sharesRole reports whether any of a user role's module roles is allowed. func sharesRole(userModuleRoles, allowed []string) bool { for _, a := range allowed { diff --git a/mdl/linter/rules/page_navigation_security_test.go b/mdl/linter/rules/page_navigation_security_test.go index 0ec8ff6e82..6866044ad7 100644 --- a/mdl/linter/rules/page_navigation_security_test.go +++ b/mdl/linter/rules/page_navigation_security_test.go @@ -220,7 +220,29 @@ func TestPageNavigationSecurity_MicroflowHomePage(t *testing.T) { r.nav.Profiles[0].HomePage = &types.NavHomePage{Microflow: "App.ACT_Home"} r.mfs = []*microflows.Microflow{{ContainerID: "mod", Name: "ACT_Home", AllowedModuleRoles: []model.ID{"App.User"}}} vs := homePageViolations(t, r) - if len(vs) != 1 || !strings.Contains(vs[0].Message, "cannot run default home microflow 'App.ACT_Home'") || !strings.Contains(vs[0].Message, "'Leftover'") { + if len(vs) != 1 || !strings.Contains(vs[0].Message, "not allowed to run its default home microflow 'App.ACT_Home'") || !strings.Contains(vs[0].Message, "'Leftover'") { t.Fatalf("want 1 microflow violation for Leftover, got %+v", vs) } } + +// At Prototype security mxbuild does not check entity access, so there is no +// CE2729, and the page still renders for the role (ako/mxcli-rest #68: 200, no +// banner, data and actions withheld). The message must state the model fact and +// not a build error or failure that does not happen at that level. +func TestPageNavigationSecurity_PrototypeLevelClaimsNoCE2729(t *testing.T) { + r := homePageFixture() + r.sec.SecurityLevel = security.SecurityLevelPrototype + vs := homePageViolations(t, r) + if len(vs) != 1 { + t.Fatalf("want 1 violation for Leftover, got %+v", vs) + } + msg := vs[0].Message + if strings.Contains(msg, "CE2729") || strings.Contains(msg, "cannot open") { + t.Errorf("Prototype-level message %q claims a consequence that does not occur", msg) + } + for _, want := range []string{"'Leftover'", "'App.Home'", "not allowed"} { + if !strings.Contains(msg, want) { + t.Errorf("message %q lacks %s", msg, want) + } + } +} diff --git a/mdl/roundtrip/allowlist_test.go b/mdl/roundtrip/allowlist_test.go index 7aaa44324c..728126edbf 100644 --- a/mdl/roundtrip/allowlist_test.go +++ b/mdl/roundtrip/allowlist_test.go @@ -107,6 +107,5 @@ var knownFailures = map[string]knownFailure{ "page MyFirstModule.Home_Web": {laws: []law{lawGetPut}, issue: "#705 #721", why: "texts and translations (#705 item 1); widget properties (#721 C)"}, // Snippets: #705 item 4 plus #721 D. - "snippet Administration.ReadMe": {laws: []law{lawGetPut}, issue: "#705 #721", why: "snippet Type (#705 item 4); widget content (#721 D)"}, - "snippet Atlas_Core.FeedbackWidget": {laws: []law{lawGetPut}, issue: "#705 #721", why: "snippet Type (#705 item 4); widget content (#721 D)"}, + "snippet Administration.ReadMe": {laws: []law{lawGetPut}, issue: "#705 #721", why: "snippet Type (#705 item 4); widget content (#721 D)"}, } diff --git a/mdl/roundtrip/flow_commit_events_rerun_test.go b/mdl/roundtrip/flow_commit_events_rerun_test.go index bace5b9857..53a530be68 100644 --- a/mdl/roundtrip/flow_commit_events_rerun_test.go +++ b/mdl/roundtrip/flow_commit_events_rerun_test.go @@ -21,6 +21,10 @@ import ( // loop and the commit together, which bypassed the "changes inside its // body" refusal: under mdl 1 the loop was silently rebuilt with new $IDs. // +// Since ako/mxcli#1106 a value edit inside the body is spliced in place, so the +// loop-body change these tests use also grows the body (growBody): a change of +// the body's shape, which is still refused. +// // The bare-commit spelling is the control for both: it always behaved. func commitEventsFlow(name, commit, value string) string { return fmt.Sprintf(`create or modify microflow MyFirstModule.%s ($L: List of System.User) @@ -35,6 +39,11 @@ end; `, name, value, commit) } +// growBody adds a statement to the loop body of a commitEventsFlow script. +func growBody(script string) string { + return strings.Replace(script, " end loop;", " log info node 'N' 'more';\n end loop;", 1) +} + func TestSpliceRerun_CommitWithEventsIsTheStoredCommit(t *testing.T) { h := newHarness(t) defer h.close() @@ -80,7 +89,7 @@ func TestSpliceRerun_CommitWithEventsIsTheStoredCommit(t *testing.T) { // replace of the loop. stored := drawnObjects(t, h.flowUnit(t, c.flow)) before = h.snapshot() - err := h.exec("mdl 1;\n" + strings.Replace(commitEventsFlow(c.flow, c.commit, "z"), "'start'", "'begin'", 1)) + err := h.exec("mdl 1;\n" + growBody(strings.Replace(commitEventsFlow(c.flow, c.commit, "z"), "'start'", "'begin'", 1))) if err == nil || !strings.Contains(err.Error(), "changes inside its body") { t.Errorf("a loop-body change: want the refusal naming the loop, got %v\n%s", err, h.out.String()) } @@ -136,7 +145,7 @@ func TestSpliceRerun_LoopBodyChangeBesideAnotherChangeIsRefused(t *testing.T) { } stored := drawnObjects(t, h.flowUnit(t, flow)) before := h.snapshot() - err := h.exec("mdl 1;\n" + commitEventsFlow(flow, c.commit, "z")) + err := h.exec("mdl 1;\n" + growBody(commitEventsFlow(flow, c.commit, "z"))) if err == nil || !strings.Contains(err.Error(), "changes inside its body") { t.Errorf("want the refusal naming the loop, got %v\n%s", err, h.out.String()) } diff --git a/mdl/roundtrip/flow_modify_test.go b/mdl/roundtrip/flow_modify_test.go index 9298a329be..758ef361e5 100644 --- a/mdl/roundtrip/flow_modify_test.go +++ b/mdl/roundtrip/flow_modify_test.go @@ -294,11 +294,13 @@ func TestFlowModify_UnspliceableChange(t *testing.T) { } } -// A change inside a loop body is not a splice: the engine does not edit -// inside a loop, and replacing the whole loop would renumber and redraw every -// node it holds — the rebuild's loss, confined to the loop but just as silent. -// So under mdl 1 it is refused, and under mdl 0 it takes the warned rebuild. -// Control: a change after the loop in the same flow is spliced. +// A change of a loop body's shape is not a splice: a body node's coordinates +// are relative to the loop box, and replacing the whole loop would renumber and +// redraw every node it holds — the rebuild's loss, confined to the loop but +// just as silent. So under mdl 1 it is refused, and under mdl 0 it takes the +// warned rebuild. A value edit of a body activity is spliced in place +// (ako/mxcli#1106), here beside an `if` in the body. Control: a change after +// the loop in the same flow is spliced. func TestFlowModify_LoopBodyChangeIsNotSpliced(t *testing.T) { h := newHarness(t) defer h.close() @@ -325,10 +327,22 @@ end;` } const target = "microflow MyFirstModule.LoopFlow" described := h.mustDescribeMdl0(t, target) - inLoop := strings.Replace(described, "'in loop'", "'in the loop'", 1) - if inLoop == described { + edited := strings.Replace(described, "'in loop'", "'in the loop'", 1) + if edited == described { t.Fatalf("describe output has no 'in loop':\n%s", described) } + if err := h.exec("mdl 1;\n" + edited); err != nil { + t.Fatalf("a value edit in the loop body under mdl 1: %v", err) + } + if !strings.Contains(h.out.String(), "(spliced: 1 replaced)") { + t.Errorf("a value edit in the loop body: want a splice, got:\n%s", h.out.String()) + } + described = h.mustDescribeMdl0(t, target) + inLoop := strings.Replace(described, "log node 'X' 'in the loop';", + "log node 'X' 'in the loop';\n log node 'X' 'grown';", 1) + if inLoop == described { + t.Fatalf("describe output has no 'in the loop':\n%s", described) + } before := h.flowUnit(t, "LoopFlow") err := h.exec("mdl 1;\n" + inLoop) @@ -345,7 +359,7 @@ end;` if !strings.Contains(h.out.String(), "Warning [MDL-V1-REBUILD]") { t.Errorf("under mdl 0 want the MDL-V1-REBUILD warning, got:\n%s", h.out.String()) } - if !strings.Contains(h.mustDescribeMdl0(t, target), "'in the loop'") { + if !strings.Contains(h.mustDescribeMdl0(t, target), "'grown'") { t.Error("under mdl 0 the rebuild did not write the change") } diff --git a/mdl/roundtrip/flow_splice_member_qualified_test.go b/mdl/roundtrip/flow_splice_member_qualified_test.go index 36f1a91f21..bae027fb4a 100644 --- a/mdl/roundtrip/flow_splice_member_qualified_test.go +++ b/mdl/roundtrip/flow_splice_member_qualified_test.go @@ -5,7 +5,6 @@ package roundtrip import ( - "bytes" "fmt" "slices" "strings" @@ -68,10 +67,6 @@ func TestFlowModify_SplicedMembersAreQualified(t *testing.T) { cases := []struct { name, from, to string - // inLoop: a change inside a loop body is not spliced (see - // TestFlowModify_LoopBodyChangeIsNotSpliced): mdl 1 refuses it and - // mdl 0 rebuilds, which must qualify the iterator's members too. - inLoop bool }{ {name: "change on a retrieve-bound variable", from: "(Name = 'r')", to: "(Name = 'r2')"}, {name: "change on a call-bound variable", from: "(Name = 'c')", to: "(Name = 'c2')"}, @@ -80,7 +75,9 @@ func TestFlowModify_SplicedMembersAreQualified(t *testing.T) { {name: "find in a call-bound list", from: "BY Name = 'x'", to: "BY Name = 'y'"}, {name: "aggregate of a call-bound list", from: "SUM $L", to: "MAXIMUM $L"}, {name: "control: change on a parameter", from: "(Name = 'p')", to: "(Name = 'p2')"}, - {name: "change on a loop iterator", from: "(Name = 'i')", to: "(Name = 'i2')", inLoop: true}, + // Spliced in place inside the loop body (ako/mxcli#1106): the + // iterator's members must be qualified there too. + {name: "change on a loop iterator", from: "(Name = 'i')", to: "(Name = 'i2')"}, } for _, header := range []string{"mdl 1;\n", ""} { for _, c := range cases { @@ -93,21 +90,12 @@ func TestFlowModify_SplicedMembersAreQualified(t *testing.T) { if edited == memberQualifiedFlow { t.Fatalf("%s: the flow has no %q", label, c.from) } - before := h.flowUnit(t, "T885_Flow") err := h.exec(header + edited) switch { - case c.inLoop && header != "": - if err == nil || !strings.Contains(err.Error(), "cannot be spliced") { - t.Errorf("%s: want the loop-body change refused, got %v", label, err) - } - if !bytes.Equal(h.flowUnit(t, "T885_Flow"), before) { - t.Errorf("%s: the refused statement wrote", label) - } - continue case err != nil: t.Errorf("%s: exec: %v", label, err) continue - case !c.inLoop && !strings.Contains(h.out.String(), "(spliced: 1 replaced)"): + case !strings.Contains(h.out.String(), "(spliced: 1 replaced)"): t.Errorf("%s: want one statement replaced by a splice, got:\n%s", label, h.out.String()) } spliced := flowMembers(t, h.flowUnit(t, "T885_Flow")) diff --git a/mdl/roundtrip/flow_splice_parity_test.go b/mdl/roundtrip/flow_splice_parity_test.go index 19f5a5ed54..2364e3bb40 100644 --- a/mdl/roundtrip/flow_splice_parity_test.go +++ b/mdl/roundtrip/flow_splice_parity_test.go @@ -150,7 +150,13 @@ func TestPedAppSpliceParity_AuthoredSpellings(t *testing.T) { // Control: a change exec refuses under mdl 1 is a refusal in diff too. t.Run("diff reports the refusal exec makes", func(t *testing.T) { - inLoop := strings.Replace(spliceParityFlow, "$Total + $R/MonthKey", "$Total + $R/MonthKey + 1", 1) + // A statement added inside the loop body: a change of its shape (a + // value edit there is spliced in place, ako/mxcli#1106). + inLoop := strings.Replace(spliceParityFlow, "set $Total = $Total + $R/MonthKey;", + "set $Total = $Total + $R/MonthKey;\n set $Total = $Total + 1;", 1) + if inLoop == spliceParityFlow { + t.Fatal("the flow has no loop-body set") + } before := h.snapshot() out := h.diff("mdl 1;\n" + inLoop) if !strings.Contains(out, "Refused: exec would stop at this error, having written nothing") || !strings.Contains(out, "cannot be spliced") { diff --git a/mdl/roundtrip/navigation_actions_980_test.go b/mdl/roundtrip/navigation_actions_980_test.go index d3f2b819a5..b144365560 100644 --- a/mdl/roundtrip/navigation_actions_980_test.go +++ b/mdl/roundtrip/navigation_actions_980_test.go @@ -20,7 +20,9 @@ import ( // is covered by the whole-fixture round trip; this adds, to the Studio // Pro-authored Responsive profile, one item per action kind in the shape // Studio Pro stores it (measured on TestApp's buttons and menu), each with -// non-default settings, plus the two properties MDL cannot spell. +// non-default settings, plus the two properties MDL cannot spell. A static +// link address carries `AttributeRef: null` here as on a page: `mx convert` +// adds it to a menu item's address written without it (measured 11.15.0). // // Each must survive describe -> exec (GetPut: nothing written) and describe // again (PutGet: the same text). Then the same description with every caption @@ -68,7 +70,7 @@ func TestNavigationMenuActionsRoundTrip_980(t *testing.T) { {Key: "$ID", Value: newID()}, {Key: "$Type", Value: "Forms$OpenLinkClientAction"}, {Key: "Address", Value: bson.D{ {Key: "$ID", Value: newID()}, {Key: "$Type", Value: "Forms$StaticOrDynamicString"}, - {Key: "IsDynamic", Value: false}, {Key: "Value", Value: "https://docs.mendix.com"}, + {Key: "AttributeRef", Value: nil}, {Key: "IsDynamic", Value: false}, {Key: "Value", Value: "https://docs.mendix.com"}, }}, {Key: "DisabledDuringExecution", Value: true}, {Key: "LinkType", Value: "Web"}, @@ -87,7 +89,7 @@ func TestNavigationMenuActionsRoundTrip_980(t *testing.T) { {Key: "$ID", Value: newID()}, {Key: "$Type", Value: "Forms$OpenLinkClientAction"}, {Key: "Address", Value: bson.D{ {Key: "$ID", Value: newID()}, {Key: "$Type", Value: "Forms$StaticOrDynamicString"}, - {Key: "IsDynamic", Value: false}, {Key: "Value", Value: "support@example.com"}, + {Key: "AttributeRef", Value: nil}, {Key: "IsDynamic", Value: false}, {Key: "Value", Value: "support@example.com"}, }}, {Key: "DisabledDuringExecution", Value: true}, {Key: "LinkType", Value: "Email"}, diff --git a/mdl/roundtrip/page_writer_carry_test.go b/mdl/roundtrip/page_writer_carry_test.go new file mode 100644 index 0000000000..e50f29081e --- /dev/null +++ b/mdl/roundtrip/page_writer_carry_test.go @@ -0,0 +1,218 @@ +// SPDX-License-Identifier: Apache-2.0 + +//go:build integration + +package roundtrip + +import ( + "fmt" + "reflect" + "sort" + "strings" + "testing" + + "go.mongodb.org/mongo-driver/v2/bson" + + "github.com/mendixlabs/mxcli/modelsdk/codec" +) + +// pageWriterShapes collects the three shapes a describe → exec of a Studio +// Pro-built page used to rewrite (ako/mxcli-ledger FINDINGS #170), in document +// order: +// +// - the key set of every Forms$StaticOrDynamicString (a static open-link +// address lost its `AttributeRef: null`); +// - every Forms$LayoutGridColumn's PreviewWidth (a stored 1 became -1); +// - the typed-array marker of every `Widgets` and `TabPages` list, tagged with +// its owner and first child (a column opening with a Forms$TabControl was +// written with 3 where Studio Pro stores 2). +// +// The pages carry other, unrelated losses (they are on the GetPut allowlist), +// so the assertion is on these shapes rather than on the unit bytes. +func pageWriterShapes(t *testing.T, raw []byte) []string { + t.Helper() + var doc bson.D + if err := bson.Unmarshal(raw, &doc); err != nil { + t.Fatalf("unmarshal unit: %v", err) + } + var out []string + var walk func(d bson.D) + walk = func(d bson.D) { + typ := "" + for _, e := range d { + if e.Key == "$Type" { + typ, _ = e.Value.(string) + } + } + switch typ { + case "Forms$StaticOrDynamicString": + var keys []string + for _, e := range d { + v := "set" + if e.Value == nil { + v = "null" + } + keys = append(keys, e.Key+"="+v) + } + sort.Strings(keys) + out = append(out, "address "+strings.Join(keys, ",")) + case "Forms$LayoutGridColumn": + for _, e := range d { + if e.Key == "PreviewWidth" { + out = append(out, fmt.Sprintf("column PreviewWidth=%v", e.Value)) + } + } + } + for _, e := range d { + switch v := e.Value.(type) { + case bson.D: + walk(v) + case bson.A: + if (e.Key == "Widgets" || e.Key == "TabPages") && len(v) > 0 { + first := "" + if len(v) > 1 { + if c, ok := v[1].(bson.D); ok { + for _, ce := range c { + if ce.Key == "$Type" { + first, _ = ce.Value.(string) + } + } + } + } + out = append(out, fmt.Sprintf("%s.%s first=%s marker=%v", typ, e.Key, first, v[0])) + } + for _, x := range v { + if c, ok := x.(bson.D); ok { + walk(c) + } + } + } + } + } + walk(doc) + return out +} + +// unitByLabel finds a unit of the committed fixture by its "$Type Name" label. +func (h *harness) unitByLabel(label string) string { + h.t.Helper() + var found string + for id, l := range h.orig.label { + if l == label { + if found != "" { + h.t.Fatalf("two units are labelled %q", label) + } + found = id + } + } + if found == "" { + h.t.Fatalf("fixture has no unit labelled %q", label) + } + return found +} + +// shapeDrift executes target's describe output and returns how the three +// shapes of unit id differ from the fixture, or nil when they do not. +func (h *harness) shapeDrift(target, id string) []string { + h.t.Helper() + defer h.restore() + described, err := h.describe(target) + if err != nil { + h.t.Fatalf("describe %s: %v", target, err) + } + if err := h.exec(described); err != nil { + h.t.Fatalf("exec describe output of %s: %v", target, err) + } + after, ok := h.snapshot().units[id] + if !ok { + h.t.Fatalf("%s disappeared", target) + } + before := pageWriterShapes(h.t, h.orig.units[id]) + got := pageWriterShapes(h.t, after) + if reflect.DeepEqual(before, got) { + return nil + } + var drift []string + n := len(before) + if len(got) > n { + n = len(got) + } + for i := 0; i < n; i++ { + var b, g string + if i < len(before) { + b = before[i] + } + if i < len(got) { + g = got[i] + } + if b != g { + drift = append(drift, fmt.Sprintf("stored %q, written %q", b, g)) + } + } + return drift +} + +// The pages are PedApp's own: MyFirstModule.Home_Web holds a static open-link +// button and the one column in the fixture whose PreviewWidth is not -1, and +// Administration.Account_Overview a layout grid column whose first widget is a +// tab control. Both are Studio Pro-authored and unmodified. +var pageWriterCarryTargets = []struct{ target, label string }{ + {"page MyFirstModule.Home_Web", "Forms$Page Home_Web"}, + {"page Administration.Account_Overview", "Forms$Page Account_Overview"}, +} + +func TestPageWriter_DescribeExecKeepsStudioProShapes(t *testing.T) { + h := newHarness(t) + defer h.close() + + // Precondition: the fixture holds every shape, or the assertion is empty. + var all []string + for _, p := range pageWriterCarryTargets { + all = append(all, pageWriterShapes(t, h.orig.units[h.unitByLabel(p.label)])...) + } + joined := strings.Join(all, "\n") + for _, want := range []string{ + "address $ID=set,$Type=set,AttributeRef=null,IsDynamic=set,Value=set", + "column PreviewWidth=1", + "Forms$LayoutGridColumn.Widgets first=Forms$TabControl marker=2", + "Forms$TabControl.TabPages first=Forms$TabPage marker=3", + } { + if !strings.Contains(joined, want) { + t.Fatalf("the fixture no longer holds %q — the test would assert nothing about it", want) + } + } + + for _, p := range pageWriterCarryTargets { + if drift := h.shapeDrift(p.target, h.unitByLabel(p.label)); drift != nil { + t.Errorf("%s: describe → exec rewrote Studio Pro's stored shapes:\n %s", p.target, strings.Join(drift, "\n ")) + } + } +} + +// CONTROL: the same comparison detects the drift when a fix is stubbed out. +// Without it the test above would pass against a comparison that never saw a +// write. Each stub reinstates the pre-fix encoder behaviour through the codec +// registry and is undone before the next. +func TestPageWriter_DescribeExecShapes_DetectsStubbedFix(t *testing.T) { + h := newHarness(t) + defer h.close() + + t.Run("static address without the AttributeRef default", func(t *testing.T) { + codec.RegisterTypeDefaults("Forms$StaticOrDynamicString", codec.TypeDefaults{}) + defer codec.RegisterTypeDefaults("Forms$StaticOrDynamicString", codec.TypeDefaults{NullFields: []string{"AttributeRef"}}) + drift := h.shapeDrift("page MyFirstModule.Home_Web", h.unitByLabel("Forms$Page Home_Web")) + if !strings.Contains(strings.Join(drift, "\n"), "AttributeRef=null") { + t.Errorf("stubbing the AttributeRef default went undetected; drift = %v", drift) + } + }) + + t.Run("Widgets marker taken from the first child", func(t *testing.T) { + // 3 is what the child-type fallback gave a Forms$TabControl first child. + codec.RegisterPropertyListMarker("Forms$LayoutGridColumn", "Widgets", 3) + defer codec.RegisterPropertyListMarker("Forms$LayoutGridColumn", "Widgets", 2) + drift := h.shapeDrift("page Administration.Account_Overview", h.unitByLabel("Forms$Page Account_Overview")) + if !strings.Contains(strings.Join(drift, "\n"), "first=Forms$TabControl marker=3") { + t.Errorf("stubbing the Widgets property marker went undetected; drift = %v", drift) + } + }) +} diff --git a/mdl/roundtrip/testapp_allowlist_test.go b/mdl/roundtrip/testapp_allowlist_test.go index 64b4c5cf7f..cccdf57d39 100644 --- a/mdl/roundtrip/testapp_allowlist_test.go +++ b/mdl/roundtrip/testapp_allowlist_test.go @@ -43,6 +43,13 @@ package roundtrip // with it, and round-trip from the start. It also struck // WorkflowCommons.ConflictedWorkflowDefinitionView_ActionConfirmation, whose only // loss was the empty OutputMappings Studio Pro stores on every nanoflow call. +// The static open-link address's null AttributeRef (ako/mxcli-ledger FINDINGS +// #170) struck WorkflowCommons.ConflictedWorkflowDefinitionView and +// WorkflowCommons.Readme, whose only loss it was. +// Keying every Widgets list's marker 2 on the property struck +// WorkflowCommons.Snip_TaskDashboard_Numbers, whose only loss it was. +// Writing an empty Widgets list as [2] on a new owner, where it had been left +// out, struck Atlas_Core.FeedbackWidget. var testAppKnownFailures = map[string]knownFailure{ "association ViewAssociations.persistent_order": {laws: []law{lawExec}, issue: "#721", why: "association: breaks exec on TestApp, measured when it joined the harness (#743); not yet triaged into #721's classes"}, "association WorkflowCommons.ConflictedWorkflowDefinitionView_WorkflowDefinition": {laws: []law{lawExec}, issue: "#721", why: "association: breaks exec on TestApp, measured when it joined the harness (#743); not yet triaged into #721's classes"}, @@ -145,8 +152,6 @@ var testAppKnownFailures = map[string]knownFailure{ "page FeedbackModule.ShareFeedback": {laws: []law{lawGetPut}, issue: "#721", why: "page: breaks getput on TestApp, measured when it joined the harness (#743); not yet triaged into #721's classes"}, "page FeedbackModule.ShareFeedback_Logo": {laws: []law{lawGetPut}, issue: "#721", why: "page: breaks getput on TestApp, measured when it joined the harness (#743); not yet triaged into #721's classes"}, "page MyFirstModule.Home_Web": {laws: []law{lawGetPut}, issue: "#721", why: "page: breaks getput on TestApp, measured when it joined the harness (#743); not yet triaged into #721's classes"}, - "page Pages.EditableLIstView": {laws: []law{lawGetPut}, issue: "#721", why: "page: breaks getput on TestApp, measured when it joined the harness (#743); not yet triaged into #721's classes"}, - "page WorkflowCommons.ConflictedWorkflowDefinitionView": {laws: []law{lawGetPut}, issue: "#721", why: "page: breaks getput on TestApp (putget struck by #721 L4), measured when it joined the harness (#743); not yet triaged into #721's classes"}, "page WorkflowCommons.DefaultWorkflowAdmin": {laws: []law{lawGetPut}, issue: "#721", why: "page: breaks getput on TestApp, measured when it joined the harness (#743); not yet triaged into #721's classes"}, "page WorkflowCommons.ManageTaskAssignments": {laws: []law{lawExec}, issue: "#721", why: "page: breaks exec on TestApp, measured when it joined the harness (#743); not yet triaged into #721's classes"}, "page WorkflowCommons.MyInitiatedWorkflowView_WithdrawConfirmation": {laws: []law{lawGetPut}, issue: "#721", why: "page: breaks getput on TestApp, measured when it joined the harness (#743); not yet triaged into #721's classes"}, @@ -180,8 +185,6 @@ var testAppKnownFailures = map[string]knownFailure{ "rule Rules.Rule2": {laws: []law{lawGetPut}, issue: "#721", why: "rule: breaks getput on TestApp, measured when it joined the harness (#743); not yet triaged into #721's classes"}, "scheduled event WorkflowCommons.SE_WorkflowAuditTrailRecord_CleanUp": {laws: []law{lawGetPut}, issue: "#721", why: "scheduled event: breaks getput on TestApp, measured when it joined the harness (#743); not yet triaged into #721's classes"}, "snippet Administration.ReadMe": {laws: []law{lawGetPut}, issue: "#721", why: "snippet: breaks getput on TestApp, measured when it joined the harness (#743); not yet triaged into #721's classes"}, - "snippet Atlas_Core.FeedbackWidget": {laws: []law{lawGetPut}, issue: "#721", why: "snippet: breaks getput on TestApp, measured when it joined the harness (#743); not yet triaged into #721's classes"}, - "snippet WorkflowCommons.Readme": {laws: []law{lawGetPut}, issue: "#721", why: "snippet: breaks getput on TestApp, measured when it joined the harness (#743); not yet triaged into #721's classes"}, "snippet WorkflowCommons.Snip_Assignee_Search": {laws: []law{lawGetPut}, issue: "#721", why: "snippet: breaks exec on TestApp, measured when it joined the harness (#743); not yet triaged into #721's classes; executes since a widget bound to a snippet parameter is described as $Param.Attr — getput not yet triaged"}, "snippet WorkflowCommons.Snip_AuditTrailViewer_Toolbar": {laws: []law{lawGetPut, lawPutGet}, issue: "#721", why: "snippet: breaks getput and putget on TestApp, measured when it joined the harness (#743); not yet triaged into #721's classes"}, "snippet WorkflowCommons.Snip_MyInitiatedWorkflowView_NotificationArea": {laws: []law{lawGetPut, lawPutGet}, issue: "#721", why: "snippet: breaks getput and putget on TestApp, measured when it joined the harness (#743); not yet triaged into #721's classes"}, @@ -191,7 +194,6 @@ var testAppKnownFailures = map[string]knownFailure{ "snippet WorkflowCommons.Snip_TaskAssignment_UserTask_Assignees": {laws: []law{lawGetPut, lawPutGet}, issue: "#721", why: "snippet: breaks getput and putget on TestApp, measured when it joined the harness (#743); not yet triaged into #721's classes"}, "snippet WorkflowCommons.Snip_TaskAssignment_UserTask_TargetUsers": {laws: []law{lawGetPut, lawPutGet}, issue: "#721", why: "snippet: breaks getput and putget on TestApp, measured when it joined the harness (#743); not yet triaged into #721's classes"}, "snippet WorkflowCommons.Snip_TaskCount_RefreshBTN": {laws: []law{lawGetPut}, issue: "#721", why: "snippet: breaks getput on TestApp, measured when it joined the harness (#743); not yet triaged into #721's classes"}, - "snippet WorkflowCommons.Snip_TaskDashboard_Numbers": {laws: []law{lawGetPut}, issue: "#721", why: "snippet: breaks getput on TestApp (putget struck by #721 L4), measured when it joined the harness (#743); not yet triaged into #721's classes"}, "snippet WorkflowCommons.Snip_TimelineViewer_Timeline_Full": {laws: []law{lawExec}, issue: "#721", why: "snippet: breaks exec on TestApp, measured when it joined the harness (#743); not yet triaged into #721's classes"}, "snippet WorkflowCommons.Snip_TimelineViewer_Timeline_Tasks": {laws: []law{lawExec}, issue: "#721", why: "snippet: breaks exec on TestApp, measured when it joined the harness (#743); not yet triaged into #721's classes"}, "snippet WorkflowCommons.Snip_UserTask_Detail": {laws: []law{lawExec}, issue: "#721", why: "snippet: breaks exec on TestApp, measured when it joined the harness (#743); not yet triaged into #721's classes"}, diff --git a/modelsdk/canon/propertyfloor.go b/modelsdk/canon/propertyfloor.go index e1028011ff..2abdbe7bdb 100644 --- a/modelsdk/canon/propertyfloor.go +++ b/modelsdk/canon/propertyfloor.go @@ -109,6 +109,18 @@ func (s *stripper) doc(d *bson.D) bool { } out := (*d)[:0] for _, e := range *d { + // A key Mendix removed at or before this version: Studio Pro discards + // it on load whatever it holds, so dropping it loses nothing and + // removes a name the merge engine does not have (#1097: an older + // writer's CloseFormAction.NumberOfPages, removed in 8.11). Measured + // rows only: the generated data's Deleted versions are wrong for keys + // Studio Pro still writes (it marks LayoutGridRow.Columns deleted in + // 8.0), and trusting it stripped ~297k live keys from a Studio + // Pro-authored 11.13 project. + if gone, ok := measuredRemoved()[typeName+"."+e.Key]; ok && s.pv.Compare(gone) >= 0 { + changed = true + continue + } since, ok := propertyFloor(typeName, e.Key) if !ok || s.pv.Compare(since) >= 0 { out = append(out, e) @@ -239,3 +251,48 @@ func parsePropertyFloors(data []byte) (map[string]version.Version, error) { } return m, nil } + +//go:embed studiopro_property_removed.json +var propertyRemovedJSON []byte + +var ( + propertyRemovedOnce sync.Once + propertyRemovedMap map[string]version.Version +) + +// propertyRemovedEntry is one measured removal: a key Mendix declared once and +// removed in Since, which `mx convert` on Seen (or later) strips. +type propertyRemovedEntry struct { + Type string `json:"type"` + Key string `json:"key"` + Since string `json:"since"` + // Seen is the version the stripping was measured on — evidence, not read. + Seen string `json:"seen"` +} + +func measuredRemoved() map[string]version.Version { + propertyRemovedOnce.Do(func() { + m, err := parsePropertyRemoved(propertyRemovedJSON) + if err != nil { + m = nil + } + propertyRemovedMap = m + }) + return propertyRemovedMap +} + +func parsePropertyRemoved(data []byte) (map[string]version.Version, error) { + var entries []propertyRemovedEntry + if err := json.Unmarshal(data, &entries); err != nil { + return nil, fmt.Errorf("removed properties: %w", err) + } + m := make(map[string]version.Version, len(entries)) + for _, e := range entries { + v := version.Parse(e.Since) + if v.IsZero() || e.Type == "" || e.Key == "" || version.Parse(e.Seen).IsZero() { + return nil, fmt.Errorf("removed properties: incomplete entry %+v", e) + } + m[e.Type+"."+e.Key] = v + } + return m, nil +} diff --git a/modelsdk/canon/propertyfloor_test.go b/modelsdk/canon/propertyfloor_test.go index 0a7eed7ae3..1481105aef 100644 --- a/modelsdk/canon/propertyfloor_test.go +++ b/modelsdk/canon/propertyfloor_test.go @@ -139,3 +139,50 @@ func TestPropertyIntroduced_FromGeneratedData(t *testing.T) { t.Error("a property with no Introduced reported a floor") } } + +// closeForm is a microflow close-page action as an older mxcli stored it: the +// string NumberOfPagesToClose plus the int NumberOfPages, which Mendix deleted +// in 8.11. Studio Pro drops the stale key on load (`mx convert`), so the name +// sets differ and `mx diff` / merge fails on the document (ako/mxcli#1097, +// measured: 4 CloseFormAction elements on an 11.15 project after every other +// key was completed). +func closeForm(t *testing.T) []byte { + t.Helper() + b, err := bson.Marshal(bson.D{ + {Key: "$ID", Value: id(1)}, + {Key: "$Type", Value: "Microflows$CloseFormAction"}, + {Key: "ErrorHandlingType", Value: "Rollback"}, + {Key: "NumberOfPages", Value: int32(1)}, + {Key: "NumberOfPagesToClose", Value: ""}, + }) + if err != nil { + t.Fatal(err) + } + return b +} + +// A key the project's version no longer declares is dropped whatever it holds: +// Mendix discards it on load, so its value reaches nothing. +func TestStripUndeclared_KeyDeletedBeforeItsVersionIsDropped(t *testing.T) { + out, err := canon.StripUndeclaredProperties(closeForm(t), ver("11.15.0")) + if err != nil { + t.Fatal(err) + } + if hasKeyAnywhere(t, out, "NumberOfPages") { + t.Error("NumberOfPages (deleted in 8.11) survived on an 11.15 project") + } + if !hasKeyAnywhere(t, out, "NumberOfPagesToClose") { + t.Error("NumberOfPagesToClose, which 11.15 declares, was dropped") + } +} + +// Control: before its deletion the key is the model's own, and stays. +func TestStripUndeclared_KeyBeforeItsDeletionStays(t *testing.T) { + out, err := canon.StripUndeclaredProperties(closeForm(t), ver("8.10.0")) + if err != nil { + t.Fatal(err) + } + if !hasKeyAnywhere(t, out, "NumberOfPages") { + t.Error("NumberOfPages was dropped on an 8.10 project, which declares it") + } +} diff --git a/modelsdk/canon/propertyset_issue1097_test.go b/modelsdk/canon/propertyset_issue1097_test.go new file mode 100644 index 0000000000..380682aa78 --- /dev/null +++ b/modelsdk/canon/propertyset_issue1097_test.go @@ -0,0 +1,46 @@ +// SPDX-License-Identifier: Apache-2.0 + +package canon + +import ( + "testing" + + "go.mongodb.org/mongo-driver/v2/bson" +) + +// ako/mxcli#1097: three keys `mx convert` added to a project built through MDL +// (Mendix 11.13.0) that the table did not have. Each value is the one Studio Pro +// itself stores on an element that does not use the property, read off the +// Studio Pro-authored fixtures: testdata/pedapp (11.13.0) holds EntityRef null on +// every unpathed AttributeRef and MessageDefinition2 "" on both FeedbackModule +// mappings; testdata/expr-checker (11.6.6) holds the same EntityRef and no +// MessageDefinition2 at all, which gen's Mappings$MappingDocument records as +// Introduced 11.10.0. +func TestCompletePropertySets_Issue1097Rows(t *testing.T) { + cases := []struct { + typ, key string + want any + since string + below string + }{ + {"DomainModels$AttributeRef", "EntityRef", nil, "11.6.6", "11.6.5"}, + {"ImportMappings$ImportMapping", "MessageDefinition2", "", "11.10.0", "11.9.9"}, + {"ExportMappings$ExportMapping", "MessageDefinition2", "", "11.10.0", "11.9.9"}, + } + for _, c := range cases { + t.Run(c.typ, func(t *testing.T) { + in := mustMarshal(t, bson.D{ + {Key: "$ID", Value: freshID()}, + {Key: "$Type", Value: "Test$Holder"}, + {Key: "El", Value: bson.D{{Key: "$ID", Value: freshID()}, {Key: "$Type", Value: c.typ}}}, + }) + got, ok := keysOf(t, CompletePropertySets(in, v(c.since)), "El")[c.key] + if !ok || got != c.want { + t.Fatalf("%s on %s: %#v (present %v), want %#v", c.key, c.since, got, ok, c.want) + } + if _, ok := keysOf(t, CompletePropertySets(in, v(c.below)), "El")[c.key]; ok { + t.Fatalf("%s added to a %s project, older than its floor %s", c.key, c.below, c.since) + } + }) + } +} diff --git a/modelsdk/canon/studiopro_property_defaults.json b/modelsdk/canon/studiopro_property_defaults.json index bf479dc81e..ec42e455fd 100644 --- a/modelsdk/canon/studiopro_property_defaults.json +++ b/modelsdk/canon/studiopro_property_defaults.json @@ -4,6 +4,7 @@ {"type":"DatabaseConnector$SimpleSqlDataType","key":"DataTypeName","since":"10.24.24","seen":"10.24.24","value":{"v":""}}, {"type":"DomainModels$AccessRule","key":"Documentation","since":"10.24.24","seen":"11.15.0","value":{"v":""}}, {"type":"DomainModels$AccessRule","key":"XPathConstraintCaption","since":"10.24.24","seen":"11.15.0","value":{"v":""}}, + {"type":"DomainModels$AttributeRef","key":"EntityRef","since":"11.6.6","seen":"11.13.0","value":{"v":null}}, {"type":"DomainModels$EntityImpl","key":"Image","since":"10.24.24","seen":"11.15.0","value":{"v":""}}, {"type":"DomainModels$EntityImpl","key":"ImageData","since":"10.24.24","seen":"11.15.0","value":{"v":{"$binary":{"base64":"","subType":"00"}}}}, {"type":"DomainModels$EntityImpl","key":"Source","since":"10.24.24","seen":"11.15.0","value":{"v":null}}, @@ -12,6 +13,7 @@ {"type":"DomainModels$RangeRuleInfo","key":"MinAttribute","since":"10.24.24","seen":"11.15.0","value":{"v":""}}, {"type":"DomainModels$RangeRuleInfo","key":"MinValue","since":"10.24.24","seen":"11.15.0","value":{"v":""}}, {"type":"DomainModels$ValidationRule","key":"Message","since":"10.24.24","seen":"11.15.0","value":{"v":{"$ID":{"$binary":{"base64":"AAAAAAAAAAAAAAAAAAAAAA==","subType":"00"}},"$Type":"Texts$Text","Items":[{"$numberInt":"3"}]}}}, + {"type":"ExportMappings$ExportMapping","key":"MessageDefinition2","since":"11.10.0","seen":"11.13.0","value":{"v":""}}, {"type":"Forms$CheckBox","key":"AriaRequired","since":"10.24.24","seen":"11.15.0","value":{"v":false}}, {"type":"Forms$CheckBox","key":"LabelPosition","since":"10.24.24","seen":"11.15.0","value":{"v":"Default"}}, {"type":"Forms$CheckBox","key":"NativeRenderMode","since":"10.24.24","seen":"11.15.0","value":{"v":"Switch"}}, @@ -39,6 +41,7 @@ {"type":"Forms$TabPage","key":"ConditionalVisibilitySettings","since":"10.24.24","seen":"11.15.0","value":{"v":null}}, {"type":"Forms$TextArea","key":"Autocomplete","since":"10.24.24","seen":"11.15.0","value":{"v":true}}, {"type":"Forms$WebLayoutContent","key":"LayoutCall","since":"10.24.24","seen":"11.15.0","value":{"v":null}}, + {"type":"ImportMappings$ImportMapping","key":"MessageDefinition2","since":"11.10.0","seen":"11.13.0","value":{"v":""}}, {"type":"Microflows$AggregateAction","key":"Expression","since":"10.24.24","seen":"11.15.0","value":{"v":""}}, {"type":"Microflows$AggregateAction","key":"ReduceInitialValueExpression","since":"10.24.24","seen":"11.15.0","value":{"v":""}}, {"type":"Microflows$AggregateAction","key":"ReduceReturnDataType","since":"10.24.24","seen":"11.15.0","value":{"v":{"$ID":{"$binary":{"base64":"AAAAAAAAAAAAAAAAAAAAAA==","subType":"00"}},"$Type":"DataTypes$BooleanType"}}}, diff --git a/modelsdk/canon/studiopro_property_removed.json b/modelsdk/canon/studiopro_property_removed.json new file mode 100644 index 0000000000..593d65a8c2 --- /dev/null +++ b/modelsdk/canon/studiopro_property_removed.json @@ -0,0 +1,3 @@ +[ + {"type":"Microflows$CloseFormAction","key":"NumberOfPages","since":"8.11.0","seen":"11.15.0"} +] diff --git a/modelsdk/canon/userroleguid.go b/modelsdk/canon/userroleguid.go new file mode 100644 index 0000000000..2b58438c69 --- /dev/null +++ b/modelsdk/canon/userroleguid.go @@ -0,0 +1,91 @@ +// SPDX-License-Identifier: Apache-2.0 + +package canon + +import ( + "go.mongodb.org/mongo-driver/v2/bson" +) + +// FillMissingUserRoleGUIDs gives every Security$UserRole in contents that has +// NO GUID key the GUID mxcli's create path mints for one: its own $ID, as +// binary subtype 0 (codec EmitGUID). A role that has a GUID keeps it, whatever +// it holds. It returns contents unchanged — the same slice — when nothing was +// filled. +// +// # Why +// +// Studio Pro gives every user role a GUID; mxcli wrote roles without one +// before the create path minted it (mendixlabs/mxcli#1373). Such a role is a +// property-name gap Mendix's merge engine throws on, and — the part a constant +// default cannot fix, so it is not a row in the CompletePropertySets table — +// with no GUID in the model mxbuild invents a RANDOM one on every build. +// Measured on Mendix 11.15.0 with PostgreSQL: the runtime matches +// system$userrole rows by modelguid first and by name second, and never +// refreshes modelguid on a name match. Filling GUID = $ID kept the row id and +// every user-to-role link. +// +// # Absent only, never changed +// +// An existing GUID is the runtime's primary match key for the role's row; when +// it matches, it is the row's identity, and changing it is #1119 for users +// instead of columns. So this fills an ABSENT key and nothing else, and the +// storage-GUID guard (which compares a GUID present on both sides) still +// refuses any write that changes one. +// +// The fill makes a later rename safe only for a database first synchronised +// after it: one synchronised before keeps the stale random modelguid and is +// still matched by name, so a rename there still deletes the row. +func FillMissingUserRoleGUIDs(contents []byte) []byte { + if len(contents) == 0 { + return contents + } + var doc bson.D + if err := bson.Unmarshal(contents, &doc); err != nil { + return contents + } + if !fillUserRoleGUIDs(&doc) { + return contents + } + out, err := bson.Marshal(doc) + if err != nil { + return contents + } + return out +} + +func fillUserRoleGUIDs(d *bson.D) bool { + changed := false + for i := range *d { + if fillUserRoleGUIDsIn(&(*d)[i].Value) { + changed = true + } + } + if t, _ := lookupString(*d, "$Type"); t != "Security$UserRole" || hasKey(*d, "GUID") { + return changed + } + id := elementIDBytes(*d) + if id == nil { + return changed + } + *d = append(*d, bson.E{Key: "GUID", Value: bson.Binary{Subtype: 0, Data: append([]byte(nil), id...)}}) + return true +} + +func fillUserRoleGUIDsIn(v *any) bool { + switch x := (*v).(type) { + case bson.D: + if fillUserRoleGUIDs(&x) { + *v = x + return true + } + case bson.A: + changed := false + for i := range x { + if fillUserRoleGUIDsIn(&x[i]) { + changed = true + } + } + return changed + } + return false +} diff --git a/modelsdk/canon/userroleguid_test.go b/modelsdk/canon/userroleguid_test.go new file mode 100644 index 0000000000..79bb5308a4 --- /dev/null +++ b/modelsdk/canon/userroleguid_test.go @@ -0,0 +1,74 @@ +// SPDX-License-Identifier: Apache-2.0 + +package canon + +import ( + "bytes" + "testing" + + "go.mongodb.org/mongo-driver/v2/bson" +) + +func projectSecurityWithRoles(t *testing.T, roles ...bson.D) []byte { + t.Helper() + list := bson.A{int32(2)} + for _, r := range roles { + list = append(list, r) + } + return mustMarshal(t, bson.D{ + {Key: "$ID", Value: freshID()}, + {Key: "$Type", Value: "Security$ProjectSecurity"}, + {Key: "UserRoles", Value: list}, + }) +} + +func userRole(name string, id bson.Binary, guid *bson.Binary) bson.D { + d := bson.D{{Key: "$ID", Value: id}, {Key: "$Type", Value: "Security$UserRole"}, {Key: "Name", Value: name}} + if guid != nil { + d = append(d, bson.E{Key: "GUID", Value: *guid}) + } + return d +} + +func roleGUIDs(t *testing.T, raw []byte) map[string]*bson.Binary { + t.Helper() + out := map[string]*bson.Binary{} + vals, _ := bson.Raw(raw).Lookup("UserRoles").Array().Values() + for _, v := range vals { + doc, ok := v.DocumentOK() + if !ok { + continue + } + name := doc.Lookup("Name").StringValue() + if g, err := doc.LookupErr("GUID"); err == nil { + st, data := g.Binary() + out[name] = &bson.Binary{Subtype: st, Data: data} + } else { + out[name] = nil + } + } + return out +} + +// A GUID-less user role gets GUID = its own $ID — what the create path mints +// (codec EmitGUID) — and a role that HAS a GUID keeps it, whatever it is: the +// runtime matches system$userrole rows by that GUID first. +func TestFillMissingUserRoleGUIDs(t *testing.T) { + bare, kept, other := freshID(), freshID(), freshID() + in := projectSecurityWithRoles(t, userRole("Bare", bare, nil), userRole("Kept", kept, &other)) + + out := FillMissingUserRoleGUIDs(in) + got := roleGUIDs(t, out) + if g := got["Bare"]; g == nil || g.Subtype != 0 || !bytes.Equal(g.Data, bare.Data) { + t.Errorf("GUID-less role: GUID = %v, want its $ID %x as subtype 0", g, bare.Data) + } + if g := got["Kept"]; g == nil || !bytes.Equal(g.Data, other.Data) { + t.Errorf("a role's existing GUID was changed: %v, want %x", g, other.Data) + } + + // Nothing to fill returns the same bytes, not a re-marshalled copy. + again := FillMissingUserRoleGUIDs(out) + if &again[0] != &out[0] { + t.Error("a unit with nothing to fill was re-marshalled") + } +} diff --git a/modelsdk/codec/defaults.go b/modelsdk/codec/defaults.go index bba6019a22..dd3c07b526 100644 --- a/modelsdk/codec/defaults.go +++ b/modelsdk/codec/defaults.go @@ -112,6 +112,29 @@ func RegisterPropertyListMarker(ownerType, key string, marker int32) { propertyListMarkers[ownerType][key] = marker } +// mandatoryPropertyLists marks the owner/key pairs Studio Pro serializes on +// every element of the owner type, empty or not. RegisterMandatoryPropertyList +// fills it; the encoder writes such a list as bson.A{marker} on a new element +// whose list is empty, where it would otherwise omit the key. +var mandatoryPropertyLists = map[string]map[string]bool{} + +// RegisterMandatoryPropertyList is RegisterPropertyListMarker for a PartList +// Studio Pro always writes: the marker applies as there, and a new element of +// ownerType with nothing in the list still writes the key as the bare marker. +// It is keyed on the owner, so it never adds the key to another type, and it +// merges with — rather than replaces — the owner's RegisterTypeDefaults. +func RegisterMandatoryPropertyList(ownerType, key string, marker int32) { + RegisterPropertyListMarker(ownerType, key, marker) + if mandatoryPropertyLists[ownerType] == nil { + mandatoryPropertyLists[ownerType] = map[string]bool{} + } + mandatoryPropertyLists[ownerType][key] = true +} + +func isMandatoryPropertyList(ownerType, key string) bool { + return mandatoryPropertyLists[ownerType][key] +} + func lookupPropertyListMarker(ownerType, key string) (int32, bool) { m, ok := propertyListMarkers[ownerType][key] return m, ok diff --git a/modelsdk/codec/encoder.go b/modelsdk/codec/encoder.go index 1585510f53..836091c242 100644 --- a/modelsdk/codec/encoder.go +++ b/modelsdk/codec/encoder.go @@ -160,6 +160,16 @@ func (e *Encoder) buildDoc(elem element.Element) (bson.D, error) { for _, prop := range elem.Properties() { idx := findRebuild(bytesOf(prop.Name())) if idx < 0 { + // An untouched PartList is empty on a new element. One Studio + // Pro always writes is emitted as its bare marker, in place. + if clp, isList := prop.(element.ChildListProperty); isList && + len(clp.ChildElements()) == 0 && + isMandatoryPropertyList(elem.TypeName(), prop.Name()) && + !e.omits(elem.TypeName(), prop.Name()) { + m, _ := lookupPropertyListMarker(elem.TypeName(), prop.Name()) + doc = append(doc, bson.E{Key: prop.Name(), Value: bson.A{m}}) + emitted[prop.Name()] = true + } continue } val, err := e.encodeEntry(elem.TypeName(), rebuild[idx]) diff --git a/modelsdk/codec/mandatory_list_test.go b/modelsdk/codec/mandatory_list_test.go new file mode 100644 index 0000000000..1ef6a4609a --- /dev/null +++ b/modelsdk/codec/mandatory_list_test.go @@ -0,0 +1,69 @@ +// SPDX-License-Identifier: Apache-2.0 + +package codec + +import ( + "testing" + + "github.com/mendixlabs/mxcli/modelsdk/element" + "github.com/mendixlabs/mxcli/modelsdk/property" + "go.mongodb.org/mongo-driver/bson" +) + +type listProbe struct { + element.Base + name *property.Primitive[string] + widgets *property.PartList[element.Element] +} + +func newListProbe(typeName string) *listProbe { + o := &listProbe{} + o.SetTypeName(typeName) + o.name = property.NewPrimitive[string]("Name", property.DecodeString) + o.name.Bind(&o.Base, 0) + o.widgets = property.NewPartList[element.Element]("Widgets") + o.widgets.Bind(&o.Base, 1) + o.SetProperties([]element.Property{o.name, o.widgets}) + o.name.Set("probe") + return o +} + +// A new element whose mandatory list is empty writes the key as the bare +// registered marker, in the property's position; an unregistered owner with +// the same empty list still omits it (Studio Pro stores the key on some owners +// and not others, so it is never a property-name rule). +func TestMandatoryPropertyListEmittedWhenEmpty(t *testing.T) { + const owner = "Probe$MandatoryWidgets" + RegisterMandatoryPropertyList(owner, "Widgets", 2) + + b, err := (&Encoder{}).Encode(newListProbe(owner)) + if err != nil { + t.Fatal(err) + } + var d bson.D + if err := bson.Unmarshal(b, &d); err != nil { + t.Fatal(err) + } + if len(d) != 4 || d[3].Key != "Widgets" { + t.Fatalf("keys = %v, want $ID $Type Name Widgets", d) + } + if a, ok := d[3].Value.(bson.A); !ok || len(a) != 1 || a[0] != int32(2) { + t.Errorf("Widgets = %v, want [2]", d[3].Value) + } + + b, err = (&Encoder{OmitKeys: map[string]map[string]bool{owner: {"Widgets": true}}}).Encode(newListProbe(owner)) + if err != nil { + t.Fatal(err) + } + if _, err := bson.Raw(b).LookupErr("Widgets"); err == nil { + t.Error("mandatory list emitted despite OmitKeys") + } + + b, err = (&Encoder{}).Encode(newListProbe("Probe$Unregistered")) + if err != nil { + t.Fatal(err) + } + if _, err := bson.Raw(b).LookupErr("Widgets"); err == nil { + t.Error("an unregistered owner gained an empty Widgets key") + } +} diff --git a/modelsdk/mpr/serialize_mappings.go b/modelsdk/mpr/serialize_mappings.go deleted file mode 100644 index 6e4bed2cec..0000000000 --- a/modelsdk/mpr/serialize_mappings.go +++ /dev/null @@ -1,397 +0,0 @@ -// SPDX-License-Identifier: Apache-2.0 - -package mpr - -import ( - "github.com/mendixlabs/mxcli/model" - - "go.mongodb.org/mongo-driver/v2/bson" -) - -// SerializeImportMapping returns BSON bytes for an import mapping unit. -func SerializeImportMapping(im *model.ImportMapping) ([]byte, error) { - elements := bson.A{int32(2)} - for _, elem := range im.Elements { - elements = append(elements, serImportMappingElement(elem, "(Object)")) - } - - exportLevel := im.ExportLevel - if exportLevel == "" { - exportLevel = "Hidden" - } - - // ParameterType is a required sub-document even when unused: an - // unparameterised mapping stores the DataTypes$UnknownType marker, and one - // declaring an input object stores a DataTypes$ObjectType naming it (#265). - // Without the property Studio Pro fails to render the schema source and - // mapping elements correctly. - parameterType := bson.D{ - {Key: "$ID", Value: idToBsonBinary(generateUUID())}, - {Key: "$Type", Value: "DataTypes$UnknownType"}, - } - if im.ParameterEntity != "" { - parameterType = bson.D{ - {Key: "$ID", Value: idToBsonBinary(generateUUID())}, - {Key: "$Type", Value: "DataTypes$ObjectType"}, - {Key: "Entity", Value: im.ParameterEntity}, - } - } - - doc := bson.D{ - {Key: "$ID", Value: idToBsonBinary(string(im.ID))}, - {Key: "$Type", Value: "ImportMappings$ImportMapping"}, - {Key: "Name", Value: im.Name}, - {Key: "Documentation", Value: im.Documentation}, - {Key: "Excluded", Value: im.Excluded}, - {Key: "ExportLevel", Value: exportLevel}, - {Key: "JsonStructure", Value: im.JsonStructure}, - {Key: "XmlSchema", Value: im.XmlSchema}, - {Key: "Elements", Value: elements}, - // Required fields with defaults — verified against Studio Pro-created BSON - {Key: "UseSubtransactionsForMicroflows", Value: false}, - {Key: "PublicName", Value: ""}, - {Key: "XsdRootElementName", Value: ""}, - {Key: "MappingSourceReference", Value: nil}, - {Key: "ParameterType", Value: parameterType}, - {Key: "OperationName", Value: ""}, - {Key: "ServiceName", Value: ""}, - {Key: "WsdlFile", Value: ""}, - } - // MessageDefinition's presence is carried: Mendix 11.15 removed the key - // (ako/mxcli#987), so it is written only when the model has it. - if im.MessageDefinition != nil { - doc = insertAfterKey(doc, "XmlSchema", bson.E{Key: "MessageDefinition", Value: *im.MessageDefinition}) - } - return bson.Marshal(doc) -} - -func serImportMappingElement(elem *model.ImportMappingElement, parentPath string) bson.D { - id := string(elem.ID) - if id == "" { - id = generateUUID() - } - - if isMappingObjectKind(elem.Kind) { - return serImportObjectElement(id, elem, parentPath) - } - return serImportValueElement(id, elem, parentPath) -} - -func serImportObjectElement(id string, elem *model.ImportMappingElement, parentPath string) bson.D { - jsonPath := elem.JsonPath - if jsonPath == "" { - if elem.ExposedName == "" { - jsonPath = parentPath - } else { - jsonPath = parentPath + "|" + elem.ExposedName - } - } - - children := bson.A{int32(2)} - for _, child := range elem.Children { - children = append(children, serImportMappingElement(child, jsonPath)) - } - - objectHandling := elem.ObjectHandling - if objectHandling == "" { - objectHandling = "Create" - } - // The backup takes {Create, Error, Ignore} only; copying the HANDLING into - // it wrote "Find"/"Custom", which occur in 0 of 1,261 real elements (#261). - objectHandlingBackup := "Create" - switch elem.ObjectHandlingBackup { - case "Create", "Error", "Ignore": - objectHandlingBackup = elem.ObjectHandlingBackup - } - if objectHandling == "FindOrCreate" { - objectHandling = "Find" - } - - return bson.D{ - {Key: "$ID", Value: idToBsonBinary(id)}, - {Key: "$Type", Value: "ImportMappings$ObjectMappingElement"}, - {Key: "Entity", Value: elem.Entity}, - {Key: "ExposedName", Value: elem.ExposedName}, - {Key: "JsonPath", Value: jsonPath}, - {Key: "XmlPath", Value: elem.XmlPath}, - {Key: "ObjectHandling", Value: objectHandling}, - {Key: "ObjectHandlingBackup", Value: objectHandlingBackup}, - {Key: "ObjectHandlingBackupAllowOverride", Value: elem.BackupAllowOverride}, - {Key: "Association", Value: elem.Association}, - {Key: "Children", Value: children}, - {Key: "MinOccurs", Value: int32(elem.MinOccurs)}, - {Key: "MaxOccurs", Value: int32(elem.MaxOccurs)}, - {Key: "Nillable", Value: elem.Nillable}, - {Key: "IsDefaultType", Value: false}, - {Key: "ElementType", Value: serElementTypeForKind(elem.Kind)}, - {Key: "Documentation", Value: ""}, - {Key: "CustomHandlerCall", Value: nil}, - } -} - -func serImportValueElement(id string, elem *model.ImportMappingElement, parentPath string) bson.D { - dataType := serMappingValueDataType(elem.DataType) - jsonPath := elem.JsonPath - if jsonPath == "" { - jsonPath = parentPath + "|" + elem.ExposedName - } - - return bson.D{ - {Key: "$ID", Value: idToBsonBinary(id)}, - {Key: "$Type", Value: "ImportMappings$ValueMappingElement"}, - {Key: "Attribute", Value: elem.Attribute}, - {Key: "ExposedName", Value: elem.ExposedName}, - {Key: "JsonPath", Value: jsonPath}, - {Key: "XmlPath", Value: elem.XmlPath}, - {Key: "IsKey", Value: elem.IsKey}, - {Key: "Type", Value: dataType}, - {Key: "MinOccurs", Value: int32(elem.MinOccurs)}, - {Key: "MaxOccurs", Value: int32(elem.MaxOccurs)}, - {Key: "Nillable", Value: elem.Nillable}, - {Key: "IsDefaultType", Value: false}, - {Key: "ElementType", Value: "Value"}, - {Key: "Documentation", Value: ""}, - {Key: "Converter", Value: elem.Converter}, - {Key: "FractionDigits", Value: int32(elem.FractionDigits)}, - {Key: "TotalDigits", Value: int32(elem.TotalDigits)}, - {Key: "MaxLength", Value: int32(elem.MaxLength)}, - {Key: "IsContent", Value: false}, - {Key: "IsXmlAttribute", Value: false}, - {Key: "OriginalValue", Value: elem.OriginalValue}, - {Key: "XmlPrimitiveType", Value: serXmlPrimitiveTypeName(elem.DataType)}, - } -} - -// SerializeExportMapping returns BSON bytes for an export mapping unit. -func SerializeExportMapping(em *model.ExportMapping) ([]byte, error) { - elements := bson.A{int32(2)} - for _, elem := range em.Elements { - elements = append(elements, serExportMappingElement(elem, "(Object)")) - } - - exportLevel := em.ExportLevel - if exportLevel == "" { - exportLevel = "Hidden" - } - - nullValueOption := em.NullValueOption - if nullValueOption == "" { - nullValueOption = "LeaveOutElement" - } - - doc := bson.D{ - {Key: "$ID", Value: idToBsonBinary(string(em.ID))}, - {Key: "$Type", Value: "ExportMappings$ExportMapping"}, - {Key: "Name", Value: em.Name}, - {Key: "Documentation", Value: em.Documentation}, - {Key: "Excluded", Value: em.Excluded}, - {Key: "ExportLevel", Value: exportLevel}, - {Key: "JsonStructure", Value: em.JsonStructure}, - {Key: "XmlSchema", Value: em.XmlSchema}, - {Key: "NullValueOption", Value: nullValueOption}, - {Key: "Elements", Value: elements}, - // Required fields with defaults — verified against Studio Pro-created BSON - {Key: "PublicName", Value: ""}, - {Key: "XsdRootElementName", Value: ""}, - {Key: "IsHeaderParameter", Value: false}, - {Key: "ParameterName", Value: ""}, - {Key: "OperationName", Value: ""}, - {Key: "ServiceName", Value: ""}, - {Key: "WsdlFile", Value: ""}, - {Key: "MappingSourceReference", Value: nil}, - } - // MessageDefinition's presence is carried: Mendix 11.15 removed the key - // (ako/mxcli#987), so it is written only when the model has it. - if em.MessageDefinition != nil { - doc = insertAfterKey(doc, "XmlSchema", bson.E{Key: "MessageDefinition", Value: *em.MessageDefinition}) - } - return bson.Marshal(doc) -} - -func serExportMappingElement(elem *model.ExportMappingElement, parentPath string) bson.D { - id := string(elem.ID) - if id == "" { - id = generateUUID() - } - - if isMappingObjectKind(elem.Kind) { - return serExportObjectElement(id, elem, parentPath) - } - return serExportValueElement(id, elem, parentPath) -} - -func serExportObjectElement(id string, elem *model.ExportMappingElement, parentPath string) bson.D { - jsonPath := elem.JsonPath - if jsonPath == "" { - if elem.ExposedName == "" { - jsonPath = parentPath - } else { - jsonPath = parentPath + "|" + elem.ExposedName - } - } - - children := bson.A{int32(2)} - for _, child := range elem.Children { - children = append(children, serExportMappingElement(child, jsonPath)) - } - - objectHandling := elem.ObjectHandling - if objectHandling == "" { - objectHandling = "Parameter" - } - - maxOccurs := int32(elem.MaxOccurs) - - return bson.D{ - {Key: "$ID", Value: idToBsonBinary(id)}, - {Key: "$Type", Value: "ExportMappings$ObjectMappingElement"}, - {Key: "Entity", Value: elem.Entity}, - {Key: "ExposedName", Value: elem.ExposedName}, - {Key: "JsonPath", Value: jsonPath}, - {Key: "XmlPath", Value: elem.XmlPath}, - {Key: "ObjectHandling", Value: objectHandling}, - // Every export object element in the demo apps stores Error (537 of - // 537), and the handling values are not in the backup enum (#261). - {Key: "ObjectHandlingBackup", Value: "Error"}, - {Key: "ObjectHandlingBackupAllowOverride", Value: false}, - {Key: "Association", Value: elem.Association}, - {Key: "Children", Value: children}, - // A schema ROOT has MinOccurs 1; hardcoding 0 lost that (#279). - {Key: "MinOccurs", Value: int32(elem.MinOccurs)}, - {Key: "MaxOccurs", Value: maxOccurs}, - {Key: "Nillable", Value: true}, - {Key: "IsDefaultType", Value: false}, - {Key: "ElementType", Value: serElementTypeForKind(elem.Kind)}, - {Key: "Documentation", Value: ""}, - {Key: "CustomHandlerCall", Value: nil}, - } -} - -func serExportValueElement(id string, elem *model.ExportMappingElement, parentPath string) bson.D { - dataType := serMappingValueDataType(elem.DataType) - jsonPath := elem.JsonPath - if jsonPath == "" { - jsonPath = parentPath + "|" + elem.ExposedName - } - - return bson.D{ - {Key: "$ID", Value: idToBsonBinary(id)}, - {Key: "$Type", Value: "ExportMappings$ValueMappingElement"}, - {Key: "Attribute", Value: elem.Attribute}, - {Key: "ExposedName", Value: elem.ExposedName}, - {Key: "JsonPath", Value: jsonPath}, - {Key: "XmlPath", Value: elem.XmlPath}, - {Key: "Type", Value: dataType}, - // A schema ROOT has MinOccurs 1; hardcoding 0 lost that (#279). - {Key: "MinOccurs", Value: int32(elem.MinOccurs)}, - // Mirror the bound schema element: Mendix cross-validates the two and - // reports CE5015 on any mismatch. Hardcoding 0 only worked while the - // JSON structure also wrote 0 for every element (#841). - {Key: "MaxOccurs", Value: int32(elem.MaxOccurs)}, - {Key: "Nillable", Value: true}, - {Key: "IsDefaultType", Value: false}, - {Key: "ElementType", Value: "Value"}, - {Key: "Documentation", Value: ""}, - {Key: "Converter", Value: elem.Converter}, - {Key: "FractionDigits", Value: int32(-1)}, - {Key: "TotalDigits", Value: int32(-1)}, - // Mirrors the bound schema element, like MaxOccurs: Studio Pro stores 0 for - // a string element and -1 for a numeric one (#277). - {Key: "MaxLength", Value: int32(elem.MaxLength)}, - // Studio Pro writes IsKey on export value elements too; omitting it was a - // divergence from the import twin (#277). - {Key: "IsKey", Value: elem.IsKey}, - {Key: "IsContent", Value: false}, - {Key: "IsXmlAttribute", Value: false}, - {Key: "OriginalValue", Value: ""}, - {Key: "XmlPrimitiveType", Value: serXmlPrimitiveTypeName(elem.DataType)}, - } -} - -// serXmlPrimitiveTypeName maps a Mendix data type name to an XML primitive type name. -func serXmlPrimitiveTypeName(dataType string) string { - switch dataType { - case "Integer", "Long": - return "Integer" - case "Decimal": - return "Decimal" - case "Boolean": - return "Boolean" - case "DateTime": - return "DateTime" - default: - return "String" - } -} - -// serElementTypeForKind maps model Kind to BSON ElementType. -func serElementTypeForKind(kind string) string { - if kind == "Array" { - return "Array" - } - if kind == "Value" { - return "Value" - } - return "Object" -} - -// serMappingValueDataType converts a simple type name to a DataTypes$ BSON object. -func serMappingValueDataType(typeName string) bson.D { - typeID := idToBsonBinary(GenerateID()) - switch typeName { - case "Integer", "Long": - return bson.D{ - {Key: "$ID", Value: typeID}, - {Key: "$Type", Value: "DataTypes$IntegerType"}, - } - case "Decimal": - return bson.D{ - {Key: "$ID", Value: typeID}, - {Key: "$Type", Value: "DataTypes$DecimalType"}, - } - case "Boolean": - return bson.D{ - {Key: "$ID", Value: typeID}, - {Key: "$Type", Value: "DataTypes$BooleanType"}, - } - case "DateTime": - return bson.D{ - {Key: "$ID", Value: typeID}, - {Key: "$Type", Value: "DataTypes$DateTimeType"}, - } - case "Binary": - return bson.D{ - {Key: "$ID", Value: typeID}, - {Key: "$Type", Value: "DataTypes$BinaryType"}, - } - default: // String - return bson.D{ - {Key: "$ID", Value: typeID}, - {Key: "$Type", Value: "DataTypes$StringType"}, - } - } -} - -// isMappingObjectKind — see the note in mdl/backend/modelsdk/mapping_write.go. -func isMappingObjectKind(kind string) bool { - switch kind { - case "Object", "Array", "Wrapper": - return true - default: - return false - } -} - -// insertAfterKey inserts e directly after the element named key, or appends it -// when the key is absent. -func insertAfterKey(doc bson.D, key string, e bson.E) bson.D { - for i, el := range doc { - if el.Key == key { - out := make(bson.D, 0, len(doc)+1) - out = append(out, doc[:i+1]...) - out = append(out, e) - return append(out, doc[i+1:]...) - } - } - return append(doc, e) -} diff --git a/modelsdk/mpr/serialize_web_services.go b/modelsdk/mpr/serialize_web_services.go deleted file mode 100644 index 16194202b6..0000000000 --- a/modelsdk/mpr/serialize_web_services.go +++ /dev/null @@ -1,785 +0,0 @@ -// SPDX-License-Identifier: Apache-2.0 - -package mpr - -import ( - "strings" - - "github.com/mendixlabs/mxcli/model" - - "go.mongodb.org/mongo-driver/v2/bson" -) - -// ============================================================================ -// Consumed OData Service — Rest$ConsumedODataService -// ============================================================================ - -// SerializeConsumedODataService returns BSON bytes for a consumed OData service unit. -func SerializeConsumedODataService(svc *model.ConsumedODataService) ([]byte, error) { - doc := bson.D{ - {Key: "$ID", Value: idToBsonBinary(string(svc.ID))}, - {Key: "$Type", Value: "Rest$ConsumedODataService"}, - {Key: "Name", Value: svc.Name}, - {Key: "Documentation", Value: svc.Documentation}, - {Key: "Version", Value: svc.Version}, - {Key: "ServiceName", Value: svc.ServiceName}, - {Key: "ODataVersion", Value: svc.ODataVersion}, - {Key: "MetadataUrl", Value: svc.MetadataUrl}, - {Key: "TimeoutExpression", Value: svc.TimeoutExpression}, - {Key: "ProxyType", Value: svc.ProxyType}, - {Key: "Description", Value: svc.Description}, - {Key: "Validated", Value: svc.Validated}, - {Key: "Excluded", Value: svc.Excluded}, - {Key: "ExportLevel", Value: "Hidden"}, - {Key: "Metadata", Value: svc.Metadata}, - {Key: "MetadataHash", Value: svc.MetadataHash}, - {Key: "MetadataReferences", Value: bson.A{int32(0)}}, - {Key: "ValidatedEntities", Value: bson.A{int32(0)}}, - {Key: "LastUpdated", Value: ""}, - {Key: "UseQuerySegment", Value: false}, - {Key: "MinimumMxVersion", Value: ""}, - {Key: "RecommendedMxVersion", Value: ""}, - } - - if svc.ConfigurationMicroflow != "" { - doc = append(doc, bson.E{Key: "ConfigurationMicroflow", Value: svc.ConfigurationMicroflow}) - } - if svc.ErrorHandlingMicroflow != "" { - doc = append(doc, bson.E{Key: "ErrorHandlingMicroflow", Value: svc.ErrorHandlingMicroflow}) - } - if svc.ProxyHost != "" { - doc = append(doc, bson.E{Key: "ProxyHost", Value: svc.ProxyHost}) - } - if svc.ProxyPort != "" { - doc = append(doc, bson.E{Key: "ProxyPort", Value: svc.ProxyPort}) - } - if svc.ProxyUsername != "" { - doc = append(doc, bson.E{Key: "ProxyUsername", Value: svc.ProxyUsername}) - } - if svc.ProxyPassword != "" { - doc = append(doc, bson.E{Key: "ProxyPassword", Value: svc.ProxyPassword}) - } - if svc.ApplicationId != "" { - doc = append(doc, bson.E{Key: "ApplicationId", Value: svc.ApplicationId}) - } - if svc.EndpointId != "" { - doc = append(doc, bson.E{Key: "EndpointId", Value: svc.EndpointId}) - } - if svc.CatalogUrl != "" { - doc = append(doc, bson.E{Key: "CatalogUrl", Value: svc.CatalogUrl}) - } - if svc.EnvironmentType != "" { - doc = append(doc, bson.E{Key: "EnvironmentType", Value: svc.EnvironmentType}) - } - - doc = append(doc, bson.E{Key: "HttpConfiguration", Value: serWebHttpConfiguration(svc.HttpConfiguration)}) - - return bson.Marshal(doc) -} - -// serWebHttpConfiguration converts an HttpConfiguration to a BSON map. -func serWebHttpConfiguration(cfg *model.HttpConfiguration) bson.D { - cfgID := generateUUID() - if cfg != nil && cfg.ID != "" { - cfgID = string(cfg.ID) - } - - useHTTPAuthentication := false - httpAuthenticationUserName := "" - httpAuthenticationPassword := "" - httpMethod := "Post" - overrideLocation := false - customLocation := "" - clientCertificate := "" - var headerEntries bson.A - - if cfg != nil { - useHTTPAuthentication = cfg.UseAuthentication - httpAuthenticationUserName = cfg.Username - httpAuthenticationPassword = cfg.Password - if cfg.HttpMethod != "" { - httpMethod = cfg.HttpMethod - } - overrideLocation = cfg.OverrideLocation - customLocation = cfg.CustomLocation - clientCertificate = cfg.ClientCertificate - - if len(cfg.HeaderEntries) > 0 { - headers := bson.A{int32(3)} - for _, h := range cfg.HeaderEntries { - hID := string(h.ID) - if hID == "" { - hID = generateUUID() - } - headers = append(headers, bson.D{ - {Key: "$ID", Value: idToBsonBinary(hID)}, - {Key: "$Type", Value: "Microflows$HttpHeaderEntry"}, - {Key: "Key", Value: h.Key}, - {Key: "Value", Value: h.Value}, - }) - } - headerEntries = headers - } - } - - doc := bson.D{ - {Key: "$ID", Value: idToBsonBinary(cfgID)}, - {Key: "$Type", Value: "Microflows$HttpConfiguration"}, - {Key: "UseHttpAuthentication", Value: useHTTPAuthentication}, - {Key: "HttpAuthenticationUserName", Value: httpAuthenticationUserName}, - {Key: "HttpAuthenticationPassword", Value: httpAuthenticationPassword}, - {Key: "HttpMethod", Value: httpMethod}, - {Key: "OverrideLocation", Value: overrideLocation}, - {Key: "CustomLocation", Value: customLocation}, - {Key: "ClientCertificate", Value: clientCertificate}, - } - - if headerEntries != nil { - doc = append(doc, bson.E{Key: "HttpHeaderEntries", Value: headerEntries}) - } - - return doc -} - -// ============================================================================ -// Published OData Service — ODataPublish$PublishedODataService2 -// ============================================================================ - -// SerializePublishedODataService returns BSON bytes for a published OData service unit. -func SerializePublishedODataService(svc *model.PublishedODataService) ([]byte, error) { - // Authentication types array (versioned: starts with int32(3)) - authTypes := bson.A{int32(3)} - for _, at := range svc.AuthenticationTypes { - authTypes = append(authTypes, at) - } - - // Serialize entity types and build ID map for entity set pointers - entityTypeIDMap := make(map[string]string) - entityTypes := bson.A{} - for _, et := range svc.EntityTypes { - etID := string(et.ID) - if etID == "" { - etID = generateUUID() - et.ID = model.ID(etID) - } - entityTypeIDMap[et.ExposedName] = etID - entityTypes = append(entityTypes, serWebPublishedEntityType(et)) - } - - entitySets := bson.A{} - for _, es := range svc.EntitySets { - esID := string(es.ID) - if esID == "" { - esID = generateUUID() - es.ID = model.ID(esID) - } - entityTypeID := entityTypeIDMap[es.EntityTypeName] - entitySets = append(entitySets, serWebPublishedEntitySet(es, entityTypeID)) - } - - doc := bson.D{ - {Key: "$ID", Value: idToBsonBinary(string(svc.ID))}, - {Key: "$Type", Value: "ODataPublish$PublishedODataService2"}, - {Key: "Name", Value: svc.Name}, - {Key: "Documentation", Value: svc.Documentation}, - {Key: "Path", Value: svc.Path}, - {Key: "Namespace", Value: svc.Namespace}, - {Key: "ServiceName", Value: svc.ServiceName}, - {Key: "Version", Value: svc.Version}, - {Key: "ODataVersion", Value: svc.ODataVersion}, - {Key: "Summary", Value: svc.Summary}, - {Key: "Description", Value: svc.Description}, - {Key: "PublishAssociations", Value: svc.PublishAssociations}, - {Key: "UseGeneralization", Value: svc.UseGeneralization}, - {Key: "AuthenticationMicroflow", Value: svc.AuthMicroflow}, - {Key: "AuthenticationTypes", Value: authTypes}, - {Key: "EntityTypes", Value: entityTypes}, - {Key: "EntitySets", Value: entitySets}, - {Key: "Excluded", Value: svc.Excluded}, - } - return bson.Marshal(doc) -} - -func serWebPublishedEntityType(et *model.PublishedEntityType) bson.D { - members := bson.A{} - for _, m := range et.Members { - members = append(members, serWebPublishedMember(m)) - } - return bson.D{ - {Key: "$ID", Value: idToBsonBinary(string(et.ID))}, - {Key: "$Type", Value: "ODataPublish$EntityType"}, - {Key: "Entity", Value: et.Entity}, - {Key: "ExposedName", Value: et.ExposedName}, - {Key: "Summary", Value: et.Summary}, - {Key: "Description", Value: et.Description}, - {Key: "ChildMembers", Value: members}, - } -} - -func serWebPublishedEntitySet(es *model.PublishedEntitySet, entityTypeID string) bson.D { - doc := bson.D{ - {Key: "$ID", Value: idToBsonBinary(string(es.ID))}, - {Key: "$Type", Value: "ODataPublish$EntitySet"}, - {Key: "ExposedName", Value: es.ExposedName}, - {Key: "UsePaging", Value: es.UsePaging}, - {Key: "PageSize", Value: int64(es.PageSize)}, - } - - if entityTypeID != "" { - doc = append(doc, bson.E{Key: "EntityTypePointer", Value: idToBsonBinary(entityTypeID)}) - } - if es.ReadMode != "" { - doc = append(doc, bson.E{Key: "ReadMode", Value: serWebReadMode(es.ReadMode)}) - } - if es.InsertMode != "" { - doc = append(doc, bson.E{Key: "InsertMode", Value: serWebChangeMode(es.InsertMode)}) - } - if es.UpdateMode != "" { - doc = append(doc, bson.E{Key: "UpdateMode", Value: serWebChangeMode(es.UpdateMode)}) - } - if es.DeleteMode != "" { - doc = append(doc, bson.E{Key: "DeleteMode", Value: serWebChangeMode(es.DeleteMode)}) - } - return doc -} - -func serWebPublishedMember(m *model.PublishedMember) bson.D { - memberID := string(m.ID) - if memberID == "" { - memberID = generateUUID() - } - - doc := bson.D{ - {Key: "$ID", Value: idToBsonBinary(memberID)}, - {Key: "ExposedName", Value: m.ExposedName}, - {Key: "Filterable", Value: m.Filterable}, - {Key: "Sortable", Value: m.Sortable}, - {Key: "IsPartOfKey", Value: m.IsPartOfKey}, - } - - switch m.Kind { - case "attribute": - doc = append(doc, bson.E{Key: "$Type", Value: "ODataPublish$PublishedAttribute"}) - doc = append(doc, bson.E{Key: "Attribute", Value: m.Name}) - case "association": - doc = append(doc, bson.E{Key: "$Type", Value: "ODataPublish$PublishedAssociationEnd"}) - doc = append(doc, bson.E{Key: "Association", Value: m.Name}) - case "id": - doc = append(doc, bson.E{Key: "$Type", Value: "ODataPublish$PublishedId"}) - doc = append(doc, bson.E{Key: "Attribute", Value: m.Name}) - default: - doc = append(doc, bson.E{Key: "$Type", Value: "ODataPublish$PublishedAttribute"}) - doc = append(doc, bson.E{Key: "Attribute", Value: m.Name}) - } - return doc -} - -func serWebReadMode(mode string) bson.D { - modeID := idToBsonBinary(generateUUID()) - switch { - case strings.EqualFold(mode, "ReadFromDatabase") || strings.EqualFold(mode, "SOURCE"): - return bson.D{{Key: "$ID", Value: modeID}, {Key: "$Type", Value: "ODataPublish$ReadSource"}} - case strings.HasPrefix(mode, "CallMicroflow:"): - return bson.D{{Key: "$ID", Value: modeID}, {Key: "$Type", Value: "ODataPublish$CallMicroflowToRead"}, {Key: "Microflow", Value: strings.TrimPrefix(mode, "CallMicroflow:")}} - case strings.HasPrefix(mode, "MICROFLOW "): - return bson.D{{Key: "$ID", Value: modeID}, {Key: "$Type", Value: "ODataPublish$CallMicroflowToRead"}, {Key: "Microflow", Value: strings.TrimPrefix(mode, "MICROFLOW ")}} - default: - return bson.D{{Key: "$ID", Value: modeID}, {Key: "$Type", Value: "ODataPublish$ReadSource"}} - } -} - -func serWebChangeMode(mode string) bson.D { - modeID := idToBsonBinary(generateUUID()) - switch { - case strings.EqualFold(mode, "ChangeFromDatabase") || strings.EqualFold(mode, "SOURCE"): - return bson.D{{Key: "$ID", Value: modeID}, {Key: "$Type", Value: "ODataPublish$ChangeSource"}} - case strings.EqualFold(mode, "NotSupported") || strings.EqualFold(mode, "NOT_SUPPORTED"): - return bson.D{{Key: "$ID", Value: modeID}, {Key: "$Type", Value: "ODataPublish$ChangeNotSupported"}} - case strings.HasPrefix(mode, "CallMicroflow:"): - return bson.D{{Key: "$ID", Value: modeID}, {Key: "$Type", Value: "ODataPublish$CallMicroflowToChange"}, {Key: "Microflow", Value: strings.TrimPrefix(mode, "CallMicroflow:")}} - case strings.HasPrefix(mode, "MICROFLOW "): - return bson.D{{Key: "$ID", Value: modeID}, {Key: "$Type", Value: "ODataPublish$CallMicroflowToChange"}, {Key: "Microflow", Value: strings.TrimPrefix(mode, "MICROFLOW ")}} - default: - return bson.D{{Key: "$ID", Value: modeID}, {Key: "$Type", Value: "ODataPublish$ChangeNotSupported"}} - } -} - -// ============================================================================ -// Consumed REST Service — Rest$ConsumedRestService -// ============================================================================ - -// SerializeConsumedRestService returns BSON bytes for a consumed REST service unit. -func SerializeConsumedRestService(svc *model.ConsumedRestService) ([]byte, error) { - doc := bson.D{ - {Key: "$ID", Value: idToBsonBinary(string(svc.ID))}, - {Key: "$Type", Value: "Rest$ConsumedRestService"}, - {Key: "Name", Value: svc.Name}, - {Key: "Documentation", Value: svc.Documentation}, - {Key: "Excluded", Value: svc.Excluded}, - {Key: "ExportLevel", Value: "Hidden"}, - {Key: "BaseUrlParameter", Value: nil}, - } - - if svc.OpenApiContent != "" { - doc = append(doc, bson.E{Key: "OpenApiFile", Value: bson.D{ - {Key: "$ID", Value: idToBsonBinary(generateUUID())}, - {Key: "$Type", Value: "Rest$OpenApiFile"}, - {Key: "Content", Value: svc.OpenApiContent}, - }}) - } - - doc = append(doc, bson.E{Key: "BaseUrl", Value: serWebValueTemplate(svc.BaseUrl)}) - - if svc.Authentication == nil { - doc = append(doc, bson.E{Key: "AuthenticationScheme", Value: nil}) - } else { - doc = append(doc, bson.E{Key: "AuthenticationScheme", Value: serWebRestAuthScheme(svc.Authentication)}) - } - - ops := bson.A{int32(2)} - for _, op := range svc.Operations { - ops = append(ops, serWebRestOperation(op)) - } - doc = append(doc, bson.E{Key: "Operations", Value: ops}) - - return bson.Marshal(doc) -} - -func serWebValueTemplate(value string) bson.D { - return bson.D{ - {Key: "$ID", Value: idToBsonBinary(generateUUID())}, - {Key: "$Type", Value: "Rest$ValueTemplate"}, - {Key: "Value", Value: value}, - } -} - -func serWebRestAuthScheme(auth *model.RestAuthentication) bson.D { - doc := bson.D{ - {Key: "$ID", Value: idToBsonBinary(generateUUID())}, - {Key: "$Type", Value: "Rest$BasicAuthenticationScheme"}, - } - doc = append(doc, bson.E{Key: "Username", Value: serWebRestValue(auth.Username)}) - doc = append(doc, bson.E{Key: "Password", Value: serWebRestValue(auth.Password)}) - return doc -} - -func serWebRestValue(value string) bson.D { - if strings.HasPrefix(value, "$") { - constRef := strings.TrimPrefix(value, "$") - return bson.D{ - {Key: "$ID", Value: idToBsonBinary(generateUUID())}, - {Key: "$Type", Value: "Rest$ConstantValue"}, - {Key: "Value", Value: constRef}, - } - } - return bson.D{ - {Key: "$ID", Value: idToBsonBinary(generateUUID())}, - {Key: "$Type", Value: "Rest$StringValue"}, - {Key: "Value", Value: value}, - } -} - -func serWebRestOperation(op *model.RestClientOperation) bson.D { - doc := bson.D{ - {Key: "$ID", Value: idToBsonBinary(generateUUID())}, - {Key: "$Type", Value: "Rest$RestOperation"}, - {Key: "Name", Value: op.Name}, - } - - timeout := int64(op.Timeout) - if timeout <= 0 { - timeout = 300 - } - doc = append(doc, bson.E{Key: "Timeout", Value: timeout}) - - tags := bson.A{int32(1)} - for _, t := range op.Tags { - tags = append(tags, t) - } - doc = append(doc, bson.E{Key: "Tags", Value: tags}) - - doc = append(doc, bson.E{Key: "Method", Value: serWebRestMethod(op)}) - doc = append(doc, bson.E{Key: "Path", Value: serWebValueTemplate(op.Path)}) - - headers := bson.A{int32(2)} - hasAccept := false - for _, h := range op.Headers { - headers = append(headers, serWebRestHeader(h)) - if strings.EqualFold(h.Name, "Accept") { - hasAccept = true - } - } - if !hasAccept { - headers = append(headers, serWebRestHeader(&model.RestClientHeader{Name: "Accept", Value: "*/*"})) - } - doc = append(doc, bson.E{Key: "Headers", Value: headers}) - - params := bson.A{int32(2)} - for _, p := range op.Parameters { - params = append(params, serWebRestParameter(p)) - } - doc = append(doc, bson.E{Key: "Parameters", Value: params}) - - queryParams := bson.A{int32(2)} - for _, q := range op.QueryParameters { - queryParams = append(queryParams, serWebRestQueryParameter(q)) - } - doc = append(doc, bson.E{Key: "QueryParameters", Value: queryParams}) - - // Case-insensitive for the same reason as the codec writer: the else-branch - // silently downgrades to Rest$NoResponseHandling (#843). - if strings.EqualFold(op.ResponseType, "MAPPING") && op.ResponseEntity != "" && len(op.ResponseMappings) > 0 { - doc = append(doc, bson.E{Key: "ResponseHandling", Value: serWebRestImplicitMappingResponse(op.ResponseEntity, op.ResponseMappings)}) - } else { - doc = append(doc, bson.E{Key: "ResponseHandling", Value: serWebRestResponseHandling(op.ResponseType)}) - } - - return doc -} - -func serWebRestMethod(op *model.RestClientOperation) bson.D { - httpMethod := serWebHttpMethodToMendix(op.HttpMethod) - - if op.BodyType != "" { - bodyDoc := bson.D{ - {Key: "$ID", Value: idToBsonBinary(generateUUID())}, - {Key: "$Type", Value: "Rest$RestOperationMethodWithBody"}, - {Key: "HttpMethod", Value: httpMethod}, - } - if strings.EqualFold(op.BodyType, "EXPORT_MAPPING") && len(op.BodyMappings) > 0 { - bodyDoc = append(bodyDoc, bson.E{Key: "Body", Value: serWebRestImplicitMappingBody(op.BodyVariable, op.BodyMappings)}) - } else { - bodyDoc = append(bodyDoc, bson.E{Key: "Body", Value: serWebRestBody(op.BodyType, op.BodyVariable)}) - } - return bodyDoc - } - - methodUpper := strings.ToUpper(op.HttpMethod) - if methodUpper == "POST" || methodUpper == "PUT" || methodUpper == "PATCH" { - bodyDoc := bson.D{ - {Key: "$ID", Value: idToBsonBinary(generateUUID())}, - {Key: "$Type", Value: "Rest$RestOperationMethodWithBody"}, - {Key: "HttpMethod", Value: httpMethod}, - } - bodyDoc = append(bodyDoc, bson.E{Key: "Body", Value: serWebRestBody("JSON", op.BodyVariable)}) - return bodyDoc - } - - return bson.D{ - {Key: "$ID", Value: idToBsonBinary(generateUUID())}, - {Key: "$Type", Value: "Rest$RestOperationMethodWithoutBody"}, - {Key: "HttpMethod", Value: httpMethod}, - } -} - -func serWebRestBody(bodyType, bodyExpr string) bson.D { - switch strings.ToUpper(bodyType) { - case "JSON": - return bson.D{ - {Key: "$ID", Value: idToBsonBinary(generateUUID())}, - {Key: "$Type", Value: "Rest$JsonBody"}, - {Key: "Value", Value: bodyExpr}, - } - case "FILE", "TEMPLATE": - return bson.D{ - {Key: "$ID", Value: idToBsonBinary(generateUUID())}, - {Key: "$Type", Value: "Rest$StringBody"}, - {Key: "ValueTemplate", Value: serWebValueTemplate(bodyExpr)}, - } - default: - return bson.D{ - {Key: "$ID", Value: idToBsonBinary(generateUUID())}, - {Key: "$Type", Value: "Rest$JsonBody"}, - {Key: "Value", Value: bodyExpr}, - } - } -} - -func serWebRestImplicitMappingBody(entity string, mappings []*model.RestResponseMapping) bson.D { - rootElement := serWebInlineMappingElement(entity, "", "", "(Object)", mappings, "ExportMappings", "Parameter") - return bson.D{ - {Key: "$ID", Value: idToBsonBinary(generateUUID())}, - {Key: "$Type", Value: "Rest$ImplicitMappingBody"}, - {Key: "RootMappingElement", Value: rootElement}, - {Key: "TestValue", Value: bson.D{ - {Key: "$ID", Value: idToBsonBinary(generateUUID())}, - {Key: "$Type", Value: "Rest$StringValue"}, - {Key: "Value", Value: ""}, - }}, - } -} - -func serWebRestImplicitMappingResponse(entity string, mappings []*model.RestResponseMapping) bson.D { - rootElement := serWebInlineMappingElement(entity, "", "", "(Object)", mappings, "ImportMappings", "Create") - return bson.D{ - {Key: "$ID", Value: idToBsonBinary(generateUUID())}, - {Key: "$Type", Value: "Rest$ImplicitMappingResponseHandling"}, - {Key: "ContentType", Value: "application/json"}, - {Key: "RootMappingElement", Value: rootElement}, - {Key: "StatusCode", Value: int32(200)}, - } -} - -func serWebInlineMappingElement(entity, association, exposedName, jsonPath string, mappings []*model.RestResponseMapping, namespace, objectHandling string) bson.D { - children := bson.A{int32(2)} - for _, m := range mappings { - if m.Entity != "" { - childJsonPath := jsonPath + "|" + m.ExposedName - child := serWebInlineMappingElement(m.Entity, m.Association, m.ExposedName, childJsonPath, m.Children, namespace, "Create") - children = append(children, child) - } else { - valueJsonPath := m.JsonPath - if valueJsonPath == "" { - valueJsonPath = jsonPath + "|" + m.ExposedName - } - attrQN := entity + "." + m.Attribute - children = append(children, bson.D{ - {Key: "$ID", Value: idToBsonBinary(generateUUID())}, - {Key: "$Type", Value: namespace + "$ValueMappingElement"}, - {Key: "Attribute", Value: attrQN}, - {Key: "ExposedName", Value: m.ExposedName}, - {Key: "JsonPath", Value: valueJsonPath}, - {Key: "XmlPath", Value: ""}, - {Key: "IsKey", Value: false}, - {Key: "Type", Value: bson.D{{Key: "$ID", Value: idToBsonBinary(generateUUID())}, {Key: "$Type", Value: "DataTypes$StringType"}}}, - {Key: "MinOccurs", Value: int32(0)}, - {Key: "MaxOccurs", Value: int32(1)}, - {Key: "Nillable", Value: true}, - {Key: "IsDefaultType", Value: false}, - {Key: "ElementType", Value: "Value"}, - {Key: "Documentation", Value: ""}, - {Key: "Converter", Value: ""}, - {Key: "FractionDigits", Value: int32(-1)}, - {Key: "TotalDigits", Value: int32(-1)}, - {Key: "MaxLength", Value: int32(0)}, - {Key: "IsContent", Value: false}, - {Key: "IsXmlAttribute", Value: false}, - {Key: "OriginalValue", Value: ""}, - {Key: "XmlPrimitiveType", Value: "String"}, - }) - } - } - - minOccurs := int32(1) - if association != "" { - minOccurs = 0 - } - - return bson.D{ - {Key: "$ID", Value: idToBsonBinary(generateUUID())}, - {Key: "$Type", Value: namespace + "$ObjectMappingElement"}, - {Key: "Entity", Value: entity}, - {Key: "ExposedName", Value: exposedName}, - {Key: "JsonPath", Value: jsonPath}, - {Key: "XmlPath", Value: ""}, - {Key: "ObjectHandling", Value: objectHandling}, - {Key: "ObjectHandlingBackup", Value: "Create"}, - {Key: "ObjectHandlingBackupAllowOverride", Value: false}, - {Key: "Association", Value: association}, - {Key: "Children", Value: children}, - {Key: "MinOccurs", Value: minOccurs}, - {Key: "MaxOccurs", Value: int32(1)}, - {Key: "Nillable", Value: true}, - {Key: "IsDefaultType", Value: false}, - {Key: "ElementType", Value: "Object"}, - {Key: "Documentation", Value: ""}, - {Key: "CustomHandlerCall", Value: nil}, - } -} - -func serWebRestHeader(h *model.RestClientHeader) bson.D { - return bson.D{ - {Key: "$ID", Value: idToBsonBinary(generateUUID())}, - {Key: "$Type", Value: "Rest$HeaderWithValueTemplate"}, - {Key: "Name", Value: h.Name}, - {Key: "Value", Value: serWebValueTemplate(h.Value)}, - } -} - -func serWebRestParameter(p *model.RestClientParameter) bson.D { - return bson.D{ - {Key: "$ID", Value: idToBsonBinary(generateUUID())}, - {Key: "$Type", Value: "Rest$OperationParameter"}, - {Key: "Name", Value: p.Name}, - {Key: "DataType", Value: serWebRestDataType(p.DataType)}, - } -} - -func serWebRestQueryParameter(p *model.RestClientParameter) bson.D { - return bson.D{ - {Key: "$ID", Value: idToBsonBinary(generateUUID())}, - {Key: "$Type", Value: "Rest$QueryParameter"}, - {Key: "Name", Value: p.Name}, - {Key: "ParameterUsage", Value: bson.D{ - {Key: "$ID", Value: idToBsonBinary(generateUUID())}, - {Key: "$Type", Value: "Rest$RequiredQueryParameterUsage"}, - }}, - } -} - -func serWebRestResponseHandling(responseType string) bson.D { - doc := bson.D{ - {Key: "$ID", Value: idToBsonBinary(generateUUID())}, - {Key: "$Type", Value: "Rest$NoResponseHandling"}, - } - switch strings.ToUpper(responseType) { - case "JSON": - doc = append(doc, bson.E{Key: "ContentType", Value: "application/json"}) - case "STRING": - doc = append(doc, bson.E{Key: "ContentType", Value: "text/plain"}) - case "FILE": - doc = append(doc, bson.E{Key: "ContentType", Value: "application/octet-stream"}) - } - return doc -} - -func serWebRestDataType(typeName string) bson.D { - bsonType := "DataTypes$StringType" - switch typeName { - case "Integer": - bsonType = "DataTypes$IntegerType" - case "Long": - bsonType = "DataTypes$IntegerType" - case "Decimal": - bsonType = "DataTypes$DecimalType" - case "Boolean": - bsonType = "DataTypes$BooleanType" - case "String": - bsonType = "DataTypes$StringType" - } - return bson.D{ - {Key: "$ID", Value: idToBsonBinary(generateUUID())}, - {Key: "$Type", Value: bsonType}, - } -} - -// ============================================================================ -// Published REST Service — Rest$PublishedRestService -// ============================================================================ - -// SerializePublishedRestService returns BSON bytes for a published REST service unit. -func SerializePublishedRestService(svc *model.PublishedRestService) ([]byte, error) { - resources := bson.A{int32(2)} - for _, res := range svc.Resources { - ops := bson.A{int32(2)} - for _, op := range res.Operations { - opDoc := bson.D{ - {Key: "$ID", Value: idToBsonBinary(GenerateID())}, - {Key: "$Type", Value: "Rest$PublishedRestServiceOperation"}, - {Key: "HttpMethod", Value: serWebHttpMethodToMendix(op.HTTPMethod)}, - {Key: "Path", Value: op.Path}, - {Key: "Microflow", Value: op.Microflow}, - {Key: "Summary", Value: op.Summary}, - {Key: "Deprecated", Value: op.Deprecated}, - {Key: "Commit", Value: "Yes"}, - {Key: "Documentation", Value: ""}, - {Key: "ExportMapping", Value: ""}, - {Key: "ImportMapping", Value: ""}, - {Key: "ObjectHandlingBackup", Value: "Create"}, - {Key: "Parameters", Value: serWebPublishedRestParams(op.Path, op.Microflow, op.Parameters)}, - } - ops = append(ops, opDoc) - } - resDoc := bson.D{ - {Key: "$ID", Value: idToBsonBinary(GenerateID())}, - {Key: "$Type", Value: "Rest$PublishedRestServiceResource"}, - {Key: "Name", Value: res.Name}, - {Key: "Documentation", Value: ""}, - {Key: "Operations", Value: ops}, - } - resources = append(resources, resDoc) - } - - doc := bson.D{ - {Key: "$ID", Value: idToBsonBinary(string(svc.ID))}, - {Key: "$Type", Value: "Rest$PublishedRestService"}, - {Key: "Name", Value: svc.Name}, - {Key: "Documentation", Value: ""}, - {Key: "Excluded", Value: svc.Excluded}, - {Key: "ExportLevel", Value: "Hidden"}, - {Key: "Path", Value: svc.Path}, - {Key: "Version", Value: svc.Version}, - {Key: "ServiceName", Value: svc.ServiceName}, - {Key: "AllowedRoles", Value: serWebMendixStringArray(svc.AllowedRoles)}, - {Key: "AuthenticationTypes", Value: bson.A{int32(2)}}, - {Key: "AuthenticationMicroflow", Value: ""}, - {Key: "CorsConfiguration", Value: nil}, - {Key: "Parameters", Value: bson.A{int32(2)}}, - {Key: "Resources", Value: resources}, - } - - return bson.Marshal(doc) -} - -// serWebPublishedRestParams builds the Parameters array for a published REST operation. -func serWebPublishedRestParams(path string, microflowQN string, _ []string) bson.A { - params := bson.A{int32(2)} - for _, name := range serWebExtractPathParams(path) { - mfParam := "" - if microflowQN != "" { - mfParam = microflowQN + "." + name - } - params = append(params, bson.D{ - {Key: "$ID", Value: idToBsonBinary(generateUUID())}, - {Key: "$Type", Value: "Rest$RestOperationParameter"}, - {Key: "Name", Value: name}, - {Key: "Type", Value: bson.D{ - {Key: "$ID", Value: idToBsonBinary(generateUUID())}, - {Key: "$Type", Value: "DataTypes$StringType"}, - }}, - {Key: "ParameterType", Value: "Path"}, - {Key: "MicroflowParameter", Value: mfParam}, - {Key: "Description", Value: ""}, - }) - } - return params -} - -// serWebExtractPathParams returns parameter names from {param} placeholders in a path. -func serWebExtractPathParams(path string) []string { - var names []string - for { - start := strings.Index(path, "{") - if start < 0 { - break - } - end := strings.Index(path[start:], "}") - if end < 0 { - break - } - names = append(names, path[start+1:start+end]) - path = path[start+end+1:] - } - return names -} - -// serWebHttpMethodToMendix converts uppercase HTTP method names to Mendix casing. -func serWebHttpMethodToMendix(method string) string { - switch strings.ToUpper(method) { - case "GET": - return "Get" - case "POST": - return "Post" - case "PUT": - return "Put" - case "PATCH": - return "Patch" - case "DELETE": - return "Delete" - case "HEAD": - return "Head" - case "OPTIONS": - return "Options" - default: - return method - } -} - -// serWebMendixStringArray builds a Mendix-style versioned string array (int32(1) prefix). -func serWebMendixStringArray(items []string) bson.A { - arr := bson.A{int32(1)} - for _, s := range items { - arr = append(arr, s) - } - return arr -} diff --git a/modelsdk/mpr/writer_core.go b/modelsdk/mpr/writer_core.go index 1e77611d6d..eb19160211 100644 --- a/modelsdk/mpr/writer_core.go +++ b/modelsdk/mpr/writer_core.go @@ -19,7 +19,6 @@ import ( "go.mongodb.org/mongo-driver/v2/bson" "github.com/mendixlabs/mxcli/modelsdk/canon" - "github.com/mendixlabs/mxcli/modelsdk/version" ) // idToBsonBinary converts a UUID string to BSON Binary format. @@ -1026,14 +1025,5 @@ func (w *Writer) UpdateUnitContainer(unitID, newContainerID string) error { // unopenable or unmergeable (mendixlabs/mxcli#1373), and a fix per writer is // the list that never ends. func (w *Writer) normalizePropertySets(unitID string, contents []byte) ([]byte, error) { - pv := w.reader.ProjectVersion() - if pv == nil || pv.MajorVersion == 0 { - return contents, nil - } - v := version.Version{Major: pv.MajorVersion, Minor: pv.MinorVersion, Patch: pv.PatchVersion} - stripped, err := canon.StripUndeclaredProperties(contents, &v) - if err != nil { - return nil, fmt.Errorf("unit %s: %w", unitID, err) - } - return canon.CompletePropertySets(stripped, &v), nil + return normalizePropertySetsFor(w.reader.ProjectVersion(), unitID, contents) } diff --git a/modelsdk/mpr/writer_property_set_repair_test.go b/modelsdk/mpr/writer_property_set_repair_test.go new file mode 100644 index 0000000000..a840ebe3a1 --- /dev/null +++ b/modelsdk/mpr/writer_property_set_repair_test.go @@ -0,0 +1,247 @@ +// SPDX-License-Identifier: Apache-2.0 + +package mpr + +import ( + "bytes" + "os" + "testing" + + "go.mongodb.org/mongo-driver/v2/bson" + + "github.com/mendixlabs/mxcli/modelsdk/mpr/version" +) + +// ako/mxcli#1097: a project built by an older mxcli keeps every property-name +// gap in the units no later statement rewrites, and Mendix's merge engine +// throws on each one. CompleteStoredPropertySets is the pass that reaches them. + +const ( + repairUnitID = "77777777-7777-4777-8777-777777777777" + repairEntityID = "88888888-8888-4888-8888-888888888888" + // A Studio Pro-style GUID, deliberately not the entity's $ID: an element + // mxcli created has GUID == $ID, and a GUID test on one detects nothing. + repairEntityGUID = "99999999-9999-4999-8999-999999999999" +) + +func pv(major, minor, patch int) *version.ProjectVersion { + return &version.ProjectVersion{MajorVersion: major, MinorVersion: minor, PatchVersion: patch} +} + +// moduleSettingsWithoutPackageKeys is a Projects$ModuleSettings as an older +// mxcli wrote it: none of the six keys Mendix 11.12+ serializes for it. +func moduleSettingsWithoutPackageKeys(t *testing.T) []byte { + t.Helper() + b, err := bson.Marshal(bson.D{ + {Key: "$ID", Value: bson.Binary{Subtype: 0, Data: uuidToBlob(repairUnitID)}}, + {Key: "$Type", Value: "Projects$ModuleSettings"}, + {Key: "BasedOnVersion", Value: ""}, + {Key: "ExportLevel", Value: "Source"}, + {Key: "ExtensionName", Value: ""}, + {Key: "JarDependencies", Value: bson.A{int32(3)}}, + {Key: "ProtectedModuleType", Value: "AddOn"}, + {Key: "SolutionIdentifier", Value: ""}, + {Key: "Version", Value: "1.0.0"}, + }) + if err != nil { + t.Fatal(err) + } + return b +} + +// domainModelWithoutEntityKeys is a domain model whose entity lacks the keys +// the table adds to DomainModels$EntityImpl, and carries a GUID that is not its +// $ID — so the test sees whether the repair moves it. +func domainModelWithoutEntityKeys(t *testing.T) []byte { + t.Helper() + bin := func(id string) bson.Binary { return bson.Binary{Subtype: 0, Data: uuidToBlob(id)} } + b, err := bson.Marshal(bson.D{ + {Key: "$ID", Value: bin(repairUnitID)}, + {Key: "$Type", Value: "DomainModels$DomainModel"}, + {Key: "Entities", Value: bson.A{int32(3), bson.D{ + {Key: "$ID", Value: bin(repairEntityID)}, + {Key: "$Type", Value: "DomainModels$EntityImpl"}, + {Key: "GUID", Value: bin(repairEntityGUID)}, + {Key: "Name", Value: "Customer"}, + }}}, + }) + if err != nil { + t.Fatal(err) + } + return b +} + +func topLevelKeys(t *testing.T, raw []byte) map[string]bool { + t.Helper() + var d bson.D + if err := bson.Unmarshal(raw, &d); err != nil { + t.Fatal(err) + } + out := map[string]bool{} + for _, e := range d { + out[e.Key] = true + } + return out +} + +var moduleSettingsPackageKeys = []string{ + "Checksum", "ConvertedChecksum", "EnableDetailedTroubleshooting", + "ModuleDependencies", "OriginalPackageId", "PackageId", +} + +// The issue's premise, measured: a rewrite does NOT elide a unit that lacks a +// table key. Normalization adds the key before the comparison, and canon.Equal +// compares strictly, so any rewrite completes the unit. The gap is reach — the +// units nothing rewrites — not elision, which is why the repair is a pass over +// every unit rather than a change to Reconcile. +func TestStoredUnitMissingAKeyIsCompletedByAnyRewrite(t *testing.T) { + stored := moduleSettingsWithoutPackageKeys(t) + w, path := newV2WriterForCommitTest(t, repairUnitID, stored) + w.reader.projectVersion = pv(11, 12, 0) + + if err := w.UpdateRawUnit(repairUnitID, stored); err != nil { + t.Fatal(err) + } + after, err := os.ReadFile(path) + if err != nil { + t.Fatal(err) + } + keys := topLevelKeys(t, after) + for _, k := range moduleSettingsPackageKeys { + if !keys[k] { + t.Errorf("rewrite of the stored bytes did not add %s — it was elided", k) + } + } +} + +func TestCompleteStoredPropertySets_RepairsAnUntouchedUnit(t *testing.T) { + stored := moduleSettingsWithoutPackageKeys(t) + w, path := newV2WriterForCommitTest(t, repairUnitID, stored) + w.reader.projectVersion = pv(11, 12, 0) + + // Dry run: the report, and no write. + gaps, err := w.reader.StoredPropertySetGaps(repairUnitID) + if err != nil { + t.Fatal(err) + } + if gaps.Added != len(moduleSettingsPackageKeys) || gaps.Stripped != 0 || gaps.Type != "Projects$ModuleSettings" { + t.Fatalf("dry run = %+v, want %d added", gaps, len(moduleSettingsPackageKeys)) + } + if b, _ := os.ReadFile(path); !bytes.Equal(b, stored) { + t.Fatal("the dry run wrote the unit") + } + + rep, err := w.CompleteStoredPropertySets(repairUnitID) + if err != nil { + t.Fatal(err) + } + if rep.Added != len(moduleSettingsPackageKeys) || !rep.Changed() { + t.Fatalf("repair = %+v", rep) + } + first, err := os.ReadFile(path) + if err != nil { + t.Fatal(err) + } + keys := topLevelKeys(t, first) + for _, k := range moduleSettingsPackageKeys { + if !keys[k] { + t.Errorf("repaired unit has no %s", k) + } + } + if _, id := bson.Raw(first).Lookup("$ID").Binary(); !bytes.Equal(id, uuidToBlob(repairUnitID)) { + t.Error("the repair changed the unit's $ID") + } + if got, want := hashOf(first), storedHash(t, w, repairUnitID); got != want { + t.Errorf("ContentsHash %s does not describe the file (%s)", want, got) + } + + // Idempotent: the second run has nothing to do and writes nothing. + rep, err = w.CompleteStoredPropertySets(repairUnitID) + if err != nil { + t.Fatal(err) + } + if rep.Changed() { + t.Fatalf("second repair = %+v, want nothing", rep) + } + if second, _ := os.ReadFile(path); !bytes.Equal(second, first) { + t.Fatal("the second repair rewrote the unit") + } +} + +// The repair adds names and nothing else: the entity keeps its $ID and, +// above all, its GUID — the database's identity for the table. +func TestCompleteStoredPropertySets_KeepsIdentities(t *testing.T) { + stored := domainModelWithoutEntityKeys(t) + w, path := newV2WriterForCommitTest(t, repairUnitID, stored) + w.reader.projectVersion = pv(11, 13, 0) + + rep, err := w.CompleteStoredPropertySets(repairUnitID) + if err != nil { + t.Fatal(err) + } + if rep.Added == 0 { + t.Fatalf("repair = %+v, want the EntityImpl keys added", rep) + } + after, err := os.ReadFile(path) + if err != nil { + t.Fatal(err) + } + ent := bson.Raw(after).Lookup("Entities").Array().Index(1).Document() + if _, got := ent.Lookup("$ID").Binary(); !bytes.Equal(got, uuidToBlob(repairEntityID)) { + t.Errorf("entity $ID moved") + } + if _, got := ent.Lookup("GUID").Binary(); !bytes.Equal(got, uuidToBlob(repairEntityGUID)) { + t.Errorf("entity GUID changed — the next deploy would drop its table") + } + if _, err := ent.LookupErr("Image"); err != nil { + t.Errorf("entity was not completed: %v", err) + } +} + +// A project older than every table row's floor gets nothing: an undeclared key +// is worse than a missing one. +func TestCompleteStoredPropertySets_RespectsTheVersionFloor(t *testing.T) { + stored := moduleSettingsWithoutPackageKeys(t) + w, path := newV2WriterForCommitTest(t, repairUnitID, stored) + w.reader.projectVersion = pv(11, 11, 0) + + rep, err := w.CompleteStoredPropertySets(repairUnitID) + if err != nil { + t.Fatal(err) + } + if rep.Changed() { + t.Fatalf("repair on 11.11 = %+v, want nothing (the keys arrive in 11.12)", rep) + } + if b, _ := os.ReadFile(path); !bytes.Equal(b, stored) { + t.Fatal("the unit was rewritten") + } +} + +// A key the version does not declare yet, holding nothing, is stripped and +// counted — the other half of the name-set difference. +func TestCompleteStoredPropertySets_StripsAnUndeclaredEmptyKey(t *testing.T) { + bin := bson.Binary{Subtype: 0, Data: uuidToBlob(repairUnitID)} + stored, err := bson.Marshal(bson.D{ + {Key: "$ID", Value: bin}, + {Key: "$Type", Value: "Navigation$NavigationProfile"}, + {Key: "Name", Value: "Responsive"}, + {Key: "ThrowPartialSyncError", Value: false}, // since 11.12.0 + }) + if err != nil { + t.Fatal(err) + } + w, path := newV2WriterForCommitTest(t, repairUnitID, stored) + w.reader.projectVersion = pv(11, 11, 0) + + rep, err := w.CompleteStoredPropertySets(repairUnitID) + if err != nil { + t.Fatal(err) + } + if rep.Stripped != 1 { + t.Fatalf("repair = %+v, want 1 stripped", rep) + } + after, _ := os.ReadFile(path) + if topLevelKeys(t, after)["ThrowPartialSyncError"] { + t.Fatal("the undeclared key is still stored") + } +} diff --git a/modelsdk/mpr/writer_property_sets.go b/modelsdk/mpr/writer_property_sets.go new file mode 100644 index 0000000000..dbd7b9241e --- /dev/null +++ b/modelsdk/mpr/writer_property_sets.go @@ -0,0 +1,181 @@ +// SPDX-License-Identifier: Apache-2.0 + +package mpr + +import ( + "fmt" + "strconv" + + "go.mongodb.org/mongo-driver/v2/bson" + + "github.com/mendixlabs/mxcli/modelsdk/canon" + "github.com/mendixlabs/mxcli/modelsdk/mpr/version" + mversion "github.com/mendixlabs/mxcli/modelsdk/version" +) + +// PropertySetRepair describes what normalizing one stored unit's property sets +// changes: how many property NAMES it adds to (canon.CompletePropertySets) and +// strips from (canon.StripUndeclaredProperties) the elements the unit already +// holds. Keys inside a value the completion adds are not counted — they belong +// to an element the stored unit did not have. +type PropertySetRepair struct { + UnitID string + Type string // the unit's $Type + Name string // the unit's Name, "" when it has none + Added int + Stripped int +} + +// Changed reports whether the unit's property-name sets differ from what is +// stored — the condition under which Mendix's merge engine refuses to compare +// it with the revision Studio Pro will save. +func (p PropertySetRepair) Changed() bool { return p.Added+p.Stripped > 0 } + +// StoredPropertySetGaps reports what CompleteStoredPropertySets would change in +// a stored unit, without writing anything. An error means the unit cannot be +// normalized — typically a key the project's version does not declare that +// holds a value, which StripUndeclaredProperties refuses rather than drop. +func (r *Reader) StoredPropertySetGaps(unitID string) (PropertySetRepair, error) { + rep, _, _, err := planPropertySetRepair(r, unitID) + return rep, err +} + +// CompleteStoredPropertySets gives a STORED unit the property set Studio Pro +// writes for each of its elements on this project's version, and writes it +// when the property-name sets changed (ako/mxcli#1097). +// +// # Why this is not just another write +// +// Every write already normalizes (normalizePropertySets), so a unit that a +// statement rewrites is completed on the way. The units nothing rewrites are +// not: a project built by an older mxcli keeps every gap in documents no +// later statement touches — a module's Projects$ModuleSettings, a microflow +// whose script was never re-run — and Mendix's merge engine throws on each +// ("Objects with ID … do not have the same properties"). This is the pass that +// reaches them. +// +// # What it writes +// +// The stored bytes plus the keys the table adds and minus the empty keys the +// version does not declare — nothing else. No element $ID moves (the content IS +// the stored content), no GUID is added or changed (the table holds none, and +// the storage-GUID guard still runs on the result), and a nested default value +// gets the derived, stable $IDs CompletePropertySets gives it. The write is +// made only when the NAME sets differ, so a second run writes nothing. +// +// It goes through the ordinary update path — canon.Reconcile and every guard +// of updateUnit, the storage-GUID guard among them — rather than around it. +// Nothing there needs bypassing: Reconcile's carries are the identity on a +// document that IS the stored one, and its elision compares the canonical form +// strictly, so a unit whose property names changed is never judged equal to +// what is stored. (Measured: TestStoredUnitMissingAKeyIsCompletedByAnyRewrite.) +// What this adds is the reach, and the decision — made on property NAMES, so +// a unit with nothing to repair is not offered at all. +func (w *Writer) CompleteStoredPropertySets(unitID string) (PropertySetRepair, error) { + rep, _, out, err := planPropertySetRepair(w.reader, unitID) + if err != nil || !rep.Changed() { + return rep, err + } + landed := w.writesLanded + if err := w.updateUnit(unitID, out); err != nil { + return rep, err + } + if w.writesLanded == landed && w.sessionBuf == nil && w.deferred == nil { + return rep, fmt.Errorf("unit %s: the property-set repair was elided as unchanged", unitID) + } + return rep, nil +} + +// planPropertySetRepair reads a stored unit, normalizes it for the project's +// version, and counts the property names that changed on the elements the +// stored unit holds. +func planPropertySetRepair(r *Reader, unitID string) (rep PropertySetRepair, stored, out []byte, err error) { + rep.UnitID = unitID + stored, err = r.GetRawUnitBytes(unitID) + if err != nil { + return rep, nil, nil, err + } + raw := bson.Raw(stored) + rep.Type, _ = raw.Lookup("$Type").StringValueOK() + rep.Name, _ = raw.Lookup("Name").StringValueOK() + + out, err = normalizePropertySetsFor(r.ProjectVersion(), unitID, stored) + if err != nil { + return rep, stored, nil, err + } + if len(out) == len(stored) && string(out) == string(stored) { + return rep, stored, out, nil + } + before, err := elementKeySets(stored) + if err != nil { + return rep, stored, nil, fmt.Errorf("unit %s: %w", unitID, err) + } + after, err := elementKeySets(out) + if err != nil { + return rep, stored, nil, fmt.Errorf("unit %s: %w", unitID, err) + } + for path, keys := range before { + now, ok := after[path] + if !ok { + // Normalization never removes an element; if it did, every key went. + rep.Stripped += len(keys) + continue + } + for k := range keys { + if !now[k] { + rep.Stripped++ + } + } + for k := range now { + if !keys[k] { + rep.Added++ + } + } + } + return rep, stored, out, nil +} + +// normalizePropertySetsFor is normalizePropertySets for a given project +// version, shared by the write path and the stored-unit repair. +func normalizePropertySetsFor(pv *version.ProjectVersion, unitID string, contents []byte) ([]byte, error) { + if pv == nil || pv.MajorVersion == 0 { + return contents, nil + } + v := mversion.Version{Major: pv.MajorVersion, Minor: pv.MinorVersion, Patch: pv.PatchVersion} + stripped, err := canon.StripUndeclaredProperties(contents, &v) + if err != nil { + return nil, fmt.Errorf("unit %s: %w", unitID, err) + } + // A GUID-less user role gets its $ID as GUID — absent only, never changed + // (canon.FillMissingUserRoleGUIDs). Not a table row: the value is per role. + return canon.FillMissingUserRoleGUIDs(canon.CompletePropertySets(stripped, &v)), nil +} + +// elementKeySets maps every document node in raw, by its containment path, to +// the set of its keys. Normalization adds and removes keys but never reorders a +// list, so the same path names the same node on both sides. +func elementKeySets(raw []byte) (map[string]map[string]bool, error) { + var doc bson.D + if err := bson.Unmarshal(raw, &doc); err != nil { + return nil, err + } + out := map[string]map[string]bool{} + var walk func(v any, path string) + walk = func(v any, path string) { + switch x := v.(type) { + case bson.D: + keys := make(map[string]bool, len(x)) + for _, e := range x { + keys[e.Key] = true + walk(e.Value, path+"/"+e.Key) + } + out[path] = keys + case bson.A: + for i, e := range x { + walk(e, path+"["+strconv.Itoa(i)+"]") + } + } + } + walk(doc, "") + return out, nil +} diff --git a/modelsdk/mpr/writer_userrole_guid_test.go b/modelsdk/mpr/writer_userrole_guid_test.go new file mode 100644 index 0000000000..525909a254 --- /dev/null +++ b/modelsdk/mpr/writer_userrole_guid_test.go @@ -0,0 +1,119 @@ +// SPDX-License-Identifier: Apache-2.0 + +package mpr + +import ( + "bytes" + "os" + "testing" + + "go.mongodb.org/mongo-driver/v2/bson" +) + +// A Security$UserRole without a GUID gets one invented by mxbuild on every +// build — random, so the runtime's system$userrole.modelguid never matches +// the model and the row is found by name only. Filling GUID = $ID (what the +// create path mints) gives the role a stable identity; an existing GUID is the +// runtime's primary match key and must never change. + +const ( + roleBareID = "aaaaaaa1-0000-4000-8000-000000000001" + roleKeptID = "aaaaaaa1-0000-4000-8000-000000000002" + roleKeptGUID = "aaaaaaa1-0000-4000-8000-000000000003" +) + +// projectSecurityRoles builds a ProjectSecurity unit with one GUID-less user +// role and one whose GUID is not its $ID (Studio Pro-style). +// With bareHasGUID the first role is given its GUID too — the baseline the +// repair count is compared with, since the table completes other keys as well. +func projectSecurityRoles(t *testing.T, bareHasGUID bool) []byte { + t.Helper() + bin := func(id string) bson.Binary { return bson.Binary{Subtype: 0, Data: uuidToBlob(id)} } + bare := bson.D{{Key: "$ID", Value: bin(roleBareID)}, {Key: "$Type", Value: "Security$UserRole"}, {Key: "Name", Value: "User"}} + if bareHasGUID { + bare = append(bare, bson.E{Key: "GUID", Value: bin(roleBareID)}) + } + b, err := bson.Marshal(bson.D{ + {Key: "$ID", Value: bin(repairUnitID)}, + {Key: "$Type", Value: "Security$ProjectSecurity"}, + {Key: "UserRoles", Value: bson.A{int32(2), + bare, + bson.D{{Key: "$ID", Value: bin(roleKeptID)}, {Key: "$Type", Value: "Security$UserRole"}, {Key: "Name", Value: "Administrator"}, + {Key: "GUID", Value: bin(roleKeptGUID)}}, + }}, + }) + if err != nil { + t.Fatal(err) + } + return b +} + +// roleGUID returns the GUID bytes of the i-th entry of UserRoles (index 0 is +// the list marker), or nil when it has none. +func roleGUID(t *testing.T, raw []byte, i int) []byte { + t.Helper() + g, err := bson.Raw(raw).Lookup("UserRoles").Array().Index(uint(i)).Document().LookupErr("GUID") + if err != nil { + return nil + } + _, data := g.Binary() + return data +} + +// The repair fills a missing UserRole GUID with the role's $ID, counts it, +// never touches an existing one, and has nothing to do on a second run. +func TestCompleteStoredPropertySets_FillsMissingUserRoleGUID(t *testing.T) { + baseW, _ := newV2WriterForCommitTest(t, repairUnitID, projectSecurityRoles(t, true)) + baseW.reader.projectVersion = pv(11, 15, 0) + base, err := baseW.reader.StoredPropertySetGaps(repairUnitID) + if err != nil { + t.Fatal(err) + } + + stored := projectSecurityRoles(t, false) + w, path := newV2WriterForCommitTest(t, repairUnitID, stored) + w.reader.projectVersion = pv(11, 15, 0) + + rep, err := w.CompleteStoredPropertySets(repairUnitID) + if err != nil { + t.Fatal(err) + } + if rep.Added != base.Added+1 { + t.Fatalf("repair = %+v, want the GUID counted on top of the %d other keys", rep, base.Added) + } + after, _ := os.ReadFile(path) + if got := roleGUID(t, after, 1); !bytes.Equal(got, uuidToBlob(roleBareID)) { + t.Errorf("GUID-less role: GUID = %x, want its $ID", got) + } + if got := roleGUID(t, after, 2); !bytes.Equal(got, uuidToBlob(roleKeptGUID)) { + t.Errorf("existing role GUID changed to %x", got) + } + + rep, err = w.CompleteStoredPropertySets(repairUnitID) + if err != nil || rep.Changed() { + t.Fatalf("second run = %+v, %v; want nothing", rep, err) + } + if again, _ := os.ReadFile(path); !bytes.Equal(again, after) { + t.Fatal("the second run rewrote the unit") + } +} + +// An ordinary rewrite of the security unit — which carries a stored role's +// bytes through as they are — fills the GUID too, and the storage-GUID guard +// does not refuse it (a GUID on one side only is not a change of identity). +func TestUpdateUnitFillsMissingUserRoleGUID(t *testing.T) { + stored := projectSecurityRoles(t, false) + w, path := newV2WriterForCommitTest(t, repairUnitID, stored) + w.reader.projectVersion = pv(11, 15, 0) + + if err := w.UpdateRawUnit(repairUnitID, stored); err != nil { + t.Fatalf("rewrite refused: %v", err) + } + after, _ := os.ReadFile(path) + if got := roleGUID(t, after, 1); !bytes.Equal(got, uuidToBlob(roleBareID)) { + t.Errorf("GUID-less role after rewrite: GUID = %x, want its $ID", got) + } + if got := roleGUID(t, after, 2); !bytes.Equal(got, uuidToBlob(roleKeptGUID)) { + t.Errorf("existing role GUID changed to %x", got) + } +} diff --git a/mods/companion/.claude-plugin/plugin.json b/mods/companion/.claude-plugin/plugin.json new file mode 100644 index 0000000000..da2da767e9 --- /dev/null +++ b/mods/companion/.claude-plugin/plugin.json @@ -0,0 +1,20 @@ +{ + "name": "mxcli-companion", + "version": "0.1.0", + "description": "Shows the Mendix app's project structure and the MDL source of its documents in a pane, from mxcli.", + "types": "./types/index.d.ts", + "userConfig": { + "mprPath": { + "type": "string", + "title": "Project file", + "description": "Path to the app's .mpr. Empty: the single .mpr in the session folder, or one level down.", + "default": "" + }, + "mxcliPath": { + "type": "string", + "title": "mxcli", + "description": "The mxcli to run. Empty: ./mxcli when present, else mxcli on PATH.", + "default": "" + } + } +} diff --git a/mods/companion/hooks/hooks.json b/mods/companion/hooks/hooks.json new file mode 100644 index 0000000000..5a63f64959 --- /dev/null +++ b/mods/companion/hooks/hooks.json @@ -0,0 +1 @@ +{ "modules": ["./register.tsx"] } diff --git a/mods/companion/hooks/register.tsx b/mods/companion/hooks/register.tsx new file mode 100644 index 0000000000..d1d6cb6684 --- /dev/null +++ b/mods/companion/hooks/register.tsx @@ -0,0 +1,414 @@ +// mxcli companion, slice 1: the Project view. +// +// A pane showing the app's structure from `mxcli project-tree`; selecting a +// document shows its MDL from `mxcli describe --json`. The mod only draws: +// every fact about the project comes from an mxcli command +// (docs/11-proposals/proposal-mxcli-companion-mod.md, P1). + +import { atom, read, update } from 'claude-code' +import type { EngineInterface as Engine, FsEntry, Register } from 'claude-code' + +import type { Project, Source, TreeNode } from '../types' +import { + assumeDescribable, + changesModel, + firstLine, + nodeId, + pageOf, + parseDescribe, + parseTree, + pickMpr, + touchesProjectFile, + visibleRows, +} from './tree' + +const PLUGIN = 'mxcli-companion' +const PANE = 'mxcli-project' + +const project = atom({ plugin: 'mxcli-companion', key: 'project' } as const, { state: 'unlocated' } as Project) +const expanded = atom({ plugin: 'mxcli-companion', key: 'expanded' } as const, [] as string[]) +const selected = atom({ plugin: 'mxcli-companion', key: 'selected' } as const, '') +const source = atom({ plugin: 'mxcli-companion', key: 'source' } as const, { state: 'none' } as Source) +// The first visible row of each column: they page on their own. +const treeTop = atom({ plugin: 'mxcli-companion', key: 'treeTop' } as const, 0) +const sourceTop = atom({ plugin: 'mxcli-companion', key: 'sourceTop' } as const, 0) +// Whether the band above the prompt offers the Project pane: once per project, +// until the person opens the pane or says not now. +const welcome = atom({ plugin: 'mxcli-companion', key: 'welcome' } as const, false) + +// project-tree reads the whole model; on a large app that is the slow call. +const TREE_TIMEOUT_MS = 120_000 +const DESCRIBE_TIMEOUT_MS = 60_000 +// Below this many columns the tree and the source stack instead of sitting +// side by side. +const SIDE_BY_SIDE_MIN_COLUMNS = 90 +// Rows the pane's own chrome takes (header, the gap, a pager line per column). +const CHROME_ROWS = 4 +// A burst of agent writes reloads once, this long after the last one. +const RELOAD_DEBOUNCE_MS = 1_500 + +const basename = (path: string) => path.split('/').pop() ?? path + +let mprOption = '' +let mxcliOption = '' +let reloadGeneration = 0 +// The store key that remembers a project was welcomed; set at session start. +let welcomedKey = '' +// MDL already described since the tree last loaded, by qualified name. A +// describe re-reads the whole model, seconds on a large app, so going back to a +// document should not pay it again; a reload (the model changed) clears it. +const described = new Map() + + +// The mxcli to run: the option, else ./mxcli (where the bootstrap script puts +// it), else whatever is on PATH. +const mxcli = async ($: Engine): Promise => { + if (mxcliOption !== '') return mxcliOption + return (await $.fs.exists('mxcli')) ? './mxcli' : 'mxcli' +} + +const run = async ($: Engine, args: string[], timeoutMs: number) => + $.process.run([await mxcli($), ...args], { env: { MXCLI_QUIET: '1' }, timeoutMs }) + +const mprFiles = async ($: Engine, dir: string): Promise => { + const entries = await $.fs.list(dir).catch((): FsEntry[] => []) + return entries + .filter(e => e.kind === 'file' && e.name.endsWith('.mpr')) + .map(e => (dir === '.' ? e.name : `${dir}/${e.name}`)) +} + +const locate = async ($: Engine): Promise => { + const here = await mprFiles($, '.') + let below: string[] = [] + if (mprOption === '' && here.length === 0) { + const dirs = (await $.fs.list('.').catch((): FsEntry[] => [])).filter( + e => e.kind === 'dir' && !e.name.startsWith('.') && e.name !== 'node_modules', + ) + for (const d of dirs) below = below.concat(await mprFiles($, d.name)) + } + const picked = pickMpr(mprOption, here, below) + if ('mpr' in picked) return { state: 'loading', mpr: picked.mpr } + if ('candidates' in picked) return { state: 'choose', candidates: picked.candidates } + return { state: 'none' } +} + +const describeSelected = async ($: Engine, node: TreeNode) => { + const current = await read($, project) + if (current.state !== 'loaded' || !node.qualifiedName) return + const qn = node.qualifiedName + const cached = described.get(qn) + if (cached !== undefined) { + await update($, source, (): Source => ({ state: 'shown', label: node.label, qn, mdl: cached })) + return + } + await update($, source, (): Source => ({ state: 'loading', label: node.label, qn })) + try { + const out = await run($, ['describe', '-p', current.mpr, node.type, qn, '--json'], DESCRIBE_TIMEOUT_MS) + let next: Source + if (out.exitCode === 0) { + const mdl = parseDescribe(out.stdout) + described.set(qn, mdl) + next = { state: 'shown', label: node.label, qn, mdl } + } else { + next = { state: 'failed', label: node.label, qn, error: firstLine(out.stderr || out.stdout) } + } + // Only land the answer if the person has not moved on meanwhile. + await update($, source, (s): Source => (s.state === 'loading' && s.qn === qn ? next : s)) + } catch (err) { + await update($, source, (s): Source => + s.state === 'loading' && s.qn === qn + ? { state: 'failed', label: node.label, qn, error: String(err) } + : s, + ) + } +} + +const findNode = (nodes: readonly TreeNode[], id: string, parent = ''): TreeNode | undefined => { + for (const node of nodes) { + const nid = nodeId(parent, node) + if (nid === id) return node + if (id.startsWith(`${nid}/`) && node.children) { + const hit = findNode(node.children, id, nid) + if (hit) return hit + } + } + return undefined +} + +const load = async ($: Engine, mpr?: string) => { + const target = mpr ? ({ state: 'loading', mpr } as Project) : await locate($) + await update($, project, () => target) + if (target.state !== 'loading') return + const startedMs = await $.clock.now() + try { + const out = await run($, ['project-tree', '-p', target.mpr], TREE_TIMEOUT_MS) + if (out.exitCode !== 0) { + await update($, project, (): Project => ({ state: 'failed', mpr: target.mpr, error: firstLine(out.stderr || out.stdout) })) + return + } + const tree = assumeDescribable(parseTree(out.stdout)) + described.clear() + const loadedMs = await $.clock.now() + await update($, project, (): Project => ({ state: 'loaded', mpr: target.mpr, tree, loadedMs, tookMs: loadedMs - startedMs })) + // A reload keeps the selection; refresh its source, which may have changed. + const sel = await read($, selected) + const node = sel === '' ? undefined : findNode(tree, sel) + if (node?.describable) await describeSelected($, node) + } catch (err) { + await update($, project, (): Project => ({ state: 'failed', mpr: target.mpr, error: String(err) })) + } +} + +const scheduleReload = async ($: Engine) => { + const current = await read($, project) + if (current.state !== 'loaded') return // never opened, or already busy + const mine = ++reloadGeneration + await $.clock.sleep(RELOAD_DEBOUNCE_MS) + if (mine === reloadGeneration) await load($, current.mpr) +} + +const press = async ($: Engine, id: string) => { + const current = await read($, project) + if (current.state !== 'loaded') return + const node = findNode(current.tree, id) + if (!node) return + if (node.children && node.children.length > 0) { + await update($, expanded, list => (list.includes(id) ? list.filter(x => x !== id) : [...list, id])) + } + if ((await read($, selected)) !== id) await update($, sourceTop, () => 0) + await update($, selected, () => id) + if (node.describable) { + await describeSelected($, node) + } else { + await update($, source, (): Source => ({ + state: 'container', + label: node.label, + type: node.type, + children: (node.children ?? []).map(c => c.label), + })) + } +} + +// background runs work nobody awaits. Its failures have nowhere to go (the +// pane shows load errors itself), and one landing after an unload must not +// surface as an unhandled rejection. +const background = (work: Promise) => { + work.catch(() => undefined) +} + +// The store outlives sessions and spans projects, so the key names the project. +const startWelcome = async ($: Engine, cwd: string) => { + welcomedKey = `welcomed:${cwd}` + if ((await $.store.get(welcomedKey)) !== true) await update($, welcome, () => true) +} + +const retireWelcome = async ($: Engine) => { + if (!(await read($, welcome))) return + await update($, welcome, () => false) + if (welcomedKey !== '') await $.store.set(welcomedKey, true) +} + +// open seats the pane and starts loading; it answers what the person should +// read, which is not always "opened": a session whose attached surfaces place +// no panes (an older desktop, some remote surfaces) keeps it waiting undrawn. +const open = async ($: Engine): Promise => { + const opened = await $.ui.open({ id: PANE, title: 'Project' }) + await retireWelcome($) + const current = await read($, project) + if (current.state === 'unlocated' || current.state === 'none' || current.state === 'failed') { + background(load($, current.state === 'failed' ? current.mpr : undefined)) + } + return opened.isPlaced + ? 'Project pane opened.' + : `The Project pane is open but not shown here: ${opened.reason}` +} + +export const register: Register = (on, options) => { + mprOption = typeof options.mprPath === 'string' ? options.mprPath : '' + mxcliOption = typeof options.mxcliPath === 'string' ? options.mxcliPath : '' + + on('session.start', async ($, e, next) => { + await $.command.register({ name: 'mxcli-project', description: 'Show the Mendix project structure and MDL source in a pane' }) + background(startWelcome($, e.cwd)) + return next(e) + }) + + // The first session after `mxcli init` installed the companion: say it is + // here and offer the pane. A band, not a pane, because a band needs no room + // and shows unasked at any width. + on('ui.render', { component: 'AbovePrompt' }, async ($, e, next) => { + if (e.props.hasSurvey || !(await read($, welcome))) return next(e) + const { Box, Text, Button } = $.ui.resolve(e) + return ( + + + mxcli companion is installed: the Project pane shows this app's structure and MDL. + +