Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{"area": "cmd/mxcli", "date": "2026-10-27", "symptom": "`mxcli test --local` fails the build with `[CE0114] A microflow that does not apply entity access can only call microflows that also do not apply entity access. — at MxTest / Microflow 'RegisterEndpoint' / Call microflow activity '<your ASU>'` whenever the project's after-startup microflow applies entity access; no test runs, and the error says it is \"in the project, not in this run's tests\"", "cause": "`GenerateEndpointMDL` wrote `MxTest.RegisterEndpoint` without `@applyentityaccess`, so it did not apply entity access, and since the 2026-08 chaining fix it calls the project's after-startup microflow. `buildFailureHint` called every error not in a `Test_*` flow \"in the project\", including ones in the two documents the run itself generates", "file": "`cmd/mxcli/testrunner/endpoint.go` (`GenerateEndpointMDL`), `cmd/mxcli/testrunner/build_attribution.go` (`buildFailureHint`, `splitRunnerDocuments`)", "insight": "Measure the rule's DIRECTION before choosing between mirroring the callee and a constant: with mx check on 11.13.0 (security Off and Production), non-applying -> applying is CE0114 while applying -> non-applying and applying -> applying are 0 errors (MDL-SEC22's table says the same for 11.14.0). So `@applyentityaccess` on the generated caller is compatible with every startup microflow, and needs no scrape of the user's flow — which the mirror fix would have added beside the DESCRIBE SETTINGS scrape that already broke once. The setting is neutral at runtime: it governs the caller's own activities only (a Java action ignores it), each callee runs with its own. Prove the chain still runs with a marker the test CONSUMES (`@cleanup none` + delete): a plain count passed on an earlier boot's marker under --skip-app-startup, i.e. the control could not fail. Generated documents with fixed names (RegisterEndpoint, RegisterTestEndpoint) are attributable exactly like the Test_* flows; say so and offer --skip-app-startup as the escape hatch", "refs": ["ako/mxcli#1654", "mendixlabs/mxcli#1645", "mendixlabs/mxcli#1500"], "ce": ["CE0114"]}
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{"area": "cmd/mxcli", "date": "2026-10-27", "symptom": "`mxcli test` stops at injection with `Parse error: line 8:75 mismatched input 'ON' expecting ';'` (and a misleading `'Error' is a keyword` hint) when a test body's call has its own `on error rollback` / `on error continue` / custom handler — no test runs", "cause": "`attachOnError` appended the runner's `ON ERROR BEGIN … END ERROR` to every line `startsErrorRaisingCall` matched, unconditionally, so a call with its own clause got a second one. The line-based join also broke on a custom handler (its body's first `;` ended the 'statement') and on a trailing `-- comment`. Both runners had their own copy of the join", "file": "`cmd/mxcli/testrunner/generator_endpoint.go` (`attachOnError`, now shared by both runners), `mdl/visitor/flow_statements.go` (`FlowStatements`), `cmd/mxcli/testrunner/results.go` (`ParseLogResults`)", "insight": "Find statements by PARSING the body (in the generator's own wrapper and language version) and read each call's own onErrorClause from the tree: a regex cannot tell a clause on the next line from `on error` in a string, a comment or a nested handler. Decide per clause from what Mendix stores: `on error rollback` is the default spelled out (same ErrorHandlingType as no clause), so it gets the runner's handler — keeping it made the same exception an ERROR, failed a matching @throws, and aborted the legacy suite; continue/custom are kept verbatim. Keeping a custom handler opened a FALSE GREEN only a runtime run showed: a handler that ends normally rejoins PAST every statement reading the failed call's output (the flow builder's skip-var rule), i.e. past the @expect checks, and the test returned PASS with nothing evaluated. Fixed fail-closed: each assertion counts itself in both branches and a guard after the last fails a short count — an assertion on what the handler set still passes. The legacy log parser had the twin: a RUN line with no verdict read as PASS (StatusPass is the zero value); a re-raising handler did exactly that. Measured verdict table on 11.13.0 in mdl-examples/bug-tests/test-runner-1448-own-error-handler.test.mdl", "refs": ["ako/mxcli#1448", "mendixlabs/mxcli#1550"]}
37 changes: 29 additions & 8 deletions .claude/skills/mendix/test-microflows/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,14 +16,14 @@ description: "Write and run MDL-based microflow tests with `mxcli test` — anno
| 180-222 | Test File Formats |
| 223-238 | string Operations |
| 239-262 | Running Tests |
| 263-450 | How It Works |
| 451-489 | Check a test file before you run it |
| 490-596 | Writing Good Tests |
| 597-611 | Test File Organization |
| 612-624 | Interpreting Failures |
| 625-646 | CI Integration |
| 647-653 | Related Skills |
| 654-656 | Warm test loop (`mxcli test --local [--watch]`, `--attach`, `run --local --test-endpoint`) |
| 263-456 | How It Works |
| 457-495 | Check a test file before you run it |
| 496-617 | Writing Good Tests |
| 618-632 | Test File Organization |
| 633-645 | Interpreting Failures |
| 646-667 | CI Integration |
| 668-674 | Related Skills |
| 675-677 | Warm test loop (`mxcli test --local [--watch]`, `--attach`, `run --local --test-endpoint`) |

<!-- /skill-index -->
# Test Microflows Skill
Expand Down Expand Up @@ -315,6 +315,12 @@ chained, `--local` ran with the app's startup logic suppressed, and a suite that
depended on startup state passed under `--attach` and failed under `--local` for
reasons unrelated to the code.

`MxTest.RegisterEndpoint` applies entity access, so it can call a startup
microflow that does or does not (CE0114 only forbids a non-applying microflow
calling an applying one); yours still runs with its own setting. A build error
located in `MxTest.RegisterEndpoint` or `MxTest.RegisterTestEndpoint` is reported
as an mxcli defect, not as an error in your project.

One thing rollback does **not** cover: whatever the startup microflow writes
happens at boot, outside any test's transaction, so `@cleanup rollback` does not
undo it. Under `--local` that lands in the scratch `<project>_test` database;
Expand Down Expand Up @@ -585,6 +591,21 @@ call microflow Sales.ValidateOrder(Total = -1);
Anything else after the tag is an error, not a message: `@throws not found` is
refused rather than read as an unquoted one.

A call that already has its own `on error` clause keeps it — the runner adds
its handler only to calls without one, so the test runs the statement exactly as
production does:

- `on error rollback` is Mendix's default spelled out and behaves like no clause:
FAIL `exception during execution: …`, or PASS under a matching `@throws`.
- `on error continue` swallows the error; the assertions decide. `@throws` FAILs
(`expected an exception but none was thrown`).
- A custom handler that ends normally skips every statement reading the failed
call's result — the assertions — so the test FAILs with `not every assertion
was evaluated`. Assert on something the handler sets instead.
- A custom handler that re-raises lets the error leave the test: ERROR with the
message, also under `@throws`. Under Docker / `--legacy-runner` it ends the
whole run.

`@throws` and `@expect` cannot be combined. A `@throws` test compiles to a
different shape — the verdict starts as a failure and only the error handler
clears it — and the `@expect` checks are not emitted into it at all, so the
Expand Down
69 changes: 66 additions & 3 deletions cmd/mxcli/testrunner/build_attribution.go
Original file line number Diff line number Diff line change
Expand Up @@ -175,28 +175,91 @@ func resultsFromFailedBuild(problems []docker.BuildProblem, suite *TestSuite) []

// buildFailureHint is appended to the error when a build failure could not be
// attributed to any test, which means it is in the project rather than in the
// suite.
// suite — or in one of the documents the run generates besides its tests.
//
// The errors themselves are already rendered by BuildFailedError.Error(), so
// this says what the reader cannot work out from them: that none of them belongs
// to a test in this run, and — for a generated microflow no current test owns —
// that it is a leftover from an earlier run, with the command to remove it. That
// last case is the one that fails every subsequent run of every test file until
// someone finds it by hand (mendixlabs/mxcli#1104).
//
// An error in the endpoint documents is mxcli's own, and says so. CE0114 in
// MxTest.RegisterEndpoint used to be reported as "in the project", which sent
// the reader through their own model for a defect in a microflow they never
// wrote (ako/mxcli#1654).
func buildFailureHint(other []docker.BuildProblem) string {
if len(other) == 0 {
return ""
}
generated, project := splitRunnerDocuments(other)
var b strings.Builder
b.WriteString("\n These errors are in the project, not in this run's tests.")
for _, name := range leftoverFlowsIn(other) {
for _, name := range generated {
b.WriteString(fmt.Sprintf("\n %s is not part of your project: mxcli generated it for this run, %s."+
" An error in it is a defect in mxcli, not in your project or your tests — please report it"+
" (`mxcli diag --bundle` collects what is needed).", name, runnerDocuments[name]))
if name == endpointStartupFlow {
b.WriteString("\n Until it is fixed, --skip-app-startup runs the tests without the call to your after-startup microflow.")
}
}
if len(project) == 0 {
return b.String()
}
if len(generated) > 0 {
b.WriteString("\n The other errors are in the project, not in this run's tests.")
} else {
b.WriteString("\n These errors are in the project, not in this run's tests.")
}
for _, name := range leftoverFlowsIn(project) {
b.WriteString(fmt.Sprintf(
"\n %s is a microflow an earlier `mxcli test` run left behind. Remove it with:"+
"\n mxcli -p <project.mpr> -c \"DROP MICROFLOW %s\"", name, name))
}
return b.String()
}

// runnerDocuments are the documents an endpoint run generates besides its test
// microflows, with what each is for. Their names are fixed, so an error located
// in one is the run's own whatever else is in the MxTest module.
var runnerDocuments = map[string]string{
endpointStartupFlow: "to register the test endpoint at startup and then run your after-startup microflow",
endpointRegisterAction: "to register the test endpoint",
}

// runnerDocumentPattern matches the `document` MxBuild reports — the document's
// kind and its bare name, `Microflow 'RegisterEndpoint'` or
// `Java action 'RegisterTestEndpoint'`; the module is carried separately.
var runnerDocumentPattern = regexp.MustCompile(`^[A-Za-z ]+ '([^']*)'$`)

// splitRunnerDocuments separates the problems located in a runner document from
// the rest, naming each runner document once, in the order MxBuild reported it.
func splitRunnerDocuments(problems []docker.BuildProblem) (generated []string, rest []docker.BuildProblem) {
seen := map[string]bool{}
for _, p := range problems {
name := ""
for _, loc := range p.Locations {
if !strings.EqualFold(loc.Module, mxTestModule) {
continue
}
if m := runnerDocumentPattern.FindStringSubmatch(loc.Document); m != nil {
if q := mxTestModule + "." + m[1]; runnerDocuments[q] != "" {
name = q
break
}
}
}
if name == "" {
rest = append(rest, p)
continue
}
if !seen[name] {
seen[name] = true
generated = append(generated, name)
}
}
return generated, rest
}

// leftoverFlowsIn names the generated test microflows among a set of build
// problems, deduplicated and in the order MxBuild reported them.
//
Expand Down
20 changes: 15 additions & 5 deletions cmd/mxcli/testrunner/endpoint.go
Original file line number Diff line number Diff line change
Expand Up @@ -67,12 +67,21 @@ func testFlowName(tc TestCase) string { return testFlowPrefix + tc.ID }
//
// chainAfterStartup, when non-empty, is a microflow the generated startup flow
// calls after registering the endpoint — the project's own after-startup
// microflow, which this one displaces.
// microflow, which this one displaces. The test runner passes it so tests see
// the app as it really boots, and "" under --skip-app-startup; a dev loop
// hosting the endpoint (`run --local --test-endpoint`) always passes it.
//
// The test runner passes "" : a test run wants a known starting state, and the
// suite is the only thing that should execute. A dev loop hosting the endpoint
// (`run --local --test-endpoint`) passes the real one, because the developer's
// app must still seed its data and do whatever else it does at boot.
// The startup flow applies entity access, whatever the chained microflow does.
// MxBuild refuses a microflow that does not apply entity access calling one
// that does (CE0114), and the flow used to be written without the setting, so
// no project whose after-startup microflow applies entity access could run a
// single test (ako/mxcli#1654). The rule has one direction — measured with mx
// check on 11.13.0, an applying microflow may call an applying or a
// non-applying one at 0 errors, under security Off and Production alike (and
// MDL-SEC22 records the same table for 11.14.0) — so a constant serves every
// project, with nothing to read from the chained microflow first. The setting
// governs only this flow's own two activities: the Java action ignores it, and
// the chained microflow runs with its own.
func GenerateEndpointMDL(chainAfterStartup string) string {
var b strings.Builder

Expand All @@ -91,6 +100,7 @@ func GenerateEndpointMDL(chainAfterStartup string) string {
b.WriteString("\n$$;\n\n")

b.WriteString("/** Registers the mxcli test endpoint at boot. Runs no tests. */\n")
b.WriteString("@applyentityaccess\n")
b.WriteString(createFlow(langver.V1) + " " + endpointStartupFlow + " ()\n")
b.WriteString("RETURNS Boolean AS $Registered\n")
b.WriteString("BEGIN\n")
Expand Down
Loading
Loading