Repository navigation
fix(test): keep a call's own on error clause; the generated startup flow applies entity access - #1748
Merged
Merged
Conversation
…or clause (#1448) A call in a test body that already had an error-handling clause (`on error rollback`, `on error continue`, a custom handler) got the runner's `ON ERROR BEGIN … END ERROR` appended as a second one. The generated MDL did not parse ("mismatched input 'ON' expecting ';'", plus a misleading hint about `Error` being a keyword) and the run stopped at injection, before the app started. Upstream: mendixlabs#1550. Cause: attachOnError (endpoint runner) and its two copies in the legacy after-startup runner wrapped every line startsErrorRaisingCall matched, unconditionally. Their line-based join also broke on a custom handler (the handler body's first `;` ended the "statement") and on a trailing comment. Fix: - One attachOnError for both runners, with and without @throws. It parses the body inside the generator's own wrapper and language version (visitor.FlowStatements, new) and reads each call's own onErrorClause from the tree — not text, so a clause on a later line counts and `on error` in a string, a comment or a nested handler does not. The set of wrapped statements is now a set of grammar rule names, pinned to the grammar. - A call with `on error continue` or a custom handler keeps it verbatim and gets no second handler. `on error rollback` is Mendix's default spelled out (the same ErrorHandlingType as no clause), so it is replaced by the runner's handler, which also rolls back: same model, same verdict. - Keeping custom handlers opened a false green that only the runtime showed: a handler that ends normally rejoins the flow past every statement reading the failed call's output — the @expect checks — and the test returned PASS with nothing evaluated. Each assertion now counts itself in both branches of its decision, and a guard after the last fails a short count ("not every assertion was evaluated"). Both runners. - Legacy log parser: a test with a RUN line and no verdict read as PASS (StatusPass is the zero value). A re-raising handler ends the legacy runner exactly there; it is now an ERROR that says so. Verification (Mendix 11.13.0, --local --db-type hsqldb, the issue's setup): the baseline binary stops with the reported parse error; the fixed one runs mdl-examples/bug-tests/test-runner-1448-own-error-handler.test.mdl: plain/rollback throwing FAIL "exception during execution: boom: a"; continue FAIL on its assertion; custom-handler-ends-normally FAIL "not every assertion was evaluated" (PASS before the guard); re-raise ERROR "boom: a"; @throws plain/rollback PASS, continue/custom FAIL "expected an exception but none was thrown", re-raise ERROR; the report's two statements PASS; an assertion on what the handler set PASS. --legacy-runner gives the same verdicts; its re-raise case was PASS and is now ERROR. Revert proofs (stubbing each guard): the always-attach stub reproduces "mismatched input 'ON'" in the unit tests; the rollback, counter and RUN-without-verdict stubs each fail their test. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…roflow applies entity access (#1654) `mxcli test --local` generates MxTest.RegisterEndpoint, the startup microflow that registers the test endpoint and then calls the project's own after-startup microflow. It was written without "apply entity access", and MxBuild refuses a microflow that does not apply entity access calling one that does (CE0114). In any project whose after-startup microflow applies entity access the build failed, no test ran, and the error said "These errors are in the project, not in this run's tests" about a microflow mxcli generated. Upstream: mendixlabs#1645. Measured direction (mx check, Mendix 11.13.0, security Off and Production): non-applying -> applying is CE0114; applying -> non-applying and applying -> applying (with a Java action call beside it) are 0 errors, and an applying microflow is accepted as the after-startup microflow. MDL-SEC22 records the same table for 11.14.0. Fix: - GenerateEndpointMDL writes `@applyentityaccess` on RegisterEndpoint. Since an applying caller may call either kind, a constant serves every project with nothing read from the user's microflow (the suggested mirror would have added a second scrape of describe output). It governs only the flow's own activities: the Java action ignores it, the chained microflow runs with its own setting. Also covers `run --local --test-endpoint`, which shares the generator. - buildFailureHint attributes an error located in MxTest.RegisterEndpoint or MxTest.RegisterTestEndpoint to mxcli — "not part of your project: mxcli generated it for this run … a defect in mxcli … please report it" — with --skip-app-startup as the escape hatch, and says "in the project" only for the errors that are. Verification: on a blank 11.13.0 app with mdl-examples/bug-tests/test-runner-1654-setup.mdl (startup microflow applies entity access and leaves a marker), the baseline binary fails with the reported CE0114; the fixed one passes both tests, the second proving the startup microflow ran at boot. Controls: the same run with --skip-app-startup fails that test ("expected $Count > 0, actual: 0") — the test consumes the marker so an earlier boot's cannot pass it; a non-applying startup microflow still passes both. A binary with only the attribution half shows the new message on the live CE0114. Revert proof (patch method): with endpoint.go and build_attribution.go checked out, TestRegisterEndpointAppliesEntityAccess and TestBuildErrorInGeneratedStartupFlowIsMxclis fail; the project-only control passes either way. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #1448 (mirror of mendixlabs#1550). Closes #1654 (mirror of mendixlabs#1645).
mxcli testgenerated a flow that could not be built, so no test ran. One commit per issue. Both verified withmxcli test --local --db-type hsqldbon a blank 11.13.0 app.#1448 — a call with its own
on errorclause got a second oneCause:
attachOnError(generator_endpoint.go) appendedON ERROR BEGIN … END ERRORto every line a regex matched, with no check for an existing clause; the legacy runner had two more copies of the same join, which also broke on a custom handler's own;and on a trailing--comment.Fix:
attachOnErrorfor both runners, with and without@throws. Newvisitor.FlowStatementsparses the body inside the generator's own wrapper and language version and returns each statement's span and its own ON ERROR clause; a body that does not parse is left unchanged. The regex is replaced byerrorRaisingCallRules, a set of grammar rule names pinned by a test.on error continueand custom handlers are kept as written.on error rollbackis replaced by the runner's handler: it is Mendix's default spelled out (the same ErrorHandlingType as no clause), and keeping it gave ERROR instead of FAIL, failed a matching@throws, and under the legacy runner aborted the suite.@expectchecks — and reported PASS with nothing evaluated. Each assertion now counts itself in both branches, and a guard fails the test with "not every assertion was evaluated" when the count is short (both runners).Verdict table (Java action that throws, endpoint runner):
@expect@throws 'boom'exception during execution: boom: aexpected $r = 'X', actual:expected an exception but none was thrownnot every assertion was evaluated(was PASS)expected … none was thrownboom: aboom: aThe issue's statements with succeeding calls PASS; an
@expecton a flag the handler sets still PASSes;--legacy-runnergives the same verdicts.#1654 — CE0114 on the generated startup flow
Measured with mx check 11.13.0, security Off and Production: a non-applying microflow calling an applying one is CE0114; an applying one calling either kind is 0 errors (MDL-SEC22 records the same for 11.14.0).
Cause:
GenerateEndpointMDLwroteMxTest.RegisterEndpointwithout the setting, and it calls the project's after-startup microflow. Separately,buildFailureHintcalled every error outside aTest_*flow "in the project".Fix: RegisterEndpoint always carries
@applyentityaccess— the measured direction makes a constant correct for every project, with nothing to read from the user's microflow. Errors located in mxcli's generatedMxTestflows are reported as mxcli's own, with--skip-app-startupas the escape hatch.Verified
own_error_handler_test.go,flow_statements_test.go,startup_entity_access_test.go; every generated script is parsed withvisitor.Build. Cases: the clause on the next line;on errorin a string, a line comment and a block comment; two calls on one line; a call nested in an IF or in the author's handler; the plain-call control.mxcli test --local: the generatedMxTest.RegisterEndpointdoes not apply entity access, so a project whose after-startup microflow does cannot be tested (CE0114) #1654 fails both its tests (the project-only control passes either way).mxcli test: a call with its ownon errorclause in a test block produces an unparsable test microflow ("mismatched input 'ON' expecting ';'") #1448) and CE0114 with "in the project" ([up#1645]mxcli test --local: the generatedMxTest.RegisterEndpointdoes not apply entity access, so a project whose after-startup microflow does cannot be tested (CE0114) #1654); fixed, both suites run with the verdicts above, and the [up#1645]mxcli test --local: the generatedMxTest.RegisterEndpointdoes not apply entity access, so a project whose after-startup microflow does cannot be tested (CE0114) #1654 marker test proves the startup microflow ran at boot (FAILs under--skip-app-startup, the control). A non-applying startup microflow still PASSes.go test ./...,go vet, gofmt and CI's doc checks pass with all five of this batch's branches merged onto main. Bug-test materialtest-runner-1448-*,test-runner-1654-*; findings recorded; test-microflows skill, docs-site test-annotations and running-tests updated.run --local --test-endpointwithtest --attachfor [up#1645]mxcli test --local: the generatedMxTest.RegisterEndpointdoes not apply entity access, so a project whose after-startup microflow does cannot be tested (CE0114) #1654 (same generator, unit test only); the Docker path; CE0114 on Mendix 10.x.Noticed, not fixed
--legacy-runner): a later assertion's PASS line overwrites an earlier FAIL (results.gocase "PASS"), so a test whose first@expectfails and second passes reports PASS. Confirmed at unit level.assignPatternmatches only upper-caseCALL|CREATE, so mdl 1's$r = call …is not renamed per test; two tests using$rfail injection (MDL063 / CE0111, measured).@throwsfalse red: an error that leaves the test microflow (a re-raising handler, or a non-call activity) is reported ERROR instead of being checked against@throws.run --local --test-endpointdoes not usebuildFailureHint, so it lacks the new attribution.🤖 Generated with Claude Code