Repository navigation
Expand file tree
/
Copy pathphpunit.xml
More file actions
111 lines (102 loc) · 6.12 KB
/
Copy pathphpunit.xml
File metadata and controls
111 lines (102 loc) · 6.12 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
<?xml version="1.0" encoding="UTF-8"?>
<phpunit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:noNamespaceSchemaLocation="vendor/phpunit/phpunit/phpunit.xsd"
bootstrap="tests/bootstrap.php"
colors="true"
failOnWarning="true"
failOnRisky="true"
enforceTimeLimit="true"
defaultTimeLimit="60"
cacheDirectory=".phpunit.cache">
<!--
defaultTimeLimit is a hang-catcher, not a performance budget. It exists
because a one-conjunct mutation in Vhs tape tokenize() can spin FOREVER on
input already present in this suite: without a limit that run never ends
and never reports, so the whole suite reads as "still going" rather than
"broken" (measured: such a run had to be killed externally at 180s).
60s was picked off measurement, not taste, and the peak it was picked
against has since moved twice — so this paragraph is re-derived from a
JUnit-logged run rather than carried forward.
RE-MEASURED 2026-09-21, all 12,061 tests on this host (PHP 8.3.6):
the peak is 18.007s (Agents\ProcessExecutorTest::testALeasedWorker-
SurvivesPastTheHeartbeatWindowWhileAlive, which waits out a real
heartbeat window), with 11 tests over 10s and 27 over 5s. The bound
therefore sits ~3.3x above the slowest real test.
WHAT THIS PARAGRAPH USED TO CLAIM, and both halves had rotted: a peak
of 9.321s over 6,465 tests, "4 tests over 5s and none over 10s", and
a ~6x margin. The suite has since almost doubled, and the same run
measured above found the real peak at 41.474s — Renderer\KeyHelpTest::
testTheCuesFitTheNarrowestBarAnyAppStateCanProduce, 2.3x the next
slowest, rebuilding a 2,400-message fixture 1,600 times. That is only
1.4x under this limit uninstrumented, and under pcov it crossed it:
`Coverage · candy-code` went red with "This test was aborted after 60
seconds". The test was made to fit (8.8s) rather than the limit
loosened to admit it — a hang-catcher that has to be raised for a test
that is merely slow has stopped catching hangs.
The named peak also moved for a second reason worth recording: the old
one, Tools\WebSearchToolTest::testHandlesRedirectResponse, was slow
because it made a real HTTP round-trip to httpbin.org. It is hermetic
now, so no test on this roster is timed by a network any more, and the
"a slow network redirect cannot trip it" clause this comment used to
need has no referent left.
Re-measure with phpunit's JUnit log flag if that peak moves again.
WHAT THE TIME LIMIT DOES AND DOES NOT BOUND, measured on this host rather
than assumed. Three runaway shapes were probed under this config: a pure
spin loop aborted at 60.09s; a loop raising a PHP warning every iteration
aborted at 61.4s having banked 2,640,658 warnings; a loop appending 4KB
strings forever aborted at 65.6s. So the alarm reports for all three. But
a reviewer measured a real tokenize() mutant that ran past 600s with RSS
over 4GB and produced ONE progress character in ten minutes — under enough
memory pressure the abort's own teardown does not complete promptly, and
the practical failure mode becomes "the CI job hangs until the CI timeout".
The limit is therefore a bound on TIME SPENT COMPUTING, not a bound on a
process that has started thrashing.
Which is why memory_limit is set below. PHP's CLI default here is -1, so
an allocating runaway grows until the machine swaps. Measured on the same
4KB-append loop: unlimited took 65.6s (38s of it in sys, i.e. paging);
at 1G it dies in 0.626s with "Allowed memory size of 1073741824 bytes
exhausted". Same defect, same red, two orders of magnitude less damage.
1G is ~4x the whole suite's measured peak of 253.25MB, so it has room to
spare for legitimate work; raise it if that peak ever approaches it.
failOnRisky is what makes the limit bite. PHPUnit records a timed-out test
as RISKY, and risky alone still exits 0 — measured: the abort printed
"This test was aborted after 60 seconds" under a green "OK, but there were
issues!" banner. So without failOnRisky the hang-catcher is a notice CI
scrolls past. Safe to turn on here because the suite reports 0 risky
tests today (6465 tests, 1 skipped, 0 risky).
No test declares #[Small]/#[Medium]/#[Large], so defaultTimeLimit is the
limit every test actually gets. Add a size attribute anywhere and that
test silently switches to PHPUnit's own 1s/10s/60s instead.
Known gap, deliberately accepted: php-invoker enforces this with
pcntl_alarm(), and tests/Agents/AgentWorkerPoolTest.php arms and cancels
pcntl_alarm() itself. Its pcntl_alarm(0) calls clear the enforcing alarm,
so the limit does not apply inside those tests. They are bounded by their
own alarms instead, which is why they are also the tests least in need of
this one.
-->
<php>
<ini name="memory_limit" value="1G"/>
</php>
<!--
E194: RuntimeNoticeSink is a process-wide static; round 47's appointment
(Chat::drainsRuntimeNotices) made the known leak unreachable but left
"clean at your start" as a property of who happens to drain. This
extension resets it before every test, making the guarantee structural.
Registered here rather than in tests/bootstrap.php so it reaches exactly
the process that emits test events - and so the guarantee is visible to
whoever reads the config. See the class doc-block for the event choice.
-->
<extensions>
<bootstrap class="SugarCraft\Code\Tests\Support\RuntimeNoticeSinkResetExtension"/>
</extensions>
<testsuites>
<testsuite name="CandyCode">
<directory>tests</directory>
</testsuite>
</testsuites>
<source>
<include>
<directory>src</directory>
</include>
</source>
</phpunit>