Skip to content

tests: build the test executables at -O1 when no build type is set - #228

Open
petlenz wants to merge 3 commits into
mainfrom
build/tests-o1
Open

petlenz wants to merge 3 commits into
mainfrom
build/tests-o1

Conversation

@petlenz

@petlenz petlenz commented Oct 28, 2026 •

Copy link
Copy Markdown
Member

Why

A local gate build directory (GCC 14, plain + ASan tests, reports) was 8.2 GB, and a dozen of them filled the disk. 8.1 GB of that is tests/.

CMAKE_BUILD_TYPE is empty by default, so every test compiles at -O0. Each of the 75 test files then instantiates tmech's expression templates and the materials without any inlining, and the ASan copy is about 4× larger again.

What

tests/CMakeLists.txt adds NUMSIM_MATERIALS_TEST_OPT_FLAGS to every test target: plain, _asan and _tsan.

  • It defaults to auto, resolved on every configure: -O1 only when no build type is set, the generator is single-config and CMAKE_CXX_FLAGS has no -O option. Any other value is used as given.
  • An explicit CMAKE_BUILD_TYPE (for example Debug) leaves the flags alone.
  • -DNUMSIM_MATERIALS_TEST_OPT_FLAGS= turns it off.

-O1 is also what the ASan documentation recommends.

The library itself is header-only and unaffected. examples/ and the material_report tool are unchanged.

Measured

-O0 (main) -O1
test_report_case.cpp.o 36 MB 6.4 MB (-O2: 6.1 MB)
gate build directory 8.2 GB 2.7 GB
full build, -j4 ~20 min ~25 min of active compiling*
ctest -j4, 1440 tests 68–85 s 71 s
warnings / failures 0 / 0 0 / 0

* The wall time of the measured run was 182 min, but no object was written for two stretches of 125 and 32 minutes, while the machine lost its network connection. The active compile time is the span minus those gaps.

One test file (test_report_case.cpp, compiled alone, same machine):

compile object
-O0 38 s 35 MB
-O1 43 s 6.3 MB
-O0 ASan 75 s 101 MB
-O1 ASan 120 s 72 MB

So -O1 is nearly free for the plain tests and costs about 60% more compile time on the ASan copy for a 30% smaller object; overall about +25% build time for a third of the disk.

Test time barely changes, because most of it is process start-up and test discovery across 150 executables.

Not here

Merging the 75 test executables into a few per area would shrink objects and link time further. That is a larger change to tests/CMakeLists.txt.

Gate

GCC 14, reports on, -DNUMSIM_PREFER_SIBLINGS=OFF: 1440/1440 tests pass, 0 warnings.

At -O0 the uninlined expression templates make each test object about 5x larger; a gate build directory drops from 8.2 GB to 2.7 GB. NUMSIM_MATERIALS_TEST_OPT_FLAGS overrides it, an explicit CMAKE_BUILD_TYPE turns it off.
…e CI leg

The cache value is now 'auto' and is resolved at each configure: -O1 only without a build type, with a single-config generator and without an -O option in CMAKE_CXX_FLAGS. Other values are used verbatim (split into arguments), empty turns it off. CI gains a clang++-19 leg without a build type so the -O1 warnings under -Werror are covered.
@petlenz

petlenz commented Oct 29, 2026 •

Copy link
Copy Markdown
Member Author

Review findings addressed in b1f5e6b.

  1. Frozen default (major). The cache value is now auto and is resolved on every configure: -O1 only when CMAKE_BUILD_TYPE is empty, the generator is single-config and CMAKE_CXX_FLAGS has no -O option. Any other value is used verbatim, empty turns it off. Reconfiguring a no-build-type directory with -DCMAKE_BUILD_TYPE=Release no longer leaves -O1 after -O3.
  2. Comment. Reworded: an empty build type is the case that enables -O1; the way to turn it off is an empty NUMSIM_MATERIALS_TEST_OPT_FLAGS.
  3. User -O in CMAKE_CXX_FLAGS (for example -O0 --coverage) is no longer overridden (covered by 1).
  4. "-O1 -g" is split with separate_arguments(... UNIX_COMMAND ...) into two arguments.
  5. CI. New matrix leg clang++-19 with an empty build type (same steps, so the tests compile at -O1 under -Werror). The job name of the existing legs is unchanged; the new one is build-and-test (clang++-19, no build type).

Configure-only check on compile_commands.json (one test TU, both the plain and the ASan entry), with CMAKE_EXPORT_COMPILE_COMMANDS=ON, no build. Script:

cmake228.sh
#!/bin/bash
# Configure-only check of the test optimisation flags in compile_commands.json (no build).
SRC=${1:-$HOME/devel/nm-o1}
W=${2:-/tmp/cmake228}
DEPS=$HOME/devel/nm-o1-build/_deps
TU=test_core_utilities.cpp
rm -rf "$W"; mkdir -p "$W"
cfg() { # dir args...
  local d=$W/$1; shift
  CXX=/usr/bin/g++-14 cmake -S "$SRC" -B "$d" -DCMAKE_EXPORT_COMPILE_COMMANDS=ON -DNUMSIM_PREFER_SIBLINGS=OFF \
    -DFETCHCONTENT_SOURCE_DIR_TMECH=$DEPS/tmech-src -DFETCHCONTENT_SOURCE_DIR_NUMSIM-CORE=$DEPS/numsim-core-src \
    -DNUMSIM_MATERIALS_BUILD_EXAMPLES=OFF "$@" > "$d.log" 2>&1 || { echo "configure failed: $d"; tail -5 "$d.log"; }
}
# prints the compile flags of every entry of $TU as one token per line, entries separated by ---
flags() {
  python3 -I - "$W/$1/compile_commands.json" "$TU" <<'PY'
import json, shlex, sys
for e in json.load(open(sys.argv[1])):
    if e["file"].endswith("/" + sys.argv[2]):
        print(" ".join("[" + t + "]" for t in shlex.split(e["command"]) if t.startswith(("-O", "-g"))) or "(none)")
PY
}
fail=0
check() { # label dir want-regex-present want-regex-absent
  local out; out=$(flags "$2")
  local ok=1
  [ -n "$out" ] || ok=0
  while IFS= read -r line; do
    [ -n "$3" ] && ! grep -qE -- "$3" <<<"$line" && ok=0
    [ -n "$4" ] && grep -qE -- "$4" <<<"$line" && ok=0
  done <<<"$out"
  echo "$1: $([ $ok = 1 ] && echo PASS || echo FAIL)"; sed 's/^/    /' <<<"$out"; [ $ok = 1 ] || fail=1
}
cfg a
check "A fresh, no build type: -O1 present" a '\[-O1\]' ''
cfg a -DCMAKE_BUILD_TYPE=Release
check "B same dir reconfigured with Release: -O1 absent" a '\[-O3\]' '\[-O1\]'
cfg c -DCMAKE_CXX_FLAGS=-O0
check "C CMAKE_CXX_FLAGS=-O0: no -O1" c '\[-O0\]' '\[-O1\]'
cfg d "-DNUMSIM_MATERIALS_TEST_OPT_FLAGS=-O1 -g"
check "D OPT_FLAGS=\"-O1 -g\": two separate args" d '\[-O1\].*\[-g\]' '\[-O1 -g\]'
cfg e -DNUMSIM_MATERIALS_TEST_OPT_FLAGS=
check "E OPT_FLAGS empty: off" e '' '\[-O'
exit $fail

Before the fix (cases B, C, D fail; E passes because the empty value was already honoured):

A fresh, no build type: -O1 present: PASS
    [-O1]
    [-O1]
B same dir reconfigured with Release: -O1 absent: FAIL
    [-O3] [-O1]
    [-O3] [-O1]
C CMAKE_CXX_FLAGS=-O0: no -O1: FAIL
    [-O0] [-O1]
    [-O0] [-O1]
D OPT_FLAGS="-O1 -g": two separate args: FAIL
    [-O1 -g]
    [-O1 -g]
E OPT_FLAGS empty: off: PASS
    (none)
    (none)

After the fix:

A fresh, no build type: -O1 present: PASS
    [-O1]
    [-O1]
B same dir reconfigured with Release: -O1 absent: PASS
    [-O3]
    [-O3]
C CMAKE_CXX_FLAGS=-O0: no -O1: PASS
    [-O0]
    [-O0]
D OPT_FLAGS="-O1 -g": two separate args: PASS
    [-O1] [-g]
    [-O1] [-g]
E OPT_FLAGS empty: off: PASS
    (none)
    (none)

Gate (GCC 14, -DCMAKE_CXX_FLAGS=-O1, reports on): BUILD=0 TEST=0, 1440/1440 tests passed, 0 warnings.

# Conflicts:
#	.github/workflows/ci.yml
#	tests/CMakeLists.txt
@petlenz

petlenz commented Oct 30, 2026

Copy link
Copy Markdown
Member Author

Merged main (#238, batched tests and split CI jobs) into this branch. Two conflicts:

  • ci.yml: the clang++-19 job without a build type is an include entry of the new {cxx} × {asan} matrix (asan: false). CI now has five jobs; the names show asan and no build type.
  • tests/CMakeLists.txt: the -O1 resolution is unchanged. A helper numsim_materials_test_options() applies it to the standalone tests and to the batch executables, each with its sanitizer variants.

The configure check from the PR description passes all five cases (fresh, Release on the same dir, -O0, "-O1 -g", empty), both for a batch (test_batch_0.cpp) and for a standalone test (test_material_conformance.cpp).

Gate: g++-14 -O1, 1486 tests, 0 warnings.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant