Merge remote-tracking branch 'origin/topic/awelzel/move-hilti-jit-parallelism-to-btest-cfg'

* origin/topic/awelzel/move-hilti-jit-parallelism-to-btest-cfg:
  testing/btest: Default to HILTI_JIT_PARALLELISM=1
  Revert "CI: Use ccache and a single CPU when building spicy analyzers for btests"
This commit is contained in:
Arne Welzel 2024-07-10 11:05:19 +02:00
commit 1c8fedaf09
4 changed files with 22 additions and 4 deletions

19
CHANGES
View file

@ -1,3 +1,22 @@
7.0.0-dev.458 | 2024-07-10 11:05:19 +0200
* testing/btest: Default to HILTI_JIT_PARALLELISM=1 (Arne Welzel, Corelight)
This is a rework of b59bed9d06f2257a6b4f6c6408b8363a277eebac moving
HILTI_JIT_PARALLELISM=1 into btest.cfg to make it default applicable to
btest -j users (and CI).
The background for this change is that spicyz may spawn up to nproc compiler
instances by default. Combined with btest -j, this may be nproc x nproc
instances worst case. Particularly with gcc, this easily overloads CI or
local systems, putting them into hard-to-recover-from thrashing/OOM states.
Exporting HILTI_JIT_PARALLELISM in the shell allows overriding.
* Revert "CI: Use ccache and a single CPU when building spicy analyzers for btests" (Arne Welzel, Corelight)
This reverts commit b59bed9d06f2257a6b4f6c6408b8363a277eebac.
7.0.0-dev.454 | 2024-07-09 15:12:19 -0700 7.0.0-dev.454 | 2024-07-09 15:12:19 -0700
* CI: Use ccache and a single CPU when building spicy analyzers for btests (Tim Wojtulewicz, Corelight) * CI: Use ccache and a single CPU when building spicy analyzers for btests (Tim Wojtulewicz, Corelight)

View file

@ -1 +1 @@
7.0.0-dev.454 7.0.0-dev.458

View file

@ -59,9 +59,7 @@ function run_btests {
pushd testing/btest pushd testing/btest
HILTI_CXX_COMPILER_LAUNCHER=ccache \ ZEEK_PROFILER_FILE=$(pwd)/.tmp/script-coverage/XXXXXX \
HILTI_JIT_PARALLELISM=1 \
ZEEK_PROFILER_FILE=$(pwd)/.tmp/script-coverage/XXXXXX \
${BTEST} -z ${ZEEK_CI_BTEST_RETRIES} -d -A -x btest-results.xml -j ${ZEEK_CI_BTEST_JOBS} || result=1 ${BTEST} -z ${ZEEK_CI_BTEST_RETRIES} -d -A -x btest-results.xml -j ${ZEEK_CI_BTEST_JOBS} || result=1
make coverage make coverage
prep_artifacts prep_artifacts

View file

@ -14,6 +14,7 @@ MinVersion = 0.63
[environment] [environment]
ZEEKPATH=`bash -c %(testbase)s/../../%(build_dir)s/zeek-path-dev` ZEEKPATH=`bash -c %(testbase)s/../../%(build_dir)s/zeek-path-dev`
HILTI_CXX_COMPILER_LAUNCHER=`f=%(testbase)s/../../%(build_dir)s/CMakeCache.txt && grep -q '^ENABLE_CCACHE:BOOL=true' $f && sed -n 's/^CCACHE_PROGRAM:FILEPATH=\(.*\)$/\1/p' $f` HILTI_CXX_COMPILER_LAUNCHER=`f=%(testbase)s/../../%(build_dir)s/CMakeCache.txt && grep -q '^ENABLE_CCACHE:BOOL=true' $f && sed -n 's/^CCACHE_PROGRAM:FILEPATH=\(.*\)$/\1/p' $f`
HILTI_JIT_PARALLELISM=`bash -c 'echo ${HILTI_JIT_PARALLELISM:-1}'`
ZEEK_SEED_FILE=%(testbase)s/random.seed ZEEK_SEED_FILE=%(testbase)s/random.seed
ZEEK_PLUGIN_PATH= ZEEK_PLUGIN_PATH=
TZ=UTC TZ=UTC