zeek/src/script_opt/CPP/maint

This is a collection of scripts to support maintenance of -O gen-C++
(and friends).  They're oriented around running against the BTest test
suite, and are currently tailored for the lead maintainer's own environment.
The scripts all assume you're running them from build/ .

If you make changes to the scripts, format them using

	shfmt -w -i 4 -ci *.sh

(or set up "pre-commit" and use "pre-commit run -a")

The maintenance workflow:

1.  Make sure the compiler can compile and execute the base scripts:

	src/zeek -O gen-C++ /dev/null
	ninja
	src/zeek -O use-C++ -r some.pcap

    and that standalone compilation works:

	rm CPP-gen.cc
	ninja
	src/zeek -b -O gen-standalone-C++ \
		--optimize-files=base/protocols/conn \
		--optimize-files=base/utils/files \
		--optimize-files=base/utils/addr \
		base/protocols/conn >my-test.zeek
	ninja
	src/zeek -b -r some.pcap my-test.zeek
	# Confirm that it generates conn.log
	rm CPP-gen.cc
	ninja

    Do this first because if it can't, you'll be making changes to the
    compiler that you'll want to subsequent run against the test suite,
    per the following.

2.  "mkdir CPP-test" - a directory for holding results relating to C++ testing

3.  Run "find-test-files.sh" to generate a list (to stdout) of all of the
    possible Zeek source files found in the test suite.

4.  For each such Zeek file, run "check-zeek.sh" to see whether Zeek can
    parse it.  This helps remove from further consideration difficult
    tests (like those that have embedded input files, or multiple separate
    scripts).  Each run writes a report into CPP-test/cz.* file.

    This step is parallelizable, say using xargs -P 10 -n 1.

5.  Run "check-CPP-gen.sh" for each Zeek file that passed "check-zeek.sh".
    This will generate a corresponding file in CPP-test/out.* indicating whether
    "-O gen-C++" can successfully run on the input.  Presently, it should
    be able to do so for all of them except a few that have conditional code,
    which I've left active (no @TEST-REQUIRES to prune) given hopes of
    soon being able to support (most) conditional code for C++ compilation.

    This step is parallelizable, say using xargs -P 10 -n 1.

6.  Copy ./src/zeek to ./zeek.HOLD.  This is used to speed up recompilation used
    in the next step.  However, it's also a headache to do development to
    fix a bug and then forget to update zeek.HOLD, which means you wind up
    running the old version.  You can combat that by removing ./zeek.HOLD
    every time you start working on fixing a bug.

7.  For every input that survives that pruning, run "do-CPP-btest.sh".
    This will generate C++ for the BTest, compile it, and run the result
    to see if it succeeds.  It populates CPP-test/diag* with the Btest
    diagnostic output (empty means success).  For non-empty output,
    either fix the problem, add 

	# @TEST-REQUIRES: test "${ZEEK_USE_CPP}" != "1"

    if the test isn't appropriate, or update the database if it's not
    readily fixable, along with the reason why.

    Note that do-CPP-btest.sh takes the same -d and -U arguments as
    does btest, for displaying differences or updating the baseline
    (which is Baseline.cpp).

FYI:
    Tests that have demonstrated variations/failures due to presumed
    race conditions:

	../testing/btest/scripts/base/utils/active-http.test