* topic/christian/broker-prometheus-cpp:
Update the scripts.base.frameworks.telemetry.internal-metrics test
Revert "Temporarily disable the scripts/base/frameworks/telemetry/internal-metrics btest"
Bump Broker to pull in new Prometheus support and pass in Zeek's registry
This now uses different record fields, and for now we no longer have CAF
telemetry. We indicate we're running under test to get reliable ordering in the
baselined output.
We would previously emit a C++ file with hooks for at least the builtin
`spicy` module even though that module like any other builtin module
never contains implementations of hooks for types in user code.
This patch adds a blocklist of builtin modules which are skipped for
generating hook files.
* origin/topic/awelzel/move-hilti-jit-parallelism-to-btest-cfg:
testing/btest: Default to HILTI_JIT_PARALLELISM=1
Revert "CI: Use ccache and a single CPU when building spicy analyzers for btests"
This is a rework of b59bed9d06 moving
HILTI_JIT_PARALLELISM=1 into btest.cfg to make it default applicable to
btest -j users (and CI).
The background for this change is that spicyz may spawn up to nproc compiler
instances by default. Combined with btest -j, this may be nproc x nproc
instances worst case. Particularly with gcc, this easily overloads CI or
local systems, putting them into hard-to-recover-from thrashing/OOM states.
Exporting HILTI_JIT_PARALLELISM in the shell allows overriding.
* cknill/topic/cknill/display_cmake_fix:
Fix for --display-cmake in configure Moved build directory creation further down in the script so that --display-cmake has a chance to happen before build tree setup.
* topic/christian/management-telemetry-additions:
Management framework: bump cluster testsuite to pull in telemetry tests
Management framework: bump zeek-client
Management framework: augment deployed configs with instance IP addresses
Management framework: add auto-enumeration of metrics ports
Management framework: propagate metrics port from agent
Management framework: add metrics port in management & Supervisor node records
Harden the telemetry manager against unset Telemetry::metrics_address
Comment-only tweaks for telemetry-related settings.
The controller learns IP addresses from agents that peer with it, but that
information has so far gotten lost when resulting configs get pushed out to the
agents. This makes these updates include that information.
This is quite redundant with the enumeration for Broker ports,
unfortunately. But the logic is subtly different: all nodes obtain a telemetry
port, while not all nodes require a Broker port, for example, and in the metrics
port assignment we also cross-check selected Broker ports. I found more unified
code actually harder to read in the end.
The logic for the two sets remains the same: from a start point, ports get
enumerated sequentially that aren't otherwise taken. These ports are assumed
available; there's nothing that checks their availability -- for now.
The default start port is 9000. I considered 9090, to align with the Prometheus
default, but counting upward from there is likely to hit trouble with the Broker
default ports (9999/9997), used by the Supervisor. Counting downward is a bit
unnatural, and shifting the Broker default ports brings subtle ordering issues.
This also changes the node ordering logic slightly since it seems more intuitive
to keep sequential ports on a given instance, instead of striping across them.
We populate that address from the ZEEK_DEFAULT_LISTEN_ADDRESS environment
variable, but weren't prepared for that not being set. We now fall back to
0.0.0.0. This may have the same IPv6 issues that we've encountered elsewhere
when doing so before (v6 interfaces need "::") -- but this is still more likely
to work than not having any string at all.
When Zeek flips roles of a HTTP connection subsequent to the HTTP analyzer
being attached, that analyzer would not update its own ContentLine analyzer
state, resulting in the wrong ContentLine analyzer being switched into
plain delivery mode.
In debug builds, this would result in assertion failures, in production
builds, the HTTP analyzer would receive HTTP bodies as individual header
lines, or conversely, individual header lines would be delivered as a
large chunk from the ContentLine analyzer.
PCAPs were generated locally using tcprewrite to select well-known-http ports
for both endpoints, then editcap to drop the first SYN packet.
Kudos to @JordanBarnartt for keeping at it.
Closes#3789
