* origin/topic/awelzel/pluggable-cluster-backends-part1:
btest: Test Broker::make_event() together with Cluster::publish_hrw()
btest: Add cluster dir, minimal test for enum value
broker: Add shim plugin adding a backend component
zeek-setup: Instantiate backend::manager
cluster: Add to src/CMakeLists.txt
cluster: Add Components and ComponentManager for new components
cluster/Backend: Interface for cluster backends
cluster/Serializer: Interface for event and log serializers
logging: Introduce logging/Types.h
SerialTypes/Field: Allow default construction and add move constructor
DebugLogger: Add cluster debugging stream
plugin: Add component enums for pluggable cluster backends
broker: Pass frame to MakeEvent()
Header and implementation for types shared between the cluster and
logging framework. The logging framework will be adapted later to
use these. For now, the new cluster components will simply reference
them.
The continued line was incorrectly marked up (as a plain comment instead
of a Zeekygen comment) which caused only half of the docstring to be
rendered by Zeekygen.
This ensures that the normal test setup is run, which enables greedy
mode on Cirrus allowing us to over-provision the testing. This should
help speed up the ZAM builds a bit.
41ec748 cmake: Use foreach range
2b90a56 tests: Update new-event to run with 7.1+, remove ip_proto elsewhere
a596f6f Makefile: Update outdated commands
b954661 Enable `cmake/FindNodejs.cmake` to find more libnode versions
This ordering fixes a test failure we're seeing on Alpine for the
signatures/tcp-end-of-match btest, since discrepancies in rule match traversal
could lead to discrepancies in corresponding event ordering.
It looks safe to rely on across platforms since the index is driven by signature
load order, which shouldn't deviate. If this somehow doesn't hold in the future,
we'll only wind up with a test failure, not incorrect match behavior.
(Correction to 2e03fbb8b0, which I pushed
accidentally.)
@Sheco reported that standalone epoch processing may exclude scheduled
events when the final sumstat epoch runs before. For example, this easily
happens when attempting to do sumstat observations within connection_state_remove().
Delay final epoch processing to zeek_done() instead.
This doesn't deal with the clustered version - this would need something
more elaborate and potentially a mechanism to delay the shutdown of
other cluster nodes until/after sumstat processing completed.
Currently, plugins do not have a way to further inspect or even mutate
script functions after ZAM optimization ran. One use-case here is
zeek-perf-support [1]. This plugin wraps Stmt instances of functions,
events and hooks hooks with a small assembly stub to support JIT map
files [2] and for integration with perf tools.
This change introduces a new InitPreExecution() hook that runs after
ZAM optimization completed, just before the zeek_init() event is enqueued.
Additionally, remove the existing CPP_activation_hook. It doesn't seem
to be used. If it becomes necessary in the future, the new
InitPreExecution() hook can be leveraged instead.
[1] https://github.com/zeek/zeek-perf-support
[2] https://github.com/torvalds/linux/blob/master/tools/perf/Documentation/jit-interface.txt
* origin/topic/christian/debug-stream-tweaks:
Make debug stream names use "-", and handle "_" transparently
Make "-B all" apply to plugin streams as well.
Sort streams in "-B help" output, and match case-insensitively throughout
We already accept that the packets coming into this analyzer won't
be processed, so forwarding out of it will just result in a failure.
Forwarding will also report a weird for every packet which just
results in extra noise.
Matching of plugins' debug streams was still case-sensitive. Also contains some
minor output tweaks.
It'd be nice to only list plugin debug streams actually _used_ by plugins. I
didn't see a quick way to do that so that's for another time.
* origin/topic/awelzel/log-broker-io-telemetry-rules-includes:
zeek-setup: Remove some unused headers
clang-format: Sort doctest header at the bottom
RuleMatcher: Move plugin/Manager.h include from .h to .cc
iosource/Manager: Remove superflous includes
telemetry/Manager: Remove broker header include, add fnmatch.h
logging/Manager: Fix using filename from input.h in debug log
GetFieldAs() does not initialize an optional or default field. Apparently,
for ZAM that makes a difference, possibly the [] record construction is
optimized, not initializing proto until actually accessed. Not quite
sure why that's not happening for classic script interpretation
though.
