This change adds two new hooks to the Intel framework that can be used
to intercept added and removed indicators and their type.
These hooks are fairly low-level. One immediate use-case is to count the
number of indicators loaded per Intel::Type and enable and disable the
corresponding event groups of the intel/seen scripts.
I attempted to gauge the overhead and while it's definitely there, loading
a file with ~500k DOMAIN entries takes somewhere around ~0.5 seconds hooks
when populated via the min_data_store store mechanism. While that
doesn't sound great, it actually takes the manager on my system 2.5
seconds to serialize and Cluster::publish() the min_data_store alone
and its doing that serially for every active worker. Mostly to say that
the bigger overhead in that area on the manager doing redundant work
per worker.
Co-authored-by: Mohan Dhawan <mohan@corelight.com>
(cherry picked from commit 3366d81e98ef381d843f6d76628834fdcd622e25)
This is a cluster-layout.zeek template that can be copied into a testing
directory if needed. The idea is that a developer sets environment
variables within their btest and the Cluster::nodes variable is
implicitly extended by appropriate nodes.
For example, using @TEST-PORT BROKER_LOGGER1_PORT will add an appropriate
logger-1 node to Cluster::nodes, based on the existence of the
BROKER_LOGGER1_PORT environment variable.
(cherry picked from commit a56744a2679c3bd42b4046f2687ccc8fb8442485)
* 'topic/mohan/intel-event-groups' of https://github.com/Mohan-Dhawan/zeek:
coalesce smtp handlers for ADDR
Add fine-grained groups for Intel events
(cherry picked from commit d5e1dc27c6)
* origin/topic/timw/ci-macos-sequoia:
ci/init-external-repo.sh: Use regex to match macos cirrus task
CI: Change macOS runner to Sequoia
(cherry picked from commit 43f108bb71)
* origin/topic/awelzel/4198-4201-quic-maintenance:
QUIC/decrypt_crypto: Rename all_data to data
QUIC: Confirm before forwarding data to SSL
QUIC: Parse all QUIC packets in a UDP datagram
QUIC: Only slurp till packet end, not till &eod
(cherry picked from commit 44304973fb)
* origin/topic/vern/ZAM-field-assign-in-op:
pre-commit: Bump spicy-format to 0.23
fix for ZAM optimization of assigning a record field to result of "in" operation
(cherry picked from commit 991bc9644d)
* security/topic/timw/7.0.5-patches:
QUIC/decrypt_crypto: Actually check if decryption was successful
QUIC/decrypt_crypto: Limit payload_length to 10k
QUIC/decrypt_crypto: Fix decrypting into too small stack buffer
Given we dynamically allocate memory for decryption, employ a limit
that is unlikely to be hit, but allows for large payloads produced
by the fuzzer or jumbo frames.
A QUIC initial packet larger than 1500 bytes could lead to crashes
due to the usage of a fixed size stack buffer for decryption.
Allocate the necessary memory dynamically on the heap instead.
* origin/topic/timw/speed-up-zam-ci-testing:
CI: Use test.sh script for running ZAM tests, but disable parts of it
(cherry picked from commit d9a74680e0)
Fixes#3994
Clang 19 with libc++ started failing to compile because the default
implementation of `std::char_traits` was removed, making uses of
`std::char_traits<unsigned char>` invalid. This was more of used for
convenience before, but it should be roughly the same behavior with
`char`.
See relevant LLVM commits:
aeecef08c308a0faf4cd
(cherry picked from commit 985f4f7c72)
* origin/topic/vern/zam-fixes-for-7.0.x:
import of GH-4022 BTest additions ZAM baseline update
fix for setting object locations to avoid use-after-free situation
fixes for script optimization of coerce-to-any expressions
porting of GH-4022
porting of GH-4016
porting of GH-4013
fixed access to uninitialized memory in ZAM's "cat" built-in
