Logs that got sent sparsely or burstily would get buffered for long
periods of time since the logic to flush them only does so on the next
log write. In the worst case, a subsequent log write could never happen
and cause a log entry to be indefinitely buffered.
This fix introduces a recurring event/timer to simply flush all pending
logs at frequency of Broker::log_batch_interval.
It may generally be better for our default use-case, as workers may
save a few percent cpu utilization as this policy does not have to
use any polling like the stealing policy does.
This also helps avoid a potential issue with the implementation of
spinlocks used in the work-stealing policy in current CAF versions,
where there's some conditions where lock contention causes a thread
to spin for long periods without relinquishing the cpu to others.
For backward compatibility when reading values, we first check
the ZEEK-prefixed value, and if not set, then check the corresponding
BRO-prefixed value.
* All "Broxygen" usages have been replaced in
code, documentation, filenames, etc.
* Sphinx roles/directives like ":bro:see" are now ":zeek:see"
* The "--broxygen" command-line option is now "--zeexygen"
Disabling this option allows one to read pcaps, but still initiate
Broker peerings and automatically exit when done processing the pcap
file. The default behavior would normally cause Broker::peer() to
prevent shutting the process down even after done reading the pcap.
Still finding it to not be foolproof enough to enable generally for all
nodes in a cluster. Specific/advanced use-cases may still consider
enabling, possibly just for specific nodes.
This enables explicit forwarding of events matching a given topic
prefix. Even if a receiving node has an event handler, it will not
be raised if the event was sent along a topic that matches a previous
call to Broker::forward().
Now defaults to a max of 4 threads typically indepedent of core
count (previously could go up to a hard cap of 8). Also now allow
controlling this setting via BRO_BROKER_MAX_THREADS environment
variable.
The former replaces the pcap vs. live versions of the same tuning
option. If a user does not change these, Bro makes some internal
decisions that may help avoid performance problems on systems with high
core counts: the number of CAF threads is capped at 8 and the maximum
sleep duration for under-utilized threads is increased to 64ms (CAF's
default is 10ms).
These may be used to change the number of scheduler threads that the
underlying CAF library creates. In pcap mode, it's currently hardcoded
to the minimal 4 threads due to potentially significant overhead in CAF.
This environment variable is now set to listen only on IPv4 loopback
when running unit tests (instead of using the default INADDR_ANY).
This also moves some of the @loads out from init-bare.bro into a new
init-frameworks-and-bifs.bro in order to better support calling BIFs
(like `getenv`) from variable initializations in those particular
frameworks.
When Bro was compiled with broker disabled, then some Bro scripts
were referencing functions and types that were not defined. Fixed
by adding @ifdefs to several scripts. Removed one @ifdef because
it was causing several unit tests to fail.
Also fixed the @TEST-REQUIRES check in tests that rely on broker so
that such tests are skipped when broker is disabled.
Also renamed the "print" function to "send_print" and the "event"
function to "send_event" because Bro shows a syntax error when a
Bro script function is named "event" or "print".
Separate the former BrokerComm and BrokerStore portions of the script
because these files will be much larger when script wrappers for BIFs are
written.
