Mirror/zeek
1
0
Fork 0
mirror of https://github.com/zeek/zeek.git synced 2025-10-02 06:38:20 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 1

logging: Dedicated log flush timer

Browse source 
Log flushing is currently triggered based on the threading heartbeat timer
of WriterBackends and the hard-coded WRITE_BUFFER_SIZE 1000.

This change introduces a separate timer that is managed by the logger
manager instead of piggy-backing on the heartbeat timer, as well as a
const &redef for the buffer size.

This allows to modify the log flush frequency and batch size independently
of the threading heartbeat interval. Later, this will allow to re-use the
buffering and flushing logic of writer frontends for non-Broker cluster
backends, too.

One change here is that even frontends that do not have a backend will
be flushed regularly. This is wanted for non-Broker backends and should be
very cheap. Possibly, Broker can piggy back on this timer down the road, too,
rather than using its own script-level timer (see Broker::log_flush()).
This commit is contained in:
Arne Welzel 2024-09-27 11:37:23 +02:00
parent 77b9510c8a
commit 0d925e935e
11 changed files with 87 additions and 23 deletions
Download patch file Download diff file Expand all files Collapse all files

27
scripts/base/init-bare.zeek
View file

@ -2903,6 +2903,33 @@ export {
} # end export
module Log;
export {
## Default interval for flushing the write buffers of all
## enabled log streams.
##
## In earlier Zeek releases this was governed by :zeek:see:`Threading::heartbeat_interval`.
## For Broker, see also :zeek::see:`Broker::log_batch_interval`.
##
## .. :zeek:see:`Log::flush`
## .. :zeek:see:`Log::set_buf`
## .. :zeek:see:`Log::write_buffer_size`
const flush_interval = 1.0sec &redef;
## Default maximum size of the log write buffer per filter/path pair.
## If this many log writes are buffered, the writer frontend flushes
## its writes to its backend before flush_interval expires.
##
## In earlier Zeek releases this was hard-coded to 1000.
##
## .. :zeek:see:`Log::flush`
## .. :zeek:see:`Log::set_buf`
## .. :zeek:see:`Log::flush_interval`
const write_buffer_size = 1000 &redef;
} # end export
module POP3;
export {