Mirror/zeek
1
0
Fork 0
mirror of https://github.com/zeek/zeek.git synced 2025-10-13 20:18:20 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 3

logging: Add telemetry for streams and log writers

Browse source 
This adds one metric per log stream and one metric per log writer (path based)
to track the number of writes on a stream level as well as on a writer level.

    $ curl -sSf localhost:8181/metrics | grep Conn
    zeek_log_writer_writes_total{endpoint="",filter-name="default",module="HTTP",path="http",stream="HTTP::LOG",writer="Log::WRITER_SQLITE"} 1 1677497572770
    zeek_log_stream_writes_total{endpoint="",module="HTTP",stream="HTTP::LOG"} 1 1677497572770

The initial version of this change also included metrics around log
write vetoes, but given no log policies exist in the default configuration
and they are mostly interesting for a few streams/writers only, skip this
for now. These can always be added by the script writer, too.

The difference between the stream level writes and concrete writers can
be used to deduce the number of vetoes (or errors) as a starting point.
This commit is contained in:
Arne Welzel 2023-02-01 17:16:11 +01:00
parent 2b4aa38315
commit 69a98e2cbb
5 changed files with 104 additions and 2 deletions
Download patch file Download diff file Expand all files Collapse all files

36
testing/btest/scripts/base/frameworks/logging/telemetry.zeek Normal file
View file

@ -0,0 +1,36 @@
# @TEST-DOC: Check telemetry.log for log stream and log filter writes.
# @TEST-EXEC: zeek -b -r ${TRACES}/wikipedia.trace %INPUT
# @TEST-EXEC: btest-diff telemetry.log
@load base/protocols/conn
@load base/protocols/dns
@load base/protocols/http
@load policy/frameworks/telemetry/log
global http_logs = 0;
hook HTTP::log_policy(rec: HTTP::Info, id: Log::ID, filter: Log::Filter)
{
if (++http_logs % 3 == 0)
break;
}
global dns_logs = 0;
global conn_logs = 0;
hook Log::log_stream_policy(rec: any, id: Log::ID)
{
if (id == DNS::LOG && ++dns_logs % 3 == 0)
break;
if (id == Conn::LOG && ++conn_logs % 7 == 0)
break;
}
hook Telemetry::log_policy(rec: Telemetry::Info, id: Log::ID, filter: Log::Filter)
{
if ( rec$prefix != "zeek" || /^log-/ !in rec$name )
break;
if ( /HTTP|DNS|Conn/ !in cat(rec$label_values) )
break;
}