Testing io_poll_interval_live tweaks with @dopheide-esnet on a Myricom based
system to reduce CPU usage showed no visible effect.
It turns out, the pkt_src->IsLive() call used to update poll_interval is only
valid *after* calling ->Register() with the source. The conditional updating
of the poll_interval introduced in 4fa3e4b9b4
never worked out how it was intended to.
The fix ensures that
* we actually use a poll_interval of 10 in the live case
* changing io_poll_interval_live does have an effect
This is a bit of a major change due to lowering the default poll_interval
by a magnitude, but that seemed to have been the intention always. It's also
tunable via redef, so worst case it can be adapted via configuration.
As reference, with the default a Pcap::non_fd_timeout of 20usec *and* a
poll_interval of 100, theoretically we'd be trying to ask a non-selectable
packet source 500000 per second for a new packet. This is not a likely packet
rate that a single worker would currently observe or manage to process.
This allows to ignore-deprecations as follows:
@pragma push ignore-deprecations
print r$deprecated;
@pragma pop
A bit of infrastructure is put in place for the push and pop, but
nothing overly generalized for future pragmas.
Removes a bit of reliance around the magic DoLog() rendering at the
cost of needing to open-code some of it. The new obj_desc_short()
helper makes that acceptable, though.
Same as obj_desc() but use the short version and do not include the
location information by default. New method instead of bool parameters
for readability.
* origin/topic/vern/when-cleanup:
test suite update for minor change in "when" error messages
removed skeletal (non-functioning) "when" support from ZAM
simplify WhenInfo and Trigger classes given removal of old capture semantics
introduced notion of light-weight Frame clones
changed function_ingredients struct to FunctionIngredients class with accessors
Renamed Frame::LightClone() to Frame::CloneForTrigger() during merge.
This is one level above the Reassembler where we still have information
about the file and source. A weird entry may looks as follows:
1679759398.237353 ... file_offset_overflow FXPLGt4SeMmlMKahJc: offset=fffffffffffffff7 len=10 F zeek HTTP
The reassembler logic isn't wrap around safe, so just truncate or
reject such blocks. For files specifically, a byte offset in the
2**64 bytes represents 16EiB which is the maximum size supported
by BTRFS or NTFS (and probably nothing we'd ever see in practice).
* origin/topic/vern/CPP-Apr23-maint:
addressed static analysis concern about possible null pointer
tweaks for "-O C++" of BTest's with conditional code
Backed out changes from at-if-lambda during merge and instead skip
test when running with ZEEK_USE_CPP.
The medium.trace in the private external test suite contains one
session/server that violates the multi-line reply protocol and
happened to work out fairly well regardless due to how we looked
up the pending commands unconditionally before.
Continue to match up reply lines that "look like they contain status codes"
even if cont_resp = T. This still improves runtime for the OSS-Fuzz
generated test case and keeps the external baselines valid.
The affected session can be extracted as follows:
zcat Traces/medium.trace.gz | tcpdump -r - 'port 1491 and port 21'
We could push this into the analyzer, too, minimally the RFC says:
> If an intermediary line begins with a 3-digit number, the Server
> must pad the front to avoid confusion.
* origin/topic/vern/ZAM-Apr23-maint:
minor ZAM BTest baseline updates
fixed type mismatch for ssl_certificate_request event
skip ZAM optimization of invalid scripts
extended script validation to be call-able on a per-function basis
Increasing this value 10x has lowered CPU usage on a Myricom based
deployment significantly with reportedly no adverse side-effects.
After reviewing the Zeek 3 IO loop, my hunch is that previously when
no packets were available, we'd sleep 20usec every loop iteration after
calling ->Process() on the packet source. With current master ->Process()
is called 10 times on a packet source before going to sleep just once
for 20 usec. Likely this explains the increased CPU usage reported.
It's probably too risky to increase the current value, so introduce
a const &redef value for advanced users to tweak it. A middle ground
might be to lower ``io_poll_interval_live`` to 5 and increase the new
``Pcap::non_fd_timeout`` setting to 100usec.
While this doesn't really fix#2296, we now have enough knobs for tweaking.
Closes#2296.
This fixes a bug for AYIYA, Geneve and VXLAN forwarding encapsulated
content only if it's longer than their header. A new weird is introduced
to indicate empty tunnels.
It turns out that for every ListVal we construct, we also allocate
and construct a new TypeList instance, even though they are all the
same. Pre-create and cache the type instances in a new TypeManager.
The following script runs ~10% faster for me after this change.
global tbl: table[string] of string;
global i = 0;
while ( ++i < 10000000 )
tbl["a"] = "a";
In #2464 the warning when overriding a packet analyzer mapping was
removed. While a warning seems indeed excessive, some info would still
be nice to have.
