@Sheco reported that standalone epoch processing may exclude scheduled
events when the final sumstat epoch runs before. For example, this easily
happens when attempting to do sumstat observations within connection_state_remove().
Delay final epoch processing to zeek_done() instead.
This doesn't deal with the clustered version - this would need something
more elaborate and potentially a mechanism to delay the shutdown of
other cluster nodes until/after sumstat processing completed.
* origin/topic/christian/debug-stream-tweaks:
Make debug stream names use "-", and handle "_" transparently
Make "-B all" apply to plugin streams as well.
Sort streams in "-B help" output, and match case-insensitively throughout
Matching of plugins' debug streams was still case-sensitive. Also contains some
minor output tweaks.
It'd be nice to only list plugin debug streams actually _used_ by plugins. I
didn't see a quick way to do that so that's for another time.
Establishing reliable ordering fixes a test failure we're seeing on Alpine for
the signatures/tcp-end-of-match btest, since discrepancies in rule match
traversal could lead to discrepancies in corresponding event ordering.
Discussed with @J-Gras, calling Broker::publish() within a scheduled
should use the "intended timestamp" implicitly.
This is subtle, but supposedly more expected when running
a pcap replay cluster.
* origin/topic/timw/remove-abspath-cleanup:
diff-remove-abspath: Add separate handling of Windows paths
diff-remove-abspath: Remove capture of windows drive letters from POSIX regex
* origin/topic/vern/zam-asan-fixes:
ZAM fixes for assignments involving "any" record fields
fixes for (mostly ZAM) vector operation issues found by ASAN
Including a fix for mmdb/explicit-open.zeek to avoid using assert.
* origin/topic/timw/update-c-ares-to-latest-release:
DNS_Mgr: Remove processing of dns aliases in general
ci: Add dnsmasq to a few platforms for testing
DNS_Mgr: Fix aliases memory issues
btest: Add integration test for DNS_Mgr
DNS_Mgr: Remove usage of ares_getsock from Lookup
DNS_Mgr: Remove usage of ares_getsock from GetNextTimeout
DNS_Mgr: Switch to ares_set_servers_csv
DNS_Mgr: Use ares_dns_record methods for queries
Update vcpkg submodule to pick up c-ares v1.34.2
Update c-ares submodule to v1.34.2
This wasn't possible before #3028 was fixed, but now it's safe to set
the value in new_connection() and allow other users access to the
field much earlier. We do not have to deal with connection_flipped()
because the community-id hash is symmetric.
So far, when Zeek didn't see a connection's regular tear-down (e.g.,
because its state timed-out before we got to the end), we'd still
signal a regular end-of-data to Spicy parsers. As a result, they would
then typically raise a parse error because they were probably still
expecting data and would now declare it missing. That's not very
useful because semantically it's not really a protocol issue if the
data just doesn't make it over to us; it's a transport-layer issue
that Zeek already handles elsewhere. So we now switch to signaling
end-of-data to Spicy analyzers only if the connection indeed shuts
down regularly. This is also matches how BinPAC handles it.
This also comes with a test exercising various combinations of
end-of-data behavior so that we ensure consistent/desired behavior.
Closes#4007.
typename(table()) apparently always resulted in a set[] type
being rendered. Make the yield type of an unspecified table
ANY so that type->IsSet() ends up false.
While at it, also render unspecified types as table(), set() and
vector() rather than vector of void, set[] or table[] of any which
IMO should help to figure out what's going.
This admittedly is a quite esoteric combination of protocols. But - as
we do correctly support them, it seems nice to have a slightly more
complete testcase that covers this.
* origin/topic/awelzel/data-end-offset-v1:
signatures: Add data_end_offset to signature_match() and custom events
Add pattern_end_offset to signature_state
This change tracks the current offset (number of bytes fed into matchers)
on the top-level RuleEndpointState such that we can compute the relative ending
for matched texts individually.
Additionally, it adds the data_end_offset as a new optional parameter to
signature_match().
This commit fixes three issues with Zeek's Modbus message logging:
1 - Some exception responses (e.g., READ_COILS_EXCEPTION) are logged
twice: once without and once with the exception message.
2 - Some exception responses (e.g., PROGRAM_484_EXCEPTION) are not
logged.
3 - Some known but reserved function codes (e.g., PROGRAM_UNITY) are
logged as unk-xxx (e.g., unk-90), while it would be possible to
log their known name.
To address these inconsistencies, the modbus parser has been updated
to parse all exception responses (i.e., all responses where the MSB
of the function code is set) using the already defined Exception
message.
Also, the Modbus main.zeek script has been updated to consistently
demand logging exception responses to the specialized
modbus_exception event, rather than logging some exception responses
in the modbus_message event and others in the modbus_exception event.
Finally, the main.zeek script has been updated to make sure that
for every known function code, the corresponding exception code was
also present, and the enumeration of known function codes in
consts.zeek has been expanded.
Closes#3984
