Motivation is basically the same as in 88bb527026.
For plugin.hooks, one example is that adding a new option in the default script
changes the baseline due registration of change handlers. Also, the connection
record is printed in various places, resulting in churn when the default
scripts change.
(cherry picked from commit 07ac6fa074)
The dump-events baseline changes are pure noise and have spurred confusion
for internal and external contributors. For example, adding new
analyzers have perturbed orderings of sets holding analyzer tags.
Running in non-bare mode, the baselines change almost whenever any of the
record types attached to connections change in the default scripts. This
causes continuous and seemingly little useful updates to the baselines.
This change switches the test to run in bare mode and explicitly loads
just base/protocols/conn and base/protocols/smtp. The primary intention
of the test should be testing the functionality of the misc/dump-events
script, not the raised events of all loaded default scripts (for that the
used PCAP is too narrow).
Protocol specific scripts that do want to leverage misc/dump-events for
baseline creation of their or their analyzer's events can add additional
specific tests with suitable PCAP files.
(cherry picked from commit 88bb527026)
* topic/christian/localversion:
Parse and store localversion string
Remove commented-out code
Check ZEEK_VERSION_LOCAL for dashes
Update version string btests for localversion
Modify version parsing for localversion
Update version used by spicyz
Update build script
Support for configurable localversion
(cherry picked from commit d09584e52e)
OSS-Fuzz managed to produce a MIME multipart message construction with
thousands of nested entities (or that's what Zeek makes out of it anyhow).
Prevent such deep analysis by capping at a nesting depth of 100,
preventing unnecessary resource usage. A new weird named exceeded_mime_max_depth
is reported when this limit is reached.
This change reduces the runtime of the OSS-Fuzz reproducer from ~45 seconds
to ~2.5 seconds.
The test PCAP was produced from a Python script using the email package
and sending the rendered version via POST to a HTTP server.
Closes#208
(cherry picked from commit 4e5849fe82c6097df5d25cd1a74d69ab4fa50f46)
The GNU and BSD versions of `touch` and `truncate` allow slightly
different arguments, change the tests so they work in both versions.
(cherry picked from commit caaffd0324)
* 'topic/xb-anssi/http_signature_body_end_match' of https://github.com/xb-anssi/zeek:
Let signature framework match HTTP body end
Test how the signature framework matches HTTP body
(cherry picked from commit 8a13155a41)
Previously, seq was computed as the result of |pending_commands|+1. This
opened the possibility to override queued commands, as well as logging
the same pending ftp reply multiple times.
For example, when commands 1, 2, 3 are pending, command 1 may be dequeued,
but the incoming command then receives seq 3 and overrides the already
pending command 3. The second scenario happens when ftp_reply() selected
command 3 as pending for logging, but is then followed by many ftp_request()
events. This resulted in command 3's response being logged for every
following ftp_request() over and over again.
Avoid both scenarios by tracking the command sequence as an absolute counter.
The previous fix also made it clear that the ssl_history field may grow
unbounded via the ssl_alert event. Prevent by capping using a configurable
limit (default 100) and raise a weird once reached.
* origin/topic/bbannier/issue-3279:
Fix handling of module scope when checking exported Spicy types for collisions
(cherry picked from commit 30dc6af831)
Setting this option to false does not count missing bytes in files towards the
extraction limits, and allows to extract data up to the desired limit,
even when partial files are written.
When missing bytes are encountered, files are now written as sparse
files.
Using this option requires the underlying storage and utilities to support
sparse files.
(cherry picked from commit afa6f3a0d3b8db1ec5b5e82d26225504c2891089)
OSS Fuzz generated a CWD request and reply followed by very many EPRT
requests. This caused Zeek to re-log the CWD request and invoke `build_url_ftp()`
over and over again resulting in long processing times.
Avoid this scenario by not logging commands that aren't pending anymore.
(cherry picked from commit b05dd31667ff634ec7d017f09d122f05878fdf65)
* origin/topic/jazoff/gh-3268:
Fix check for emailed notices
Changes: Added a test-case printing email_delay_tokens to compare email vs
non-email notice types. Previously, both notice types would have email
delay tokens at that point in the flow.
(cherry picked from commit 7e11501d3c)
* origin/topic/awelzel/3278-spicy-fix-port-range-off-by-one:
spicy: Do not register port N+1 for port N in .evt file
(cherry picked from commit 6e6a2bee8a)
Avoids loosing state on a connection value when a connection is flipped.
Fixes up the NTP baseline as well where this was visible: analyzer_confirmation_info()
was raised for a connection value which was immediately forgotten due to
the subsequent connection flipping.
Closed#3028
(cherry picked from commit a2214ad611)
* origin/topic/johanna/3205-do-not-parse-udp-padding:
Do not forward more than the remaining data to downstream UDP analyzer
(cherry picked from commit 3c7a52d0a7)
* origin/topic/awelzel/3145-dcerpc-state-clean:
dce-rpc: Test cases for unbounded state growth
dce-rpc: Handle smb2_close_request() in scripts
smb/dce-rpc: Cleanup DCE-RPC analyzers when fid is closed and limit them
dce-rpc: Do not repeatedly register removal hooks
(cherry picked from commit f9904511ab)
* origin/topic/awelzel/3112-log-suffix-left-over-log-rotation-fixup:
tests: Do not use archiver_rotation_format_func as postprocessor
(cherry picked from commit 9e2e6d0174)
* topic/awelzel/3112-log-suffix-left-over-log-rotation:
cluster/logger: Fix leftover-log-rotation in multi-logger setups
cluster/logger: Fix global var reference
(cherry picked from commit f53aefdd5b)
* origin/topic/awelzel/cluster-at-if-removal:
test-all-policy: Do not load nodes-experimental/manager.zeek
cluster/main: Remove extra @if ( Cluster::is_enabled() )
(cherry picked from commit 98e44ee14f)
After the introduction of @if ... analyze, a lot of warnings were
triggered due to nested @if and @if .. analyze usage.
Add a test for coverage of all policy scripts in cluster mode
for the usual node types so this does not happen again.
* origin/topic/vern/at-if-analyze:
updates reflecting review comments
change base scripts to use run-time if's or @if ... &analyze
a number of BTests updated with @if ... &analyze
update for scripting coverage BTest demonstrating utility of @if ... &analyze
BTests for new @if ... &analyze functionality
"if ( ... ) &analyze" language feature
classes for tracking "@if (...) &analyze" notion of code being/not being "activated"
RemoveGlobal() method for Scope class + simplifying interfaces
When a Spicy unit type was used in an EVT event definition, it could
end up being cleared out at the time the logic for `export` got to see it.
Closes#3083.
Note: This needs backporting to the legacy plugin.
