Due to the asynchronous behavior of the input framework and broker
communication, change handlers were previously called even after
zeek_done() event processing completed and also broker shutdown.
Accessing broker store handles within change handlers this late
triggered invalid Broker store handle messages:
error in ././my_option_store.zeek, line 13: invalid Broker store handle (Broker::put(Test::store, to_any_coercemy_option, to_any_coerceTest::new_value, 0 secs) and broker::store::{})
Fixes#2010
A flood of DHCP traffic can result if very large log entries consisting
of many uids and/or msg_types. Such large log entries can disrupt a SIEM
ingestion pipeline. This change forcing a log entry to be written when
the number of uids or the number of msg_Types exceed a certain value.
The values are treated as options for easy configuration.
Add the `pdu_type` field to Modbus over TCP logs to indicate whether the Modbus
message was a request or a response. Due to the client/server nature of Modbus
over TCP/IP, all messages from the TCP session originator are requests, while
all messages from the TCP session responder are responses.
Adding this information to the default log surfaces protocol metadata in a way
that doesn't require users to understand the Modbus over TCP protocol.
The previous way of splitting strings would break if the last string in
the line was an empty string, and it would return one fewer fields than
it should have. This was breaking the last line in the
scripts.base.framework.input.ascii.setspecialcases once the bug fixed in
GH #1628 was fixed.
* origin/topic/robin/gh-2280-modifiable-const-via-loops-merge:
Add help function to check loop variable
Fix local const variables can be modified via loops
Edits: Slight tweaking, plus a simple test.
* 'topic/amazingpp/modifiable-const-via-loops' of ssh://github.com/AmazingPP/zeek:
Add help function to check loop variable
Fix local const variables can be modified via loops
When a CREATE request contains the FILE_DELETE_ON_CLOSE option and
the subsequent CREATE response indicates success, we now raise the
smb2_file_delete event to log a delete action in smb_files.log and
also give users a way to handle this scenario.
The provided pcap was generated locally by recording a smbtorture run
of the smb2.delete-on-close-perms test case.
Placed the create_options into the CmdInfo record for potential
exposure in smb_cmd.log (wasn't sure how that would look so left it
for the future).
Fixes#2276.
* topic/christian/gh-2239-stdin-ctrl-c:
Stop signal-masking upon running unit tests
Pause signal-masking during script parsing
Add btests to verify Zeek's handling of SIGTERM and reading stdin
Add procps/procps-ng to several CI Docker images
While writing a test for the new "tail -F semantics" I found that
the $want_record=F case was broken (errno 25). So instead of opening
/dev/null when the input file is missing change READER_RAW to avoid
I/O until it can be opened.
Add two tests, one for when the event handler is called with a
record and one for when it's called with a string.
The sequence of
- adding a new file
- deleting an existing one
- waiting for Zeek to notice the addition
- re-adding the pre-existing file
was prone to a race: it could happen that Zeek's directory observation would see
the new file in one round, and by the time the next round happens the removal
and re-addition had already happened, thus missing the change and failing the
test.
This avoids the race by placing the removal of the existing file before the
addition of the new one, ensuring that when Zeek notices the addition (and
pushes the test to the next round), it has also seen the removal, so the
re-addition cannot get lost.
The core.load-stdin test checks Zeek's ability to read scripts from stdin.
core.sigterm-regular and core.sigterm-stdin verify that SIGTERM shuts down a
Zeek process during normal operation and while reading script content from
stdin, respectively. For technical reasons we don't test with SIGINT, as ctrl-c
would trigger -- see comments for details.
One change: turns out that zeek-config isn't in PATH, so I prefixed it
with $BUILD.
* origin/topic/awelzel/913-plugin-debug-stream-validation:
zeek-setup: Validate plugin debug streams during startup
Providing an unknown plugin debug stream with -B was previously silently
accepted. This caused user confusing as the behavior is "no output, but
seems to work".
Check the enabled debug streams once all plugins have been loaded and
exit early on for invalid streams.
$ ZEEK_PLUGIN_PATH=./build zeek -B plugin-zeek-myplugin -e 'print zeek_version();'
error in <command line>, line 3: No plugin debug stream 'plugin-zeek-myplugin' found
Closes#913.
Observed .sqlite-journal files and missing reporter.sqlite files
in CI runs. Subsequently reading the ./test.sqlite file is more
reliable and should be good enough.
Also modify FormatRotationPath to keep rotated logs within
Log::default_logdir unless the rotation function explicitly
set dir, e.g. by when the user redef'ed default_rotation_interval.
With the introduction of LogAscii::logdir, log filenames can now include
parent directories rather than being plain basenames. Enabling log rotation,
leftover log rotation and setting LogAscii::logdir broke due to not
handling this situation.
This change ensures that .shadow files are placed within the directory where
the respective .log file is created. Previously, the .shadow. (or .tmp.shadow.)
prefix was simply prepended, yielding non-sensical paths such as
.tmp.shadow.foo/bar/packet_filter.log for a logdir of foo/bar.
Additionally, respect LogAscii::logdir when searching for leftover log files
rather than defaulting to the current working directory.
The following quirk exist around LogAscii::logdir, but will be addressed
in a follow-up.
* By default, logs are currently rotated into the working directory of the
process, rather than staying confined within LogAscii::logdir. One of
the added tests shows this behavior.
