Commit graph

5073 commits

Author SHA1 Message Date
Tim Wojtulewicz
d57ad3e405 Fix a bug with splitting strings in Ascii input reader
The previous way of splitting strings would break if the last string in
the line was an empty string, and it would return one fewer fields than
it should have. This was breaking the last line in the
scripts.base.framework.input.ascii.setspecialcases once the bug fixed in
GH #1628 was fixed.
2022-07-21 11:53:47 -07:00
Tim Wojtulewicz
5665696a05 Merge remote-tracking branch 'amazingpp/topic/amazingpp/bitshift-operators'
* amazingpp/topic/amazingpp/bitshift-operators:
  Add bitshift operators
2022-07-21 09:06:41 -07:00
AmazingPP
f2bfa61fc6 Add bitshift operators 2022-07-21 11:12:30 +08:00
Robin Sommer
ad2557755d Merge remote-tracking branch 'origin/topic/robin/gh-2267-doctest'
* origin/topic/robin/gh-2267-doctest:
  Add btest that runs the doctest-based unit tests.
2022-07-19 15:21:39 +02:00
Robin Sommer
d5db2623dc Merge remote-tracking branch 'origin/topic/robin/gh-2280-modifiable-const-via-loops-merge'
* origin/topic/robin/gh-2280-modifiable-const-via-loops-merge:
  Add help function to check loop variable
  Fix local const variables can be modified via loops
2022-07-19 15:20:34 +02:00
Robin Sommer
f40ca42590 Merge branch 'topic/amazingpp/modifiable-const-via-loops' of ssh://github.com/AmazingPP/zeek
Edits: Slight tweaking, plus a simple test.

* 'topic/amazingpp/modifiable-const-via-loops' of ssh://github.com/AmazingPP/zeek:
  Add help function to check loop variable
  Fix local const variables can be modified via loops
2022-07-19 14:26:59 +02:00
Robin Sommer
265fa83a24 Merge remote-tracking branch 'origin/topic/awelzel/2276-smb2-delete-on-close'
* origin/topic/awelzel/2276-smb2-delete-on-close:
  smb2: Raise smb2_file_delete for CREATE with FILE_DELETE_ON_CLOSE
2022-07-19 12:02:16 +02:00
Robin Sommer
202037181c Add btest that runs the doctest-based unit tests.
Closes #2267.
2022-07-19 11:35:09 +02:00
Tim Wojtulewicz
05bb509789 GH-636: Fix regex to handle commas at the end of config parser lines 2022-07-18 08:54:06 -07:00
Michael Torres
bab2036aa4 Add modbus transaction and unit ids to logs
Add transaction IDs and unit IDs to default modbus over TCP/IP logs.
Update the relevant testing baselines to account for the extra fields.
2022-07-17 21:02:37 +00:00
Arne Welzel
3dae8ab086 smb2: Raise smb2_file_delete for CREATE with FILE_DELETE_ON_CLOSE
When a CREATE request contains the FILE_DELETE_ON_CLOSE option and
the subsequent CREATE response indicates success, we now raise the
smb2_file_delete event to log a delete action in smb_files.log and
also give users a way to handle this scenario.

The provided pcap was generated locally by recording a smbtorture run
of the smb2.delete-on-close-perms test case.

Placed the create_options into the CmdInfo record for potential
exposure in smb_cmd.log (wasn't sure how that would look so left it
for the future).

Fixes #2276.
2022-07-16 17:14:13 +02:00
AmazingPP
a76dc6a389 Add pow function 2022-07-14 22:13:34 +08:00
Christian Kreibich
8d10cbfb36 Fix requirement check in a logging framework / sqlite btest 2022-07-13 17:20:03 -07:00
Christian Kreibich
48486b4156 Merge branch 'topic/christian/gh-2239-stdin-ctrl-c'
* topic/christian/gh-2239-stdin-ctrl-c:
  Stop signal-masking upon running unit tests
  Pause signal-masking during script parsing
  Add btests to verify Zeek's handling of SIGTERM and reading stdin
  Add procps/procps-ng to several CI Docker images
2022-07-13 11:57:52 -07:00
Tim Wojtulewicz
ed2934d345 Merge remote-tracking branch 'origin/topic/bbannier/format-btest_testing_plugins'
* origin/topic/bbannier/format-btest_testing_plugins:
  Use clang-format for all files in `testing/btest/plugins`.
2022-07-13 11:23:45 -07:00
Tim Wojtulewicz
7e56605d83 GH-696: Add bytestring_to_float BIF 2022-07-13 10:44:24 -07:00
Benjamin Bannier
489534bd74 Use clang-format for all files in testing/btest/plugins.
This is a fixup commit for dc65b6248c.
2022-07-13 17:58:53 +02:00
Johanna Amann
96a14b39fa Merge remote-tracking branch 'origin/topic/neverlord/gh-2192'
* origin/topic/neverlord/gh-2192:
  Avoid race conditions in broker.ssl-mismatch test
2022-07-13 09:34:43 +01:00
Craig Leres
44ba9138c2 Fix tail -F semantics when want_record=F and add tests for the new features
While writing a test for the new "tail -F semantics" I found that
the $want_record=F case was broken (errno 25). So instead of opening
/dev/null when the input file is missing change READER_RAW to avoid
I/O until it can be opened.

Add two tests, one for when the event handler is called with a
record and one for when it's called with a string.
2022-07-12 17:57:50 -07:00
Tim Wojtulewicz
1496b99a34 Deprecate HOOK_BRO_OBJ_DTOR and related methods 2022-07-12 12:01:23 -07:00
Tim Wojtulewicz
f624c18383 Deprecate bro_int_t and bro_uint_t 2022-07-12 12:01:23 -07:00
Tim Wojtulewicz
dc65b6248c Run clang-format on all of our plugin test c++ files 2022-07-12 11:59:55 -07:00
Christian Kreibich
b59bfe5558 Merge branch 'topic/christian/gh-2259-fix-dir-test'
* topic/christian/gh-2259-fix-dir-test:
  Fix race condition in scripts.base.utils.dir test
  Remove needless "touch" in scripts.base.utils.dir test
2022-07-11 17:29:42 -07:00
Christian Kreibich
3d87400f1d Fix race condition in scripts.base.utils.dir test
The sequence of

- adding a new file
- deleting an existing one
- waiting for Zeek to notice the addition
- re-adding the pre-existing file

was prone to a race: it could happen that Zeek's directory observation would see
the new file in one round, and by the time the next round happens the removal
and re-addition had already happened, thus missing the change and failing the
test.

This avoids the race by placing the removal of the existing file before the
addition of the new one, ensuring that when Zeek notices the addition (and
pushes the test to the next round), it has also seen the removal, so the
re-addition cannot get lost.
2022-07-11 16:26:45 -07:00
Christian Kreibich
e2a7b890ed Remove needless "touch" in scripts.base.utils.dir test
This touch happened after teardown of Zeek, so didn't add to the test.
2022-07-11 16:05:37 -07:00
Tim Wojtulewicz
1226aa538a Merge remote-tracking branch 'origin/topic/vern/zam-vector-cond-fix'
* origin/topic/vern/zam-vector-cond-fix:
  Fix for ZAM optimization of '?' conditional with vector-valued result
2022-07-11 09:44:05 -07:00
Dominik Charousset
d3ad3c618a Avoid race conditions in broker.ssl-mismatch test 2022-07-10 15:47:43 +02:00
Christian Kreibich
5beb68194d Add btests to verify Zeek's handling of SIGTERM and reading stdin
The core.load-stdin test checks Zeek's ability to read scripts from stdin.
core.sigterm-regular and core.sigterm-stdin verify that SIGTERM shuts down a
Zeek process during normal operation and while reading script content from
stdin, respectively. For technical reasons we don't test with SIGINT, as ctrl-c
would trigger -- see comments for details.
2022-07-09 11:17:14 -07:00
AmazingPP
ba552ceeaf Add table_keys function 2022-07-09 01:11:31 +08:00
AmazingPP
cb71b15eab Add table_values function 2022-07-09 00:19:06 +08:00
Robin Sommer
e8e6fa8fed Merge remote-tracking branch 'origin/topic/awelzel/913-plugin-debug-stream-validation'
One change: turns out that zeek-config isn't in PATH, so I prefixed it
with $BUILD.

* origin/topic/awelzel/913-plugin-debug-stream-validation:
  zeek-setup: Validate plugin debug streams during startup
2022-07-08 10:00:39 +02:00
Vern Paxson
0706e8869d Fix for ZAM optimization of '?' conditional with vector-valued result 2022-07-07 13:08:28 -07:00
Arne Welzel
d703033ae6 zeek-setup: Validate plugin debug streams during startup
Providing an unknown plugin debug stream with -B was previously silently
accepted. This caused user confusing as the behavior is "no output, but
seems to work".

Check the enabled debug streams once all plugins have been loaded and
exit early on for invalid streams.

    $ ZEEK_PLUGIN_PATH=./build zeek -B plugin-zeek-myplugin -e 'print zeek_version();'
    error in <command line>, line 3: No plugin debug stream 'plugin-zeek-myplugin' found

Closes #913.
2022-07-07 17:56:19 +02:00
Arne Welzel
a2bcb1bf28 sqlite default-logdir test: Remove ls ./logs baseline
Observed .sqlite-journal files and missing reporter.sqlite files
in CI runs. Subsequently reading the ./test.sqlite file is more
reliable and should be good enough.
2022-07-06 22:57:14 +02:00
Arne Welzel
93584c7c7f logging/sqlite: Recognize Log::default_logdir and place files there if set 2022-07-06 18:54:29 +02:00
Arne Welzel
aaa47a709c logging: Introduce Log::default_logdir deprecate LogAscii::logdir and per writer logdir
Also modify FormatRotationPath to keep rotated logs within
Log::default_logdir unless the rotation function explicitly
set dir, e.g. by when the user redef'ed default_rotation_interval.
2022-07-06 18:54:29 +02:00
Arne Welzel
513ea7e04f logging/ascii: Fix .shadow paths when using LogAscii::logdir
With the introduction of LogAscii::logdir, log filenames can now include
parent directories rather than being plain basenames. Enabling log rotation,
leftover log rotation and setting LogAscii::logdir broke due to not
handling this situation.

This change ensures that .shadow files are placed within the directory where
the respective .log file is created. Previously, the .shadow. (or .tmp.shadow.)
prefix was simply prepended, yielding non-sensical paths such as
.tmp.shadow.foo/bar/packet_filter.log for a logdir of foo/bar.

Additionally, respect LogAscii::logdir when searching for leftover log files
rather than defaulting to the current working directory.

The following quirk exist around LogAscii::logdir, but will be addressed
in a follow-up.

* By default, logs are currently rotated into the working directory of the
  process, rather than staying confined within LogAscii::logdir. One of
  the added tests shows this behavior.
2022-07-06 13:21:21 +02:00
Johanna Amann
6e1e6fefe5 Merge remote-tracking branch 'origin/topic/johanna/2198'
* origin/topic/johanna/2198:
  SSL/GH-2211: Address review feedback, remove USE_FLIPPED
  SSL Analyzer: track connection direction by messages
2022-07-05 15:09:34 +01:00
Tim Wojtulewicz
509718b51c GH-2229: Fix some typos in weak-keys.zeek 2022-06-30 15:12:10 -07:00
Tim Wojtulewicz
fb16ce3711 Remove other general deprecations 2022-06-30 19:17:13 +00:00
Tim Wojtulewicz
6130d32440 Remove some deprecated ocsp/ssl base scripts 2022-06-30 19:17:08 +00:00
Tim Wojtulewicz
644ed5d314 Remove deprecated protocol_confirmation/violation events and methods 2022-06-30 19:11:57 +00:00
Tim Wojtulewicz
70e63d4749 Remove deprecated MemoryAllocation() methods and related code 2022-06-30 18:56:52 +00:00
Tim Wojtulewicz
9e953f50cb Merge remote-tracking branch 'origin/topic/vern/remove-deprecated-closures'
* origin/topic/vern/remove-deprecated-closures:
  removed deprecated capture-by-reference closures
2022-06-27 13:04:57 -07:00
Tim Wojtulewicz
7c4fd382d9 Code modernization: Convert from deprecated C standard library headers 2022-06-27 09:47:31 -07:00
Johanna Amann
e14eddeb97 SSL Analyzer: track connection direction by messages
This PR changes the way in which the SSL analyzer tracks the direction
of connections. So far, the SSL analyzer assumed that the originator of
a connection would send the client hello (and other associated
client-side events), and that the responder would be the SSL servers.

In some circumstances this is not true, and the initiator of a
connection is the server, with the responder being the client. So far
this confused some of the internal statekeeping logic and could lead to
mis-parsing of extensions.

This reversal of roles can happen in DTLS, if a connection uses STUN -
and potentially in some StartTLS protocols.

This PR tracks the direction of a TLS connection using the hello
request, client hello and server hello handshake messages. Furthermore,
it changes the SSL events from providing is_orig to providing is_client,
where is_client is true for the client_side of a connection. Since the
argument positioning in the event has not changed, old scripts will
continue to work seamlessly - the new semantics are what everyone
writing SSL scripts will have expected in any case.

There is a new event that is raised when a connection is flipped. A
weird is raised if a flip happens repeatedly.

Addresses GH-2198.
2022-06-24 18:35:44 +01:00
Vern Paxson
5a0a88526e removed deprecated capture-by-reference closures 2022-06-17 15:55:43 -07:00
Vern Paxson
847963e94d flip connections that begin with SYN-ACKs if subsequent acks or data seen 2022-06-15 14:01:32 -07:00
Vern Paxson
1724ddfebf ZAM test suite updates due to recent changes 2022-06-03 14:58:22 -07:00
Tim Wojtulewicz
48a858f5db Update local-compat test for 5.1 2022-06-03 12:38:20 -07:00