Commit graph

16250 commits

Author SHA1 Message Date
Arne Welzel
6867eda621 Merge remote-tracking branch 'origin/topic/awelzel/fix-assert-cond-twice'
* origin/topic/awelzel/fix-assert-cond-twice:
  Stmt: Fix assert evaluating cond twice
2024-03-04 18:22:43 +01:00
Robin Sommer
ce344d3f84
Merge remote-tracking branch 'origin/topic/robin/node-rewrite'
* origin/topic/robin/node-rewrite:
  Port Spicy integration to new AST API.
2024-03-04 15:49:29 +01:00
Robin Sommer
db98dc4193
Port Spicy integration to new AST API. 2024-03-04 15:48:39 +01:00
Arne Welzel
6572324b8c Stmt: Fix assert evaluating cond twice
Since 81a9745fb3, the assert condition is
evaluated twice. This leads to unexpected behavior when cond has a side
effect like publishing a message or creating a log stream or filter.

Found while using the following in ad-hoc testing code and wondering
why two messages were published.

    assert publish(Cluster::worker_topic, hello, "abc")
2024-03-04 15:30:09 +01:00
Arne Welzel
724daa2792 Merge remote-tracking branch 'origin/topic/awelzel/zeekjs-0.12.0'
* origin/topic/awelzel/zeekjs-0.12.0:
  Bump zeekjs to 0.12.0
2024-02-28 17:10:44 +01:00
Arne Welzel
e3fab940b3 Bump zeekjs to 0.12.0
0a89ca6 doc: Expand zeek.as() description and add an example
  7e814d7 Types: Implement basic pattern support
  43df9d2 Update docs to provide example of shared Node.js openSSL configuration
  3bf2ea5 lsan suppressions: Add some for 21.11
  640affa zeek.global_vars: Remove leftover internal field usage
  3ee53c7 zeek.global_vars: Implement setter
  8144061 zeek.as: Support more Zeek types
  b453483 zeek.as: Fix crash for non-atomic types
2024-02-28 11:10:56 +01:00
zeek-bot
45152e4879 Update doc submodule [nomail] [skip ci] 2024-02-28 00:17:13 +00:00
Arne Welzel
e33bf64dc9 Merge remote-tracking branch 'origin/topic/awelzel/3619-zeekygen-spicyc-mtime'
* origin/topic/awelzel/3619-zeekygen-spicyc-mtime:
  spicy/zeekygen: Remove mtime from generated code
2024-02-27 15:18:58 +01:00
Arne Welzel
ecdd2b0b29 spicy/zeekygen: Remove mtime from generated code
Zeekygen implements its own make-style update logic to prevent
re-creation of files that have not changed. To fulfill this, we
currently encode the current time into spicyz generated .cc files.

This degrades ccache efficiency for built-in analyzers and also
for all .evt files compiled during testing. Switch SpicyModuleInfo
to return current time instead. This results in the re-generation
of documentation files unconditionally when running Zeekygen, but
that seems more acceptable IMO.

Generally wonder if Zeekygen should produce output unconditionally
and if we need to clobber prevention, compare with the content of
the existing file.

Closes #3619
2024-02-27 15:06:02 +01:00
zeek-bot
961bac0220 Update doc submodule [nomail] [skip ci] 2024-02-27 00:12:07 +00:00
Arne Welzel
d728981b77 Merge remote-tracking branch 'origin/topic/awelzel/move-iso-9660-sig-to-policy'
* origin/topic/awelzel/move-iso-9660-sig-to-policy:
  signatures/iso-9660: Add \x01 suffix to CD001
  test-all-policy: Do not load iso-9660.zeek
  signatures: Move ISO 9660 signature to policy
2024-02-26 21:16:48 +01:00
Arne Welzel
e87272f5a7 Merge remote-tracking branch 'origin/topic/awelzel/3490-global-ids-type-name-change'
* origin/topic/awelzel/3490-global-ids-type-name-change:
  global_ids: Align script_id$type_name field with type_name()
2024-02-26 21:15:25 +01:00
Arne Welzel
77d57fccfd Merge remote-tracking branch 'origin/topic/awelzel/find-in-zeekpath'
* origin/topic/awelzel/find-in-zeekpath:
  NEWS: Mention can_load() and find_in_zeekpath()
  utils: Introduce packages.zeek with can_load() helper
  zeek.bif: Add find_in_zeekpath() helper
2024-02-26 21:14:59 +01:00
Arne Welzel
f4ed1e05fc signatures/iso-9660: Add \x01 suffix to CD001
As discussed with Tim, that should make it a bit more robust against
false positives.
2024-02-26 21:00:01 +01:00
Tim Wojtulewicz
c315202ece Merge remote-tracking branch 'origin/topic/johanna/update-tls-ca-stuff'
* origin/topic/johanna/update-tls-ca-stuff:
  Update Mozilla CA and Google CT list and related tests.
2024-02-26 11:10:32 -07:00
Arne Welzel
e11c20e1eb test-all-policy: Do not load iso-9660.zeek
Changing the default_file_bof_buffer_size has subtle impact on
MIME type detection and changed the zeek-testing baseline. Do
not load this new script via test-all-policy to avoid this.

The new test was mainly an aid to understand what is actually going on.
In short, if default_file_bof_buffer_size is larger than the file MIME
detection only runs when the buffer is full, or when the file is removed.
When a file transfer happens over multiple HTTP connections, only
some or one of the http.log entries will have a proper response MIME type.

PCAP extracted from 2009-M57-day11-18.trace.gz.
2024-02-26 17:58:26 +01:00
Arne Welzel
1a5ce65e3d signatures: Move ISO 9660 signature to policy
The previous "fix" caused significant performance degradation without
the signature ever having a chance to trigger. Moving it to policy
seems the best compromise, the alternative being outright removing it.
2024-02-26 13:35:23 +01:00
Arne Welzel
1c2d1b3717 global_ids: Align script_id$type_name field with type_name()
Populate script_id$type_name with what the type_name() bif
would produce for the same identifier.

Closes #3490
2024-02-23 17:26:56 +01:00
Johanna Amann
c81274b549 Update Mozilla CA and Google CT list and related tests.
Test updates are necessary due to removed CT logs.

Fixes GH-3620
2024-02-23 11:50:31 +00:00
zeek-bot
f96600391a Update doc submodule [nomail] [skip ci] 2024-02-23 00:12:41 +00:00
Arne Welzel
0dfee33c83 Merge remote-tracking branch 'origin/topic/awelzel/3139-zeek-version-spicy-ccache-busting'
* origin/topic/awelzel/3139-zeek-version-spicy-ccache-busting:
  spicy: Remove Zeek version from spicyz/config.h.in
2024-02-22 17:00:46 +01:00
Arne Welzel
02703eeb9d spicy: Remove Zeek version from spicyz/config.h.in
It seems that Zeek's version number and string only need to be
available at runtime, so this change removes it from spicyz/configh.in
to avoid needlessly busting ccache for the src/spicyz tree for on a
Zeek version bump.

Closes #3139.
2024-02-22 16:59:46 +01:00
Arne Welzel
5742f7cc21 spicy: Drop ZEEK_VERSION_NUMBER ifdef'ery
Given src/spicy is part of the Zeek tree, these are not necessary and
are probably just remains from the spicy-plugin era.
2024-02-22 13:00:15 +01:00
Arne Welzel
08bf8d47f4 Merge remote-tracking branch 'origin/topic/awelzel/iso-9660-image-signature'
* origin/topic/awelzel/iso-9660-image-signature:
  signatures: Fix ISO 9960 signature
2024-02-22 12:38:55 +01:00
Arne Welzel
d2409dd432 signatures: Fix ISO 9960 signature
This signature only really works when default_file_bof_buffer_size is bumped
to a sufficient value (40k).
2024-02-22 12:37:40 +01:00
Arne Welzel
036bcfe919 Merge remote-tracking branch 'origin/topic/awelzel/ftp-clear-fuid-after-logging'
* origin/topic/awelzel/ftp-clear-fuid-after-logging:
  testing: Update zeek-testing-private commit hash
  ftp: Reset fuid after logging
2024-02-22 12:23:21 +01:00
Arne Welzel
e9e930bdfa Merge remote-tracking branch 'origin/topic/awelzel/delete-stmt-table-set'
* origin/topic/awelzel/delete-stmt-table-set:
  Allow delete statement for tables, sets and vectors
2024-02-22 12:02:37 +01:00
Arne Welzel
83bbcc1269 NEWS: Mention can_load() and find_in_zeekpath() 2024-02-22 11:41:54 +01:00
Arne Welzel
947294efab utils: Introduce packages.zeek with can_load() helper 2024-02-22 11:41:50 +01:00
Arne Welzel
704f75a214 zeek.bif: Add find_in_zeekpath() helper
Relates to #3594. This helper can be used to determine the path that
will be used by @load, if at all.
2024-02-22 11:33:03 +01:00
Tim Wojtulewicz
4a9b580dab Update cmake submodule [nomail] 2024-02-21 12:20:28 -07:00
Tim Wojtulewicz
cddd5eb3e0 Update cmake submodule [nomail] 2024-02-21 11:12:31 -07:00
Tim Wojtulewicz
e3e29b3bd4 Point cmake submodule at master branch 2024-02-21 11:08:45 -07:00
Arne Welzel
2f1893bc58 Allow delete statement for tables, sets and vectors
Relates to #3472. This allow "delete tbl" as an alternative for
clear_table(tbl). Also works for vectors.
2024-02-21 18:00:00 +01:00
Arne Welzel
bf8f428c8b testing: Update zeek-testing-private commit hash 2024-02-21 12:57:49 +01:00
Arne Welzel
31b548babc ftp: Reset fuid after logging
A user reported being confused about the fuid association of subsequent
FTP commands when a data transfer has completed. It seems reasonable to
unset fuid upon logging a FTP command which had a fuid.

The current behavior results in the PORT or PASV commands after a RETR or STOR
to have the fuid of the prior file transfer. Similarly, any CWD or DEL commands
following a file transfer will unnecessarily be logged with the fuid of the
prior file transfer.

This tickles the baselines for the private testing PCAP a lot, primarily
because there data connections in that pcap are never established properly.
E.g, the fuids FzDzid1Dxm9srVKHXf and FEfYX73q5C6GEQZXX9 have been re-used
for multiple commands.

This may look like we're losing information, but the fuids vanishing
in the normal btests belong to a LIST command that isn't logged by
default into ftp.log. If it was, the fuid would be attached to it.
2024-02-21 12:41:32 +01:00
Tim Wojtulewicz
6d86a48a6a Start of 7.0.0 development 2024-02-20 11:51:06 -07:00
Tim Wojtulewicz
5102cb8392 Update zkg submodule to v3.0.1 2024-02-20 10:58:38 -07:00
Tim Wojtulewicz
04d965a13a Merge branch 'topic/neverlord/broker-variant'
* topic/neverlord/broker-variant:
  Pull in variant-enabled Broker
2024-02-20 10:57:42 -07:00
Dominik Charousset
0a84aad417 Pull in variant-enabled Broker
Switch the Broker submodule to a version with `broker::variant` enabled
and provide new overloads for `broker::variant` in the Broker manager.
2024-02-20 09:47:32 -07:00
Tim Wojtulewicz
6de51f0d7a Merge remote-tracking branch 'origin/topic/timw/update-zkg'
* origin/topic/timw/update-zkg:
  Bump zkg to v3.0.0
2024-02-15 11:08:30 -05:00
Tim Wojtulewicz
8bef179ab1 Merge remote-tracking branch 'origin/topic/bbannier/bump-spicy'
* origin/topic/bbannier/bump-spicy:
  Bump Spicy to latest dev snapshot.
2024-02-15 11:07:25 -05:00
Tim Wojtulewicz
3a4a20c815 Bump zkg to v3.0.0 2024-02-15 10:21:23 -05:00
Benjamin Bannier
7c459b94c6 Bump Spicy to latest dev snapshot. 2024-02-15 15:20:46 +01:00
Arne Welzel
72eb565c5e Merge remote-tracking branch 'origin/topic/awelzel/cheaper-hex-printing'
* origin/topic/awelzel/cheaper-hex-printing:
  zeek.bif: Booleanate String constructor calls
  Conn/net_utils/fmt_mac: Avoid snprintf(), memcpy() and allocation
  bytestring_to_hexstr: Use bytetohex() instead of snprintf()
  digest/digest_print: Use bytetohex() instead of snprintf()
2024-02-13 20:10:15 +01:00
Arne Welzel
485950fcff zeek.bif: Booleanate String constructor calls 2024-02-13 20:08:31 +01:00
Arne Welzel
29f5b507b6 Conn/net_utils/fmt_mac: Avoid snprintf(), memcpy() and allocation
The fmt_mac() function returning a std::string means the resulting mac
is copied at least once upon returning. Then, the Assign() in GetVal()
taking a std::string internally allocates a new zeek::String which
hits a malloc (no short-string optimization for zeek::String) and then
also copies the content from the std::string into the malloced memory.

Save a few cycles by directly using the allocated memory with the
String instance. This change improves runtime for a SYN-only pcap
with just base/protocols/conn loaded by some 1-2%.
2024-02-13 19:37:15 +01:00
Arne Welzel
c41977057a bytestring_to_hexstr: Use bytetohex() instead of snprintf()
...also avoid the extra memcpy and memory allocation.
2024-02-13 19:37:12 +01:00
Arne Welzel
4fc08d8fc2 digest/digest_print: Use bytetohex() instead of snprintf() 2024-02-13 19:32:16 +01:00
zeek-bot
32a0260291 Update doc submodule [nomail] [skip ci] 2024-02-09 00:23:46 +00:00