Commit graph

16422 commits

Author SHA1 Message Date
Arne Welzel
947294efab utils: Introduce packages.zeek with can_load() helper 2024-02-22 11:41:50 +01:00
Arne Welzel
704f75a214 zeek.bif: Add find_in_zeekpath() helper
Relates to #3594. This helper can be used to determine the path that
will be used by @load, if at all.
2024-02-22 11:33:03 +01:00
Tim Wojtulewicz
4a9b580dab Update cmake submodule [nomail] 2024-02-21 12:20:28 -07:00
Tim Wojtulewicz
cddd5eb3e0 Update cmake submodule [nomail] 2024-02-21 11:12:31 -07:00
Tim Wojtulewicz
e3e29b3bd4 Point cmake submodule at master branch 2024-02-21 11:08:45 -07:00
Arne Welzel
2f1893bc58 Allow delete statement for tables, sets and vectors
Relates to #3472. This allow "delete tbl" as an alternative for
clear_table(tbl). Also works for vectors.
2024-02-21 18:00:00 +01:00
Arne Welzel
bf8f428c8b testing: Update zeek-testing-private commit hash 2024-02-21 12:57:49 +01:00
Arne Welzel
31b548babc ftp: Reset fuid after logging
A user reported being confused about the fuid association of subsequent
FTP commands when a data transfer has completed. It seems reasonable to
unset fuid upon logging a FTP command which had a fuid.

The current behavior results in the PORT or PASV commands after a RETR or STOR
to have the fuid of the prior file transfer. Similarly, any CWD or DEL commands
following a file transfer will unnecessarily be logged with the fuid of the
prior file transfer.

This tickles the baselines for the private testing PCAP a lot, primarily
because there data connections in that pcap are never established properly.
E.g, the fuids FzDzid1Dxm9srVKHXf and FEfYX73q5C6GEQZXX9 have been re-used
for multiple commands.

This may look like we're losing information, but the fuids vanishing
in the normal btests belong to a LIST command that isn't logged by
default into ftp.log. If it was, the fuid would be attached to it.
2024-02-21 12:41:32 +01:00
Tim Wojtulewicz
6d86a48a6a Start of 7.0.0 development 2024-02-20 11:51:06 -07:00
Tim Wojtulewicz
5102cb8392 Update zkg submodule to v3.0.1 2024-02-20 10:58:38 -07:00
Tim Wojtulewicz
04d965a13a Merge branch 'topic/neverlord/broker-variant'
* topic/neverlord/broker-variant:
  Pull in variant-enabled Broker
2024-02-20 10:57:42 -07:00
Dominik Charousset
0a84aad417 Pull in variant-enabled Broker
Switch the Broker submodule to a version with `broker::variant` enabled
and provide new overloads for `broker::variant` in the Broker manager.
2024-02-20 09:47:32 -07:00
Tim Wojtulewicz
6de51f0d7a Merge remote-tracking branch 'origin/topic/timw/update-zkg'
* origin/topic/timw/update-zkg:
  Bump zkg to v3.0.0
2024-02-15 11:08:30 -05:00
Tim Wojtulewicz
8bef179ab1 Merge remote-tracking branch 'origin/topic/bbannier/bump-spicy'
* origin/topic/bbannier/bump-spicy:
  Bump Spicy to latest dev snapshot.
2024-02-15 11:07:25 -05:00
Tim Wojtulewicz
3a4a20c815 Bump zkg to v3.0.0 2024-02-15 10:21:23 -05:00
Benjamin Bannier
7c459b94c6 Bump Spicy to latest dev snapshot. 2024-02-15 15:20:46 +01:00
Arne Welzel
72eb565c5e Merge remote-tracking branch 'origin/topic/awelzel/cheaper-hex-printing'
* origin/topic/awelzel/cheaper-hex-printing:
  zeek.bif: Booleanate String constructor calls
  Conn/net_utils/fmt_mac: Avoid snprintf(), memcpy() and allocation
  bytestring_to_hexstr: Use bytetohex() instead of snprintf()
  digest/digest_print: Use bytetohex() instead of snprintf()
2024-02-13 20:10:15 +01:00
Arne Welzel
485950fcff zeek.bif: Booleanate String constructor calls 2024-02-13 20:08:31 +01:00
Arne Welzel
29f5b507b6 Conn/net_utils/fmt_mac: Avoid snprintf(), memcpy() and allocation
The fmt_mac() function returning a std::string means the resulting mac
is copied at least once upon returning. Then, the Assign() in GetVal()
taking a std::string internally allocates a new zeek::String which
hits a malloc (no short-string optimization for zeek::String) and then
also copies the content from the std::string into the malloced memory.

Save a few cycles by directly using the allocated memory with the
String instance. This change improves runtime for a SYN-only pcap
with just base/protocols/conn loaded by some 1-2%.
2024-02-13 19:37:15 +01:00
Arne Welzel
c41977057a bytestring_to_hexstr: Use bytetohex() instead of snprintf()
...also avoid the extra memcpy and memory allocation.
2024-02-13 19:37:12 +01:00
Arne Welzel
4fc08d8fc2 digest/digest_print: Use bytetohex() instead of snprintf() 2024-02-13 19:32:16 +01:00
zeek-bot
32a0260291 Update doc submodule [nomail] [skip ci] 2024-02-09 00:23:46 +00:00
Arne Welzel
382121fd8c Merge remote-tracking branch 'origin/topic/awelzel/stmt-extern'
* origin/topic/awelzel/stmt-extern:
  Bump zeekjs for STMT_ANY deprecation
  Stmt: Deprecate STMT_ANY
  Stmt: Introduce STMT_EXTERN
2024-02-08 15:12:33 +01:00
zeek-bot
8f56140f73 Update doc submodule [nomail] [skip ci] 2024-02-08 00:18:14 +00:00
Tim Wojtulewicz
5db7b3d567 Merge remote-tracking branch 'origin/topic/timw/update-vcpkg'
* origin/topic/timw/update-vcpkg:
  Update .gitignore to add Emacs and Vim temp files
  Update vcpkg submodule to pull in c-ares 1.26.0 to match other platforms
2024-02-07 12:50:07 -07:00
Tim Wojtulewicz
5fa606ec01 Update .gitignore to add Emacs and Vim temp files 2024-02-07 12:12:58 -07:00
Tim Wojtulewicz
2426cef41c Update vcpkg submodule to pull in c-ares 1.26.0 to match other platforms 2024-02-07 12:12:58 -07:00
Christian Kreibich
1f2821dfe3 Merge branch 'topic/christian/news-additions'
* topic/christian/news-additions:
  Some NEWS updates for 6.2 [skip ci]
2024-02-07 09:36:36 -08:00
Arne Welzel
fa97e08f60 Merge remote-tracking branch 'origin/topic/awelzel/websocket-spicy-analyzer'
* origin/topic/awelzel/websocket-spicy-analyzer:
  websocket: Implement Spicy's unmask in C++
  websocket: Add Spicy parser version, too.
2024-02-07 13:15:59 +01:00
Arne Welzel
bd26d41d6b websocket: Implement Spicy's unmask in C++
This improves runtime for the test pcap I have from 22 seconds to
12 seconds, about 1.8x times faster.
2024-02-07 12:47:24 +01:00
Christian Kreibich
2e66faba76 Some NEWS updates for 6.2 [skip ci] 2024-02-06 17:38:07 -08:00
zeek-bot
26a5ebe246 Update doc submodule [nomail] [skip ci] 2024-02-07 00:26:16 +00:00
Arne Welzel
c1a685a05d websocket: Add Spicy parser version, too.
The Spicy analyzer is added as a child analyzer when enabled and the
WebSocket.cc logic dispatches between the BinPac and Spicy version.

It substantially slower when tested against a somewhat artificial
2.4GB PCAP. The first flamegraph indicates that the unmask() function
stands out with 35% of all samples, and above it shared_ptr samples.
2024-02-06 17:29:55 +01:00
Arne Welzel
8ad954cfd4 Bump zeekjs for STMT_ANY deprecation 2024-02-06 15:09:08 +01:00
Arne Welzel
4019e9917a Stmt: Deprecate STMT_ANY
This isn't used in-tree and has been misused by some external plugins
of mine (zeekjs and zeek-perf-support) for their own Stmt subclasses.
These plugins should be updated to use the new STMT_EXTERN statement.

Handle STMT_ANY explicitly in stmt_name() for the time being
to fix #3529 until we remove STMT_ANY for good.
2024-02-06 14:40:23 +01:00
Arne Welzel
a5c941139c Stmt: Introduce STMT_EXTERN
It's currently possible for plugin's to implement their own statement
subclasses and override the Exec() implementation. This has been leveraged
by ZeekJS [1] and zeek-perf-support [2] as well as a private WASM plugin.
All of these used STMT_ANY as the tag of their own statement subclasses.

With STMT_EXTERN, we make the possibility to add external code into the AST
somewhat more supported. It's all in detail space and plugin authors have
no guarantee for stability, but it seems such a powerful extension point
that IMO we should keep it.

I'm conscious there's the broader topic how this interacts with ZAM
optimization like in-lining or rewriting of statements. However, this
already applies to the STMT_ANY usage of the mentioned plugins.

[1] https://github.com/corelight/zeekjs
[2] https://github.com/zeek/zeek-perf-support
2024-02-06 14:40:20 +01:00
Johanna Amann
160ccda45f Merge remote-tracking branch 'origin/topic/johanna/netcontrol-updates'
* origin/topic/johanna/netcontrol-updates:
  Netcontrol: add rule_added_policy
  Netcontrol: more logging in catch-and-release
  Netcontrol: allow supplying explicit name to Debug plugin
2024-02-06 11:54:26 +00:00
Arne Welzel
f8a7f3699c Merge remote-tracking branch 'origin/topic/vern/complete-ast-locs'
* origin/topic/vern/complete-ast-locs:
  Obj: Implement with_location_of() as template
  fixes & enhancements to location information associated w/ AST nodes & ZAM optimization
2024-02-06 11:22:58 +01:00
Arne Welzel
caa1c7493f Obj: Implement with_location_of() as template 2024-02-06 11:03:06 +01:00
Vern Paxson
e5bb63c662 fixes & enhancements to location information associated w/ AST nodes & ZAM optimization 2024-02-06 11:03:02 +01:00
Arne Welzel
7a283afe00 Merge remote-tracking branch 'origin/topic/christian/cluster-pool-nodetype-default'
* origin/topic/christian/cluster-pool-nodetype-default:
  Do not default PoolSpec topics to the empty string.
  Do not default to proxy nodes in Broker::PoolSpec
2024-02-06 10:11:45 +01:00
Arne Welzel
ee5579c40a Merge remote-tracking branch 'origin/topic/timw/fix-alpine-ci-image'
* origin/topic/timw/fix-alpine-ci-image:
  Add missing include for int32_t in DebugCmds.h
  CI: Remove emacs backup file of fedora-38 Dockerfile
  CI: Fix building of alpine image with regards to python packages
2024-02-06 09:55:21 +01:00
Christian Kreibich
873d734c79 Do not default PoolSpec topics to the empty string.
Similar to `node_topic`, we already spell out a topic in the existing use and
there's no obviously meaningful default value.
2024-02-05 18:03:08 -08:00
Christian Kreibich
8437012346 Do not default to proxy nodes in Broker::PoolSpec
This requires pool creation to spell out a spec explicitly, which the only code
using these types already does. There's no reason for pools to automatically
refer to proxies.
2024-02-05 17:51:11 -08:00
Christian Kreibich
42f0a10cb0 Bump cluster testsuite to latest main [skip ci] 2024-02-05 17:19:38 -08:00
Tim Wojtulewicz
01d11d6c0b Add missing include for int32_t in DebugCmds.h 2024-02-05 14:38:16 -07:00
Tim Wojtulewicz
60da270800 CI: Remove emacs backup file of fedora-38 Dockerfile 2024-02-05 13:33:52 -07:00
Tim Wojtulewicz
2736e8e527 CI: Fix building of alpine image with regards to python packages 2024-02-05 13:32:19 -07:00
Tim Wojtulewicz
4e589121c8 Merge remote-tracking branch 'origin/topic/vern/script-opt-maint.Feb24'
* origin/topic/vern/script-opt-maint.Feb24:
  minor updates and fixes for script optimization maintenance
2024-02-05 12:51:01 -07:00
Johanna Amann
737f828bcb Netcontrol: add rule_added_policy
rule_added_policy allows the modification of rules just after they have
been added. This allows the implementation of some more complex features
- like changing rule states depending on insertion in other plugins.
2024-02-05 18:52:27 +00:00