Commit graph

15389 commits

Author SHA1 Message Date
Benjamin Bannier
ddac7201df Bump spicy-plugin to v1.4.0.
spicy-plugin-1.4.0 changes behavior, so it is not backwards-compatible
with the spicy-plugin-1.3 series. This patch should not be backported to
LTS release branches.
2022-10-28 11:56:26 +02:00
Tim Wojtulewicz
2ed42ef771 Merge remote-tracking branch 'origin/topic/bbannier/bump-spicy-plugin'
* origin/topic/bbannier/bump-spicy-plugin:
  Bump spicy-plugin to v1.3.21.
2022-10-27 17:11:21 -07:00
Tim Wojtulewicz
3891a113dd Merge remote-tracking branch 'origin/topic/timw/btest-egrep-obsolete'
* origin/topic/timw/btest-egrep-obsolete:
  egrep reported as obsolete by opensuse-tumbleweed builds
2022-10-27 17:10:17 -07:00
Tim Wojtulewicz
d442ea1bb9 egrep reported as obsolete by opensuse-tumbleweed builds 2022-10-27 11:48:43 -07:00
Tim Wojtulewicz
a96b7e6563 Updating submodule(s) [nomail] 2022-10-27 08:48:00 -07:00
Arne Welzel
a5f04b6270 cat_sep: Make fully vararg and do explicit runtime type checks
Using positional and vararg arguments for BIFs, it's not possible to do
proper runtime type checking on them as discussed in #2425. The bifcl produced
code unconditionally attempts to convert the positional arguments to StringVals,
but nothing ever type checks them. Instead of improving the vararg support in
Zeek script and bifcl, align cat_sep() with fmt() in making it fully vararg
and do implement type checks by hand.

With this change, passing wrong types for the separator and default argument
isn't a fatal error anymore and the error messages are also more descriptive.

It's a bit of a crutch working around varargs limitations.

Fixes #2425
2022-10-27 13:06:06 +02:00
Christian Kreibich
096ff41966 Update doc submodule [nomail] [skip ci] 2022-10-26 17:32:39 -07:00
Christian Kreibich
3237ad0ba2 Merge remote-tracking branch 'origin/topic/awelzel/builtin-af-packet'
* origin/topic/awelzel/builtin-af-packet:
  Include in Jan's AF_PACKET plugin as builtin plugin
2022-10-26 16:48:37 -07:00
Arne Welzel
699949cd40 bifs/record_fields: Include actual enum name in type_name
One more from @stevesmoot. The record_fields() BIF produced "enum" as
type_name for fields of type enum.

Extend container_type_name() to append the actual name of the enum.

This is changing the format and may break consumers, but those are
likely in a category that are happy to adapt. Not having the actual
enum name available wasn't very helpful.

We could alternatively render only the actual type_name without the
prefixed "enum", but that isn't how it's done for record types currently
and it would make it more difficult to decide which subsequent BIFs to
use for further introspection, like enum_names().
2022-10-26 20:21:20 +02:00
Benjamin Bannier
08bfcff702 Bump spicy-plugin to v1.3.21. 2022-10-26 16:17:02 +02:00
Robin Sommer
bf4bbd0b81
Update doc submodule. 2022-10-26 11:38:57 +02:00
Robin Sommer
2959afba8a
Merge branch 'topic/amazingpp/broker-debug-output-truncated' of ssh://github.com/AmazingPP/zeek
* 'topic/amazingpp/broker-debug-output-truncated' of ssh://github.com/AmazingPP/zeek:
  Use json_escape_utf8 in broker debug output
2022-10-26 11:38:41 +02:00
Robin Sommer
c44899156a
Merge remote-tracking branch 'origin/topic/awelzel/event-groups-v2'
* origin/topic/awelzel/event-groups-v2:
  Reintroduce event groups
2022-10-26 10:58:59 +02:00
zeek-bot
2407a22b3e Update doc submodule [nomail] [skip ci] 2022-10-26 00:55:15 +00:00
Arne Welzel
2ad609cbbb Reintroduce event groups
This started with reverting commit 52cd02173d
and then rewriting it to be per handler rather than handler identifier
and adding support for hooks as well as adding implicit module groups.
2022-10-25 18:03:26 +02:00
Arne Welzel
f3f593c523 IPBasedAnalyzer/TCPSessionAdapter: Fix TCP reassembly decision for known port analyzers
This seems to be an age-old bug. Reported by mchen on discourse [1].

The TCPSessionAdapter decides in AddExtraAnalyzers() whether to enable
reassembly or not. When dpd_reassemble_first_packets is F, this boils down to
! GetChildren().empty(). The intention being that if any analyzers have been
added to the connection based on known ports, reassembly is to be enabled.

However, GetChildren() does not take into account new_children and so
! GetChildren().empty() is always false here and reassembly solely
based on dpd_reassemble_first_packets=F (or the tcp_content... options).
Ouch.

Call AppendNewChildren() before AddExtraAnalyzers() as a fix. Without this,
the new test does not produce an http.log and service "http" isn't in conn.log.

[1] https://community.zeek.org/t/how-to-activate-an-application-layer-analyzer-when-signature-dpd-reassemble-first-packets-is-off/6763
2022-10-25 16:07:40 +02:00
Robin Sommer
5aa7d80e88
Merge remote-tracking branch 'origin/topic/awelzel/enum-names-with-strings'
* origin/topic/awelzel/enum-names-with-strings:
  enum_names: Support naming types with a string
2022-10-25 12:37:33 +02:00
Robin Sommer
9f3a234f40
Merge remote-tracking branch 'origin/topic/awelzel/blank-identifer'
* origin/topic/awelzel/blank-identifer:
  scripts: Migrate table iteration to blank identifiers
  Introduce special treatment for the blank identifier _
2022-10-25 12:36:23 +02:00
Christian Kreibich
92c758bd44 Management framework: bump zeek-client and cluster testsuite 2022-10-24 16:00:58 -07:00
Christian Kreibich
108da36b82 Add new zeek-client dependency to Dockerfile: python3-websocket
This is a required dependency of zeek-client, and available as a .deb on Debian.
2022-10-24 15:59:26 -07:00
Christian Kreibich
147283c8f5 Management framework: add websocket support to controller
The controller now listens on an additional port, defaulting to 2149, for Broker
connections via websockets. Configuration works as for the existing traditional
Broker port (2150), via ZEEK_CONTROLLER_WEBSOCKET_ADDR and
ZEEK_CONTROLLER_WEBSOCKET_PORT environment variables, as well as corresponding
redef'able constants.

To disable the websockets feature, leave ZEEK_CONTROLLER_WEBSOCKET_PORT unset
and redefine Management::Controller::default_port_websocket to 0/unknown.
2022-10-24 15:59:26 -07:00
Tim Wojtulewicz
aa00a5cf20 Merge remote-tracking branch 'origin/topic/timw/ci-benchmarks'
* origin/topic/timw/ci-benchmarks:
  Re-enable ci-based benchmarking
2022-10-24 08:10:16 -07:00
Arne Welzel
8c5896a74d scripts: Migrate table iteration to blank identifiers
No obvious hot-cases. Maybe the describe_file() ones or the intel ones
if/when there are hot intel hits.
2022-10-24 10:36:09 +02:00
Arne Welzel
46334f8b59 Introduce special treatment for the blank identifier _
Mostly: Do not instantiate variables within for loops and allow
reusing differently typed blanks which previously wasn't possible.

This may be missing some corner-cases, but the added tests seem
to work as expected and nothing else fell apart it seems.
2022-10-24 10:36:01 +02:00
zeek-bot
9f3c7cb517 Update doc submodule [nomail] [skip ci] 2022-10-22 00:56:36 +00:00
Christian Kreibich
0a7e64c316 Merge branch 'topic/christian/coverity-drop-external-caf'
* topic/christian/coverity-drop-external-caf:
  Remove external CAF build for Coverity setup
2022-10-21 16:29:55 -07:00
Christian Kreibich
efc0a1f8d8 Remove external CAF build for Coverity setup
The Coverity build used an external CAF setup because the bundled build used to
cause internal errors in Coverity's tooling. zeek/broker#285 causes a problem
with the external CAF setup, plus Coverity now looks happy with the usual
bundled build, so this commit simply drops the external CAF approach.
2022-10-21 15:46:34 -07:00
Tim Wojtulewicz
d116983d04 Merge remote-tracking branch 'origin/topic/timw/2132-bpf-filtering-error-messages'
* origin/topic/timw/2132-bpf-filtering-error-messages:
  Remove cmake tests for LIBPCAP_PCAP_COMPILE_NOPCAP_HAS_ERROR_PARAMETER
  Rework the non-pcap version of BPF_Program::Compile
  Add btest that exercises the pcap filter warnings
  Update docs submodule
  Restore reporting messages for pcap filter issues
  Add column to packet_filter.log for failure reason
  Store error message from BPF compilation
  Minor cleanup in BPF filtering code
2022-10-21 12:24:39 -07:00
Arne Welzel
a00cef9920 enum_names: Support naming types with a string
In his ZeekWeek 2022 presentation, @stevesmoot mentioned that he had a
difficult time looking up enum names when all he had was a string
naming the type.

Add support to enum_names() to transparently lookup the type if a string
is provided. This is similar in how record_fields() behaves when being
passed a string.
2022-10-21 20:09:48 +02:00
Tim Wojtulewicz
5cf84c24f8 Remove cmake tests for LIBPCAP_PCAP_COMPILE_NOPCAP_HAS_ERROR_PARAMETER 2022-10-21 10:50:00 -07:00
Tim Wojtulewicz
dbb1980ade Rework the non-pcap version of BPF_Program::Compile 2022-10-21 10:50:00 -07:00
Tim Wojtulewicz
6bb6fed49f Add btest that exercises the pcap filter warnings 2022-10-21 10:50:00 -07:00
Tim Wojtulewicz
145abedbdf Update docs submodule 2022-10-21 10:50:00 -07:00
Tim Wojtulewicz
81357853ed Restore reporting messages for pcap filter issues 2022-10-21 10:50:00 -07:00
Tim Wojtulewicz
5e4db6d0c4 Add column to packet_filter.log for failure reason 2022-10-21 10:50:00 -07:00
Tim Wojtulewicz
82adecb2ad Store error message from BPF compilation 2022-10-21 10:09:56 -07:00
Tim Wojtulewicz
767c83ede8 Minor cleanup in BPF filtering code 2022-10-21 10:09:56 -07:00
Tim Wojtulewicz
072158d4b6 Merge remote-tracking branch 'origin/topic/timw/remove-nbdns-leftovers'
* origin/topic/timw/remove-nbdns-leftovers:
  Reorder includes in Options.cc
  Remove some leftover nb_dns stuff from the DNS_Mgr rework
2022-10-21 10:09:24 -07:00
Tim Wojtulewicz
05ba11c72a Reorder includes in Options.cc 2022-10-21 10:08:34 -07:00
Tim Wojtulewicz
e8507f3a95 Remove some leftover nb_dns stuff from the DNS_Mgr rework 2022-10-21 10:08:34 -07:00
Christian Kreibich
bf892370ef Update broker submodule [nomail] 2022-10-20 16:59:07 -07:00
Tim Wojtulewicz
0e8318ff1a Merge remote-tracking branch 'origin/topic/timw/ms-testing'
* origin/topic/timw/ms-testing:
  Update bifcl, binpac, broker, cmake, gen-zam, and paraglob submodules
2022-10-20 13:03:58 -07:00
Tim Wojtulewicz
d875bfb7bd Update bifcl, binpac, broker, cmake, gen-zam, and paraglob submodules 2022-10-19 20:16:01 -07:00
Tim Wojtulewicz
f9c1d5a756 Re-enable ci-based benchmarking 2022-10-19 17:47:16 -07:00
zeek-bot
7f7c77ab07 Update doc submodule [nomail] [skip ci] 2022-10-19 00:57:23 +00:00
Tim Wojtulewicz
2159df41cb Merge remote-tracking branch 'jasonlue/topic/jasonlu/dict-clone-performance'
* jasonlue/topic/jasonlu/dict-clone-performance:
  reset performance metrics every SizeUp()
  reduce space_distance_samples from int64_6 to uint32_t
  add performance based growth strategy
2022-10-18 08:53:38 -07:00
Tim Wojtulewicz
0143d0cb86 Merge remote-tracking branch 'origin/topic/vern/CPP-Oct22-maint'
* origin/topic/vern/CPP-Oct22-maint:
  -O C++ maintenance updates to test suite
  -O gen-C++ maintenance: fixes for attributes & initializing globals
2022-10-17 16:19:04 -07:00
Tim Wojtulewicz
ea57cac871 Merge remote-tracking branch 'origin/topic/timw/surface-better-sanitizer-flag-error'
* origin/topic/timw/surface-better-sanitizer-flag-error:
  Surface a better CMake error if the user passes an invalid sanitizer name
2022-10-17 08:50:17 -07:00
Tim Wojtulewicz
0ba53de80e Surface a better CMake error if the user passes an invalid sanitizer name 2022-10-17 08:47:50 -07:00
Tim Wojtulewicz
3656699e2f Merge remote-tracking branch 'origin/topic/awelzel/2403-reject-conflicting-plugins'
* origin/topic/awelzel/2403-reject-conflicting-plugins:
  plugins: Reject dynamic plugins matching names of built-in ones
2022-10-16 17:49:32 -07:00