Commit graph

16055 commits

Author SHA1 Message Date
Arne Welzel
1fecdfd815 cmake: Introduce bro_PLUGIN_LINK_LIBS and add to link command.
When using --include-plugins with plugins that depend other libraries
using zeek_plugin_link_library(), these libraries were stashed into
bro_SUBDIR_LIBS that is used in add_executable(). However, that does not
work for shared libraries.

Explicitly track library dependencies in bro_PLUGIN_LINK_LIBS now and
use target_link_libraries() with Zeek (there's a change in cmake to
make zeek_plugin_link_library() populate bro_PLUGIN_LINK_LIBS instead).

Currently, zeekdeps needs to come after bro_PLUGIN_LINK_LIBS due to how
spicy-plugin (and other external plugins) fiddle with zeekdeps in order
to stash dependencies into the PARENT_SCOPE.

bro_PLUGIN_LINK_LIBS will continue to work with static libraries.

The failure occurred when attempting to include bro-http2 [1].

Fixes #2483

[1] https://github.com/MITRECND/bro-http2
2022-11-02 12:21:44 +01:00
Arne Welzel
4414c9a49f cmake: Do not use include_directories() for included plugins
There's a cmake repo change that does the include_directories() in
zeek_plugin_begin() unconditionally instead.
2022-11-02 12:21:44 +01:00
Arne Welzel
79e9384218 cmake: Bump submodule 2022-11-02 12:21:44 +01:00
Robin Sommer
f8eb2d9241
Merge remote-tracking branch 'origin/topic/awelzel/2440-break-next-usage-validation'
* origin/topic/awelzel/2440-break-next-usage-validation:
  parse.y: Traverse AST post parsing to detect break/next usage issues
2022-11-02 10:59:29 +01:00
Tim Wojtulewicz
bc0284aefa Merge remote-tracking branch 'origin/topic/awelzel/2446-runtime-error-for-variadic-functions'
* origin/topic/awelzel/2446-runtime-error-for-variadic-functions:
  Func: Do not crash on va_args confusion for script funcs
2022-11-01 07:47:50 -07:00
Arne Welzel
4314467e44 Func: Do not crash on va_args confusion for script funcs
Script and BIF functions with a single any parameter are excluded from
type checking regarding arguments. This makes it possible to call a
ScriptFunc with more arguments than it actually has parameters and frame
space for, causing heap-buffer-overflows.

This change runtime checks expected parameters and provided arguments
and short-circuits execution as well as logging runtime expression errors.

Fixes #2446
2022-11-01 08:55:14 +01:00
Tim Wojtulewicz
6dec7d6027 Merge remote-tracking branch 'origin/topic/awelzel/record-fields-not-just-enum'
* origin/topic/awelzel/record-fields-not-just-enum:
  bifs/record_fields: Include actual enum name in type_name
2022-10-31 15:18:11 -07:00
Tim Wojtulewicz
352705d316 Merge remote-tracking branch 'origin/topic/awelzel/no-reassembly-for-known-ports'
* origin/topic/awelzel/no-reassembly-for-known-ports:
  IPBasedAnalyzer/TCPSessionAdapter: Fix TCP reassembly decision for known port analyzers
2022-10-31 14:57:39 -07:00
Tim Wojtulewicz
e2a384848e Update cmake submodule [nomail] [skip ci] 2022-10-31 11:38:29 -07:00
Tim Wojtulewicz
b6826a43e4 Updating submodule(s) [nomail] 2022-10-29 19:14:05 -07:00
zeek-bot
9a2a086f98 Update doc submodule [nomail] [skip ci] 2022-10-30 00:42:24 +00:00
Christian Kreibich
95151e1da9 Merge remote-tracking branch 'origin/topic/awelzel/2425-cat-sep-fully-varargs'
* origin/topic/awelzel/2425-cat-sep-fully-varargs:
  cat_sep: Make fully vararg and do explicit runtime type checks
2022-10-28 16:55:29 -07:00
Tim Wojtulewicz
05ef4a36f1 Update cmake submodule [nomail] 2022-10-28 15:51:31 -07:00
Tim Wojtulewicz
28c3cd086a Merge remote-tracking branch 'origin/topic/bbannier/spicy-bumps'
* origin/topic/bbannier/spicy-bumps:
  Update Spicy to the latest development snapshot.
  Bump spicy-plugin to v1.4.0.
2022-10-28 15:45:08 -07:00
Arne Welzel
850aaaa5a8 parse.y: Traverse AST post parsing to detect break/next usage issues
Seemed easiest to do it via the traversal infrastructure as we do not
otherwise track enough context/scope when instantiating break or next
statements.

Might be worth moving this out of src/parse.y, but didn't exactly know
where. Or maybe we wait until there's more such trivial validations
popping up

Fixes #2440
2022-10-28 12:53:37 +02:00
Benjamin Bannier
976f5cb2d9 Update Spicy to the latest development snapshot. 2022-10-28 11:56:28 +02:00
Benjamin Bannier
ddac7201df Bump spicy-plugin to v1.4.0.
spicy-plugin-1.4.0 changes behavior, so it is not backwards-compatible
with the spicy-plugin-1.3 series. This patch should not be backported to
LTS release branches.
2022-10-28 11:56:26 +02:00
Tim Wojtulewicz
2ed42ef771 Merge remote-tracking branch 'origin/topic/bbannier/bump-spicy-plugin'
* origin/topic/bbannier/bump-spicy-plugin:
  Bump spicy-plugin to v1.3.21.
2022-10-27 17:11:21 -07:00
Tim Wojtulewicz
3891a113dd Merge remote-tracking branch 'origin/topic/timw/btest-egrep-obsolete'
* origin/topic/timw/btest-egrep-obsolete:
  egrep reported as obsolete by opensuse-tumbleweed builds
2022-10-27 17:10:17 -07:00
Tim Wojtulewicz
d442ea1bb9 egrep reported as obsolete by opensuse-tumbleweed builds 2022-10-27 11:48:43 -07:00
Tim Wojtulewicz
a96b7e6563 Updating submodule(s) [nomail] 2022-10-27 08:48:00 -07:00
Arne Welzel
a5f04b6270 cat_sep: Make fully vararg and do explicit runtime type checks
Using positional and vararg arguments for BIFs, it's not possible to do
proper runtime type checking on them as discussed in #2425. The bifcl produced
code unconditionally attempts to convert the positional arguments to StringVals,
but nothing ever type checks them. Instead of improving the vararg support in
Zeek script and bifcl, align cat_sep() with fmt() in making it fully vararg
and do implement type checks by hand.

With this change, passing wrong types for the separator and default argument
isn't a fatal error anymore and the error messages are also more descriptive.

It's a bit of a crutch working around varargs limitations.

Fixes #2425
2022-10-27 13:06:06 +02:00
Christian Kreibich
096ff41966 Update doc submodule [nomail] [skip ci] 2022-10-26 17:32:39 -07:00
Christian Kreibich
3237ad0ba2 Merge remote-tracking branch 'origin/topic/awelzel/builtin-af-packet'
* origin/topic/awelzel/builtin-af-packet:
  Include in Jan's AF_PACKET plugin as builtin plugin
2022-10-26 16:48:37 -07:00
Arne Welzel
699949cd40 bifs/record_fields: Include actual enum name in type_name
One more from @stevesmoot. The record_fields() BIF produced "enum" as
type_name for fields of type enum.

Extend container_type_name() to append the actual name of the enum.

This is changing the format and may break consumers, but those are
likely in a category that are happy to adapt. Not having the actual
enum name available wasn't very helpful.

We could alternatively render only the actual type_name without the
prefixed "enum", but that isn't how it's done for record types currently
and it would make it more difficult to decide which subsequent BIFs to
use for further introspection, like enum_names().
2022-10-26 20:21:20 +02:00
Benjamin Bannier
08bfcff702 Bump spicy-plugin to v1.3.21. 2022-10-26 16:17:02 +02:00
Robin Sommer
bf4bbd0b81
Update doc submodule. 2022-10-26 11:38:57 +02:00
Robin Sommer
2959afba8a
Merge branch 'topic/amazingpp/broker-debug-output-truncated' of ssh://github.com/AmazingPP/zeek
* 'topic/amazingpp/broker-debug-output-truncated' of ssh://github.com/AmazingPP/zeek:
  Use json_escape_utf8 in broker debug output
2022-10-26 11:38:41 +02:00
Robin Sommer
c44899156a
Merge remote-tracking branch 'origin/topic/awelzel/event-groups-v2'
* origin/topic/awelzel/event-groups-v2:
  Reintroduce event groups
2022-10-26 10:58:59 +02:00
zeek-bot
2407a22b3e Update doc submodule [nomail] [skip ci] 2022-10-26 00:55:15 +00:00
Arne Welzel
2ad609cbbb Reintroduce event groups
This started with reverting commit 52cd02173d
and then rewriting it to be per handler rather than handler identifier
and adding support for hooks as well as adding implicit module groups.
2022-10-25 18:03:26 +02:00
Arne Welzel
f3f593c523 IPBasedAnalyzer/TCPSessionAdapter: Fix TCP reassembly decision for known port analyzers
This seems to be an age-old bug. Reported by mchen on discourse [1].

The TCPSessionAdapter decides in AddExtraAnalyzers() whether to enable
reassembly or not. When dpd_reassemble_first_packets is F, this boils down to
! GetChildren().empty(). The intention being that if any analyzers have been
added to the connection based on known ports, reassembly is to be enabled.

However, GetChildren() does not take into account new_children and so
! GetChildren().empty() is always false here and reassembly solely
based on dpd_reassemble_first_packets=F (or the tcp_content... options).
Ouch.

Call AppendNewChildren() before AddExtraAnalyzers() as a fix. Without this,
the new test does not produce an http.log and service "http" isn't in conn.log.

[1] https://community.zeek.org/t/how-to-activate-an-application-layer-analyzer-when-signature-dpd-reassemble-first-packets-is-off/6763
2022-10-25 16:07:40 +02:00
Robin Sommer
5aa7d80e88
Merge remote-tracking branch 'origin/topic/awelzel/enum-names-with-strings'
* origin/topic/awelzel/enum-names-with-strings:
  enum_names: Support naming types with a string
2022-10-25 12:37:33 +02:00
Robin Sommer
9f3a234f40
Merge remote-tracking branch 'origin/topic/awelzel/blank-identifer'
* origin/topic/awelzel/blank-identifer:
  scripts: Migrate table iteration to blank identifiers
  Introduce special treatment for the blank identifier _
2022-10-25 12:36:23 +02:00
Christian Kreibich
92c758bd44 Management framework: bump zeek-client and cluster testsuite 2022-10-24 16:00:58 -07:00
Christian Kreibich
108da36b82 Add new zeek-client dependency to Dockerfile: python3-websocket
This is a required dependency of zeek-client, and available as a .deb on Debian.
2022-10-24 15:59:26 -07:00
Christian Kreibich
147283c8f5 Management framework: add websocket support to controller
The controller now listens on an additional port, defaulting to 2149, for Broker
connections via websockets. Configuration works as for the existing traditional
Broker port (2150), via ZEEK_CONTROLLER_WEBSOCKET_ADDR and
ZEEK_CONTROLLER_WEBSOCKET_PORT environment variables, as well as corresponding
redef'able constants.

To disable the websockets feature, leave ZEEK_CONTROLLER_WEBSOCKET_PORT unset
and redefine Management::Controller::default_port_websocket to 0/unknown.
2022-10-24 15:59:26 -07:00
Tim Wojtulewicz
aa00a5cf20 Merge remote-tracking branch 'origin/topic/timw/ci-benchmarks'
* origin/topic/timw/ci-benchmarks:
  Re-enable ci-based benchmarking
2022-10-24 08:10:16 -07:00
Arne Welzel
8c5896a74d scripts: Migrate table iteration to blank identifiers
No obvious hot-cases. Maybe the describe_file() ones or the intel ones
if/when there are hot intel hits.
2022-10-24 10:36:09 +02:00
Arne Welzel
46334f8b59 Introduce special treatment for the blank identifier _
Mostly: Do not instantiate variables within for loops and allow
reusing differently typed blanks which previously wasn't possible.

This may be missing some corner-cases, but the added tests seem
to work as expected and nothing else fell apart it seems.
2022-10-24 10:36:01 +02:00
zeek-bot
9f3c7cb517 Update doc submodule [nomail] [skip ci] 2022-10-22 00:56:36 +00:00
Christian Kreibich
0a7e64c316 Merge branch 'topic/christian/coverity-drop-external-caf'
* topic/christian/coverity-drop-external-caf:
  Remove external CAF build for Coverity setup
2022-10-21 16:29:55 -07:00
Christian Kreibich
efc0a1f8d8 Remove external CAF build for Coverity setup
The Coverity build used an external CAF setup because the bundled build used to
cause internal errors in Coverity's tooling. zeek/broker#285 causes a problem
with the external CAF setup, plus Coverity now looks happy with the usual
bundled build, so this commit simply drops the external CAF approach.
2022-10-21 15:46:34 -07:00
Tim Wojtulewicz
d116983d04 Merge remote-tracking branch 'origin/topic/timw/2132-bpf-filtering-error-messages'
* origin/topic/timw/2132-bpf-filtering-error-messages:
  Remove cmake tests for LIBPCAP_PCAP_COMPILE_NOPCAP_HAS_ERROR_PARAMETER
  Rework the non-pcap version of BPF_Program::Compile
  Add btest that exercises the pcap filter warnings
  Update docs submodule
  Restore reporting messages for pcap filter issues
  Add column to packet_filter.log for failure reason
  Store error message from BPF compilation
  Minor cleanup in BPF filtering code
2022-10-21 12:24:39 -07:00
Arne Welzel
a00cef9920 enum_names: Support naming types with a string
In his ZeekWeek 2022 presentation, @stevesmoot mentioned that he had a
difficult time looking up enum names when all he had was a string
naming the type.

Add support to enum_names() to transparently lookup the type if a string
is provided. This is similar in how record_fields() behaves when being
passed a string.
2022-10-21 20:09:48 +02:00
Tim Wojtulewicz
5cf84c24f8 Remove cmake tests for LIBPCAP_PCAP_COMPILE_NOPCAP_HAS_ERROR_PARAMETER 2022-10-21 10:50:00 -07:00
Tim Wojtulewicz
dbb1980ade Rework the non-pcap version of BPF_Program::Compile 2022-10-21 10:50:00 -07:00
Tim Wojtulewicz
6bb6fed49f Add btest that exercises the pcap filter warnings 2022-10-21 10:50:00 -07:00
Tim Wojtulewicz
145abedbdf Update docs submodule 2022-10-21 10:50:00 -07:00
Tim Wojtulewicz
81357853ed Restore reporting messages for pcap filter issues 2022-10-21 10:50:00 -07:00