Commit graph

15378 commits

Author SHA1 Message Date
Robin Sommer
2959afba8a
Merge branch 'topic/amazingpp/broker-debug-output-truncated' of ssh://github.com/AmazingPP/zeek
* 'topic/amazingpp/broker-debug-output-truncated' of ssh://github.com/AmazingPP/zeek:
  Use json_escape_utf8 in broker debug output
2022-10-26 11:38:41 +02:00
Robin Sommer
c44899156a
Merge remote-tracking branch 'origin/topic/awelzel/event-groups-v2'
* origin/topic/awelzel/event-groups-v2:
  Reintroduce event groups
2022-10-26 10:58:59 +02:00
zeek-bot
2407a22b3e Update doc submodule [nomail] [skip ci] 2022-10-26 00:55:15 +00:00
Arne Welzel
2ad609cbbb Reintroduce event groups
This started with reverting commit 52cd02173d
and then rewriting it to be per handler rather than handler identifier
and adding support for hooks as well as adding implicit module groups.
2022-10-25 18:03:26 +02:00
Arne Welzel
f3f593c523 IPBasedAnalyzer/TCPSessionAdapter: Fix TCP reassembly decision for known port analyzers
This seems to be an age-old bug. Reported by mchen on discourse [1].

The TCPSessionAdapter decides in AddExtraAnalyzers() whether to enable
reassembly or not. When dpd_reassemble_first_packets is F, this boils down to
! GetChildren().empty(). The intention being that if any analyzers have been
added to the connection based on known ports, reassembly is to be enabled.

However, GetChildren() does not take into account new_children and so
! GetChildren().empty() is always false here and reassembly solely
based on dpd_reassemble_first_packets=F (or the tcp_content... options).
Ouch.

Call AppendNewChildren() before AddExtraAnalyzers() as a fix. Without this,
the new test does not produce an http.log and service "http" isn't in conn.log.

[1] https://community.zeek.org/t/how-to-activate-an-application-layer-analyzer-when-signature-dpd-reassemble-first-packets-is-off/6763
2022-10-25 16:07:40 +02:00
Robin Sommer
5aa7d80e88
Merge remote-tracking branch 'origin/topic/awelzel/enum-names-with-strings'
* origin/topic/awelzel/enum-names-with-strings:
  enum_names: Support naming types with a string
2022-10-25 12:37:33 +02:00
Robin Sommer
9f3a234f40
Merge remote-tracking branch 'origin/topic/awelzel/blank-identifer'
* origin/topic/awelzel/blank-identifer:
  scripts: Migrate table iteration to blank identifiers
  Introduce special treatment for the blank identifier _
2022-10-25 12:36:23 +02:00
Christian Kreibich
92c758bd44 Management framework: bump zeek-client and cluster testsuite 2022-10-24 16:00:58 -07:00
Christian Kreibich
108da36b82 Add new zeek-client dependency to Dockerfile: python3-websocket
This is a required dependency of zeek-client, and available as a .deb on Debian.
2022-10-24 15:59:26 -07:00
Christian Kreibich
147283c8f5 Management framework: add websocket support to controller
The controller now listens on an additional port, defaulting to 2149, for Broker
connections via websockets. Configuration works as for the existing traditional
Broker port (2150), via ZEEK_CONTROLLER_WEBSOCKET_ADDR and
ZEEK_CONTROLLER_WEBSOCKET_PORT environment variables, as well as corresponding
redef'able constants.

To disable the websockets feature, leave ZEEK_CONTROLLER_WEBSOCKET_PORT unset
and redefine Management::Controller::default_port_websocket to 0/unknown.
2022-10-24 15:59:26 -07:00
Tim Wojtulewicz
aa00a5cf20 Merge remote-tracking branch 'origin/topic/timw/ci-benchmarks'
* origin/topic/timw/ci-benchmarks:
  Re-enable ci-based benchmarking
2022-10-24 08:10:16 -07:00
Arne Welzel
8c5896a74d scripts: Migrate table iteration to blank identifiers
No obvious hot-cases. Maybe the describe_file() ones or the intel ones
if/when there are hot intel hits.
2022-10-24 10:36:09 +02:00
Arne Welzel
46334f8b59 Introduce special treatment for the blank identifier _
Mostly: Do not instantiate variables within for loops and allow
reusing differently typed blanks which previously wasn't possible.

This may be missing some corner-cases, but the added tests seem
to work as expected and nothing else fell apart it seems.
2022-10-24 10:36:01 +02:00
zeek-bot
9f3c7cb517 Update doc submodule [nomail] [skip ci] 2022-10-22 00:56:36 +00:00
Christian Kreibich
0a7e64c316 Merge branch 'topic/christian/coverity-drop-external-caf'
* topic/christian/coverity-drop-external-caf:
  Remove external CAF build for Coverity setup
2022-10-21 16:29:55 -07:00
Christian Kreibich
efc0a1f8d8 Remove external CAF build for Coverity setup
The Coverity build used an external CAF setup because the bundled build used to
cause internal errors in Coverity's tooling. zeek/broker#285 causes a problem
with the external CAF setup, plus Coverity now looks happy with the usual
bundled build, so this commit simply drops the external CAF approach.
2022-10-21 15:46:34 -07:00
Tim Wojtulewicz
d116983d04 Merge remote-tracking branch 'origin/topic/timw/2132-bpf-filtering-error-messages'
* origin/topic/timw/2132-bpf-filtering-error-messages:
  Remove cmake tests for LIBPCAP_PCAP_COMPILE_NOPCAP_HAS_ERROR_PARAMETER
  Rework the non-pcap version of BPF_Program::Compile
  Add btest that exercises the pcap filter warnings
  Update docs submodule
  Restore reporting messages for pcap filter issues
  Add column to packet_filter.log for failure reason
  Store error message from BPF compilation
  Minor cleanup in BPF filtering code
2022-10-21 12:24:39 -07:00
Arne Welzel
a00cef9920 enum_names: Support naming types with a string
In his ZeekWeek 2022 presentation, @stevesmoot mentioned that he had a
difficult time looking up enum names when all he had was a string
naming the type.

Add support to enum_names() to transparently lookup the type if a string
is provided. This is similar in how record_fields() behaves when being
passed a string.
2022-10-21 20:09:48 +02:00
Tim Wojtulewicz
5cf84c24f8 Remove cmake tests for LIBPCAP_PCAP_COMPILE_NOPCAP_HAS_ERROR_PARAMETER 2022-10-21 10:50:00 -07:00
Tim Wojtulewicz
dbb1980ade Rework the non-pcap version of BPF_Program::Compile 2022-10-21 10:50:00 -07:00
Tim Wojtulewicz
6bb6fed49f Add btest that exercises the pcap filter warnings 2022-10-21 10:50:00 -07:00
Tim Wojtulewicz
145abedbdf Update docs submodule 2022-10-21 10:50:00 -07:00
Tim Wojtulewicz
81357853ed Restore reporting messages for pcap filter issues 2022-10-21 10:50:00 -07:00
Tim Wojtulewicz
5e4db6d0c4 Add column to packet_filter.log for failure reason 2022-10-21 10:50:00 -07:00
Tim Wojtulewicz
82adecb2ad Store error message from BPF compilation 2022-10-21 10:09:56 -07:00
Tim Wojtulewicz
767c83ede8 Minor cleanup in BPF filtering code 2022-10-21 10:09:56 -07:00
Tim Wojtulewicz
072158d4b6 Merge remote-tracking branch 'origin/topic/timw/remove-nbdns-leftovers'
* origin/topic/timw/remove-nbdns-leftovers:
  Reorder includes in Options.cc
  Remove some leftover nb_dns stuff from the DNS_Mgr rework
2022-10-21 10:09:24 -07:00
Tim Wojtulewicz
05ba11c72a Reorder includes in Options.cc 2022-10-21 10:08:34 -07:00
Tim Wojtulewicz
e8507f3a95 Remove some leftover nb_dns stuff from the DNS_Mgr rework 2022-10-21 10:08:34 -07:00
Christian Kreibich
bf892370ef Update broker submodule [nomail] 2022-10-20 16:59:07 -07:00
Tim Wojtulewicz
0e8318ff1a Merge remote-tracking branch 'origin/topic/timw/ms-testing'
* origin/topic/timw/ms-testing:
  Update bifcl, binpac, broker, cmake, gen-zam, and paraglob submodules
2022-10-20 13:03:58 -07:00
Tim Wojtulewicz
d875bfb7bd Update bifcl, binpac, broker, cmake, gen-zam, and paraglob submodules 2022-10-19 20:16:01 -07:00
Tim Wojtulewicz
f9c1d5a756 Re-enable ci-based benchmarking 2022-10-19 17:47:16 -07:00
zeek-bot
7f7c77ab07 Update doc submodule [nomail] [skip ci] 2022-10-19 00:57:23 +00:00
Tim Wojtulewicz
2159df41cb Merge remote-tracking branch 'jasonlue/topic/jasonlu/dict-clone-performance'
* jasonlue/topic/jasonlu/dict-clone-performance:
  reset performance metrics every SizeUp()
  reduce space_distance_samples from int64_6 to uint32_t
  add performance based growth strategy
2022-10-18 08:53:38 -07:00
Tim Wojtulewicz
0143d0cb86 Merge remote-tracking branch 'origin/topic/vern/CPP-Oct22-maint'
* origin/topic/vern/CPP-Oct22-maint:
  -O C++ maintenance updates to test suite
  -O gen-C++ maintenance: fixes for attributes & initializing globals
2022-10-17 16:19:04 -07:00
Tim Wojtulewicz
ea57cac871 Merge remote-tracking branch 'origin/topic/timw/surface-better-sanitizer-flag-error'
* origin/topic/timw/surface-better-sanitizer-flag-error:
  Surface a better CMake error if the user passes an invalid sanitizer name
2022-10-17 08:50:17 -07:00
Tim Wojtulewicz
0ba53de80e Surface a better CMake error if the user passes an invalid sanitizer name 2022-10-17 08:47:50 -07:00
Tim Wojtulewicz
3656699e2f Merge remote-tracking branch 'origin/topic/awelzel/2403-reject-conflicting-plugins'
* origin/topic/awelzel/2403-reject-conflicting-plugins:
  plugins: Reject dynamic plugins matching names of built-in ones
2022-10-16 17:49:32 -07:00
Tim Wojtulewicz
5dbe98290b Merge remote-tracking branch 'origin/topic/timw/2208-ordered-tables'
* origin/topic/timw/2208-ordered-tables:
  Add &ordered attribute for tables/sets
2022-10-16 17:48:36 -07:00
Tim Wojtulewicz
bd7df9e2bf Add &ordered attribute for tables/sets 2022-10-16 15:31:42 -07:00
Tim Wojtulewicz
e3682a09e0 Merge remote-tracking branch 'origin/topic/timw/update-kqueue-rapidjson'
* origin/topic/timw/update-kqueue-rapidjson:
  Update zeek-3rdparty submodule for doctest/sqlite updates
  Update libkqueue submodule to latest upstream master
  Update rapidjson submodule to latest upstream master
  Update highwayhash submodule to latest upstream master
2022-10-16 15:30:38 -07:00
Tim Wojtulewicz
456aa1771d Update zeek-3rdparty submodule for doctest/sqlite updates 2022-10-16 14:22:51 -07:00
Tim Wojtulewicz
0d7d490a15 Update libkqueue submodule to latest upstream master 2022-10-16 14:22:51 -07:00
Tim Wojtulewicz
3cc1f3c645 Update rapidjson submodule to latest upstream master 2022-10-16 14:22:51 -07:00
Tim Wojtulewicz
af1b65dc2c Update highwayhash submodule to latest upstream master 2022-10-16 14:22:51 -07:00
Jason Lu
bf2896f33c reset performance metrics every SizeUp() 2022-10-14 09:55:36 -07:00
Jason Lu
a716d766a9 reduce space_distance_samples from int64_6 to uint32_t 2022-10-14 09:43:41 -07:00
Vern Paxson
f4c1309dfc -O C++ maintenance updates to test suite 2022-10-14 08:38:55 -05:00
Vern Paxson
26dae9a5ae -O gen-C++ maintenance: fixes for attributes & initializing globals 2022-10-14 08:37:45 -05:00