Commit graph

16174 commits

Author SHA1 Message Date
Christian Kreibich
873d734c79 Do not default PoolSpec topics to the empty string.
Similar to `node_topic`, we already spell out a topic in the existing use and
there's no obviously meaningful default value.
2024-02-05 18:03:08 -08:00
Christian Kreibich
8437012346 Do not default to proxy nodes in Broker::PoolSpec
This requires pool creation to spell out a spec explicitly, which the only code
using these types already does. There's no reason for pools to automatically
refer to proxies.
2024-02-05 17:51:11 -08:00
Christian Kreibich
42f0a10cb0 Bump cluster testsuite to latest main [skip ci] 2024-02-05 17:19:38 -08:00
Tim Wojtulewicz
4e589121c8 Merge remote-tracking branch 'origin/topic/vern/script-opt-maint.Feb24'
* origin/topic/vern/script-opt-maint.Feb24:
  minor updates and fixes for script optimization maintenance
2024-02-05 12:51:01 -07:00
Vern Paxson
9e32443504 minor updates and fixes for script optimization maintenance 2024-02-04 07:57:45 -08:00
zeek-bot
50c674b670 Update doc submodule [nomail] [skip ci] 2024-02-04 00:14:18 +00:00
zeek-bot
e521526cba Update doc submodule [nomail] [skip ci] 2024-02-03 00:14:08 +00:00
Tim Wojtulewicz
9609f1fe1a Merge branch 'topic/timw/vcpkg'
* topic/timw/vcpkg:
  Fix searching for libpcap installed by vcpkg
  Switch all of the conan configuration to vcpkg
2024-02-02 15:27:40 -07:00
Tim Wojtulewicz
a610b8488d Fix searching for libpcap installed by vcpkg 2024-02-02 14:52:16 -07:00
Tim Wojtulewicz
674816a01d Switch all of the conan configuration to vcpkg 2024-02-02 14:52:16 -07:00
Christian Kreibich
9da9d0d7a8 Merge branch 'topic/christian/docs-fix'
* topic/christian/docs-fix:
  Fix markup typo in the Intel::seen_policy hook docstring [skip ci]
2024-02-02 13:50:20 -08:00
Christian Kreibich
2cbc41a70c Fix markup typo in the Intel::seen_policy hook docstring [skip ci] 2024-02-02 12:30:39 -08:00
Tim Wojtulewicz
104116e782 Merge remote-tracking branch 'origin/topic/timw/update-c-ares'
* origin/topic/timw/update-c-ares:
  Configure c-ares before libkqueue
  Update 3rdparty submodule to update sqlite to 3.45.0
  Upgrade rapidjson to current upstream master
  Upgrade c-ares to 1.26.0
2024-02-01 11:22:18 -07:00
Tim Wojtulewicz
9cabff1da6 Configure c-ares before libkqueue
On platforms without a native libkqueue, c-ares is using the existing
value for HAVE_KQUEUE that was set during the libkqueue setup. We don't
pass the libkqueue information down to the c-ares cmake run so it won't
have the paths or library when it builds.
2024-02-01 10:06:28 -07:00
Tim Wojtulewicz
4e6faea8fe Update 3rdparty submodule to update sqlite to 3.45.0 2024-01-31 10:45:45 -07:00
Tim Wojtulewicz
ad712d65de Upgrade rapidjson to current upstream master 2024-01-31 10:45:45 -07:00
Tim Wojtulewicz
4c864beeec Upgrade c-ares to 1.26.0 2024-01-31 10:45:45 -07:00
Arne Welzel
072fe660bc Merge remote-tracking branch 'origin/topic/awelzel/spicy-diff-remove-timestamps-fix'
* origin/topic/awelzel/spicy-diff-remove-timestamps-fix:
  spicy/diff-remove-timestamp: Fix missing -e
2024-01-31 14:27:59 +01:00
Arne Welzel
3f7d225cee spicy/diff-remove-timestamp: Fix missing -e
This got lost when converting to sed -E.
2024-01-31 14:06:32 +01:00
zeek-bot
86c91a6ec9 Update doc submodule [nomail] [skip ci] 2024-01-31 00:15:04 +00:00
Arne Welzel
42d05d2e09 Merge remote-tracking branch 'origin/topic/awelzel/fix-server-initial-with-ack'
* origin/topic/awelzel/fix-server-initial-with-ack:
  quic: Log client's source connection id, too.
  quic: Continue decryption attempts for later INITIAL packets
  quic: Rename set_conn() to set_session()
2024-01-30 21:47:16 +01:00
Arne Welzel
1c516f738c quic: Log client's source connection id, too.
Seem reasonable give we log the server SCID. Interestingly, the Chromium
examples actually have zero length (empty) source connection IDs. I wonder
if that's part of their "protocol ossification avoidance" effort.
2024-01-30 21:46:38 +01:00
Arne Welzel
33acfdc630 quic: Continue decryption attempts for later INITIAL packets
The original logic stopped decrypting any INITIAL packets after the
first. The Firefox/cloudflare pcaps actually show that the server
replies with a QUIC INITAL packet containing just ACK frames and no
CRYPTO frames. Only the second QUIC INITIAL packet from the server
then contains the CRYPTO frames.

There's no good reason to stop decryption attempts, either we succeed
down the road and then stop, or we fail and raise analyzer violations.
2024-01-30 21:46:38 +01:00
Arne Welzel
ac524b4a33 quic: Rename set_conn() to set_session()
set_conn() should be about the c$conn record. Most other base scripts
for protocols use set_session(), so do the same.
2024-01-30 21:46:38 +01:00
Robin Sommer
5c3bb6cd97
Merge remote-tracking branch 'origin/topic/robin/bump-spicy'
* origin/topic/robin/bump-spicy:
  Bump Spicy to current `main`.
2024-01-29 10:30:43 +01:00
Robin Sommer
1837e0289a
Bump Spicy to current main.
This requires a few baseline updates, and a tweak to one test
canonifier.
2024-01-29 09:59:46 +01:00
Christian Kreibich
318f8295ea Merge branch 'topic/christian/mmdb-fix'
* topic/christian/mmdb-fix:
  Move GeoIP availability test in btests to `zeek-config --have-geoip`
  Fix MMDB::Lookup() to check result status correctly
  Add btest for succeeding/failing IPv4/IPv6 lookups
  Add an IPv6 range to the test MMDB DBs
2024-01-26 17:14:18 -08:00
Christian Kreibich
173f5e7137 Move GeoIP availability test in btests to zeek-config --have-geoip 2024-01-26 14:27:19 -08:00
Tim Wojtulewicz
5efaae0a7e Merge remote-tracking branch 'origin/topic/timw/windows-openssl-version'
* origin/topic/timw/windows-openssl-version:
  Force Windows CI to use OpenSSL 3.1.1
2024-01-26 15:08:47 -07:00
Tim Wojtulewicz
5a7c482a0f Force Windows CI to use OpenSSL 3.1.1
There's something wrong with chocolatey's OpenSSL 3.2.0 package that
causes cmake to not be able to find libcrypto even though it's clearly
in the directory. Pinning to 3.1.1 fixes the build issue.
2024-01-26 13:10:51 -07:00
Christian Kreibich
c2e791287e Fix MMDB::Lookup() to check result status correctly
This function confused checking the return value of MMDB_lookup_sockaddr() with
testing the value of the returned result.found_entry bit when that call
succeeds. Both need to happen.
2024-01-25 23:54:22 -08:00
Christian Kreibich
1044f4db0d Add btest for succeeding/failing IPv4/IPv6 lookups
The existing tests happen to test only succeeding IP addresses.
2024-01-25 23:54:20 -08:00
Christian Kreibich
975a6ae981 Add an IPv6 range to the test MMDB DBs 2024-01-25 23:43:34 -08:00
zeek-bot
4d6ec9a533 Update doc submodule [nomail] [skip ci] 2024-01-26 00:23:47 +00:00
Arne Welzel
b586b59b69 Merge remote-tracking branch 'origin/topic/vern/zam-rec-constr-opt2'
* origin/topic/vern/zam-rec-constr-opt2:
  ZAM optimizations for record creation
2024-01-25 20:55:16 +01:00
Vern Paxson
91cab9931d ZAM optimizations for record creation
includes reworking of managing "auxiliary" information for ZAM instructions
2024-01-25 20:49:12 +01:00
Arne Welzel
9f94360cfc testing/external: Revert commit hash for zeek-testing 2024-01-25 12:38:47 +01:00
Arne Welzel
e318b82032 Merge remote-tracking branch 'origin/topic/awelzel/intel-seen-hook'
* origin/topic/awelzel/intel-seen-hook:
  Intel: Introduce Intel::seen_policy() hook
2024-01-25 12:27:34 +01:00
Arne Welzel
62e0dc94db Intel: Introduce Intel::seen_policy() hook
This introduces a new hook into the Intel::seen() function that allows
users to directly interact with the result of a find() call via external
scripts.

This should solve the use-case brought up by @chrisanag1985 in
discussion #3256: Recording and acting on "no intel match found".

@Canon88 was recently asking on Slack about enabling HTTP logging for a
given connection only when an Intel match occurred and found that the
Intel::match() event would only occur on the manager. The
Intel::match_remote() event might be a workaround, but possibly running a
bit too late and also it's just an internal "detail" event that might not
be stable.

Another internal use case revolved around enabling packet recording
based on Intel matches which necessarily needs to happen on the worker
where the match happened. The proposed workaround is similar to the above
using Intel::match_remote().

This hook also provides an opportunity to rate-limit heavy hitter intel
items locally on the worker nodes, or even replacing the event approach
currently used with a customized approach.
2024-01-25 12:22:47 +01:00
Arne Welzel
ba510f018c Merge remote-tracking branch 'origin/topic/awelzel/websocket-continuation-opcode-fix'
* origin/topic/awelzel/websocket-continuation-opcode-fix:
  websocket: Fix opcode for continuation frames
2024-01-25 12:21:37 +01:00
zeek-bot
e3717f5476 Update doc submodule [nomail] [skip ci] 2024-01-25 00:12:17 +00:00
Christian Kreibich
46eba4811e Merge branch 'topic/neverlord/broker-variant-prep'
* topic/neverlord/broker-variant-prep:
  Prepare Broker manager for broker::variant
2024-01-24 15:25:27 -08:00
Dominik Charousset
4e3acfe8fc Prepare Broker manager for broker::variant
- With `broker::data`, we always have actual `std::string` objects that
  we can pass to C functions expecting a null-terminated string.
  However, `broker::variant` will return a `std::string_view` where we
  have previously received a `std::string`. Hence, we add an extra level
  of indirection that ensures that views are converted to
  null-terminated strings and also use `c_str()` where we have
  previously used `data()`. The former is not present on a
  `std::string_view`. Using this member function instead acts as an
  extra level of insurance that we do not accidentally pass the bytes
  from a view to a C function.
- Switch from error and status views to actual error and status objects.
  The view types from Broker only work with `broker::data` and thus
  won't be available with `broker::variant`.
2024-01-24 15:13:56 -08:00
Arne Welzel
a16179eae7 Merge remote-tracking branch 'origin/topic/awelzel/sed-E-everywhere'
* origin/topic/awelzel/sed-E-everywhere:
  testing/scripts: Use sed -E everywhere
2024-01-24 23:05:01 +01:00
Arne Welzel
77ef7f4ad9 Merge remote-tracking branch 'origin/topic/awelzel/bdat-websocket-fixlets'
* origin/topic/awelzel/bdat-websocket-fixlets:
  HTTP: Coverity std::move suggestion
  WebSocket: Coverity std::move suggestion
  SMTP/BDAT: Fix SonarQube reported issues
2024-01-24 23:04:27 +01:00
Arne Welzel
fb7799bdf6 websocket: Fix opcode for continuation frames
A continuation frame has the same type as the first frame, but that
information wasn't used nor kept, resulting payload of continuation
frames not being forwarded. The pcap was created with a fake Python
server and a bit of message crafting.
2024-01-24 22:57:24 +01:00
Arne Welzel
73458927ba testing/scripts: Use sed -E everywhere
I'm always a bit worried to use sed -E anywhere, because the canonifiers
give the impression it won't work everywhere consistently. My manpage says
sed -E should be preferred for portability, so lets remove the
sed -r / sed -E differentiation assuming it's just a thing from the past.
2024-01-24 11:19:17 +01:00
Arne Welzel
11e0322f0f HTTP: Coverity std::move suggestion 2024-01-24 10:50:42 +01:00
Arne Welzel
96376fe487 WebSocket: Coverity std::move suggestion 2024-01-24 10:50:42 +01:00
Arne Welzel
b5b30eb58d SMTP/BDAT: Fix SonarQube reported issues 2024-01-24 10:50:04 +01:00