Commit graph

19582 commits

Author SHA1 Message Date
Arne Welzel
d122894d0d cluster: Rename setup-connections to setup-subscriptions
Now that all the logic about establishing connections among nodes has
moved to the broker policy script, the setup-connections name seems
unfortunate, transition away from it.
2025-09-25 19:29:55 +02:00
Arne Welzel
351bd7b81b cluster: Move Broker connect logic into policy package
Place a package into policy/frameworks/cluster/backend/broker and move
Broker's connection specific logic there.
2025-09-25 19:29:55 +02:00
Arne Welzel
2f15f5ce6a cluster: Introduce pubsub.zeek and types.zeek
Allow access to the Cluster's subscribe(), unsubscribe(), publish(),
publish_hrw() and publish_rr() methods by loading only the
base/frameworks/cluster/pubsub, rather than everything that
__load__.zeek or also main.zeek pulls in.

This can be used by other scripts to use these functions without
relying or expecting the rest of the cluster framework to be loaded
already. Concretely, this is needed to move the Supervisor framework
from Broker to Cluster.
2025-09-25 19:29:55 +02:00
Arne Welzel
ba8305e1e6 cluster: Add None backend
Mostly so we have the CLUSTER_BACKEND_NONE enum value available, but
potentially useful for testing, too. The rough idea is that Zeek by
default has Cluster::backend set to CLUSTER_BACKEND_NONE, and script
packages in frameworks/cluster/backend redef it accordingly.
2025-09-25 19:29:55 +02:00
Arne Welzel
8777c1522d btest/broker/remote_event_ts_compat: Replace 15 second sleep with wait-for-file
Just so the test runs faster when it can.
2025-09-25 19:29:55 +02:00
Arne Welzel
e3373d31f0 btest/broker: Remove get_broker_stats() print
They seem mostly irrelevant for the tests and depending on what is
logged, the num_logs_outgoing changes, so just stop including them
in the baseline for now.
2025-09-25 19:29:55 +02:00
Arne Welzel
0ad1c78bae btest/Files/zeromq/metrics: Add missing load for telemetry framework 2025-09-25 19:29:55 +02:00
Arne Welzel
01666df3d7 cluster/zeromq: Fix Cluster::subscribe() block if not initialized
If Cluster::init() hasn't been invoked yet, Cluster::subscribe() with the
ZeroMQ backend would block because the main_inproc socket didn't
yet have a connection from the child thread. Prevent this by connecting
the main and child socket pair at construction time.

This will queue the subscriptions and start processing them once the
child thread has started.
2025-09-25 18:58:35 +02:00
Arne Welzel
d2cda5a68c Merge remote-tracking branch 'origin/topic/awelzel/zeromq-handle-eintr'
Some checks are pending
pre-commit / pre-commit (push) Waiting to run
* origin/topic/awelzel/zeromq-handle-eintr:
  cluster/zeromq: Improve EINTR handling
2025-09-25 13:53:04 +02:00
Arne Welzel
a318463c1c cluster/zeromq: Improve EINTR handling
When using ZeroMQ also within the Supervisor process, zmq::poll() and
recv() were observed to return EINTR, handle these.
2025-09-25 13:52:12 +02:00
Tim Wojtulewicz
69b7bcc323 Merge remote-tracking branch 'origin/topic/vern/no-opt'
Some checks failed
pre-commit / pre-commit (push) Has been cancelled
* origin/topic/vern/no-opt:
  BTests & baselines for testing selective skipping of script optimization
  added &no_ZAM_opt/&no_CPP_opt attributes and --no-opt-files/--no-opt-funcs for controlling skipping script optimization
2025-09-23 09:06:02 -07:00
Vern Paxson
dc7f4dca1c BTests & baselines for testing selective skipping of script optimization 2025-09-23 09:05:40 -07:00
Vern Paxson
441baaf2c1 added &no_ZAM_opt/&no_CPP_opt attributes and --no-opt-files/--no-opt-funcs for controlling skipping script optimization 2025-09-23 09:05:40 -07:00
Arne Welzel
2bde66c9d6 Merge remote-tracking branch 'origin/topic/awelzel/4842-decompose-uri-ipv6'
* origin/topic/awelzel/4842-decompose-uri-ipv6:
  utils/decompose_uri: Support URIs containing IPv6 addresses
2025-09-23 13:08:14 +02:00
Arne Welzel
b3b0c3db7d utils/decompose_uri: Support URIs containing IPv6 addresses
An URI containing a bracketed or non-bracketed IPv6 address of the form
http://[::1]:42 was previously split on the first colon for port extraction,
causing a subsequent to_count() call to fail. Harden this to check for a
digits in the last :[0-9]+ component.

Fixes #4842
2025-09-23 13:04:29 +02:00
Tim Wojtulewicz
416d997263 Merge remote-tracking branch 'origin/topic/timw/restore-settype-methods'
Some checks are pending
pre-commit / pre-commit (push) Waiting to run
* origin/topic/timw/restore-settype-methods:
  Restore the SetType constructor and destructor
2025-09-22 07:46:06 -07:00
Tim Wojtulewicz
c741e8f489 Merge remote-tracking branch 'origin/topic/timw/update-libkqueue'
* origin/topic/timw/update-libkqueue:
  Bump libkqueue to latest upstream master
2025-09-22 07:34:03 -07:00
Tim Wojtulewicz
5f8a89ea40 Bump libkqueue to latest upstream master 2025-09-22 07:33:13 -07:00
Tim Wojtulewicz
210737f36e Restore the SetType constructor and destructor 2025-09-21 17:13:14 +00:00
zeek-bot
d6c7a479e2 Update doc submodule [nomail] [skip ci] 2025-09-20 00:13:40 +00:00
Tim Wojtulewicz
9fb0f5e76d Fix clang-tidy finding in recent script_opt changes 2025-09-17 16:34:26 -07:00
Tim Wojtulewicz
ef9ffda2ef Merge remote-tracking branch 'origin/topic/vern/standalone-event-groups'
* origin/topic/vern/standalone-event-groups:
  tracking of event groups for compilation to standalone-C++
2025-09-17 14:28:44 -07:00
Vern Paxson
973cc00c42 tracking of event groups for compilation to standalone-C++ 2025-09-17 14:28:13 -07:00
Tim Wojtulewicz
1567fe8c09 Merge remote-tracking branch 'origin/topic/vern/remove-SetType'
* origin/topic/vern/remove-SetType:
  Deprecate SetType, as it can be replaced by TableType
2025-09-17 14:26:42 -07:00
Vern Paxson
b442c25389 Deprecate SetType, as it can be replaced by TableType 2025-09-17 14:09:58 -07:00
Tim Wojtulewicz
938073dfed Merge remote-tracking branch 'origin/topic/timw/4742-skip-storage-thread'
* origin/topic/timw/4742-skip-storage-thread:
  Avoid starting up storage expiration thread with zero backends
  Remove a couple of obsolete TODOs in storage manager
2025-09-17 08:44:55 -07:00
Tim Wojtulewicz
067c257480 Merge remote-tracking branch 'origin/topic/timw/windows-zeromq-support'
* origin/topic/timw/windows-zeromq-support:
  Reword comment about when ZeroMQ is required for the build
  CI: Add label for enabling the Windows build in PRs
  Enable building ZeroMQ support on Windows
2025-09-16 13:31:48 -07:00
Tim Wojtulewicz
723b64d81f Reword comment about when ZeroMQ is required for the build 2025-09-16 13:30:14 -07:00
Tim Wojtulewicz
37cc4999b3 CI: Add label for enabling the Windows build in PRs 2025-09-16 13:30:14 -07:00
Tim Wojtulewicz
4705537002 Enable building ZeroMQ support on Windows 2025-09-16 13:30:14 -07:00
Tim Wojtulewicz
5cc5f2a275 Merge remote-tracking branch 'origin/topic/timw/4285-sqlite-threadsafe-check'
* origin/topic/timw/4285-sqlite-threadsafe-check:
  Force SQLite to be in thread-safe mode during CMake
2025-09-16 12:42:04 -07:00
Tim Wojtulewicz
dd14e380a1 Force SQLite to be in thread-safe mode during CMake
This allows us to remove the need to check for thread-safe mode in the
various SQLite plugins. See https://www.sqlite.org/compile.html#threadsafe
for why `1` is a good choice here.
2025-09-16 12:19:09 -07:00
Tim Wojtulewicz
bd60c6fc15 Merge remote-tracking branch 'origin/topic/vern/gen-C++-global-characteristics'
* origin/topic/vern/gen-C++-global-characteristics:
  full tracking of the characteristics of globals when compiling scripts to C++
2025-09-16 11:31:13 -07:00
Tim Wojtulewicz
71eca7c9e6 Merge remote-tracking branch 'origin/topic/vern/standalone-CPP-global-attrs'
* origin/topic/vern/standalone-CPP-global-attrs:
  fix for associating attributes with globals for -O gen-standalone-C++
2025-09-16 11:02:44 -07:00
Tim Wojtulewicz
aebeae8f8a Merge remote-tracking branch 'origin/topic/vern/standalone-id-tracking-fix'
* origin/topic/vern/standalone-id-tracking-fix:
  fix for tracking identifiers and aggregates when compiling to standalone-C++
2025-09-16 10:53:11 -07:00
Tim Wojtulewicz
00ba3483bb Merge remote-tracking branch 'origin/topic/vern/gen-CPP-precedence-fix'
* origin/topic/vern/gen-CPP-precedence-fix:
  fix for '?' operator precedence when compiling scripts to C++
2025-09-16 10:52:53 -07:00
Vern Paxson
0e5042e451 fix for associating attributes with globals for -O gen-standalone-C++ 2025-09-15 14:28:07 -07:00
Vern Paxson
bfc6508dff full tracking of the characteristics of globals when compiling scripts to C++ 2025-09-15 14:21:32 -07:00
Vern Paxson
6a9175da7d fix for '?' operator precedence when compiling scripts to C++ 2025-09-15 14:18:16 -07:00
Tim Wojtulewicz
50581bfd3c Avoid starting up storage expiration thread with zero backends 2025-09-15 14:18:03 -07:00
Tim Wojtulewicz
8bc849d425 Remove a couple of obsolete TODOs in storage manager 2025-09-15 14:17:41 -07:00
Vern Paxson
18d435ac12 fix for tracking identifiers and aggregates when compiling to standalone-C++ 2025-09-15 13:57:35 -07:00
Tim Wojtulewicz
b25a844210 Merge remote-tracking branch 'origin/topic/vern/standalone-skip-fatal'
* origin/topic/vern/standalone-skip-fatal:
  for -O gen-standalone-C++, make the presence of uncompilable functions fatal unless -O allow-cond is used
2025-09-12 09:24:58 -07:00
Tim Wojtulewicz
489681e0c3 Merge remote-tracking branch 'origin/topic/timw/bump-zeekctl'
* origin/topic/timw/bump-zeekctl:
  Bump zeekctl for new trace-summary/pysubnettree versions
2025-09-12 09:24:26 -07:00
Tim Wojtulewicz
3750fb11f4 Bump zeekctl for new trace-summary/pysubnettree versions 2025-09-12 09:23:55 -07:00
Vern Paxson
e5210160ea for -O gen-standalone-C++, make the presence of uncompilable functions fatal unless -O allow-cond is used 2025-09-11 13:30:40 -06:00
Tim Wojtulewicz
e1c8cbe25a Merge remote-tracking branch 'origin/topic/timw/dns-analyzer-type-consistency'
* origin/topic/timw/dns-analyzer-type-consistency:
  Clean up initialization of DNS_Interpreter
  Switch types used in DNS analyzer to be more consistent
2025-09-11 10:57:29 -07:00
Tim Wojtulewicz
f056df1301 Clean up initialization of DNS_Interpreter 2025-09-11 10:56:57 -07:00
Tim Wojtulewicz
501160c976 Switch types used in DNS analyzer to be more consistent 2025-09-11 10:56:57 -07:00
Tim Wojtulewicz
c6cf1ee3ae Merge remote-tracking branch 'origin/topic/awelzel/btest-suspend-processing-websocket'
* origin/topic/awelzel/btest-suspend-processing-websocket:
  btest/core/suspend_processing: Add WebSocket example
2025-09-08 11:19:45 -07:00