Merge remote-tracking branch 'origin/topic/awelzel/pluggable-cluster-backends-part4'

* origin/topic/awelzel/pluggable-cluster-backends-part4:
  Update ZAM BiF-tracking
  configure: Add --disable-cluster-backend-zeromq
  CMakeLists: Cluster backends output
  zeromq: Conditionally enable by default
  btest/generic: Add publish_hrw(), publish_rr() and logging tests
  generate-docs: Run on Ubuntu 24.04, add cppzmq
  docker: Add cppzmq/libzmq dependencies
  tsan_suppressions: Add called_from_lib: libzmq
  ci: Add cppzmq and libzmq to most platforms
  cluster/backend/zeromq: Add ZeroMQ based cluster backend
  cluster/backend/zeromq: Add cppzmq submodule
  cluster: Add Cluster::node_id(), allow redef of node_topic(), nodeid_topic()
  cluster: Move publish_hrw() and publish_rr() to cluster.bif
This commit is contained in:
Arne Welzel 2024-12-11 09:28:04 +01:00
commit 4ee2f9256b
89 changed files with 3035 additions and 109 deletions

24
NEWS
View file

@ -39,6 +39,12 @@ New Functionality
This entire feature can be disabled by loading the new
``policy/protocols/conn/disable-unknown-ip-proto-support.zeek`` policy script.
- New ``Cluster::publish()``, ``Cluster::subscribe()`` and ``Cluster::unsubscribe()``
functions have been added. In contrast to their ``Broker`` counterparts, these
will operator on whichever cluster backend is enabled. Going forward, in-tree
``Broker::publish()`` usages will be replaced with ``Cluster::publish()`` and
script writers should opt to prefer these over the Broker-specific functions.
- Zeek now includes a PostgreSQL protocol analyzer. This analyzer is enabled
by default. The analyzer's events and its ``postgresql.log`` should be
considered preliminary and experimental until the arrival of Zeek's next
@ -148,6 +154,24 @@ New Functionality
This comes with performance caveats: For use-cases with high-data rates
a native protocol analyzer with dedicated events will be far more efficient.
- Experimental support for pluggable cluster backends has been added. New plugin
components have been introduced to support switching Zeek's Broker-based
publish-subscribe and remote logging functionality to alternative implementations.
redef Cluster::backend = Cluster::CLUSTER_BACKEND_ZEROMQ;
Besides the backend, the serialization format used for events and log-writes
has become pluggable as well.
- The Zeek distribution now includes an experimental ZeroMQ based cluster backend.
To experiment with it, load the following script on each cluster node.
@load frameworks/cluster/backend/zeromq/connect
Note that Broker-dependent scripts or integrations will become non-functional
when doing so as Zeek nodes will not listen on Broker ports anymore, nor will
they establish a peering to other nodes.
Changed Functionality
---------------------