Commit graph

13 commits

Author SHA1 Message Date
Arne Welzel
9c5c0f40e1 cluster/zeromq: Fix Unsubscribe() bug caused by \x00 prefix 2025-02-05 10:39:56 +01:00
Arne Welzel
6d1259423e cluster/serializer/broker: Fix handler lookup
Handler overwrites operator bool, so need to explicitly test for nullptr
rather than not having any handlers defined.
2025-02-05 10:39:56 +01:00
Arne Welzel
21e33fdcd9 btest/cluster: Bump timeouts to 30 seconds
ZAM startup may take a long time, particularly in CI environments, so
bump it up from 10 to 30 seconds.
2024-12-13 18:28:43 +01:00
Arne Welzel
fdf783df65 cluster/Backend: Handle unspecified table/set
Same as what we do in Broker. Use the expected type if publishing
a table() or set() parameter.

This fixes issues when switching sumstats to Cluster::publish()
2024-12-12 17:54:42 +01:00
Arne Welzel
d9a74cf32d cluster: Fix Cluster::publish() of Broker::Data
The broker serializer leverages the existing data_to_val() function.
During unserialization, if the destination type is any, the logic
simply wraps the broker::data value into a Broker::Data record.
Therefore, events with any parameters are currently exposed to
the Broker::Data type.

There is a bigger issue in that re-publishing such Broker::Data
instances would encode them as a normal record. Explicitly prevent
this by serializing the contained data value directly instead, similar
to what Broker already did when publishing a record.
2024-12-12 17:54:37 +01:00
Arne Welzel
0ad3210177 Broker::publish: Warn on using Broker::publish() when inactive
This is mostly for transitioning base scripts to Cluster::publish() and
avoid silent surprises why certain things don't work when using ZeroMQ.
2024-12-11 17:20:42 +01:00
Arne Welzel
d816bfb249 btest/generic: Add publish_hrw(), publish_rr() and logging tests
They currently use zeromq, but technically they should be valid for
any other backend, too, even broker.
2024-12-10 20:33:02 +01:00
Arne Welzel
35c79ab2e3 cluster/backend/zeromq: Add ZeroMQ based cluster backend
This is a cluster backend implementation using a central XPUB/XSUB proxy
that by default runs on the manager node. Logging is implemented leveraging
PUSH/PULL sockets between logger and other nodes, rather than going
through XPUB/XSUB.

The test-all-policy-cluster baseline changed: Previously, Broker::peer()
would be called from setup-connections.zeek, causing the IO loop to be
alive. With the ZeroMQ backend, the IO loop is only alive when
Cluster::init() is called, but that doesn't happen anymore.
2024-12-10 20:33:02 +01:00
Arne Welzel
210b54799e cluster: Move publish_hrw() and publish_rr() to cluster.bif
From this point on, Cluster::publish_hrw() and Cluster::publish_rr()
go through cluster/Backend.cc code.
2024-12-10 20:33:02 +01:00
Arne Welzel
fdde1e9841 cluster/serializer: Add binary-serialization-format
This is a serializer for log records that is using SerialTypes
for serializing and un-serializing rather. Essentially, this is
similar to what broker does except for the envelope.
2024-12-04 12:40:35 +01:00
Arne Welzel
9ec872d161 cluster/serializer: Add Broker based event serializers
This adds the first event serializers that use
broker functionality. Binary and JSON formats.
2024-11-26 12:58:23 +01:00
Arne Welzel
ef04a199c8 cluster: Add Cluster scoped bifs
... and a broker based test using Cluster::publish() and
Cluster::subscribe().
2024-11-26 12:58:23 +01:00
Arne Welzel
de9d39cd01 btest: Add cluster dir, minimal test for enum value 2024-11-22 10:43:55 +01:00