This hooks into Telemetry::sync() to update Broker-level metrics tracking the
peerings' send buffer state. We do this in the cluster framework so we can label
the resulting metrics with Zeek cluster node names, not Broker's endpoint IDs.
This implements basic counts for each peering's fill level and overflows. We
don't use proper telemetry here since doing so makes more sense at the Zeek
cluster level, so we can label updates with cluster node names. See subsequent
commit for that part.
On very busy machines the hardwired scheduling of the ping batches could move
around among the arriving pongs, causing baseline deviations. We now wait for
each batch to complete before triggering the next one.
If the server doesn't include a responseToken within negTokenResp,
assume there won't be signing or sealing happening on the
connection. Don't switch into MS_KRB5 mode.
Closes#4275
* origin/topic/awelzel/generic-metadata-pre-work:
cluster/Backend: Add name and lookup component tag
cluster/Event: Hide members behind accessors
cluster/PublishEvent:: Make event non-const
broker/Manager: Re-use broker serializer for conversion
EventMgr: Add Dispatch() with handler and args
plugin/Manager: Fix MetaHookPre and MetaHookPost using HOOK_CALL_FUNCTION
For scheduled events, the event timestamp is the intended timestamp. If
we force timer expiration, the timestamp might be in the future. Today,
this happens on shutdown. This change guarantees that event timestamps
are never set beyond network time.
Allow users to call event_mgr.Dispatch(handler, args) instead of
constructing the Event instance themselves. Deprecate the old API
and replace users.
There's a subtle change that net_done() may be propagated via
auto_publish() now, but that still needs opt-in from script land
and likely no one did that, or else they'd expected to have it
work anyhow.
This test is very sensitive to new enum values that perturb the final
order of the output. Listing the commits that modify that file in the
past clearly shows changes to the baseline happen when a new component
with an enum value is introduced.
40f04d4ccf broker: Add WebSocketShim backend
51836d08ae protocol: Add StreamEvent analyzer
fdde1e9841 cluster/serializer: Add binary-serialization-format
6fb73aa9da broker: Add shim plugin adding a backend component
85ca59484b postgresql: Initial parser implementation
efc2681152 WebSocket: Introduce new analyzer and log
...
That's just distraction. Point fix the test to sort the analyzers before
writing the final log line.
I've previously attempted this in a more complete fashion in PR #3365
(issue #3364), but seems this direct fix might just make more sense.
This test has become flaky after #4295. It hasn't been failing regularly
enough locally to make it approachable. Annotate with
BTEST_BG_RUN_SLEEP=1 to revert to pre #4295 behavior, hopefully
fixing the flakiness as a side-effect.
* origin/topic/timw/4281-unconditionally-forward-out-of-snap:
Make SNAP analyzer use both OUI and protocol for forwarding
Change packet analyzer identifiers to be 64-bit
The logic was template-based so far, which wasn't great because: (1)
conceptually, it models the Spicy types at the wrong layer (C++ rather
than HILTI types), and (2) stopped working with some recent Spicy
updates (which we have temporarily reverted in the meantime to keep
Zeek working).
The new code is based on HILTI's runtime type information and the
corresponding introspection API, pretty much like `spicy-dump` works
as well. This is the recommended approach for working with HILTI
values, and generally much cleaner.
This is on top of https://github.com/zeek/zeek/pull/4300.
Calling collect_metrics() from a script would not invoke metric
callbacks, resulting in most of the process metrics to be zero
when a Zeek process isn't scraped via Prometheus.
Fixes#4309
Due to prefix matching, worker-1's node_topic() also matched worker-10,
worker-11, etc. Suffix the node topic with a `.`. The original implementation
came from NATS, where subjects are separated by `.`.
Adapt nodeid_topic() for consistency.
These test were very sensible to the speed at which ZeroMQ distributes
subscriptions in the cluster and showed to be unreliably when testing with
zeek/btest#113.
The main fix here is to have individual WebSocket clients subscribe to unique
topics, e.g /test/client-0 and /test/client-1, instead of just a shared topic.
This ensures the WebSocket handshake completes only when they observed their
own subscriptions and not prematurely when observing the shared topic.
This seems mainly relevant for tests: In the real world one shouldn't
rely on subscription visibility - you miss messages if you're too late
to the party.
When two workers connect to zeek.cluster.worker, the central ZeroMQ
proxy would not propagate unsubscription information to other nodes
once they both left. Set ZMQ_XPUB_VERBOSER on the proxies XPUB socket
for visibility.
This btest uses the exit() BIF to shut down, which immediately calls
::exit() and kills Zeek without doing any shutdown. This will sometimes
leave the thread running the storage manager, which causes TSan to
complain about a thread leak. Switch to use the terminate() BIF instead
which cleanly shuts down all of Zeek.
Bump to the latest btest master version and set BTEST_BG_RUN_SLEEP
environment variable and to 0. This makes btest-bg-run return
immediately instead of delaying by 1 second.
* origin/topic/vern/CPP-maint.Mar25:
minor baseline updates for -O gen-C++
-O gen-C++ support for pattern-to-pattern comparison
-O gen-C++ support for pattern vector comparisons
-O gen-C++ support for type expressions
* origin/topic/johanna/ssh-server-banners-can-be-wild:
SSH analyzer - tiny aesthetic fixes
SSH: make banner parsing more robust
SSH: split banner into client/server parts
These functions reside in base/utils/addrs.zeek and were not tested. We don't
use has_valid_octets() anywhere right now, normalize_mac() is used in the RADIUS
analyzer, and addr_to_uri() is used in the signatures/dpd.zeek tests.
