* origin/topic/timw/storage-timeout-testing:
Fix handling of timeout conditions from storage backends
Reformat plugin.storage btest to be more consistent with other storage tests
Calling collect_metrics() from a script would not invoke metric
callbacks, resulting in most of the process metrics to be zero
when a Zeek process isn't scraped via Prometheus.
Fixes#4309
The busy polling wasn't clever and usually resulted in delays. For now,
switch to mutex/condition variable and log an error if the timeouts are
immense.
Due to prefix matching, worker-1's node_topic() also matched worker-10,
worker-11, etc. Suffix the node topic with a `.`. The original implementation
came from NATS, where subjects are separated by `.`.
Adapt nodeid_topic() for consistency.
WebSocket clients that connected with the wrong URL do not have
a backend attached. If a dispatcher is terminated while these
clients are still connected, a null deref would happen.
This was found while running all cluster/websocket tests in a loop
for a long time, tickling a segfault during the bad-url test.
These test were very sensible to the speed at which ZeroMQ distributes
subscriptions in the cluster and showed to be unreliably when testing with
zeek/btest#113.
The main fix here is to have individual WebSocket clients subscribe to unique
topics, e.g /test/client-0 and /test/client-1, instead of just a shared topic.
This ensures the WebSocket handshake completes only when they observed their
own subscriptions and not prematurely when observing the shared topic.
This seems mainly relevant for tests: In the real world one shouldn't
rely on subscription visibility - you miss messages if you're too late
to the party.
When two workers connect to zeek.cluster.worker, the central ZeroMQ
proxy would not propagate unsubscription information to other nodes
once they both left. Set ZMQ_XPUB_VERBOSER on the proxies XPUB socket
for visibility.
* origin/topic/timw/storage-framework-followup:
Redis: Handle other errors from requests, fix KEY_EXISTS for put operations
SQLite: handle existing keys when overwrite=F correctly
Remove unnecessary type aliases from storage btests
Avoid thread-leak in scripts.base.frameworks.file-analysis.bifs.enable-disable btest
Fix data-race with calling DBG_LOG from a separate thread
Fix data-race with ReturnCode objects in Sqlite::DoExpire
Fix data race with calling run_state::network_time from a separate thread
Add NEWS entry for Storage, reduce CHANGES spam
Fix Coverity findings in the SQLite backend
Remove inclusion of non-existent expected-lite path during dynamic plugin builds
Squash code from OperationResultCallback into ResultCallback
Add hiredis to generate-docs workflow to enable Redis backend
This btest uses the exit() BIF to shut down, which immediately calls
::exit() and kills Zeek without doing any shutdown. This will sometimes
leave the thread running the storage manager, which causes TSan to
complain about a thread leak. Switch to use the terminate() BIF instead
which cleanly shuts down all of Zeek.
Bump to the latest btest master version and set BTEST_BG_RUN_SLEEP
environment variable and to 0. This makes btest-bg-run return
immediately instead of delaying by 1 second.
* origin/topic/vern/CPP-maint.Mar25:
minor baseline updates for -O gen-C++
-O gen-C++ support for pattern-to-pattern comparison
-O gen-C++ support for pattern vector comparisons
-O gen-C++ support for type expressions
* origin/topic/johanna/ssh-server-banners-can-be-wild:
SSH analyzer - tiny aesthetic fixes
SSH: make banner parsing more robust
SSH: split banner into client/server parts
These functions reside in base/utils/addrs.zeek and were not tested. We don't
use has_valid_octets() anywhere right now, normalize_mac() is used in the RADIUS
analyzer, and addr_to_uri() is used in the signatures/dpd.zeek tests.
