This is a bit annoying: The logging framework looks at Cluster::backend
to decide what should happen with log writes and only do the right thing
if it was switched to broker.
Now that Cluster::backend defaults to NONE by default, some tests won't
just work anymore for various reasons (mostly due to the logging
framework not sending log writes to broker anymore).
Load the new frameworks/cluster/backend/broker package for these to keep them
functional. Also add the @load to the common broker/cluster-layout.zeek file
to avoid changing all of the files that use that layout.
Now that all the logic about establishing connections among nodes has
moved to the broker policy script, the setup-connections name seems
unfortunate, transition away from it.
Allow access to the Cluster's subscribe(), unsubscribe(), publish(),
publish_hrw() and publish_rr() methods by loading only the
base/frameworks/cluster/pubsub, rather than everything that
__load__.zeek or also main.zeek pulls in.
This can be used by other scripts to use these functions without
relying or expecting the rest of the cluster framework to be loaded
already. Concretely, this is needed to move the Supervisor framework
from Broker to Cluster.
They seem mostly irrelevant for the tests and depending on what is
logged, the num_logs_outgoing changes, so just stop including them
in the baseline for now.
If Cluster::init() hasn't been invoked yet, Cluster::subscribe() with the
ZeroMQ backend would block because the main_inproc socket didn't
yet have a connection from the child thread. Prevent this by connecting
the main and child socket pair at construction time.
This will queue the subscriptions and start processing them once the
child thread has started.
An URI containing a bracketed or non-bracketed IPv6 address of the form
http://[::1]:42 was previously split on the first colon for port extraction,
causing a subsequent to_count() call to fail. Harden this to check for a
digits in the last :[0-9]+ component.
Fixes#4842
Add a test/example forwarding all new_connection() events produced during
`zeek -r wikipedia.trace` as my_new_connection() to a WebSocket client.
This is mostly to demonstrate and verify usage of suspend_processing(),
websocket_client_added(), resume_processing(), Pcap::file_done(),
websocket_client_lost() and terminate() together.
* origin/topic/vern/emphasize-IDPtrs:
fixup! fixup! shift much of the internal use of ID* identifier pointers over to IDPtr objects
fixup! shift much of the internal use of ID* identifier pointers over to IDPtr objects
fixup! shift much of the internal use of ID* identifier pointers over to IDPtr objects
fixup! shift much of the internal use of ID* identifier pointers over to IDPtr objects
annotate a number of BTests as unsuited for -O gen-C++ testing due to multiple Zeek runs
BTest baseline updates for -O gen-C++ - all minor tweaks
BTest updates for script optimization tracking of BiFs
regression test for former ASAN issue with script optimization of lambdas
shift much of the internal use of ID* identifier pointers over to IDPtr objects
maintenance update for script optimization's knowledge of BiFs
logger fix for interoperability with -O gen-C++ code
Add full support for RFC 9460's SvcParams list.
Amend the existing `dns_svcb_rr` record by a vector of new
`dns_svcb_param` records containing aptly typed SvcParamKey and
SvcParamValue pairs. Example output:
```
@load base/protocols/dns
event dns_HTTPS( c: connection , msg: dns_msg , ans: dns_answer , https: dns_svcb_rr ) {
for (_, param in https$svc_params)
print to_json(param); # filter uninitialised values
}
```
```
$ dig https cloudflare-ech.com +short | tr [:space:] \\n
1
.
alpn="h3,h2"
ipv4hint=104.18.10.118,104.18.11.118
ech=AEX+DQBBHgAgACBGL2e9TiFwjK/w1Zg9AmRm7mgXHz3PjffP0mTFNMxmDQAEAAEAAQASY2xvdWRmbGFyZS1lY2guY29tAAA=
ipv6hint=2606:4700::6812:a76,2606:4700::6812:b76
```
```
{"key":1,"alpn":["h3","h2"]}
{"key":4,"hint":["104.18.10.118","104.18.11.118"]}
{"key":5,"ech":"AEX+DQBBHgAgACBGL2e9TiFwjK/w1Zg9AmRm7mgXHz3PjffP0mTFNMxmDQAEAAEAAQASY2xvdWRmbGFyZS1lY2guY29tAAA="}
{"key":6,"hint":["2606:4700::6812:a76","2606:4700::6812:b76"]}
```
Values with malformed data or belonging to invalid/reserved keys
are passed raw bytes in network order for script-level inspection.
Follow up to "Initial Support to DNS SVCB/HTTPS RR"
https://github.com/zeek/zeek/pull/1808
* origin/topic/timw/readability-container-contains:
Fix a few more random clang-tidy findings
Use std::numbers::pi instead of hard-coded value
Use std::scoped_lock instead of std::lock_guard
Use .contains() instead of .find() or .count()
We install test data which we also make available under an alternative
path for backwards compatibility. The installation of this symlink did
not take `DESTDIR` installs like used by Zeek's packaging into account
which caused installations from packages to behave different from
installs from source.
This patch fixes the symlink to respect a possible `DESTDIR`.
Closes#3266.
When a WebSocket client sends an event to Zeek without explicit network
timestamp metadata, Zeek would use -1.0 as a timestamp for any events
published while handling this event. Instead, it seems far more sensible
to use the current network time in that scenario.
Event when there's no metadata attached to an event, we'd still use the
constructor passing an empty metadata vector, resulting in an on-the-wire
representation with an empty trailing vector.
Particularly visible when just snooping events via websocat. There also
seems to be some bug with the timestamp -1 handling.
* 'master' of https://github.com/blightzero/zeek:
Changed behavior of var-extraction-uri.zeek from policy/protocol/http to extract only the URI parameter names. Do not include the path in the first parameter name. Only extract uri vars if parameters actually exist.
Previously, Zeek treated the receipt of `AuthenticationOk` as a
successful login. However, according to the PostgreSQL
Frontend/Backend Protocol, the startup phase is not complete until
the server sends `ReadyForQuery`. It is still possible for the server
to emit an `ErrorResponse` (e.g. ERRCODE_INVALID_AUTHORIZATION_SPECIFICATION)
after `AuthenticationOk` but before `ReadyForQuery`.
This change updates the PostgreSQL analyzer to defer reporting login
success until `ReadyForQuery` is observed. This prevents false
positives in cases where authentication succeeds but session startup
fails.
The previous approach ignored the fact that nested / inner values might
also be Broker::Data values. I'm not super sure about the validity of
the test, because it's essentially demonstrating any-nesting, but
it's not leading to extra Broker::Data encoding.
Parameters relied on is_record for a couple of validations, but they are
not records and should not be treated as such. This way we can validate
&optional better.
There was some confusing behavior with &optional and locals, so this
should get rid of that by making it an error. However, there is a case
where function parameters are still allowed to have &optional - this is
because there are checks for &default in parameters as well.
