Add full support for RFC 9460's SvcParams list.
Amend the existing `dns_svcb_rr` record by a vector of new
`dns_svcb_param` records containing aptly typed SvcParamKey and
SvcParamValue pairs. Example output:
```
@load base/protocols/dns
event dns_HTTPS( c: connection , msg: dns_msg , ans: dns_answer , https: dns_svcb_rr ) {
for (_, param in https$svc_params)
print to_json(param); # filter uninitialised values
}
```
```
$ dig https cloudflare-ech.com +short | tr [:space:] \\n
1
.
alpn="h3,h2"
ipv4hint=104.18.10.118,104.18.11.118
ech=AEX+DQBBHgAgACBGL2e9TiFwjK/w1Zg9AmRm7mgXHz3PjffP0mTFNMxmDQAEAAEAAQASY2xvdWRmbGFyZS1lY2guY29tAAA=
ipv6hint=2606:4700::6812:a76,2606:4700::6812:b76
```
```
{"key":1,"alpn":["h3","h2"]}
{"key":4,"hint":["104.18.10.118","104.18.11.118"]}
{"key":5,"ech":"AEX+DQBBHgAgACBGL2e9TiFwjK/w1Zg9AmRm7mgXHz3PjffP0mTFNMxmDQAEAAEAAQASY2xvdWRmbGFyZS1lY2guY29tAAA="}
{"key":6,"hint":["2606:4700::6812:a76","2606:4700::6812:b76"]}
```
Values with malformed data or belonging to invalid/reserved keys
are passed raw bytes in network order for script-level inspection.
Follow up to "Initial Support to DNS SVCB/HTTPS RR"
https://github.com/zeek/zeek/pull/1808
While we support initializing records via coercion from an expression
list, e.g.,
local x: X = [$x1=1, $x2=2];
this can sometimes obscure the code to readers, e.g., when assigning to
value declared and typed elsewhere. The language runtime has a similar
overhead since instead of just constructing a known type it needs to
check at runtime that the coercion from the expression list is valid;
this can be slower than just writing the readible code in the first
place, see #4559.
With this patch we use explicit construction, e.g.,
local x = X($x1=1, $x2=2);
By default, dns_skip_all_addl is set to false. This causes several
events to not be raised. This change emits warnings when a user defines
event handlers for events that will not be raised.
Furthermore, it adds notes about this behavior to the documentation. We
also introduce a new BIF, `is_event_handled`, which checks if an event
is handled.
Fixes GH-4061
There was a misunderstanding whether to include them by default in
the dns.log, so remove them again.
There had also been a discussion and quirk that AD of a request would
always be overwritten by reply in the dns.log unless the reply is
missing. For now, let users extend dns.log themselves for what best
fits their requirements, rather than adding these flags by default.
Add a btest to print AD and CD flags for smoke testing still.
* 'dnssec-flag-parse' of github.com:micrictor/zeek-codespace:
Update external testing commit hash for DNS flag changes
Parse DNSSEC AD and CD bits
Updated dump-events baseline which seemed unrelated.
Parse authentic data (AD) and checking disabled (CD) bits according to
RFC 2535. Leaves the Z field as-is, in case users are already handling
this elsewhere and depend on the value being the integer for all 3 bits.
https://www.rfc-editor.org/rfc/rfc2535#section-6.1Fixes#2672
Merge includes small changes, e.g. fixing the comsumption of remaining
raw data.
* 'master' of https://github.com/FlyingWithJerome/zeek:
remove excussive fields in dns_svcb_rr
address code reviews (formatting and type and intrusiveptr)
newlines at the end of test outputs
lazy commit
use tabs in init-bare.zeek
add svcb test case
add a dns https test case
remove test logs
fix a few syntax errors
initial commit for SVCB/HTTPS records
Fixes to `decode_netbios_name`:
* Improve validation that input string is a NetBIOS encoding
(32 bytes, with characters ranging from 'A' to 'P'). This helps
prevent Undefined Behavior of left-shifting negative values.
Invalid encodings now cause a return-value of an empty string.
* More liberal in what decoded characters are allowed. Namely,
spaces are now allowed (but any trailing null-bytes and spaces
are trimmed, similar to before).
Fixes to `decode_netbios_name_type`:
* Improve validation that input string is a NetBIOS encoding
(32 bytes, with characters ranging from 'A' to 'P'). This helps
prevent Undefined Behavior of left-shifting negative values and
a heap-buffer-overread when the input string is too small.
Invalid encodings now cause a return-value of 256.
This adds a "policy" hook into the logging framework's streams and
filters to replace the existing log filter predicates. The hook
signature is as follows:
hook(rec: any, id: Log::ID, filter: Log::Filter);
The logging manager invokes hooks on each log record. Hooks can veto
log records via a break, and modify them if necessary. Log filters
inherit the stream-level hook, but can override or remove the hook as
needed.
The distribution's existing log streams now come with pre-defined
hooks that users can add handlers to. Their name is standardized as
"log_policy" by convention, with additional suffixes when a module
provides multiple streams. The following adds a handler to the Conn
module's default log policy hook:
hook Conn::log_policy(rec: Conn::Info, id: Log::ID, filter: Log::Filter)
{
if ( some_veto_reason(rec) )
break;
}
By default, this handler will get invoked for any log filter
associated with the Conn::LOG stream.
The existing predicates are deprecated for removal in 4.1 but continue
to work.
This adds two new functions: `Conn::register_removal_hook()` and
`Conn::unregister_removal_hook()` for registering a hook function to be
called back during `connection_state_remove`. The benefit of using hook
callback approach is better scalability: the overhead of unrelated
protocols having to dispatch no-op `connection_state_remove` handlers is
avoided.
Doesn't typically cause any problems since the loop breaks after
deleting, except there's now an assert in debug builds catching
potential problems like this.
And switch Zeek's base scripts over to using it in place of
"connection_state_remove". The difference between the two is
that "connection_state_remove" is raised for all events while
"successful_connection_remove" excludes TCP connections that were never
established (just SYN packets). There can be performance benefits
to this change for some use-cases.
There's also a new event called ``connection_successful`` and a new
``connection`` record field named "successful" to help indicate this new
property of connections.
SPF response records are identical to TXT records in structure, and
can be parsed and interpreted the same way. However, they have a
different RR type, so they would generate weird events and not be
parsed by Zeek before this change.
Even though they're the same as TXT records from a protocol stance, I
created a new event type (dns_SPF_reply), and call the records out as
SPF in the logs, instead of as TXT records, since the distinction
could be important for detection purposes.
SPF records have been obsoleted, but continue to be seen in the wild.
* All "Broxygen" usages have been replaced in
code, documentation, filenames, etc.
* Sphinx roles/directives like ":bro:see" are now ":zeek:see"
* The "--broxygen" command-line option is now "--zeexygen"
get_conn_transport_proto needs to use sessions->FindConnection and do a
hash lookup to find the connection while get_port_transport_proto just
looks at the port directly.
* origin/master:
Fix typo in Sessions.h
Update baselines for SSH capabilities fix
Fix SSH analyzer bug where is_server in capabilities is wrong.
Revert DNS query 255 from ANY to *
Missing commas...
Fix typo in dce-rpc consts
Refactor to use consistent numeric type in dce_rpc. Add missing DNS query type codes
Add DCE_RPC exchange_mapi operations to relevant consts.bro file
GH-186: fix JSON formatting of timestamps before Unix epoch
Fix test baseline for plugin skeleton update
Fix crash when modifying a table from within its &expire_func
Convert site::local_nets, etc. into options.
GH-184: add `bro-config --build_type`, outputs CMake build type
Updating submodule(s).
Add return value checks for some RPC parsing functions
Add 'fallthrough' comment to a switch/case block
Improve broker.remote_id unit test
Increase broker unit test timeout intervals
* 'master' of https://github.com/spitfire55/bro:
Revert DNS query 255 from ANY to *
Missing commas...
Fix typo in dce-rpc consts
Refactor to use consistent numeric type in dce_rpc. Add missing DNS query type codes
I added back in DNS constants for PTR, EDNS, and ANY to avoid breaking
code for any people that use them.
Also omitted the DNP3 function code 0x83 name change from
"AUTHENTICATE_RESP" to "AUTHENTICATE_RESPONSE", again to avoid
potentially breaking code unnecessarily: "RESP" vs. "RESPONSE" is not
wrong in any sense, just maybe a matter of clarify.
* 'fatemabw/bro' of https://github.com/fatemabw/bro:
DNSSEC support in Bro
I made several changes:
- renamed event/record types
- reformatted the info added to dns.log
- removed the "addl" scripts that added extended dnssec info to dns.log
- simplifications/improvements to the internal parsing logic
