Anchors within pattern passed to sub() or gsub() were previously ignored,
replacing any occurrence of '<text>' even when '^<text>' was used as a
pattern.
This is a pretty user-visible change (and we even have anchored patterns
within the base scripts), but seems "the right thing to do".
Relates to #3455
Ad-hoc include module names in the global_ids() table. Table values will
have the type_name field set to "module" and their key in the table is
prefixed with "module " to avoid clashes with existing global identifiers
shadowing module names (Management::Node being an existing example).
Closes#3136
* origin/topic/vern/at-if-analyze:
updates reflecting review comments
change base scripts to use run-time if's or @if ... &analyze
a number of BTests updated with @if ... &analyze
update for scripting coverage BTest demonstrating utility of @if ... &analyze
BTests for new @if ... &analyze functionality
"if ( ... ) &analyze" language feature
classes for tracking "@if (...) &analyze" notion of code being/not being "activated"
RemoveGlobal() method for Scope class + simplifying interfaces
* amazing-pp/topic/fupeng/from_json_bif:
Implement from_json bif
Minor updates during merge: Moved ValFromJSON into zeek::detail for the
time being, removed gotos, normalized some error messages to lower case,
minimal test extension and added a raw reader input framework test reading
"json lines" as a demo, adding notes about the implicit type
conversions.
"Community ID" has become an established flow hash for connection correlation
across different monitoring and storage systems. Other NSMs have had native
and built-in support for Community ID since late 2018. And even though the
roots of "Community ID" are very close to Zeek, Zeek itself has never provided
out-of-the-box support and instead required users to install an external plugin.
While we try to make that installation as easy as possible, an external plugin
always sets the bar higher for an initial setup and can be intimidating.
It also requires a rebuild operation of the plugin during upgrades. Nothing
overly complicated, but somewhat unnecessary for such popular functionality.
This isn't a 1:1 import. The options are parameters and the "verbose"
functionality has been removed. Further, instead of a `connection`
record, the new bif works with `conn_id`, allowing computation of the
hash with little effort on the command line:
$ zeek -e 'print community_id_v1([$orig_h=1.2.3.4, $orig_p=1024/tcp, $resp_h=5.6.7.8, $resp_p=80/tcp])'
1:RcCrCS5fwYUeIzgDDx64EN3+okU
Reference: https://github.com/corelight/zeek-community-id/
It wasn't possible from script land to determine the total size
of the cache table held by the DNS_Mgr. Add the total and also
also the TEXT entries count.
This commit introduces parsing of the CertificateRequest message in the
TLS handshake. It introduces a new event ssl_certificate_request, as
well as a new function parse_distinguished_name, which can be used to
parse part of the ssl_certificate_request event parameters.
This commit also introduces a new policy script, which appends
information about the CAs a TLS server requests in the
CertificateRequest message, if it sends it.
A baseline difference between arm64 and x86 showed up. We would
print a wrong character as negative value on x86 due to chars
being signed by default. Force an unsigned interpretation which
is also more reasonable because we'd have never indexed the
base64 table with -112
-XXXXXXXXXX.XXXXXX XXXXXXXXXXX 131.243.99.154 3288 193.159.183.138 80 base64_illegal_encoding character -112 ignored by Base64 decoding F zeek -
+XXXXXXXXXX.XXXXXX XXXXXXXXXXX 131.243.99.154 3288 193.159.183.138 80 base64_illegal_encoding character 144 ignored by Base64 decoding F zeek -
Fixes more of #2742
The buf[i] < 3 condition in use previously allowed all chars (signed on x86)
through that had the 0x80 high-bit set after reconstructing from the two
bytes of the netbios name, resulting in escaped non-ascii content in
the logs.
Fixes more of #2742
When disabling_analyzer() was introduced, it was added to the GLOBAL
module. The awkward side-effect is that implementing a hook handler
in another module requires to prefix it with GLOBAL. Alternatively, one
can re-open the GLOBAL module and implement the handler in that scope.
Both are not great, and prefixing with GLOBAL is ugly, so move the
identifier to the Analyzer module and ask users to prefix with Analyzer.
When passing invalid IPs or an out-of range port to parse_eftp()
a warning or error was generated on stderr (in addition to setting
the $valid field to F). Prevent the output by adding safe-guarding
and using IPAddr::ConvertString() instead.
Using positional and vararg arguments for BIFs, it's not possible to do
proper runtime type checking on them as discussed in #2425. The bifcl produced
code unconditionally attempts to convert the positional arguments to StringVals,
but nothing ever type checks them. Instead of improving the vararg support in
Zeek script and bifcl, align cat_sep() with fmt() in making it fully vararg
and do implement type checks by hand.
With this change, passing wrong types for the separator and default argument
isn't a fatal error anymore and the error messages are also more descriptive.
It's a bit of a crutch working around varargs limitations.
Fixes#2425
One more from @stevesmoot. The record_fields() BIF produced "enum" as
type_name for fields of type enum.
Extend container_type_name() to append the actual name of the enum.
This is changing the format and may break consumers, but those are
likely in a category that are happy to adapt. Not having the actual
enum name available wasn't very helpful.
We could alternatively render only the actual type_name without the
prefixed "enum", but that isn't how it's done for record types currently
and it would make it more difficult to decide which subsequent BIFs to
use for further introspection, like enum_names().
In his ZeekWeek 2022 presentation, @stevesmoot mentioned that he had a
difficult time looking up enum names when all he had was a string
naming the type.
Add support to enum_names() to transparently lookup the type if a string
is provided. This is similar in how record_fields() behaves when being
passed a string.
Ran into this when using to_port(getenv(...)) for an undefined/empty
environment variable. ASAN doesn't like that the slash variable ends
up being access behind the string.
This hook can be used to coordinate disabling an analyzer for a given
connection. The contract is simple: Any script can veto a disable_analyzer()
call by breaking from this hook. The decision is local to the script taking
into account any state attached to the connection object or script specific
state stored elsewhere.
A script breaking from the hook takes over the responsibility to call
disable_analyzer() at a later point when it finds the condition due to which
it vetoed fulfilled (which may be never).
Signature:
disabling_analyzer: hook(c: connection, atype: AllAnalyzers::Tag, aid: count);
Example use-cases are keeping the SSL analyzer enabled for finger-printing
until a certain amount of bytes or packets have been transferred or
similarly the connection duration exceed a certain threshold.
Other example use-cases might be keeping analyzers for SSH, RDP or SSL
enabled for connections from specific subnets.
It's a bit quirky as it makes disable_analyzer() a maybe operation. While log
policy hooks and/or the notice hook have similar semantics, they are not as
stateful. It still seems like a quite powerful primitive.
The disable_analyzer() call in dpd/main.zeek may motivate the addition of a
force flag as a follow-up for situations where the caller "knows better" or
absolutely wants to override.
Closes#1678#1593.
