* origin/topic/jazoff/gh-3268:
Fix check for emailed notices
Changes: Added a test-case printing email_delay_tokens to compare email vs
non-email notice types. Previously, both notice types would have email
delay tokens at that point in the flow.
(cherry picked from commit 7e11501d3c)
* origin/topic/vern/at-if-analyze:
updates reflecting review comments
change base scripts to use run-time if's or @if ... &analyze
a number of BTests updated with @if ... &analyze
update for scripting coverage BTest demonstrating utility of @if ... &analyze
BTests for new @if ... &analyze functionality
"if ( ... ) &analyze" language feature
classes for tracking "@if (...) &analyze" notion of code being/not being "activated"
RemoveGlobal() method for Scope class + simplifying interfaces
While writing documentation about troubleshooting and looking a bit
at the older stats.log, realized we don't have the packet lag metric
exposed as metric/telemetry. Add it.
This is a Zeek instance lagging behind in network time ~6second because
it's very overloaded:
zeek_net_packet_lag_seconds{endpoint=""} 6.169406 1684848998092
Issue #3028 tracks how a flipped connections reset a connection's value
including any state set during new_connection(). For the time being,
update community-id functionality back to the original connection_state_remove()
approach to avoid missing community_ids on flipped connections.
* origin/topic/awelzel/add-community-id:
testing/external: Bump hashes for community_id addition
NEWS: Add entry for Community ID
policy: Import zeek-community-id scripts into protocols/conn frameworks/notice
Add community_id_v1() based on corelight/zeek-community-id
This commit introduces parsing of the CertificateRequest message in the
TLS handshake. It introduces a new event ssl_certificate_request, as
well as a new function parse_distinguished_name, which can be used to
parse part of the ssl_certificate_request event parameters.
This commit also introduces a new policy script, which appends
information about the CAs a TLS server requests in the
CertificateRequest message, if it sends it.
The seen/file-names script relies on f$info$filename to be populated.
For HTTP and other network protocols, however, this field is only
populated during file_over_new_connection() that's running after
file_new().
Use the file_new() event only for files without connections and
file_over_new_connection() implies that f$conns is populated, anyway.
Special case SMB to avoid finding files twice, because there's a
custom implementation in seen/smb-filenames.zeek.
Fixes#2647
The controller now listens on an additional port, defaulting to 2149, for Broker
connections via websockets. Configuration works as for the existing traditional
Broker port (2150), via ZEEK_CONTROLLER_WEBSOCKET_ADDR and
ZEEK_CONTROLLER_WEBSOCKET_PORT environment variables, as well as corresponding
redef'able constants.
To disable the websockets feature, leave ZEEK_CONTROLLER_WEBSOCKET_PORT unset
and redefine Management::Controller::default_port_websocket to 0/unknown.
Introduce two new events for analyzer confirmation and analyzer violation
reporting. The current analyzer_confirmation and analyzer_violation
events assume connection objects and analyzer ids are available which
is not always the case. We're already passing aid=0 for packet analyzers
and there's not currently a way to report violations from file analyzers
using analyzer_violation, for example.
These new events use an extensible Info record approach so that additional
(optional) information can be added later without changing the signature.
It would allow for per analyzer extensions to the info records to pass
analyzer specific info to script land. It's not clear that this would be
a good idea, however.
The previous analyzer_confirmation and analyzer_violation events
continue to exist, but are deprecated and will be removed with Zeek 6.1.
* origin/topic/awelzel/dpd-analyzer-merger:
analyzer/dpd: Address review comments
Remove @load base/frameworks/dpd from tests
frameworks/dpd: Move to frameworks/analyzer/dpd, load by default
scripts/dce-rpc,ntlm: Do not load base/frameworks/dpd
btest: Remove unnecessary loading of frameworks/dpd
* Because frameworks/analyzer is loaded via init-frameworks-and-bifs the
dpd functionality (really just dpd.log and disabling of analyzers) is
now enabled even in bare mode.
* Not sure we need to keep frameworks/base/dpd/__load__.zeek around
or can just remove it right away.
This is a script-only change that unrolls File::Info records into
multiple files.log entries if the same file was seen over different
connections by single worker. Consequently, the File::Info record
gets the commonly used uid and id fields added. These fields are
optional for File::Info - a file may be analyzed without relation
to a network connection (e.g by using Input::add_analysis()).
The existing tx_hosts, rx_hosts and conn_uids fields of Files::Info
are not meaningful after this change and removed by default. Therefore,
files.log will have them removed, too.
The tx_hosts, rx_hosts and conn_uids fields can be revived by using the
policy script frameworks/files/deprecated-txhosts-rxhosts-connuids.zeek
included in the distribution. However, with v6.1 this script will be
removed.
This erroneously used connectedness of instances, not presence of a deployed
cluster. Without a deployment, there's no point in trying to retrieve global ID
values.
Adds base/frameworks/telemetry with wrappers around telemetry.bif
and updates telemetry/Manager to support collecting metrics from
script land.
Add policy/frameworks/telemetry/log for logging of metrics data
into a new telemetry.log and telemetry_histogram.log and add into
local.zeek by default.
Ensure the framework's log stream exists prior to using it in zeek_init(), and
use a node-is-live message similar to those in agent and controller also in
launched nodes.
Failing to do so could open a race condition in which a quickly connecting
controller could send instructions whose resulting Supervisor interactions got
lost.
The following two patterns were identified while reviewing patterns that
match on any characters. The intention likely was to match actual
literal dots.
This PR changes the way in which the SSL analyzer tracks the direction
of connections. So far, the SSL analyzer assumed that the originator of
a connection would send the client hello (and other associated
client-side events), and that the responder would be the SSL servers.
In some circumstances this is not true, and the initiator of a
connection is the server, with the responder being the client. So far
this confused some of the internal statekeeping logic and could lead to
mis-parsing of extensions.
This reversal of roles can happen in DTLS, if a connection uses STUN -
and potentially in some StartTLS protocols.
This PR tracks the direction of a TLS connection using the hello
request, client hello and server hello handshake messages. Furthermore,
it changes the SSL events from providing is_orig to providing is_client,
where is_client is true for the client_side of a connection. Since the
argument positioning in the event has not changed, old scripts will
continue to work seamlessly - the new semantics are what everyone
writing SSL scripts will have expected in any case.
There is a new event that is raised when a connection is flipped. A
weird is raised if a flip happens repeatedly.
Addresses GH-2198.
