* topic/christian/management-cluster-dirs:
Management framework: bump zeek-client to pull in instance serialization fixes
Management framework: bump external cluster testsuite
Management framework: update agent-checkin test to reflect recent changes
Management framework: place each Zeek process in its own working dir
Management framework: set defaults for log rotation and persistent state
Management framework: add spool and state directory config settings
Management framework: establish stdout/stderr files also for cluster nodes
Management framework: default to having agents check in with the (local) controller
Management framework: move role variable from logging into framework-wide config
Management framework: distinguish supervisor/supervisee when loading agent/controller
Management framework: simplify agent and controller stdout/stderr files
Management framework: prefix the management logs with "management-"
Management framework: comment and layouting tweaks, no functional change
Management framework: rename env var that labels agents/controllers
Management framework: increase robustness of agent/controller naming
Includes submodule bumps for Broker (to pull in better handling of data
structures that are difficult to unserialize in Python), zeek-client (for the
get-config command), and a commit hash update for the external testsuite.
* topic/christian/management-print-cmd:
Management framework: bump external testsuite
Management framework: allow selecting cluster nodes in get_id_value
Management framework: minor tweaks to logging component
Management framework: bump zeek-client to pull in get-id-value command
Avoid whitespace around function type strings in JSON rendering
Management framework: improve handling of node run states
Management framework: add get_id_value dispatch
Management framework: allow dispatching "actions" on cluster nodes.
Management framework: some renaming to avoid the term "data cluster"
Management framework: allow agents to communicate with cluster nodes
* topic/christian/controller-renaming:
Bump external cluster testsuite to reflect Management framework reorg
Bump zeek-client to reflect Management framework reorg
Reorg of the cluster controller to new "Management framework" layout
* topic/christian/cluster-controller-get-nodes:
Bump external cluster testsuite
Bump zeek-client for the get-nodes command
Add ClusterController::API::get_nodes_request/response event pair
Support optional listening ports for cluster nodes
Don't auto-publish Supervisor response events in the cluster agent
Make members of the ClusterController::Types::State enum all-caps
Be more conservative with triggering request timeout events
Move redefs of ClusterController::Request::Request to their places of use
Simplify ClusterController::API::set_configuration_request/response
This commit changes DPD matching for TLS connections. A one-sided match
is enough to enable DPD now.
This commit also removes DPD for SSLv2 connections. SSLv2 connections do
basically no longer happen in the wild. SSLv2 is also really finnicky to
identify correctly - there is very little data required to match it, and
basically all matches today will be false positives. If DPD for SSLv2 is
still desired, the optional signature in policy/protocols/ssl/dpd-v2.sig
can be loaded.
Fixes GH-1952
This job runs in sequence after the image build one, using its resulting image.
The actual tests live in the external zeek-testing-cluster testsuite, which
the new job clones and runs.
To specify a version of the testsuite to use, testing/external/ has a new
commit-hash.zeek-testing-cluster file that tracks the testsuite's relevant
commit ref
We used to attempt to remove any port specification before recording
HTTP host headers in logs. Doing so would (1) remove potentially useful
information, (2) not match what the documentation seemed to suggest, and
(3) fail for IP6 addresses containing colons.
We now record the original HOST header as is.
Addresses #1844.
All changes in this patch were performed automatically with `shfmt` with
configuration flags specified in `.pre-commit-config.yaml`.
In addition to fixing whitespace the roundtrip through shfmt's AST also
transforms command substitutions
`cmd`
# becomes
$(cmd)
and some redirects
>&2 echo "msg"
# becomes
echo >&2 "msg"
* origin/topic/seth/tsv-logs-utf8-by-default:
Fix mis-usage of string::append that leads to an overflow
Use json_escape_utf8 for all utf8 data in ODesc
Switch the TSV Zeek logs to be UTF8 by default.
Delete the unused Baseline folder, remove a Bro-era name from gitignore,
change create-new-repo to work with a second argument that is actually
a URL, and tweak whitespace. Expand description in README and update the
explanation of OPENSSL_ENABLE_MD5_VERIFY in subdir-btest.cfg.
* origin/topic/johanna/gh-859:
Add X509/SSL changes to NEWS
X509: add check if function succeeds
GH-1634: Address feedback
Small indentation fixes in ssl-log-ext.zeek
Fix memory leak in x509_check_cert_hostname bif
Small bugfix and updates for external test hashes (SSL/X509)
Baseline updates for recent SSL changes.
Add ability to check if hostname is valid for a specific cert
Add ssl_history field to ssl.log
Add policy script suppressing certificate events
Add new ssl-log-ext policy script
Deprecate extract-certs-pem.zeek and add log-certs-base64.zeek
Implement X509 certificate log caching
Deprecate ICSI SSL notary script.
Change SSL and X.509 logging format
Enable OCSP logging by default.
Split the code that handles X509 event hashing into its own file
Closes GH-859
Due to different double precision on M1, file IDs for SMB could end up
changing on M1 because the access time of a file goes into their
computation. The real solution for this would be changing Zeek's
internal "time" representation to uint64; that's planned, but requires
major surgery. For now, this PR changes the SMB code to also pass SMB's
original time representation (which is a uint64) into script-land, and
then use that for computing the file ID.
Closes#1406
Fixes to `decode_netbios_name`:
* Improve validation that input string is a NetBIOS encoding
(32 bytes, with characters ranging from 'A' to 'P'). This helps
prevent Undefined Behavior of left-shifting negative values.
Invalid encodings now cause a return-value of an empty string.
* More liberal in what decoded characters are allowed. Namely,
spaces are now allowed (but any trailing null-bytes and spaces
are trimmed, similar to before).
Fixes to `decode_netbios_name_type`:
* Improve validation that input string is a NetBIOS encoding
(32 bytes, with characters ranging from 'A' to 'P'). This helps
prevent Undefined Behavior of left-shifting negative values and
a heap-buffer-overread when the input string is too small.
Invalid encodings now cause a return-value of 256.
* 'topic/oakljon/gh-1352-smtp-header-parsing' of https://github.com/theavgjojo/zeek:
GH-1352: Added flag to stop processing SMTP headers in attached messages
Changes \x00-\x37 ranges to \x00-\x1f with assumption that the former
was attempting to match ASCII control characters, but mistook an octal
range for hex. This change reduces some false positives.
