Merge includes small changes, e.g. fixing the comsumption of remaining
raw data.
* 'master' of https://github.com/FlyingWithJerome/zeek:
remove excussive fields in dns_svcb_rr
address code reviews (formatting and type and intrusiveptr)
newlines at the end of test outputs
lazy commit
use tabs in init-bare.zeek
add svcb test case
add a dns https test case
remove test logs
fix a few syntax errors
initial commit for SVCB/HTTPS records
* origin/topic/vern/remove-uu:
fix up for linking w/ doc update
documentation update
script simplification that removes an unnecessary &is_assigned
removing -uu functionality and associated script analysis now no longer needed
Changes during merge:
- Add dedicated test (w/ trace "client_timestamp_enabled.pcapng" from Cloudshark)
- Change types from signed to unsigned.
- Add cast for bit-shifting operand.
- clang-format run
Add a small cache in front of the parse method. This cache should
reduce most of the calls to parse, and ultimately save memory because
redundant versions of the parsed strings will not be created in memory.
Move the parsing itself to the proxies where the caching can be more
efficient.
This could lead to a noticeable (single-percent) performance
improvement.
Most of the functionality for this is in the packet analyzers that now
cache ignore_chesksums_nets.
Based on a patch by Arne Welzel (Corelight).
* origin/topic/vlad/bogon:
Zeekygen doesn't support comments on set members
Add 6to4 addresses, if the matching IPv4 address is private
Update Site::private_address_space to latest IANA networks
* topic/christian/cluster-controller:
Add a cluster controller testcase for agent-controller checkin
Add zeek-client via new submodule
Update baselines affected by cluster controller changes
Introduce cluster controller and cluster agent scripting
Establish a separate init script when using the supervisor
Add optional bare-mode boolean flag to Supervisor's node configuration
Add support for making the supervisor listen for requests
Add support for setting environment variables via supervisor
The supervisor does not require the full weight of scripts that
init-default.zeek brings with it. The new file, init-supervisor.zeek, contains
only what's required by the supervisor in addition to the other always-loaded
init files.
When omitted, the node inherits the Supervisor's bare-mode
status. When true/false, the new Zeek node will enable/disable bare
mode, respectively. It continues to load any scripts passed at the
command line and in the additional scripts list already provided in
the node configuration.
Includes testcase.
The supervisor now starts listening on the configured Broker default address and
port when the new boolean SupervisorControl::enable_listen is T. Listening
remains disabled by default. Listening allows nodes to communicate with the
supervisor via the events laid out in control.zeek, to conduct further node
management.
The NodeConfig record now has a table for specifying environment variable names
and values, which the supervisor sets in the created node.
This also repositions the cpu_affinity member to keep the order the same in
the corresponding script-layer and in-core types.
Includes testcase.
* origin/topic/johanna/gh-859:
Add X509/SSL changes to NEWS
X509: add check if function succeeds
GH-1634: Address feedback
Small indentation fixes in ssl-log-ext.zeek
Fix memory leak in x509_check_cert_hostname bif
Small bugfix and updates for external test hashes (SSL/X509)
Baseline updates for recent SSL changes.
Add ability to check if hostname is valid for a specific cert
Add ssl_history field to ssl.log
Add policy script suppressing certificate events
Add new ssl-log-ext policy script
Deprecate extract-certs-pem.zeek and add log-certs-base64.zeek
Implement X509 certificate log caching
Deprecate ICSI SSL notary script.
Change SSL and X.509 logging format
Enable OCSP logging by default.
Split the code that handles X509 event hashing into its own file
Closes GH-859
This addresses the need for a central hook on any log write, which
wasn't previously doable without a lot of effort. The log manager
invokes the new Log::log_stream_policy hook prior to any filter-specific
hooks. Like filter-level hooks, it may veto a log write. Even when
it does, filter-level hooks still get invoked, but cannot "un-veto".
Includes test cases.
This commit switches to only allowing the CT logs that are currently
accepted by Google Chrome - which makes much more sense for us since
this is (potentially) used for validation.
Additional CT logs can be added in user-scripts.
Due to different double precision on M1, file IDs for SMB could end up
changing on M1 because the access time of a file goes into their
computation. The real solution for this would be changing Zeek's
internal "time" representation to uint64; that's planned, but requires
major surgery. For now, this PR changes the SMB code to also pass SMB's
original time representation (which is a uint64) into script-land, and
then use that for computing the file ID.
Closes#1406
This commit adds two new bifs, x509_check_hostname and
x509_check_cert_hostname. These bifs can be used to check if a given
hostname which can, e.g., be sent in a SNI is valid for a specific
certificate.
This PR furthermore modifies the ssl logs again, and adds information
about this to the log-file. Furthermore we now by default remove the
server certificate information from ssl.log - I doubt that this is often
looked at, it is not present in TLS 1.3, we do still have the SNI, and
if you need it you have the information in x509.log.
This also fixes a small potential problem in X509.cc assuming there
might be SAN-entries that contain null-bytes.
Baseline update will follow in another commit.
This is the equivalent to a connection history for SSL - and contains
information about which protocol messages were exchanged in which order.
Tests currently don't pass - I will update the ssl.log baselines after
doing another a bit invasive change that will change all the logs.
