This commit adds two new bifs, x509_check_hostname and
x509_check_cert_hostname. These bifs can be used to check if a given
hostname which can, e.g., be sent in a SNI is valid for a specific
certificate.
This PR furthermore modifies the ssl logs again, and adds information
about this to the log-file. Furthermore we now by default remove the
server certificate information from ssl.log - I doubt that this is often
looked at, it is not present in TLS 1.3, we do still have the SNI, and
if you need it you have the information in x509.log.
This also fixes a small potential problem in X509.cc assuming there
might be SAN-entries that contain null-bytes.
Baseline update will follow in another commit.
This is the equivalent to a connection history for SSL - and contains
information about which protocol messages were exchanged in which order.
Tests currently don't pass - I will update the ssl.log baselines after
doing another a bit invasive change that will change all the logs.
The added disable-certificate-events-known-certs.zeek disables repeated
X509 events in SSL connections, given that the connection terminates at
the same server and used the samt SNI as a previously seen connection
with the same certificate.
For people that see significant amounts of TLS 1.2 traffic, this could
reduce the amount of raised events significantly - especially when a
lot of connections are repeat connections to the same servers.
The practical impact of not raising these events is actually very little
- unless a script directly interacts with the x509 events, everything
works as before - the x509 variables in the connection records are still
being set (from the cache).
By default, each certificate is now output only once per hour. This also
should work in cluster mode, where we use the net broker-table-syncing
feature to distribute the information about already seen certificates
across the entire cluster.
Log caching is also pretty configureable and can be changed using a
range of confiuration options and hooks.
Note that this is currently completely separate from X509 events
caching, which prevents duplicate parsing of X509 certificates.
This commit changes the SSL and X.509 logging formats to something that,
hopefully, slowly approaches what they will look like in the future.
X.509 log is not yet deduplicated; this will come in the future.
This commit introduces two new options, which determine if certificate
issuers and subjects are still logged in ssl.log. The default is to have
the host subject/issuer logged, but to remove client-certificate
information. Client-certificates are not a typically used feature
nowadays.
In the past I thought that this is not super interesting. However, it
turns out that this can actually contain a slew of interresting
information - like operating systems querying for the revocation of
software signing certificates, e.g.
So - let's just enable this as a default log for the future.
* origin/topic/vlad/page_hostnames:
Expand the tests, and have email_admin email src contacts in one e-mail, and dst contacts in another.
Simplify the logic a big by making Notice$email_dest have a default rather than be optional
Missed some baselines
Update hashes in external testing repos
Update notice.log baselines
Remove unneccessary @load
Add tests for e-mail actions, and cleanup the new logic a bit.
Fixup priorities
Add a new field to the notice, which defines where to send the e-mail
Add page and email_admin to hostnames extension
* origin/topic/bbannier/issue-1590:
Change SSH version field to be `&optional`.
Reformat function in SSH base script.
Changes during merge:
* rename weirds to fit into our naming scheme
* add NEWS entry
Closes GH-1590
In 3769ed6c66 we added handling for SSH version 1.99 which unsed a SSH
version of 0 to indicate weird cases where no version could be
determined.
This patch is a fixup for that patch. Instead of using a magic version
of 0 we now use an `&optional` version value. If no SSH version can be
extracted the version will be unset; additionally a `conn_weird` event
will be raised.
Closes#1590.
* 'logging/script-logdir' of https://github.com/kramse/zeek:
Copy of ascii-empty test, just changed path in the beginning
Logdir: Change requested by 0xxon, no problem
Introduce script-land variable that can be used to set logdir.
Closes GH-772
This commit removes the stepping stone analyzer. It has been deactivated
by default since at least Zeek 2.0, is dysfunctional in cluster settings
and has a bunch of other issued.
Relates to GH-1573
Fixes to `decode_netbios_name`:
* Improve validation that input string is a NetBIOS encoding
(32 bytes, with characters ranging from 'A' to 'P'). This helps
prevent Undefined Behavior of left-shifting negative values.
Invalid encodings now cause a return-value of an empty string.
* More liberal in what decoded characters are allowed. Namely,
spaces are now allowed (but any trailing null-bytes and spaces
are trimmed, similar to before).
Fixes to `decode_netbios_name_type`:
* Improve validation that input string is a NetBIOS encoding
(32 bytes, with characters ranging from 'A' to 'P'). This helps
prevent Undefined Behavior of left-shifting negative values and
a heap-buffer-overread when the input string is too small.
Invalid encodings now cause a return-value of 256.
This patch adds the ability to decap Geneve packets to process the inner
payload. The structure of the analyzer borrows heavily from the VXLAN
analyzer.
- Minor adjustments to header includes, whitespace, and a
compiler warning fix during merge
* origin/topic/neverlord/gh-1408:
Add new Timer class to the telemetry API
Fix build with latest VectorVal API
Update baselines
Prefix telemetry BIFs with __
Expose telemetry API to scripting land
Add handle types for histogram metrics
Move duplicated code to detail header
Adhere to Zeek coding style
Apply suggestions from code review
Add telemetry Manager to Zeek setup / teardown
Add missing whitespaces for Zeek coding style
Add gauge metric types
Add scaffold for new metrics API with counters
- Restored a deprecated version of 'supervisor_rotation_format_func'
during merge.
* origin/topic/vlad/expose_supervisor_rotation_func:
Rename supervisor_rotation_format_func to archiver_rotation_format_func, and expose it for non-supervised setups
* origin/topic/vern/zval: (42 commits)
whitespace tweaks
resolved some TODO comments
remove unnecessary casts, and change necessary ones to use static_cast<>
explain cmp_func default
change functions for ZVal type management to static members
fix some unsigned/signed integer warnings
address lint concern about uninitialized variable
Remove use of obsolete forward-declaration macros
fix #include's that lack zeek/ prefixes
explicitly populate holes created in vectors
fixes for now-incorrect assumption that GetField always returns an existing ValPtr
memory management for assignment to vector elements
memory management for assignment to record fields
destructor cleanup from ZAM_vector/ZAM_record
fix #include's that lack zeek/ prefixes
overlooked another way in which vector holes can be created
initialize vector holes to the correct corresponding type
explicitly populate holes created in vectors
fix other instances of GetField().get() assuming long-lived ValPtr's
fix for now-incorrect assumption that GetField always returns an existing ValPtr
...
* origin/topic/vern/cpp-prep-fixes:
fix for associating current scope with the name of enums; name tidying
avoid infinite recursion in same_type() if it is analyzing recursive types
remove iffy reliance on type punning that relies on interpreter's behavior
fixes for propagating optimization options, and pruning script function analysis
This avoids a problem identified by amanbansal2709 in pull
request #1288. I fixed it in a different way than that pull request
by making sure the ts field is always set so that this isssue doesn't
return in the future.
Broker::create_master() and Broker::create_clone() now return
a valid value even when there's a failure to open the backend database
(e.g. SQLite filesystem error). In that case, the returned value can
still be passed into other data store operations, but they'll fail
immediately with an error. Broker::is_closed() can now also be used to
determine whether the data store creation calls failed.
