publish_hrw() and publish_rr() are excluded from type checking due to their
variadic nature. Passing a wrong type for the pool argument previously triggered
an abort, now the result is runtime errors. This isn't great, but it's
better than crashing Zeek.
Closes#2935
The known_log_certs table is populated asynchronously via broker after a
Broker::peer_added. It may take a variable amount of time depending on where
we run this test and it has been observed flaky specifically for the
arm_debian11 task. Instead of racing, give worker-2 3 seconds for receiving
the expected table content before continuing.
Fixes#2885
While working on a rotation format function, ran into Zeek crashing
when not returning a value from it, fix and recover the same way as
for scripting errors.
* security/topic/timw/154-rdp-timeout:
RDP: Instantiate SSL analyzer instead of PIA
RDP: add some enforcement to required values based on MS-RDPBCGR docs
* security/topic/awelzel/152-smtp-validate-mail-transactions:
smtp: Validate mail transaction and disable SMTP analyzer if excessive
generic-analyzer-fuzzer: Detect disable_analyzer() from scripts
* security/topic/awelzel/148-ftp-skip-get-pending-commands-multi-line-response:
ftp/main: Special case for intermediate reply lines
ftp/main: Skip get_pending_command() for intermediate reply lines
An invalid mail transaction is determined as
* RCPT TO command without a preceding MAIL FROM
* a DATA command without a preceding RCPT TO
and logged as a weird.
The testing pcap for invalid mail transactions was produced with a Python
script against a local exim4 configured to accept more errors and unknown
commands than 3 by default:
# exim4.conf.template
smtp_max_synprot_errors = 100
smtp_max_unknown_commands = 100
See also: https://www.rfc-editor.org/rfc/rfc5321#section-3.3
Intermediate lines of multiline replies usually do not contain valid status
codes (even if servers may opt to include them). Their content may be anything
and likely unrelated to the original command. There's little reason for us
trying to match them with a corresponding command.
OSS-Fuzz generated a large command reply with very many intermediate lines
which caused long processing times due to matching every line with all
currently pending commands.
This is a DoS vector against Zeek. The new ipv6-multiline-reply.trace and
ipv6-retr-samba.trace files have been extracted from the external ipv6.trace.
This has been around since Zeek v4.1, so it was warned about in Zeek 5.0
LTS and we could've removed it with 5.1.
Also removed merge_top_scope() from the zeek::detail namespace, it's
unused now.
Updated the when-aggregates test somehow. I'm not quite sure what had
been tested there :-/
There was a misunderstanding whether to include them by default in
the dns.log, so remove them again.
There had also been a discussion and quirk that AD of a request would
always be overwritten by reply in the dns.log unless the reply is
missing. For now, let users extend dns.log themselves for what best
fits their requirements, rather than adding these flags by default.
Add a btest to print AD and CD flags for smoke testing still.
OpenSSL 3.1 switched from outputting UNDEF to not giving a short name in
this case. Luckily this only requires a tiny test change.
We might consider pulling this into older versions, for ease of CI
testing.
Fixes GH-2869
This was exposed by OSS-Fuzz after the HTTP/0.9 changes in zeek/zeek#2851:
We do not check the result of parsing the from and last bytes of a
Content-Range header and would reference uninitialized values on the stack
if these were not valid.
This doesn't seem as bad as it sounds outside of yielding non-sensible values:
If the result was negative, we weird/bailed. If the result was positive, we
already had to treat it with suspicion anyway and the SetPlainDelivery()
logic accounts for that.
OSS-Fuzz tickled an assert when sending a HTTP response before a HTTP/0.9
request. Avoid this by resetting reply_message upon seeing a HTTP/0.9 request.
PCAP was generated artificially: Server sending a reply providing a
Content-Length. Because HTTP/0.9 processing would remove the ContentLine
support analyzer, more data was delivered to the HTTP_Message than
expected, triggering an assert.
This is a follow-up for zeek/zeek#2851.
This commit introduces parsing of the CertificateRequest message in the
TLS handshake. It introduces a new event ssl_certificate_request, as
well as a new function parse_distinguished_name, which can be used to
parse part of the ssl_certificate_request event parameters.
This commit also introduces a new policy script, which appends
information about the CAs a TLS server requests in the
CertificateRequest message, if it sends it.
This change exposes the signature tyope inside the signed portion of an
X.509 certificate. In the past, we only exposed the signature type that
is contained inside the signature, which is outside the signed portion
of the X.509 certificate.
In theory, both signature fields should have the same value; it is,
however, possible to encode differing values in both fields. The new
field is not logged by default.
This adds one metric per log stream and one metric per log writer (path based)
to track the number of writes on a stream level as well as on a writer level.
$ curl -sSf localhost:8181/metrics | grep Conn
zeek_log_writer_writes_total{endpoint="",filter-name="default",module="HTTP",path="http",stream="HTTP::LOG",writer="Log::WRITER_SQLITE"} 1 1677497572770
zeek_log_stream_writes_total{endpoint="",module="HTTP",stream="HTTP::LOG"} 1 1677497572770
The initial version of this change also included metrics around log
write vetoes, but given no log policies exist in the default configuration
and they are mostly interesting for a few streams/writers only, skip this
for now. These can always be added by the script writer, too.
The difference between the stream level writes and concrete writers can
be used to deduce the number of vetoes (or errors) as a starting point.
Put the IntCounter into a std::optional rather than initializing
it at EventHandler construction time as that will currently expose
a time series per event handler through the Prometheus API.
The user and password fields are replicated to each of the ftp.log
entries. Using a very large username (100s of KBs) allows to bloat
the log without actually sending much traffic. Further, limit the
arg and reply_msg columns to large, but not unbounded values.
This avoids interference from other log streams in the policy hook test cases,
which could cause deviations in output vs baselines depending on build
configuration.
* security/topic/awelzel/125-ftp-timeout-three:
testing/ftp: Add tests and pcaps with invalid reply lines
ftp: Harden reply handing a bit and don't raise bad replies to script-land
ftp: ignore invalid commands
As initial examples, this branch ports the Syslog and Finger analyzers
over. We leave the old analyzers in place for now and activate them
iff we compile without any Spicy.
Needs `zeek-spicy-infra` branches in `spicy/`, `spicy-plugin/`,
`CMake/`, and `zeek/zeek-testing-private`.
Note that the analyzer events remain associated with the Spicy plugin
for now: that's where they will show up with `-NN`, and also inside
the Zeekygen documentation.
We switch CMake over to linking the runtime library into the plugin,
vs. at the top-level through object libraries.
b41a4bf06d removed a field from this record
because it had a duplicate name as another field. The field does need to
exist, but it needs the correct name.
This instantiates the SSL analyzer when the client requests SSL
so that Zeek now has a bit more visibility into encrypted MySQL
connections.
The pattern used is the same as in the IMAP, POP or XMPP analyzer.
Not sure this adds much more coverage then there was, but minimally
more recent software versions.
The instances/passwords were ephemeral, so hostname and password hashes
etc aren't useful to anyone.
We were parsing MySQL using bigendian even though the protocol is
specified as with "least significant byte first" [1]. This is most
problematic when parsing length encoded strings with 2 byte length
fields...
Further, I think, the EOF_Packet parsing was borked, either due to
testing the CLIENT_DEPRECATE_EOF with the wrong endianness, or due to
the workaround in Resultset processing raising mysql_ok(). Introduce a
new mysql_eof() that triggers for EOF_Packet's and remove the fake
mysql_ok() Resultset invocation to fix. Adapt the mysql script and tests
to account for the new event.
This is a quite backwards incompatible change on the event level, but
due to being quite buggy in general, doubt this matters to many.
I think there is more buried, but this fixes the violation of the simple
"SHOW ENGINE INNODB STATUS" and the existing tests continue to
succeed...
[1] https://dev.mysql.com/doc/dev/mysql-server/latest/page_protocol_basic_dt_integers.html
These have been created artificially. The tests show that for an
invalid reply line without a numeric code, with a numeric code < 100
or a numeric code not followed by a space we now raise an analyzer
violation and disable the analyzer.
When disabling_analyzer() was introduced, it was added to the GLOBAL
module. The awkward side-effect is that implementing a hook handler
in another module requires to prefix it with GLOBAL. Alternatively, one
can re-open the GLOBAL module and implement the handler in that scope.
Both are not great, and prefixing with GLOBAL is ugly, so move the
identifier to the Analyzer module and ask users to prefix with Analyzer.
* origin/topic/awelzel/analyzer-log:
btest/net-control: Use different expiration times for rules
analyzer: Add analyzer.log for logging violations/confirmations
