b41a4bf06d removed a field from this record
because it had a duplicate name as another field. The field does need to
exist, but it needs the correct name.
This instantiates the SSL analyzer when the client requests SSL
so that Zeek now has a bit more visibility into encrypted MySQL
connections.
The pattern used is the same as in the IMAP, POP or XMPP analyzer.
Not sure this adds much more coverage then there was, but minimally
more recent software versions.
The instances/passwords were ephemeral, so hostname and password hashes
etc aren't useful to anyone.
We were parsing MySQL using bigendian even though the protocol is
specified as with "least significant byte first" [1]. This is most
problematic when parsing length encoded strings with 2 byte length
fields...
Further, I think, the EOF_Packet parsing was borked, either due to
testing the CLIENT_DEPRECATE_EOF with the wrong endianness, or due to
the workaround in Resultset processing raising mysql_ok(). Introduce a
new mysql_eof() that triggers for EOF_Packet's and remove the fake
mysql_ok() Resultset invocation to fix. Adapt the mysql script and tests
to account for the new event.
This is a quite backwards incompatible change on the event level, but
due to being quite buggy in general, doubt this matters to many.
I think there is more buried, but this fixes the violation of the simple
"SHOW ENGINE INNODB STATUS" and the existing tests continue to
succeed...
[1] https://dev.mysql.com/doc/dev/mysql-server/latest/page_protocol_basic_dt_integers.html
These have been created artificially. The tests show that for an
invalid reply line without a numeric code, with a numeric code < 100
or a numeric code not followed by a space we now raise an analyzer
violation and disable the analyzer.
When disabling_analyzer() was introduced, it was added to the GLOBAL
module. The awkward side-effect is that implementing a hook handler
in another module requires to prefix it with GLOBAL. Alternatively, one
can re-open the GLOBAL module and implement the handler in that scope.
Both are not great, and prefixing with GLOBAL is ugly, so move the
identifier to the Analyzer module and ask users to prefix with Analyzer.
* 'topic/fox-ds/ssh-key-init-events' of github.com:fox-ds/zeek:
Added several events for detailed info on the SSH2 key init directions
* Straightened out the zeek:see lines in events.bif to be the same across all events.
oss-fuzz produced FTP traffic with a ~550KB long FTP command. Cap FTP command
length at 100 bytes, log a weird if a command is larger than that and move
on to the next. Likely it's not actual FTP traffic, but raising an
analyzer violation would allow clients an easy way to disable the analyzer
by sending an overly long command.
The added test PCAP was generated using a fake Python socket server/client.
oss-fuzz generated "HTTP traffic" containing 250k+ sequences of "T<space>\r\r"
which Zeek then logged as individual HTTP requests. Add a heuristic to bail
on such request lines. It's a bit specific to the test case, but should work.
There are more issues around handling HTTP/0.9, e.g. triggering
"not a http reply line" when HTTP/0.9 never had such a thing, but
I don't think that's worth fixing up.
Fixes#119
This uses the v3 json as a source for the first time. The test needed
some updating because Google removed a couple more logs - in the future
this should hopefully not be neccessary anymore because I think v3
should retain all logs.
In theory this might be neat in 5.1.
The current_entity tracking in HTTP assumes that client/server never
send HTTP entities at the same time. The attached pcap (generated
artificially) violates this and triggers:
1663698249.307259 expression error in <...>base/protocols/http/./entities.zeek, line 89: field value missing (HTTP::c$http$current_entity)
For the http-no-crlf test, include weird.log as baseline. Now that weird is
@load'ed from http, it is actually created and seems to make sense
to btest-diff it, too.
Now that it's loaded in bare mode, no need to load it explicitly.
The main thing that tests were relying on seems to be tracking of
c$service for conn.log baselines. Very few were actually checking
for dpd.log
This is a script-only change that unrolls File::Info records into
multiple files.log entries if the same file was seen over different
connections by single worker. Consequently, the File::Info record
gets the commonly used uid and id fields added. These fields are
optional for File::Info - a file may be analyzed without relation
to a network connection (e.g by using Input::add_analysis()).
The existing tx_hosts, rx_hosts and conn_uids fields of Files::Info
are not meaningful after this change and removed by default. Therefore,
files.log will have them removed, too.
The tx_hosts, rx_hosts and conn_uids fields can be revived by using the
policy script frameworks/files/deprecated-txhosts-rxhosts-connuids.zeek
included in the distribution. However, with v6.1 this script will be
removed.
This hook can be used to coordinate disabling an analyzer for a given
connection. The contract is simple: Any script can veto a disable_analyzer()
call by breaking from this hook. The decision is local to the script taking
into account any state attached to the connection object or script specific
state stored elsewhere.
A script breaking from the hook takes over the responsibility to call
disable_analyzer() at a later point when it finds the condition due to which
it vetoed fulfilled (which may be never).
Signature:
disabling_analyzer: hook(c: connection, atype: AllAnalyzers::Tag, aid: count);
Example use-cases are keeping the SSL analyzer enabled for finger-printing
until a certain amount of bytes or packets have been transferred or
similarly the connection duration exceed a certain threshold.
Other example use-cases might be keeping analyzers for SSH, RDP or SSL
enabled for connections from specific subnets.
It's a bit quirky as it makes disable_analyzer() a maybe operation. While log
policy hooks and/or the notice hook have similar semantics, they are not as
stateful. It still seems like a quite powerful primitive.
The disable_analyzer() call in dpd/main.zeek may motivate the addition of a
force flag as a follow-up for situations where the caller "knows better" or
absolutely wants to override.
Closes#1678#1593.
* origin/topic/vern/bit-shift-fixes:
btest portability fix address review comment about shifting corner-case
canonicalize filenames for new vector deprecation btest
updates for gen-C++ maintenance, including skipping some inappropriate tests
fix for profiling "when" statements
gen-C++ support for vector bit-shift operations
corrected wording in some btest comments
make gen-C++ maintenance scripts directly executable
ZAM support for bit-shifting
don't allow deprecated-style mixing of vectors and scaling for shifting leverage restrictions placed on shifting (RHS is always unsigned) split deprecated vector operations into separate test, with separate ZAM baseline
ZAM fix for vector "in" operator
ensure that language tests pay attention to .stderr
fix vector tests, including checking for errors
A flood of DHCP traffic can result if very large log entries consisting
of many uids and/or msg_types. Such large log entries can disrupt a SIEM
ingestion pipeline. This change forcing a log entry to be written when
the number of uids or the number of msg_Types exceed a certain value.
The values are treated as options for easy configuration.
When a CREATE request contains the FILE_DELETE_ON_CLOSE option and
the subsequent CREATE response indicates success, we now raise the
smb2_file_delete event to log a delete action in smb_files.log and
also give users a way to handle this scenario.
The provided pcap was generated locally by recording a smbtorture run
of the smb2.delete-on-close-perms test case.
Placed the create_options into the CmdInfo record for potential
exposure in smb_cmd.log (wasn't sure how that would look so left it
for the future).
Fixes#2276.
This PR changes the way in which the SSL analyzer tracks the direction
of connections. So far, the SSL analyzer assumed that the originator of
a connection would send the client hello (and other associated
client-side events), and that the responder would be the SSL servers.
In some circumstances this is not true, and the initiator of a
connection is the server, with the responder being the client. So far
this confused some of the internal statekeeping logic and could lead to
mis-parsing of extensions.
This reversal of roles can happen in DTLS, if a connection uses STUN -
and potentially in some StartTLS protocols.
This PR tracks the direction of a TLS connection using the hello
request, client hello and server hello handshake messages. Furthermore,
it changes the SSL events from providing is_orig to providing is_client,
where is_client is true for the client_side of a connection. Since the
argument positioning in the event has not changed, old scripts will
continue to work seamlessly - the new semantics are what everyone
writing SSL scripts will have expected in any case.
There is a new event that is raised when a connection is flipped. A
weird is raised if a flip happens repeatedly.
Addresses GH-2198.
This commit changes DPD matching for TLS connections. A one-sided match
is enough to enable DPD now.
This commit also removes DPD for SSLv2 connections. SSLv2 connections do
basically no longer happen in the wild. SSLv2 is also really finnicky to
identify correctly - there is very little data required to match it, and
basically all matches today will be false positives. If DPD for SSLv2 is
still desired, the optional signature in policy/protocols/ssl/dpd-v2.sig
can be loaded.
Fixes GH-1952
Our test trace is extracted from https://www.cloudshark.org/captures/b9089aac6eee.
There actually seems to be a bug in the existing code: the URI passed to
bt_tracker_request() includes a partial HTTP version. This commits
includes the baseline as the current code produces it, we'll fix that in
a subsequent comment.
This commit fixes the compile-time warnings that OpenSSL 3.0 raises for
our source-code. For the cases where this was necessary we now have two
implementations - one for OpenSSL 1.1 and earlier, and one for OpenSSL
3.0.
This also makes our testsuite pass with OpenSSL 3.0
Relates to GH-1379
Changes during merge:
- Add dedicated test (w/ trace "client_timestamp_enabled.pcapng" from Cloudshark)
- Change types from signed to unsigned.
- Add cast for bit-shifting operand.
- clang-format run
