This uses the v3 json as a source for the first time. The test needed
some updating because Google removed a couple more logs - in the future
this should hopefully not be neccessary anymore because I think v3
should retain all logs.
In theory this might be neat in 5.1.
Introduce two new events for analyzer confirmation and analyzer violation
reporting. The current analyzer_confirmation and analyzer_violation
events assume connection objects and analyzer ids are available which
is not always the case. We're already passing aid=0 for packet analyzers
and there's not currently a way to report violations from file analyzers
using analyzer_violation, for example.
These new events use an extensible Info record approach so that additional
(optional) information can be added later without changing the signature.
It would allow for per analyzer extensions to the info records to pass
analyzer specific info to script land. It's not clear that this would be
a good idea, however.
The previous analyzer_confirmation and analyzer_violation events
continue to exist, but are deprecated and will be removed with Zeek 6.1.
The current_entity tracking in HTTP assumes that client/server never
send HTTP entities at the same time. The attached pcap (generated
artificially) violates this and triggers:
1663698249.307259 expression error in <...>base/protocols/http/./entities.zeek, line 89: field value missing (HTTP::c$http$current_entity)
For the http-no-crlf test, include weird.log as baseline. Now that weird is
@load'ed from http, it is actually created and seems to make sense
to btest-diff it, too.
...the only known cases where the `-` for `connection$service` was
handled is to skip/ignore these analyzers.
Slight suspicion that join_string_set() should maybe become a bif
now determine_service() runs once for each connection.
Closes#2388
* micrictor/master:
Add a field to Modbus/TCP log to indicate the Modbus PDU type
Add modbus transaction and unit ids to logs
Enable modbus logging for requests
The next patch will have a test script rely on c$ssl$analyzer_id staying
around when disable_analyzer() wasn't successful.
I was tempted to remove the `delete` completely as neither RDP nor SSH
have that and not sure why SSL is special here.
A flood of DHCP traffic can result if very large log entries consisting
of many uids and/or msg_types. Such large log entries can disrupt a SIEM
ingestion pipeline. This change forcing a log entry to be written when
the number of uids or the number of msg_Types exceed a certain value.
The values are treated as options for easy configuration.
Add the `pdu_type` field to Modbus over TCP logs to indicate whether the Modbus
message was a request or a response. Due to the client/server nature of Modbus
over TCP/IP, all messages from the TCP session originator are requests, while
all messages from the TCP session responder are responses.
Adding this information to the default log surfaces protocol metadata in a way
that doesn't require users to understand the Modbus over TCP protocol.
When a CREATE request contains the FILE_DELETE_ON_CLOSE option and
the subsequent CREATE response indicates success, we now raise the
smb2_file_delete event to log a delete action in smb_files.log and
also give users a way to handle this scenario.
The provided pcap was generated locally by recording a smbtorture run
of the smb2.delete-on-close-perms test case.
Placed the create_options into the CmdInfo record for potential
exposure in smb_cmd.log (wasn't sure how that would look so left it
for the future).
Fixes#2276.
This PR changes the way in which the SSL analyzer tracks the direction
of connections. So far, the SSL analyzer assumed that the originator of
a connection would send the client hello (and other associated
client-side events), and that the responder would be the SSL servers.
In some circumstances this is not true, and the initiator of a
connection is the server, with the responder being the client. So far
this confused some of the internal statekeeping logic and could lead to
mis-parsing of extensions.
This reversal of roles can happen in DTLS, if a connection uses STUN -
and potentially in some StartTLS protocols.
This PR tracks the direction of a TLS connection using the hello
request, client hello and server hello handshake messages. Furthermore,
it changes the SSL events from providing is_orig to providing is_client,
where is_client is true for the client_side of a connection. Since the
argument positioning in the event has not changed, old scripts will
continue to work seamlessly - the new semantics are what everyone
writing SSL scripts will have expected in any case.
There is a new event that is raised when a connection is flipped. A
weird is raised if a flip happens repeatedly.
Addresses GH-2198.
* origin/topic/vern/table-attr-fixes:
updates for btests - new cases to check, new baselines
updates for btests - new cases to check, new baselines
fix for ill-formed (complex) &default function
type-checking for use of empty table constructors in expressions
catch empty constructors used for type inference suppress repeated error messages
factoring to make checking of &default attributes externally accessible
bug fix for empty table constructors with &default attributes (plus a typo)
This commit changes DPD matching for TLS connections. A one-sided match
is enough to enable DPD now.
This commit also removes DPD for SSLv2 connections. SSLv2 connections do
basically no longer happen in the wild. SSLv2 is also really finnicky to
identify correctly - there is very little data required to match it, and
basically all matches today will be false positives. If DPD for SSLv2 is
still desired, the optional signature in policy/protocols/ssl/dpd-v2.sig
can be loaded.
Fixes GH-1952
* origin/topic/vern/when-lambda:
explicitly provide the frame for evaluating a "when" timeout expression
attempt to make "when" btest deterministic
tests for new "when" semantics/errors
update existing test suite usage of "when" statements to include captures
update uses of "when" in base scripts to include captures
captures for "when" statements update Triggers to IntrusivePtr's and simpler AST traversal introduce IDSet type, migrate associated "ID*" types to "const ID*"
logic (other than in profiling) for assignments that yield separate values
option for internal use to mark a function type as allowing non-expression returns
removed some now-obsolete profiling functionality
minor commenting clarifications
We used to attempt to remove any port specification before recording
HTTP host headers in logs. Doing so would (1) remove potentially useful
information, (2) not match what the documentation seemed to suggest, and
(3) fail for IP6 addresses containing colons.
We now record the original HOST header as is.
Addresses #1844.
Merge includes small changes, e.g. fixing the comsumption of remaining
raw data.
* 'master' of https://github.com/FlyingWithJerome/zeek:
remove excussive fields in dns_svcb_rr
address code reviews (formatting and type and intrusiveptr)
newlines at the end of test outputs
lazy commit
use tabs in init-bare.zeek
add svcb test case
add a dns https test case
remove test logs
fix a few syntax errors
initial commit for SVCB/HTTPS records
* origin/topic/johanna/gh-859:
Add X509/SSL changes to NEWS
X509: add check if function succeeds
GH-1634: Address feedback
Small indentation fixes in ssl-log-ext.zeek
Fix memory leak in x509_check_cert_hostname bif
Small bugfix and updates for external test hashes (SSL/X509)
Baseline updates for recent SSL changes.
Add ability to check if hostname is valid for a specific cert
Add ssl_history field to ssl.log
Add policy script suppressing certificate events
Add new ssl-log-ext policy script
Deprecate extract-certs-pem.zeek and add log-certs-base64.zeek
Implement X509 certificate log caching
Deprecate ICSI SSL notary script.
Change SSL and X.509 logging format
Enable OCSP logging by default.
Split the code that handles X509 event hashing into its own file
Closes GH-859
This commit switches to only allowing the CT logs that are currently
accepted by Google Chrome - which makes much more sense for us since
this is (potentially) used for validation.
Additional CT logs can be added in user-scripts.
Due to different double precision on M1, file IDs for SMB could end up
changing on M1 because the access time of a file goes into their
computation. The real solution for this would be changing Zeek's
internal "time" representation to uint64; that's planned, but requires
major surgery. For now, this PR changes the SMB code to also pass SMB's
original time representation (which is a uint64) into script-land, and
then use that for computing the file ID.
Closes#1406
