This adds a protocol parser for the PostgreSQL protocol and a new
postgresql.log similar to the existing mysql.log.
This should be considered preliminary and hopefully during 7.1 and 7.2
with feedback from the community, we can improve on the events and logs.
Even if most PostgreSQL communication is encrypted in the real-world, this
will minimally allow monitoring of the SSLRequest and hand off further
analysis to the SSL analyzer.
This originates from github.com/awelzel/spicy-postgresql, with lots of
polishing happening in the past two days.
The current implementation would only log, if the password contains a
colon, the part before the first colon (e.g., the password
`password:password` would be logged as `password`).
A test has been added to confirm the expected behaviour.
It turns out that, for probably a long time, we have reported an
incorrect version when parsing an SSLv2 client hello. We always reported
this as SSLv2, no matter which version the client hello actually
contained.
This bug probably went unnoticed for a long time, as SSLv2 is
essentially unused nowadays, and as this field does not show up in the
default logs.
This was found due to a baseline difference when writing the Spicy SSL
analyzer.
This avoids the earlier problem of not tracking ports correctly in
scriptland, while still supporting `port` in EVT files and `%port` in
Spicy files.
As it turns out we are already following the same approach for file
analyzers' MIME types, so I'm applying the same pattern: it's one
event per port, without further customization points. That leaves the
patch pretty small after all while fixing the original issue.
* origin/topic/johanna/ssl-history-also-for-sslv2-not-only-for-things-that-use-the-more-modern-handshake:
Make ssl_history work for SSLv2 handshakes/connections
* origin/topic/vern/zam-regularization: (33 commits)
simpler and more robust identification of function parameters for AST profiling
fixes to limit AST traversal in the face of recursive types
address some script optimization compiler warnings under Linux
fix for -O C++ construction of variable names that use multiple module namespaces
fix for script optimization of "opaque" values that are run-time constants
fix for script optimization of nested switch statements
script optimization fix for complex "in" expressions in conditionals
updates to typos allow-list reflecting ZAM regularization changes
BTest updates for ZAM regularization changes
convert new ZAM operations to use typed operands
complete migration of ZAM to use only public ZVal methods
"-O validate-ZAM" option to validate generated ZAM instructions
internal option to suppress control-flow optimization
exposing some functionality for greater flexibility in structuring run-time execution
rework ZAM compilation of type switches to leverage value switches
add tracking of control flow information
factoring of ZAM operation specifications into separate files
updates to ZAM operations / gen-zam regularization, other than the operations themselves
type-checking fix for vector-of-string operations
ZVal constructor for booleans
...
* origin/topic/vern/cpp-init:
Func: Add SetCapturesVec()
marked some recently added BTests as not suitable for -O gen-C++ testing
robustness improvements for -O gen-C++ generation of lambdas / "when"s
speedups for compilation of initializers in -O gen-C++ generated code
fixes for -O gen-C++ generation of floating point constants
-O gen-C++ fix for dealing with use of more than one module qualifier
header tweaks to provide gen-C++ script optimization with more flexibility
fix for script optimization of constants of type "opaque"
fix for script optimization of "in" operations
some minor tidying of -O gen-C++ sources
This reworks the parser such that COM_CHANGE_USER switches the
connection back into the CONNECTION_PHASE so that we can remove the
EXPECT_AUTH_SWITCH special case in the COMMAND_PHASE. Adds two pcaps
produced with Python that actually do COM_CHANGE_USER as it seems
not possible from the MySQL CLI.
It turns out that the ssl_history field never was populated with C/S for
SSLv2 connections, or connections using the SSLv2 handshake. In our
testcases, the latter is especially common - with connections up to TLS1
using the old SSLv2 client hello for backwards compatibility.
This change resolves this issue. As the history is not by default
enabled in a lot of locations, baseline impact is minor.
* jgras/topic/jgras/packet-analyzer-history-rework:
Deprecate old unknown_protocol event signature
Use raw pointer for packet analyzer history
Update external baselines for analyzer history
Update btest baselines for analyzer history
Add packet analyzer history
The current test attempts to instantiate two spicy::SSH_1 protocol
analyzers in the .evt file. The intention likely was to use two
distinct protocol analyzer both trying to replace the builtin SSH
analyzer.
Coincidentally, fixing this happens to workaround TSAN errors tickled
by the FatalError() call while loading the .hlto with two identically
named analyzers.
$ cat .tmp/spicy.replaces-conflicts/output
error: redefinition of protocol analyzer spicy::SSH_1
ThreadSanitizer: main thread finished with ignores enabled
One of the following ignores was not ended (in order of probability)
Ignore was enabled at:
#0 __llvm_gcov_init __linker___d192e45c25d5ee23-484d3e0fc2caf5b4.cc (ssh.hlto+0x34036) (BuildId: 091934ca4da885e7)
#1 __llvm_gcov_init __linker___d192e45c25d5ee23-484d3e0fc2caf5b4.cc (ssh.hlto+0x34036) (BuildId: 091934ca4da885e7)
...
I was tempted to replace FatalError() with Error() and rely on
zeek-setup.cc's early exiting on any reporter errors, but this
seems easier for now.
Relates to #3865.
This avoids the callbacks from being processed on the worker thread
spawned by Civetweb. It fixes data race issues with lookups involving
global variables, amongst other threading issues.
The ctu-sme-11-win7ad-1-ldap-tcp-50041.pcap file was harvested
from the CTU-SME-11 (Experiment-VM-Microsoft-Windows7AD-1) dataset
at https://zenodo.org/records/7958259 (DOI 10.5281/zenodo.7958258).
Closes#3853
Pcap was generated as follows. Doesn't seem wireshark even parses
this properly right now.
with common.get_connection() as c:
with c.cursor() as cur:
date1 = datetime.date(1987, 10, 18)
datetime1 = datetime.datetime(1990, 9, 26, 12, 13, 14)
cur.add_attribute("number1", 42)
cur.add_attribute("string1", "a string")
cur.add_attribute("date1", date1)
cur.add_attribute("datetime1", datetime1)
cur.execute("SELECT version()")
result = cur.fetchall()
print("result", result)
Remove caching_sha2_password parsing/state from the analyzer and implement
the generic events. If we actually want to peak into the authentication
mechanism, we could write a separate analyzer for it. For now, treat it
as opaque values that are exposed to script land.
The added tests show the --get-server-public-key in use where
mysql_auth_more_data contains an RSA public key.
* origin/topic/vern/script-opt-maint.Aug24:
minor optimization of boolean comparisons
fix & regression test for GH-3839 (spurious warnings for "when" constructs)
