* origin/topic/awelzel/script-opt-less-includes:
script_opt/ZAM/IterInfo.h: Add missing Dict.h dependency
script_opt/ZAM: ZBody.h / Support.h: Cleanup includes, use forward declarations
script_opt/ZAM/Profile: Remove Zeek header includes
script_opt: Extend Support.h to break include dependencies
script_opt: Do not include ZBody.h in ZAM/Validate.cc
This adds a protocol parser for the PostgreSQL protocol and a new
postgresql.log similar to the existing mysql.log.
This should be considered preliminary and hopefully during 7.1 and 7.2
with feedback from the community, we can improve on the events and logs.
Even if most PostgreSQL communication is encrypted in the real-world, this
will minimally allow monitoring of the SSLRequest and hand off further
analysis to the SSL analyzer.
This originates from github.com/awelzel/spicy-postgresql, with lots of
polishing happening in the past two days.
We previously would ignore any files under `auxil/`. Since pre-commit
only formats files tracked in this repo, and `auxil/` today contains
only submodules this exclusion did not exclude anything; if however we
ever end up adding C++ files under `auxil/` we would like them to be
formatted consistently.
This patch contains a bump of the configured clang-format version from
17.0.3 to 18.1.8 and automatically generated C++ source updates. The
main difference we are seeing from this is fixes for previously
incomplete reformats.
* origin/topic/awelzel/less-threading-star-star:
logging/WriterFrontend: No need for explicit CleanupWriteBuffer()
logging: Switch index-assignment of raw pointers to emplace_back()
broker/logging: Change threading::Value** usage std::vector instead
threading/Value: Support move and copy constructors
Rebuilding ZBody.cc is annoyingly slow with UBSAN or ASAN enabled. A rebuild
is currently triggered when touching Manager.h files like logging/Manager.h,
broker/Manager.h or file_analysis/Manager.h and various other headers that are
included from there or directly in ZBody.h.
Add tiny wrapper functions to Support.h that encapsulate the actual Zeek
functionality and move the includes to Support.cc instead. This will
cause frequent Support.cc rebuilds, but that is decently fast.
This comes at the expense of one indirect function call for some ops, but
its overhead should be negligible in comparison to a lookup for a
connection or component or a log_mgr->Write() call.
This allows to leverage automatic memory management, less allocations
and using move semantics for expressing ownership.
This breaks the existing logging and broker API, but keeps the plugin
DoWrite() and HookLogWrite() methods functioning.
It further changes ValToLogVal to return a threading::Value rather than
a threading::Value*. The vector_val and set_val fields unfortunately
use the same pointer-to-array-of-pointers approach. this can'tbe changed
as it'd break backwards compatibility for plugin provided input readers
and log writers.
Touching broker/Manager.h or logging/Manager.h or anything that's
included in ZBody.h results in a rebuild of a bunch of files in ZAM.
Particularly, ZAM/Validate.cc takes much too long to recompile for
me (45+ seconds) and apparently that's mostly by accident.
Clean up the ZAM/Validate.cc includes to prevent this.
The current implementation would only log, if the password contains a
colon, the part before the first colon (e.g., the password
`password:password` would be logged as `password`).
A test has been added to confirm the expected behaviour.
Depending on an analyzer's implementation, its Done() method may
attempt to access analyzer or connection state when executing.
When this happens in the destructor of the parent analyzer during
the process of destructing a connection, this state may have been
deleted, resulting in use-after-free crashes or worse memory
corruption.
The following cases have been observed in the wild for when this happens.
* PIA matching during Done() for undelivered TCP data enables a Spicy
based analyzer which in turn attempts to raise an analyzer violation
during Done()->EndOfData().
* Spicy analyzers attaching new analyzers during their Done() processing
which in turn attempt to use TCP() (to call FindChild()) during Done()
while the analyzer tree / connection is being destructed.
The second scenario was previously found to happen in the HTTP analyzer
and fixed with 6ef9423f3c.
Plug these scenarios by short-circuiting AddChildAnalyzer() if the analyzer
or connection have finished or are being finished.
It turns out that, for probably a long time, we have reported an
incorrect version when parsing an SSLv2 client hello. We always reported
this as SSLv2, no matter which version the client hello actually
contained.
This bug probably went unnoticed for a long time, as SSLv2 is
essentially unused nowadays, and as this field does not show up in the
default logs.
This was found due to a baseline difference when writing the Spicy SSL
analyzer.
This avoids the earlier problem of not tracking ports correctly in
scriptland, while still supporting `port` in EVT files and `%port` in
Spicy files.
As it turns out we are already following the same approach for file
analyzers' MIME types, so I'm applying the same pattern: it's one
event per port, without further customization points. That leaves the
patch pretty small after all while fixing the original issue.
* origin/topic/johanna/ssl-history-also-for-sslv2-not-only-for-things-that-use-the-more-modern-handshake:
Make ssl_history work for SSLv2 handshakes/connections
