* origin/master: (27 commits)
Update doc submodule [nomail] [skip ci]
btest/ldap: Add regression test for #3919
postgresql: Simplify SSL buffering and forwarding
postgresql: Initial parser implementation
testing/external: Update private baselines
analyzer/syslog: Reformat with spicy-format
analyzer/finger: Reformat with spicy-format
scripts/spicy: Reformat with spicy-format
pre-commit: Add spicy-format
Check for netbios to avoid reporting extra bad DNS opcodes
Add weird for unhandled opcodes in DNS analyzer
Bump zeek-aux for zeek/zeek-aux#57
Remove pre-commit exclusions for clang-format
Bump clang-format
Bump auxil/spicy to latest development snapshot
RunState: Drop broker_mgr->Active() usage
script_opt/ZAM/IterInfo.h: Add missing Dict.h dependency
script_opt/ZAM: ZBody.h / Support.h: Cleanup includes, use forward declarations
script_opt/ZAM/Profile: Remove Zeek header includes
script_opt: Extend Support.h to break include dependencies
...
* origin/topic/awelzel/script-opt-less-includes:
script_opt/ZAM/IterInfo.h: Add missing Dict.h dependency
script_opt/ZAM: ZBody.h / Support.h: Cleanup includes, use forward declarations
script_opt/ZAM/Profile: Remove Zeek header includes
script_opt: Extend Support.h to break include dependencies
script_opt: Do not include ZBody.h in ZAM/Validate.cc
This adds a protocol parser for the PostgreSQL protocol and a new
postgresql.log similar to the existing mysql.log.
This should be considered preliminary and hopefully during 7.1 and 7.2
with feedback from the community, we can improve on the events and logs.
Even if most PostgreSQL communication is encrypted in the real-world, this
will minimally allow monitoring of the SSLRequest and hand off further
analysis to the SSL analyzer.
This originates from github.com/awelzel/spicy-postgresql, with lots of
polishing happening in the past two days.
We previously would ignore any files under `auxil/`. Since pre-commit
only formats files tracked in this repo, and `auxil/` today contains
only submodules this exclusion did not exclude anything; if however we
ever end up adding C++ files under `auxil/` we would like them to be
formatted consistently.
This patch contains a bump of the configured clang-format version from
17.0.3 to 18.1.8 and automatically generated C++ source updates. The
main difference we are seeing from this is fixes for previously
incomplete reformats.
* origin/topic/awelzel/less-threading-star-star:
logging/WriterFrontend: No need for explicit CleanupWriteBuffer()
logging: Switch index-assignment of raw pointers to emplace_back()
broker/logging: Change threading::Value** usage std::vector instead
threading/Value: Support move and copy constructors
Rebuilding ZBody.cc is annoyingly slow with UBSAN or ASAN enabled. A rebuild
is currently triggered when touching Manager.h files like logging/Manager.h,
broker/Manager.h or file_analysis/Manager.h and various other headers that are
included from there or directly in ZBody.h.
Add tiny wrapper functions to Support.h that encapsulate the actual Zeek
functionality and move the includes to Support.cc instead. This will
cause frequent Support.cc rebuilds, but that is decently fast.
This comes at the expense of one indirect function call for some ops, but
its overhead should be negligible in comparison to a lookup for a
connection or component or a log_mgr->Write() call.
This allows to leverage automatic memory management, less allocations
and using move semantics for expressing ownership.
This breaks the existing logging and broker API, but keeps the plugin
DoWrite() and HookLogWrite() methods functioning.
It further changes ValToLogVal to return a threading::Value rather than
a threading::Value*. The vector_val and set_val fields unfortunately
use the same pointer-to-array-of-pointers approach. this can'tbe changed
as it'd break backwards compatibility for plugin provided input readers
and log writers.
Touching broker/Manager.h or logging/Manager.h or anything that's
included in ZBody.h results in a rebuild of a bunch of files in ZAM.
Particularly, ZAM/Validate.cc takes much too long to recompile for
me (45+ seconds) and apparently that's mostly by accident.
Clean up the ZAM/Validate.cc includes to prevent this.
The current implementation would only log, if the password contains a
colon, the part before the first colon (e.g., the password
`password:password` would be logged as `password`).
A test has been added to confirm the expected behaviour.
The reason that this is necessary is the end-of-connection-handling of
spicy. If spicy is in the middle of parsing some bytes while the
connection ends, an error is raised. This behavior cannot be changed,
and means that there will be a DPD-log entry, etc. for connections that
are completely valid TLS connections - that just happen to be truncated
and end in the middle.
* origin/master:
Update doc submodule [nomail] [skip ci]
Analyzer: Do not add child analyzers when finished
Fix parsing of version field in SSLv2 client hello
TCP_Reassembler: Fix IsOrig() position in Match() call
Spicy: Register well-known ports through an event handler.
Update doc submodule [nomail] [skip ci]
Revert "Remove deprecated port/ports fields for spicy analyzers"
Make ssl_history work for SSLv2 handshakes/connections
Depending on an analyzer's implementation, its Done() method may
attempt to access analyzer or connection state when executing.
When this happens in the destructor of the parent analyzer during
the process of destructing a connection, this state may have been
deleted, resulting in use-after-free crashes or worse memory
corruption.
The following cases have been observed in the wild for when this happens.
* PIA matching during Done() for undelivered TCP data enables a Spicy
based analyzer which in turn attempts to raise an analyzer violation
during Done()->EndOfData().
* Spicy analyzers attaching new analyzers during their Done() processing
which in turn attempt to use TCP() (to call FindChild()) during Done()
while the analyzer tree / connection is being destructed.
The second scenario was previously found to happen in the HTTP analyzer
and fixed with 6ef9423f3c.
Plug these scenarios by short-circuiting AddChildAnalyzer() if the analyzer
or connection have finished or are being finished.
