* origin/topic/awelzel/3504-ldap-logs-scalars:
Update external baselines
ldap: Use scalar values in logs where appropriate
ldap: Rename LDAP::search_result to LDAP::search_result_entry
* origin/topic/bbannier/issue-3234:
Introduce dedicated `LDAP::Info`
Remove redundant storing of protocol in LDAP logs
Use LDAP `RemovalHook` instead of implementing `connection_state_remove`
Tidy up LDAP code by using local references
Pluralize container names in LDAP types
Move LDAP script constants to their own file
Name `LDAP::Message` and `LDAP::Search` `*Info`
Make ports for LDAP analyzers fully configurable
Require have-spicy for tests which log spicy-ldap information
Fix LDAP analyzer setup for when Spicy analyzers are disabled
Bump zeek-testing-private
Integrate spicy-ldap test suite
Move spicy-ldap into Zeek protocol analyzer tree
Explicitly use all of spicy-ldap's modules
Explicitly list `asn1.spicy` as spicy-ldap source
Remove uses of `zeek` module in spicy-ldap
Fix typos in spicy-ldap
Remove project configuration files in spicy-ldap
Integrate spicy-ldap into build
Import zeek/spicy-ldap@57b5eff988
This moves the ports the LDAP analyzers should be triggered on from the
EVT file to the Zeek module. This gives users full control over which
ports the analyzers are registered for while previously they could only
register them for additional ports (there is no Zeek script equivalent
of `Manager::UnregisterAnalyzerForPort`).
The analyzers could still be triggered via DPD, but this is intentional.
To fully disable analyzers users can use e.g.,
```zeek
event zeek_init()
{
Analyzer::disable_analyzer(Analyzer::ANALYZER_LDAP_TCP);
}
```
The diffs produced by telemetry.log when introducing a weird or
removing/adding protocol specific logs is overwhelming and distracting
without providing value. Exclude telemetry.log similar to how we already
exclude stats.log.
Some more targeted telemetry.log tests exists in the normal testing/btest
suite and that appears more sensible.
This change makes the community-id script that adds the community id to
notice.log automatically load the main script if this was not already
loaded.
In the past, the script just did not perform any action if the main
script was not loaded.
This change also makes the notice script respect the seed/base64
settings that were set in the main script.
Fixes GH-3242
* origin/topic/vern/zam-feature-complete: (23 commits)
documentation updates
updates to ZAM-specific BTest baseline
Address PR review feedback on zam-feature-complete
Updates to BTest baselines due to previous BTest tweaks
tweaks to BTests for ZAM feature completeness; new ZAM-only btest
removal of unused functionality and some follow-on simplifications
feature completeness for ZAM
-O gen-C++ tweaks to be compatible with ZAM changes
ZAM support for "when" statements
ZAM changes intermixed with lambda and "when" support
WhenStmt/WhenInfo restructuring in support of ZAM "when" statements
ZAM support for lambdas
ZAM internals have a notion of "captures" as global-like variables
AST profiling enhnacements in support of script optimization for lambdas/whens
refinements to LambdaExpr's to provide flexibility, support for ZVal captures
support in ScriptFunc class for ZVal-oriented vector of captures
simplifications to the Frame class now that it no longer has to support old-style captures
use Ingredients directly for constructing functions
the "Capture" struct is now a class
more debugging information when dumping script optimization data structures
...
* origin/topic/timw/2914-truncated-tunnel-plus-things:
Address wire/capture length feedback
packet_analysis/TCP: Do not use untrusted len for DeliverPacket()
Add regression test using pcap from GH-2683
Add btest to test Geneve->VXLAN->Truncated inner packet
IP: Update packet->len with accumulated fragment size
UDP: Forward any remaining data (also empty) to session-analysis
IPTunnel: Compute inner wire length based on cap_len differences.
IP: fix weird name to not be ipv6 specific
UDP: don't validate checksum if caplen < len
PIA: Modernize how struct initialization is done
* origin/topic/awelzel/add-community-id:
testing/external: Bump hashes for community_id addition
NEWS: Add entry for Community ID
policy: Import zeek-community-id scripts into protocols/conn frameworks/notice
Add community_id_v1() based on corelight/zeek-community-id
There was a misunderstanding whether to include them by default in
the dns.log, so remove them again.
There had also been a discussion and quirk that AD of a request would
always be overwritten by reply in the dns.log unless the reply is
missing. For now, let users extend dns.log themselves for what best
fits their requirements, rather than adding these flags by default.
Add a btest to print AD and CD flags for smoke testing still.
* topic/christian/gh-2761-default-localnets:
Update NEWS to cover new Site::local_nets behavior.
Bump external testsuites to reflect updated baselines
Add btests for new Site::local_nets behavior
Update plugins.hooks baseline to reflect added config framework activity
Update scripts.policy.misc.dump-events baseline
Update btest baselines to reflect the use of local address ranges.
Tighten local-nets filtering in the logging framework's path-func-column-demote test
Fix scripts.policy.frameworks.intel.removal test given address locality info
Treat private address space as site-local by default
Provide a mechanism to suppress logging of internal config framework activity
* 'dnssec-flag-parse' of github.com:micrictor/zeek-codespace:
Update external testing commit hash for DNS flag changes
Parse DNSSEC AD and CD bits
Updated dump-events baseline which seemed unrelated.
Mostly, treat HTTP0.9 completely separate. Because we're doing raw
delivery of a body directly, fake enough (connection_close=1, and finish
headers manually) so that the MIME infrastructure thinks it is seeing a
body.
This deals better with the body due to accounting for the first line. Also
it avoids the content line analyzer to strip CRLF/LF and the analyzer
then adding CRLF unconditionally by fully bypassing the content line
analyzer.
Concretely, the vlan-mpls test case contains a HTTP response with LF only,
but the previous implementation would use CRLF, accounting for two many bytes.
Same for the http.no-version test which would previously report a body
length of 280 and now is at 323 (which agrees with wireshark).
Further, the mime_type detection for the http-09 test case works because
it's now seeing the full body.
Drawback: We don't extract headers when a server actually replies with
a HTTP/1.1 message, but grrr, something needs to give I guess.
This commit introduces parsing of the CertificateRequest message in the
TLS handshake. It introduces a new event ssl_certificate_request, as
well as a new function parse_distinguished_name, which can be used to
parse part of the ssl_certificate_request event parameters.
This commit also introduces a new policy script, which appends
information about the CAs a TLS server requests in the
CertificateRequest message, if it sends it.
* origin/topic/awelzel/2572-event-handler-stats:
Bump external baselines
external/testing-setup: Less telemetry.log volume
EventHandler: Use telemetry framework for EventHandler.call_count
The user and password fields are replicated to each of the ftp.log
entries. Using a very large username (100s of KBs) allows to bloat
the log without actually sending much traffic. Further, limit the
arg and reply_msg columns to large, but not unbounded values.
* origin/topic/awelzel/arm64-container-follow-ups-2:
Bump private testsuite for char handling fixes
Base64: report byte as positive integer
netbios_decode: use unsigned char for result
analyzer/http: Do not assume char is signed
cirrus: Run tests in a Debian 11 container, too
As initial examples, this branch ports the Syslog and Finger analyzers
over. We leave the old analyzers in place for now and activate them
iff we compile without any Spicy.
Needs `zeek-spicy-infra` branches in `spicy/`, `spicy-plugin/`,
`CMake/`, and `zeek/zeek-testing-private`.
Note that the analyzer events remain associated with the Spicy plugin
for now: that's where they will show up with `-NN`, and also inside
the Zeekygen documentation.
We switch CMake over to linking the runtime library into the plugin,
vs. at the top-level through object libraries.
