* origin/topic/seth/tsv-logs-utf8-by-default:
Fix mis-usage of string::append that leads to an overflow
Use json_escape_utf8 for all utf8 data in ODesc
Switch the TSV Zeek logs to be UTF8 by default.
Due to a bug (or intentional code) in SQLite, we disabled enabling the shared cache
in sqlite3 if running under ThreadSanitizer (see cf1fefbe0b0a6163b389cc92b5a6878c7fc95f1f).
Unfortunately, this has the side-effect of breaking the simultaneous-writes test because
the shared cache is disabled. This is hopefully a temporary fix until SQLite fixes the
issue on their side.
Delete the unused Baseline folder, remove a Bro-era name from gitignore,
change create-new-repo to work with a second argument that is actually
a URL, and tweak whitespace. Expand description in README and update the
explanation of OPENSSL_ENABLE_MD5_VERIFY in subdir-btest.cfg.
This adds a new function for validating UTF-8 sequences by converting to
UTF-32. This allows us to also check for various blocks of codepointsi
that we consider invalid while checking for valid sequences in general.
Broker::NetworkInfo$address is of type string. The Broker::peers() BiF treated
it as an address, which caused ZVal's type validation to flag a
mismatch and abort.
Includes a small tweak to the broker.unpeer test that makes it invoke
Broker::peers(), which breaks in absence of the fix.
We previously computed the length of the Geneve options field
incorrectly which lead to us passing data at an incorrect offset to
inner analyzers. With this patch we now interpret the length field
correctly, according the the spec
https://datatracker.ietf.org/doc/html/rfc8926#section-3.4.
Closes#1726.
This could lead to a noticeable (single-percent) performance
improvement.
Most of the functionality for this is in the packet analyzers that now
cache ignore_chesksums_nets.
Based on a patch by Arne Welzel (Corelight).
When cloning TableVals, a new timer was created for the wrong object
(the existing TableVal, not the clone). This lead to the already
existing timer being no longer accessible. Which, in turn, leads to an
abandoned timer reading into no longer allocated data when the original
TableVal is deleted.
Fixes GH-1687
With the recent packet manager work, it broke to register a protocol
analyzer for a specific port from inside a plugin's initialization code.
That's because that registration now depends on the packet manager being
set up, which isn't case at that time a plugin's `InitPostInit()` runs.
This fix contains two parts:
- Initialize the packet manager before the analyzer manager, so that
the latter's `InitPostScript()` can rely on the former being
ready.
- Change the analyzer manager to (only) record port registrations
happening before it's fully initialized. Its `InitPostScript()`
then performs the actual registrations, knowing it can use the
packet manager now.
This comes with a `cmake/` to add a missing include directory.
* origin/topic/vlad/dns_rr_fixes:
Add a TODO to return a correct status for ParseRR_WKS
Remove unnecessary if
Generate dns_WKS_reply event
dns_HINFO_reply event was never being generated.
On FreeBSD, this test showed two problems: (1) reordering problems
based on writing the predicate, event, and end-of-data updates into a
single file, (2) a race condition based on printing the entirety of
the table description argument in update events. The description
contains the destination table, and its content at the time an update
event gets processed isn't deterministic: depending on the number
of updates the reader thread has sent, the table will contain a
varying number of entries.
The invalidset and invalidtext tests loaded an input file via table
and event reads, in parallel. On FreeBSD this triggers an occasional
reordering of messages coming out of the reader thread vs the input
managers. This commit makes the table and event reads sequential,
avoiding the race.
When omitted, the node inherits the Supervisor's bare-mode
status. When true/false, the new Zeek node will enable/disable bare
mode, respectively. It continues to load any scripts passed at the
command line and in the additional scripts list already provided in
the node configuration.
Includes testcase.
The NodeConfig record now has a table for specifying environment variable names
and values, which the supervisor sets in the created node.
This also repositions the cpu_affinity member to keep the order the same in
the corresponding script-layer and in-core types.
Includes testcase.
* origin/topic/johanna/gh-859:
Add X509/SSL changes to NEWS
X509: add check if function succeeds
GH-1634: Address feedback
Small indentation fixes in ssl-log-ext.zeek
Fix memory leak in x509_check_cert_hostname bif
Small bugfix and updates for external test hashes (SSL/X509)
Baseline updates for recent SSL changes.
Add ability to check if hostname is valid for a specific cert
Add ssl_history field to ssl.log
Add policy script suppressing certificate events
Add new ssl-log-ext policy script
Deprecate extract-certs-pem.zeek and add log-certs-base64.zeek
Implement X509 certificate log caching
Deprecate ICSI SSL notary script.
Change SSL and X.509 logging format
Enable OCSP logging by default.
Split the code that handles X509 event hashing into its own file
Closes GH-859
This addresses the need for a central hook on any log write, which
wasn't previously doable without a lot of effort. The log manager
invokes the new Log::log_stream_policy hook prior to any filter-specific
hooks. Like filter-level hooks, it may veto a log write. Even when
it does, filter-level hooks still get invoked, but cannot "un-veto".
Includes test cases.
The framework so far populated data structures with missing fields
even when those fields are defined without the &optional
attribute. When using the attribute, such entries continue to get
populated.
Update tests to reflect focus on unset fields.
* origin/topic/seth/small-builtin-plugin-fixes:
Fix tests
Removed a non-functional builtin plugin preload loading mechanism
Load the builtin-plugin preload files and fix the dev path.
