* origin/topic/vern/type-names:
regularize deprecation warning
make DoDescribe protected for a tidier interface mark use of DESC_PORTABLE as deprecated
when printing a description of a type, use its name if available
* origin/topic/vern/footprint:
new environment variable to enable BTests to skip ASAN checks
skip new BiF test for ASAN CI runs
btest update to include recursive value that doesn't require a record
to avoid recursion, track all aggregates, not just records isolate the internal methods
simpler public calling interface for computing footprint
use stack-based set to prevent infinite recursion rather than a static one
change value_footprint() to val_footprint() to be more similar to val_size()
make including count of container elements non-optional
btest for mutually-recursive case
fix for tracking footprints of mutually-recursive records
added value_footprint() and global_container_footprints() BiFs
* origin/topic/vern/zam-maint-01May22:
fix for coverage reporting for functions that use "when" statements
ZAM maintenance for recent changes and some newly exercised corner cases
This opens up the possibility of storing other request types outside
of T_A, T_PTR and T_TXT without requiring redoing the caching. It
also fixes the caching code in DNS_Mapping, adding a version number
to the start of the cache file so the cache structure can be modified
and old caches invalidated more easily.
* topic/christian/management-print-cmd:
Management framework: bump external testsuite
Management framework: allow selecting cluster nodes in get_id_value
Management framework: minor tweaks to logging component
Management framework: bump zeek-client to pull in get-id-value command
Avoid whitespace around function type strings in JSON rendering
Management framework: improve handling of node run states
Management framework: add get_id_value dispatch
Management framework: allow dispatching "actions" on cluster nodes.
Management framework: some renaming to avoid the term "data cluster"
Management framework: allow agents to communicate with cluster nodes
This provides Broker-level plumbing that allows agents to reach out to their
managed Zeek nodes and collect responses.
As a first event, it establishes Management::Node::API::notify_agent_hello,
to notify the agent when the cluster node is ready to communicate.
Also a bit of comment rewording to replace use of "data cluster" with simply
"cluster", to avoid ambiguity with data nodes in SumStats, and expansion of
test-all-policy.zeek and related/dependent tests, since we're introducing new
scripts.
#1835 subtly changed the semantics of the `LoadFile` plugin hook to no
longer have the current script location available for signature files
being loaded through `@load-sigs`. This was undocumented behavior, so
it's technically not a regression, but since at least one external
plugin is depending on it, this change restores the old behavior.
* origin/topic/vern/table-attr-fixes:
updates for btests - new cases to check, new baselines
updates for btests - new cases to check, new baselines
fix for ill-formed (complex) &default function
type-checking for use of empty table constructors in expressions
catch empty constructors used for type inference suppress repeated error messages
factoring to make checking of &default attributes externally accessible
bug fix for empty table constructors with &default attributes (plus a typo)
* ynadji/raw_bytes_to_v6_addr:
add raw_bytes_to_v6_addr in docs when raw_bytes_to_v4_addr is present
Zero out bytes by default for consistent return value on error
Add tests for raw_bytes_to_v6_addr
Add raw_bytes_to_v6_addr function
* origin/topic/vern/rec-constr-check:
associated btest
fix base scripts to include mandatory fields in record constructors
restored record constructor checking for missing-but-mandatory fields
* origin/topic/christian/gh-1982:
The is_num(), is_alpha(), and is_alnum() BiFs now return F on empty string
Expand testcases around is_num(), is_alpha(), is_alnum(), is_ascii() BiFs
Clarify is_ascii() BiF docstring re behavior on empty strings
Fix minor indentation bugs in strings.bif
Documentation is missing and will be added in the next couple of hours.
* origin/topic/johanna/tls12-decryption: (24 commits)
TLS decryption: add test, fix small issues
Address PR feedback
TLS decryption: refactoring, more comments, less bare pointers
Small code fix and test baseline update.
SSL decryption: refactor TLS12_PRF
SSL decryption: small style changes, a bit of documentation
Deprecation and warning fixes
Clang-format updates
add missing call to EVP_KDF_CTX_set_params
TLS decryption: remove payload from ssl_encrypted_data again.
TLS 1.2 decryption: adapt OpenSSL 3.0 changes for 1.1
ssl: adapt TLS-PRF to openSSL 3.0
ssl/analyzer: potentially fix memory leaks caused by bytestrings
analyzer/ssl: several improvements
analyzer/ssl: defensive key length check + more debug logging
testing: feature gate ssl/decryption test
testing: add ssl/decryption test
analyzer/ssl: handle missing <openssl/kdf.h>
analyzer/ssl: silence warning in DTLS analyzer
analyzer/ssl: move proc-{client,server}-hello into the respective analyzers
...
- This gives the cluster controller and agent the common name "Management
framework" and changes the start directory of the sources from
"policy/frameworks/cluster" to "policy/frameworks/management". This avoids
ambiguity with the existing cluster framework.
- It renames the "ClusterController" and "ClusterAgent" script modules to
"Management::Controller" and "Management::Agent", respectively. This allows us
to anchor tooling common to both controller and agent at the "Management"
module.
- It moves common configuration settings, logging, requests, types, and
utilities to the common "Management" module.
- It removes the explicit "::Types" submodule (so a request/response result is
now a Management::Result, not a Management::Types::Result), which makes
typenames more readable.
- It updates tests that depend on module naming and full set of scripts.
This commit changes DPD matching for TLS connections. A one-sided match
is enough to enable DPD now.
This commit also removes DPD for SSLv2 connections. SSLv2 connections do
basically no longer happen in the wild. SSLv2 is also really finnicky to
identify correctly - there is very little data required to match it, and
basically all matches today will be false positives. If DPD for SSLv2 is
still desired, the optional signature in policy/protocols/ssl/dpd-v2.sig
can be loaded.
Fixes GH-1952
Conceptually, a TCP-based application analyzer should not need any
knowledge about the underlying TCP analysis; it's supposed to just
process its reassembled input stream as it's handed over. But our
analyzers break that assumption at a few places because sometimes
knowledge about the TCP state of the connection can be helpful for
heuristics. This is fine as long as there actually *is* a TCP parent
analyzer available. Sometimes, however, there isn't: if the payload
stream is encapsulated inside another application-layer protocol, the
semantic link to TCP is broken. And if the outer connection is even
UDP, then we don't have a TCP analyzer at all.
We didn't handle this situation well so far. Most analyzers needing
TCP state would just crash if there's no TCP analyzer (in debug mode
with an `assert`, in release mode with a null pointer deref ...). Only
HTTP did the right thing already: check if TCP is available and adapt
accordingly.
We know extend that check to all other analyzers as well: all accesses
to `TCP()` are guarded, with reasonable defaults if not available.
It's actually a pretty small change overall, which is evidence for how
little this layering violation actually matters.
The existing behavior is what's causing
https://github.com/corelight/zeek-spicy-openvpn/issues/3.
Intersecting two bloom filters yields a bloom filter that returns true
when an element was contained in both bloom filters. The false positive
rate is potentially a bit higher than in the original bloom filters.
This operation also works for counting bloom filters, however the
counters are discarded and the bloomfilters are converted to basic bloom
filters. The reason is that there is no obvious meaning to the counters
when two bloom filters are intersected - besides the fact if an element
was inserted at all.
