Matching of plugins' debug streams was still case-sensitive. Also contains some
minor output tweaks.
It'd be nice to only list plugin debug streams actually _used_ by plugins. I
didn't see a quick way to do that so that's for another time.
With this commit, the entire Zeek test suite passes using spicy TLS.
Tests that either use a SSLv2 handshake, or DTLS are skipped, as the
parser currently does not support either.
Similarly, tests that rely on behavior we cannot replicate (baseline,
hooks, exact error messages) are passed. Other than that, all the
TLS-based tests pass with 100% the exact same baseline results.
This necessitated a couple of small tweaks to the spicy file - the
testcases uncovered several small problems.
This commit also enables cirrus tests for Spicy SSL/TLS.
The variant ended up conflicting with std::bind, which resulted in failures
on the btest invoking it. Change back to a single function that takes a
flow, and default it to a value in Exec.
Add a new overload to `copy_string` that takes the input characters plus
size. The new overload avoids inefficient scanning of the input for the
null terminator in cases where we know the size beforehand. Furthermore,
this overload *must* be used when dealing with input character sequences
that may have no null terminator, e.g., when the input is from a
`std::string_view` object.
This largely copies over Spicy's `.clang-format` configuration file. The
one place where we deviate is header include order since Zeek depends on
headers being included in a certain order.
Motivation is basically the same as in 88bb527026.
For plugin.hooks, one example is that adding a new option in the default script
changes the baseline due registration of change handlers. Also, the connection
record is printed in various places, resulting in churn when the default
scripts change.
Dot directories rarely contain anything we would want to load as a
dynamic plugin. Even worse, they likely contain files with externally
controlled lifetimes which might be removed while we are using them
(see e.g., zeek/btest#98).
With this patch we do not search _discovered_ dot directories anymore.
We continue to load from a user-specified `ZEEK_PLUGIN_PATH`, even if
its name starts with a dot.
Since this patch changes previous behavior it is a **BREAKING CHANGE**.
This was brought up on Slack as a potential source of confusion during
development as it's not visible when plugin directories are ignored outside
of looking into the plugin debug stream. I'd actually prefer to just
FatalError() this, but a warning seems reasonably visible for interactive
usage.
* origin/topic/neverlord/version-header:
testing/zeek-version-link: Assume nm is there
Drop dependency for zeek_dynamic_plugin_base
Fixup ifdef check in Plugin.h
Update cmake_minimum_required() in test plugins
testing: Add zeek-version-link tests
Clean up ZEEK_CONFIG_SKIP_VERSION_H issues
cmake-format during merge
zeek.on('zeek_init', () => {
console.log('Hello, Zeek!');
});
For interaction with external systems and HTTP APIs, JavaScript and the
Node.js ecosystem beat Zeek script. Make it more easily accessible by
including ZeekJS with Zeek directly.
When a recent enough libnode version is found on the build system, ZeekJS is
added as a builtin plugin. This behavior can be disabled via
``--disable-javascript``. Linux distributions providing such a package are
Ubuntu (22.10) and Debian (testing/bookworm) as libnode-dev.
Fedora provides it as nodejs-devel.
This plugin takes over loading of .js or .cjs files. When no such files
are provided to Zeek, Node and the V8 engine are not initialized and
should not get into the way.
This should be considered experimental.
This is mostly for documentation/verification purposes of how the IO loop
currently does draining and when it picks up FD based (non packet) IO
source. For example, it shows that currently FD based sources are processed
fairly delayed and that we now also process two timeout sources that are ready.
This has come up a few times and the motivation is mainly better "first timer"
experience with Zeek. Concretely, if one wants to run a Zeek cluster with
multiple workers and reasonable load balancing on Linux, AF_PACKET is a decent
start. Without AF_PACKET support being built into Zeek, however, a new user's
next experience is that of setting up a development environment in order
to compile an external plugin (think compiler, kernel headers, zkg, ...).
Only to get what could be termed basic functionality.
This is using the ZEEK_INCLUDE_PLUGINS infrastructure. I've used the all
upper case spelling of AF_PACKET in the help output because it seems everyone
else references/writes it like that. I think we should also write it
like that in the docs.
This goes the hard-exit on conflicts route as IMO it provides better
messaging that something is wrong, rather than defaulting to something
the user may not expect.
Fixes#2403
Add a test parsing a malformed PE file showing that analyzer_violation_info
is raised with the fa_file object set.
It could be interesting to pass through an optional connection if one
exists, but access is provided through f$conns, too.
Adds base/frameworks/telemetry with wrappers around telemetry.bif
and updates telemetry/Manager to support collecting metrics from
script land.
Add policy/frameworks/telemetry/log for logging of metrics data
into a new telemetry.log and telemetry_histogram.log and add into
local.zeek by default.
One change: turns out that zeek-config isn't in PATH, so I prefixed it
with $BUILD.
* origin/topic/awelzel/913-plugin-debug-stream-validation:
zeek-setup: Validate plugin debug streams during startup
Providing an unknown plugin debug stream with -B was previously silently
accepted. This caused user confusing as the behavior is "no output, but
seems to work".
Check the enabled debug streams once all plugins have been loaded and
exit early on for invalid streams.
$ ZEEK_PLUGIN_PATH=./build zeek -B plugin-zeek-myplugin -e 'print zeek_version();'
error in <command line>, line 3: No plugin debug stream 'plugin-zeek-myplugin' found
Closes#913.
This was excercising hooks to validate that they were called. For that
it uses pretty verbose logging. Since version numbers are not stable and
hard to canonify this script already unloads the version module (there
is e.g., code in `base/misc/version` which splits the version strings
and works on single components which might appear like pretty ordinary
floating point numbers in the log).
This test however worked under the assumption that nothing in bare mode
loads `base/misc/version` which is hard to guarantee, especially
considering that plugins can be embedded and might load that module
themself.
With this patch we now make the logging slightly less verbose so that
function call arguments are not logged anymore.
#1835 subtly changed the semantics of the `LoadFile` plugin hook to no
longer have the current script location available for signature files
being loaded through `@load-sigs`. This was undocumented behavior, so
it's technically not a regression, but since at least one external
plugin is depending on it, this change restores the old behavior.
Conceptually, a TCP-based application analyzer should not need any
knowledge about the underlying TCP analysis; it's supposed to just
process its reassembled input stream as it's handed over. But our
analyzers break that assumption at a few places because sometimes
knowledge about the TCP state of the connection can be helpful for
heuristics. This is fine as long as there actually *is* a TCP parent
analyzer available. Sometimes, however, there isn't: if the payload
stream is encapsulated inside another application-layer protocol, the
semantic link to TCP is broken. And if the outer connection is even
UDP, then we don't have a TCP analyzer at all.
We didn't handle this situation well so far. Most analyzers needing
TCP state would just crash if there's no TCP analyzer (in debug mode
with an `assert`, in release mode with a null pointer deref ...). Only
HTTP did the right thing already: check if TCP is available and adapt
accordingly.
We know extend that check to all other analyzers as well: all accesses
to `TCP()` are guarded, with reasonable defaults if not available.
It's actually a pretty small change overall, which is evidence for how
little this layering violation actually matters.
The existing behavior is what's causing
https://github.com/corelight/zeek-spicy-openvpn/issues/3.
