Commit graph

194 commits

Author SHA1 Message Date
Arne Welzel
85b8c8866b testing/btest/*zeek: Comment all @TEST lines 2025-04-17 16:30:23 +02:00
Tim Wojtulewicz
cb1ef47a31 Add STORAGE_ prefixes for backends and serializers 2025-04-14 10:11:13 -07:00
Tim Wojtulewicz
1169fcf2a2 Move byte_buffer types from cluster and storage into util 2025-04-14 10:11:13 -07:00
Tim Wojtulewicz
88786a28a2 Add JSON storage serializer, use with existing backends/tests 2025-04-14 10:11:13 -07:00
Arne Welzel
6d97d5526a plugin/Manager: Fix MetaHookPre and MetaHookPost using HOOK_CALL_FUNCTION 2025-04-11 09:58:07 +02:00
Arne Welzel
cb610bdea2 btest/plugin: Add plugin testing enum identifiers
This plugin has a generic name, but for now just tests the API around
enum types and enum vals at InitPostScript() time.
2025-04-04 15:36:42 +02:00
Tim Wojtulewicz
32ae8f4eaa Make storage events take a tag for the backend instead of a string 2025-03-27 16:12:24 -07:00
Tim Wojtulewicz
85701e4514 Fix handling of timeout conditions from storage backends 2025-03-27 13:52:11 -07:00
Tim Wojtulewicz
046f32a6df Reformat plugin.storage btest to be more consistent with other storage tests 2025-03-27 13:52:11 -07:00
Tim Wojtulewicz
ba9cf1e4db Remove unnecessary type aliases from storage btests 2025-03-21 11:56:27 -07:00
Tim Wojtulewicz
e5b06367f7 Squash code from OperationResultCallback into ResultCallback 2025-03-21 11:44:39 -07:00
Tim Wojtulewicz
e6f1eea1b7 Remove default argument for callbacks, reorder function arguments 2025-03-18 10:20:34 -07:00
Tim Wojtulewicz
9ed3e33f97 Completely rework return values from storage operations 2025-03-18 10:20:33 -07:00
Tim Wojtulewicz
e766af7322 Split sync/async handling into the BIF methods 2025-03-18 10:20:33 -07:00
Tim Wojtulewicz
a485b1d237 Make backend options a record, move actual options to be sub-records 2025-03-18 10:20:33 -07:00
Tim Wojtulewicz
28951dccf1 Split sync and async into separate script-land namespaces 2025-03-18 10:20:33 -07:00
Tim Wojtulewicz
f1a7376e0a Return generic result for get operations that includes error messages 2025-03-18 09:32:34 -07:00
Tim Wojtulewicz
4695060d75 Allow opening and closing backends to be async 2025-03-18 09:32:34 -07:00
Tim Wojtulewicz
7ad6a05f5b Add infrastructure for asynchronous storage operations 2025-03-18 09:32:34 -07:00
Tim Wojtulewicz
d07d27453a Add infrastructure for automated expiration of storage entries
This is used for backends that don't support expiration natively.
2025-03-18 09:32:34 -07:00
Tim Wojtulewicz
8dee733a7d Change args to Storage::put to be a record
The number of args being passed to the put() methods was getting to be
fairly long, with more on the horizon. Changing to a record means simplifying
things a little bit.
2025-03-18 09:32:34 -07:00
Tim Wojtulewicz
69d940533d Pass key/value types for validation when opening backends 2025-03-18 09:32:34 -07:00
Tim Wojtulewicz
2ea0f3e70a Lay out initial parts for the Storage framework
This includes a manager, component manager, BIF and script code, and
parts to support new storage backend plugins.
2025-03-18 09:32:34 -07:00
Arne Welzel
ab99f8e233 scan.l: Fix @load-plugin scripts loading
For a plugin loaded via @load-plugin, create a YY_BUFFER_STATE holding
the required loads for the implicitly loaded files. In loaded scripts,
this generated file will show up with a path of the shared object file
of the plugin with the __preload__.zeek and __load__.zeek files loaded
by it.

Closes #2311
2025-03-04 09:35:00 +01:00
Arne Welzel
927a06b9ab logging: Fix HookLogInit() and HookLogWrite() info usage
There's two instances of WriterBackend::WriterInfo for a given
writer. One in Manager::WriterInfo that's accessible via
stream.writers and a copy within WriterFrontend.

Commit 78999d147d switched to use the
address of the frontend's info instance for HookLogWrite() invocations,
breaking users using the address for identification purposes.
2025-01-14 10:44:50 +01:00
Arne Welzel
85189ca918 btest: Avoid loading ZeroMQ if not compiled in
...at the same time, add some `TEST-REQUIRES: have-zeromq` which
unfortunately means that developers will usually want libzmq
installed on their system.
2024-12-11 18:08:43 +01:00
Arne Welzel
39b43e8dd6 btest/plugins: Add a plugin testing Init and Done hooks
Follow-up for #4047.
2024-11-22 13:37:03 +01:00
Christian Kreibich
b1e0e26484 Make debug stream names use "-", and handle "_" transparently
This looks consistent, and doesn't break any existing uses with underscores.
2024-11-15 17:53:29 -08:00
Christian Kreibich
7563af4906 Make "-B all" apply to plugin streams as well. 2024-11-15 13:53:06 -08:00
Christian Kreibich
5e4942bcc0 Sort streams in "-B help" output, and match case-insensitively throughout
Matching of plugins' debug streams was still case-sensitive. Also contains some
minor output tweaks.

It'd be nice to only list plugin debug streams actually _used_ by plugins. I
didn't see a quick way to do that so that's for another time.
2024-11-15 12:50:13 -08:00
Arne Welzel
d18045ee16 testing: Add have-spicy-ssl helper and update tests 2024-10-29 09:30:01 +01:00
Johanna Amann
cf1074518e Spicy SSL analyzer - address feedback
Minor proposed changes from https://github.com/zeek/zeek/pull/3765,
mostly cosmetic.

Changes CI to be part of an already existing spicy CI job.
2024-09-11 11:30:16 +02:00
Johanna Amann
84c4d53a4e Spicy TLS - full test suite pass
With this commit, the entire Zeek test suite passes using spicy TLS.
Tests that either use a SSLv2 handshake, or DTLS are skipped, as the
parser currently does not support either.

Similarly, tests that rely on behavior we cannot replicate (baseline,
hooks, exact error messages) are passed. Other than that, all the
TLS-based tests pass with 100% the exact same baseline results.

This necessitated a couple of small tweaks to the spicy file - the
testcases uncovered several small problems.

This commit also enables cirrus tests for Spicy SSL/TLS.
2024-08-13 14:41:37 +01:00
Tim Wojtulewicz
9de1dd16d5 Remove variant from StdFunctionStmt
The variant ended up conflicting with std::bind, which resulted in failures
on the btest invoking it. Change back to a single function that takes a
flow, and default it to a value in Exec.
2024-03-19 09:07:45 -07:00
Tim Wojtulewicz
39890fda5e Add a method to register an event handler to a std::function via C++ 2024-03-18 16:36:25 -07:00
Dominik Charousset
cebb85b1e8 Fix unsafe and inefficient uses of copy_string
Add a new overload to `copy_string` that takes the input characters plus
size. The new overload avoids inefficient scanning of the input for the
null terminator in cases where we know the size beforehand. Furthermore,
this overload *must* be used when dealing with input character sequences
that may have no null terminator, e.g., when the input is from a
`std::string_view` object.
2023-11-03 15:25:38 +01:00
Benjamin Bannier
f5a76c1aed Reformat Zeek in Spicy style
This largely copies over Spicy's `.clang-format` configuration file. The
one place where we deviate is header include order since Zeek depends on
headers being included in a certain order.
2023-10-30 09:40:55 +01:00
Arne Welzel
07ac6fa074 btest/plugins/hooks: Run in bare mode
Motivation is basically the same as in 88bb527026.
For plugin.hooks, one example is that adding a new option in the default script
changes the baseline due registration of change handlers. Also, the connection
record is printed in various places, resulting in churn when the default
scripts change.
2023-10-09 16:13:59 +02:00
Benjamin Bannier
0896f09081 Exclude dot directories when searching ZEEK_PLUGIN_PATH
Dot directories rarely contain anything we would want to load as a
dynamic plugin. Even worse, they likely contain files with externally
controlled lifetimes which might be removed while we are using them
(see e.g., zeek/btest#98).

With this patch we do not search _discovered_ dot directories anymore.
We continue to load from a user-specified `ZEEK_PLUGIN_PATH`, even if
its name starts with a dot.

Since this patch changes previous behavior it is a **BREAKING CHANGE**.
2023-09-26 11:56:47 +02:00
Arne Welzel
5a9c6d4633 Merge remote-tracking branch 'origin/topic/awelzel/duplicate-plugin-no-debug'
* origin/topic/awelzel/duplicate-plugin-no-debug:
  plugin/Manager: Warn when plugin with the same name is found
2023-06-15 18:39:49 +02:00
Tim Wojtulewicz
4229af6820 Remove deprecations tagged for v6.1 2023-06-14 10:07:22 -07:00
Arne Welzel
fdd6dfd096 plugin/Manager: Warn when plugin with the same name is found
This was brought up on Slack as a potential source of confusion during
development as it's not visible when plugin directories are ignored outside
of looking into the plugin debug stream. I'd actually prefer to just
FatalError() this, but a warning seems reasonably visible for interactive
usage.
2023-06-14 17:36:18 +02:00
Robin Sommer
d8f7329227
Modernize plugin test.
Not using the `zeek/` include style could lead to path problems.
2023-05-15 16:08:47 +02:00
Arne Welzel
8c0e7cd8bc Merge remote-tracking branch 'origin/topic/neverlord/version-header'
* origin/topic/neverlord/version-header:
  testing/zeek-version-link: Assume nm is there
  Drop dependency for zeek_dynamic_plugin_base
  Fixup ifdef check in Plugin.h
  Update cmake_minimum_required() in test plugins
  testing: Add zeek-version-link tests
  Clean up ZEEK_CONFIG_SKIP_VERSION_H issues

cmake-format during merge
2023-05-10 23:33:11 +02:00
Arne Welzel
ca75834c83 testing/zeek-version-link: Assume nm is there 2023-05-10 17:37:32 +02:00
Arne Welzel
85934ec632 Update cmake_minimum_required() in test plugins 2023-05-10 15:53:13 +02:00
Arne Welzel
460dc16d32 testing: Add zeek-version-link tests 2023-05-10 15:53:10 +02:00
Tim Wojtulewicz
84e3f414a7 Use the same rules as cmake submodule to reformat Zeek 2023-05-09 08:31:43 -07:00
Tim Wojtulewicz
c5b8603218 Remove non-standard way of forwarding out of the Ethernet analyzer 2023-04-25 12:29:55 -07:00
Arne Welzel
f0b9c59adb Add experimental JavaScript support when libnode is available
zeek.on('zeek_init', () => {
        console.log('Hello, Zeek!');
    });

For interaction with external systems and HTTP APIs, JavaScript and the
Node.js ecosystem beat Zeek script. Make it more easily accessible by
including ZeekJS with Zeek directly.

When a recent enough libnode version is found on the build system, ZeekJS is
added as a builtin plugin. This behavior can be disabled via
``--disable-javascript``. Linux distributions providing such a package are
Ubuntu (22.10) and Debian (testing/bookworm) as libnode-dev.
Fedora provides it as nodejs-devel.

This plugin takes over loading of .js or .cjs files. When no such files
are provided to Zeek, Node and the V8 engine are not initialized and
should not get into the way.

This should be considered experimental.
2023-04-14 11:26:41 +02:00