This allows to read Zeek global variables from inside Spicy code. The
main challenge here is supporting all of Zeek's data type in a
type-safe manner.
The most straight-forward API is a set of functions
`get_<type>(<id>)`, where `<type>` is the Zeek-side type
name (e.g., `count`, `string`, `bool`) and `<id>` is the fully scoped
name of the Zeek-side global (e.g., `MyModule::Boolean`). These
functions then return the corresponding Zeek value, converted in an
appropriate Spicy type. Example:
Zeek:
module Foo;
const x: count = 42;
const y: string = "xxx";
Spicy:
import zeek;
assert zeek::get_count("Foo::x") == 42;
assert zeek::get_string("Foo::y") == b"xxx"; # returns bytes(!)
For container types, the `get_*` function returns an opaque types that
can be used to access the containers' values. An additional set of
functions `as_<type>` allows converting opaque values of atomic
types to Spicy equivalents. Example:
Zeek:
module Foo;
const s: set[count] = { 1, 2 };
const t: table[count] of string = { [1] = "One", [2] = "Two" }
Spicy:
# Check set membership.
local set_ = zeek::get_set("Foo::s");
assert zeek::set_contains(set_, 1) == True
# Look up table element.
local table_ = zeek::get_table("Foo::t");
local value = zeek::table_lookup(t, 1);
assert zeek::as_string(value) == b"One"
There are also functions for accessing elements of Zeek-side vectors
and records.
If any of these `zeek::*` conversion functions fails (e.g., due to a
global of that name not existing), it will throw an exception.
Design considerations:
- We support only reading Zeek variables, not writing. This is
both to simplify the API, and also conceptually to avoid
offering backdoors into Zeek state that could end up with a very
tight coupling of Spicy and Zeek code.
- We accept that a single access might be relatively slow due to
name lookup and data conversion. This is primarily meant for
configuration-style data, not for transferring lots of dynamic
state over.
- In that spirit, we don't support deep-copying complex data types
from Zeek over to Spicy. This is (1) to avoid performance
problems when accidentally copying large containers over,
potentially even at every access; and (2) to avoid the two sides
getting out of sync if one ends up modifying a container without
the other being able to see it.
This reverts part of commit a0888b7e36 due
to inhibiting analyzer violations when parsing non SSH traffic when
the &restofdata path is entered.
@J-Gras reported the analyzer not being disabled when sending HTTP
traffic on port 22.
This adds the verbose analyzer.log baselines such that future improvements
of these scenarios become visible.
We now reject EVT files that attempt to replace the same built-in
analyzer multiple times as doing so would be ill-defined and not very
intuitive in what exactly it means.
Closes#3783.
This replaces generic reporter->Error() calls with the builtin-specific variety,
which gives better context in the resulting error messages (such as the script
and line causing it).
Includes corresponding baseline update in one affected btest.
* origin/topic/robin/gh-3561-forward-to-udp:
Update docs.
Add explicit children life-cycle management method to analyzers.
Spicy: Support UDP in Spicy's `protocol_*` runtime functions.
Add method to analyzer to retrieve direct child by name.
Extend PIA's `FirstPacket` API.
Spicy: Prepare for supporting forwarding to protocols other than TCP.
* origin/topic/christian/fix-zeekygen-crash:
Avoid segfault when generating Zeekygen docs on Zeek-internal identifiers.
Add btest for Zeekygen docs extraction on identifiers defined by the Zeek core.
This extends the ability to feed new payload back into Zeek's analyzer
pipeline from TCP to now also UDP.
Note: We don't extend this further to ICMP because the ICMP analyzer
cannot be dynamically instantiated (Zeek aborts when trying so). As
ICMP isn't very interesting from use-case perspective anyways, that
seems fine.
Closes#3561.
* origin/topic/robin/gh-3573-replaces-cleanup:
Fix packet analyzer replacement.
Spicy: Wenn replacing an analyzer add a component mapping.
Add component API to transparently remap one component to another one.
Move enabled/disabled functionality from analyzers into `Component` base class API.
This uses the new API to replace components internally.
With these changes in place, replacing protocol analyzers now don't
need to register their ports anymore if they match what the original
analyzer was using (because the old one's registrations will map
over).
Packet analyzer replacement doesn't quite work yet but will be fixed
in next commit.
Closes#3573.
Like traditional file analyzers, we now query Zeek's
`get_file_handle()` event for handles when a connection begins
analyzing an embedded file. That means that Spicy-side protocol
analyzers that are forwarding data into file analysis now need to call
Zeek's `Files::register_protocol()` and provide a callback for
computing file handles. If that's missing, Zeek will now issue a
warning. This aligns with the requirements Zeek's traditional protocol
analyzers. (If the EVT file defines a protocol analyzer to `replace`
an existing one, that one's `register_protocol()` will be consulted.)
Because Zeek's `get_file_handle()` event requires a current
connection, if a Spicy file analyzer isn't directly part of a
connection context (e.g., with nested files), we continue to use
hardcoded, built-in file handle. Scriptland won't be consulted in
that case, just like before.
Closes#3440.
Mainly, this should not crash. :-) The baseline output file also should remain
empty, since Zeekygen's staleness test should conclude that it's sufficiently
fresh.
