Allow spicy parsers to generate their own file IDs and provide them to
Zeek. This duplicates functionality that is currently possible (and
used) by some binpac-based analyzers. One example for an analyzer
creating its own file IDs is the SSL analyzer.
```
## Tells Zeek to skip sending any further input data to the current analyzer.
## This is supported for protocol and file analyzers.
public function skip_input() : void;
```
Closes#3443.
Without this, Zeekygen won't generate documentation about exported
enum types as it can not resolve the identifier. Also, only register a
type as item with the Spicy plugin if there's no _module_info currently
active.
So far we had trouble documenting Spicy analyzers through Zeekygen
because they would show up as components belonging to the
`Zeek::Spicy` plugin; whereas traditional analyzers would be their own
plugins and hence documented individually on their own. This commit
teaches Zeekygen to track Spicy analyzers separately inside their own
`Info` instances. This information isn't further used in this commit
yet, but will be merged with the plugin output in a subsequent change
to get the expected joint output.
To pass additional information to Zeekygen, EVT files now also support
two new tags for Zeekygen purposes:
- `%doc-id = ID;` defines the global ID under which everything inside
the EVT file will be documented by Zeekygen, conceptually comparable
to plugin names (e.g., `Zeek::Syslog`).
- `%doc-description = "text" provides additional text to go into the
documentation (comparable to plugin descriptions).
This information is carried through into the HLTO runtime
initialization code, from where it's registered with Zeekygen.
This commit also removes a couple of previous hacks of how Spicy
integrated with Zeekygen which (1) ended up generating broken doc output
for Spicy components, and (2) don't seem to be necessary anymore
anyways.
In 36a6770e98 we changed the way Spicy
fields exported as
record fields are represented. This broke the `zeek` debug log which
hardcoded a different representation.
This patch brings the generation of the debug log in line with the
actual AST structure. We also enable debug logging in one test to
validate that it works, at least for the case we hit here.
When checking exported Spicy types for collisions with existing Zeek
types we previously would also check whether they collide with names in
global scope, i.e., we didn't provide a `no_global` arg to
`detail::lookup_ID` which defaulted to false (since we also provided a
module name I'd argue that the behavior of that function is confusing
and probably error-prone -- like seen here).
This meant that e.g., a Spicy enum `foo::Direction` (automatically in
implicit Spicy module scope) would be detected to collide with the
existing Zeek `Direction` enum.
With this patch we use the `lookup_ID` API correctly and do not check
against potential collisions with globals anymore since it is not
needed.
Closes#3279.
We now support selecting which fields of a unit type get exported into
the automatically created Zeek record; as well as selecting which
fields get a `&log` attribute added automatically to either all fields
or to selected fields.
Syntax:
- To export only selected fields:
export Foo::X with { field1, field3 };
- To export all but selected fields:
export Foo::X without { field2, field3 };
- To `&log` all fields:
export Foo::X &log;
- To `&log` only selected fields:
export Foo::X with { field1 &log, field3 }; # exports (only) field1 and field3, and marks field1 for logging
Syntax is still subject to change.
Closes#3218.
Closes#3219.
When a Spicy unit type was used in an EVT event definition, it could
end up being cleared out at the time the logic for `export` got to see it.
Closes#3083.
Note: This needs backporting to the legacy plugin.
This patch changes invocations of `spicyz` and similar Spicy tools in
tests which perform compilation to use debug mode via passing `-d`. This
in turn leads to Spicy compiling generated C++ code in debug as opposed
to release mode which typically seems to require less CPU time and RAM.
For a local test running with `btest -j 16` and no caching via
`HILTI_CXX_COMPILER_LAUNCER` this sped up running of BTests under
`spicy/` by about 40s on my machine (120s vs 160s).
This moves `auxil/spicy/spicy` to `auxil/spicy`. It relocates the
pieces previously inside that intermediary directory. The main change
is that now tweak the compilation flags for Spicy through target
options.
This is on top of `topic/robin/spicy-plugin`.
This reflects the `spicy-plugin` code as of `d8c296b81cc2a11`.
In addition to moving the code into Zeek's source tree, this comes
with a couple small functional changes:
- `spicyz` no longer tries to infer if it's running from the build
directory. Instead `ZEEK_SPICY_LIBRARY` can be set to a custom
location. `zeek-set-path.sh` does that now.
- ZEEK_CONFIG can be set to change what `spicyz -z` print out. This is
primarily for backwards compatibility.
Some further notes on specifics:
- We raise the minimum Spicy version to 1.8 (i.e., current `main`
branch).
- Renamed the `compiler/` subdirectory to `spicyz` to avoid
include-path conflicts with the Spicy headers.
- In `cmake/`, the corresponding PR brings a new/extended version of
`FindZeek`, which Spicy analyzer packages need. We also now install
some of the files that the Spicy plugin used to bring for testing,
so that existing packages keep working.
- For now, this all remains backwards compatible with the current
`zkg` analyzer templates so that they work with both external and
integrated Spicy support. Later, once we don't need to support any
external Spicy plugin versions anymore, we can clean up the
templates as well.
- All the plugin's tests have moved into the standard test suite. They
are skipped if configure with `--disable-spicy`.
This holds off on adapting the new code further to Zeek's coding
conventions, so that it remains easier to maintain it in parallel to
the (now legacy) external plugin. We'll make a pass over the
formatting for (presumable) Zeek 6.1.
We previously would include any and all output from stderr during
compilation in the test baseline. Depending on the used compiler this
output may contain C++ compilation warnings which are uninteresting for
the behavior under test.
(cherry picked from commit 5221edf474)
As initial examples, this branch ports the Syslog and Finger analyzers
over. We leave the old analyzers in place for now and activate them
iff we compile without any Spicy.
Needs `zeek-spicy-infra` branches in `spicy/`, `spicy-plugin/`,
`CMake/`, and `zeek/zeek-testing-private`.
Note that the analyzer events remain associated with the Spicy plugin
for now: that's where they will show up with `-NN`, and also inside
the Zeekygen documentation.
We switch CMake over to linking the runtime library into the plugin,
vs. at the top-level through object libraries.
