* origin/topic/christian/fix-zeekygen-crash:
Avoid segfault when generating Zeekygen docs on Zeek-internal identifiers.
Add btest for Zeekygen docs extraction on identifiers defined by the Zeek core.
* origin/topic/robin/gh-3573-replaces-cleanup:
Fix packet analyzer replacement.
Spicy: Wenn replacing an analyzer add a component mapping.
Add component API to transparently remap one component to another one.
Move enabled/disabled functionality from analyzers into `Component` base class API.
This uses the new API to replace components internally.
With these changes in place, replacing protocol analyzers now don't
need to register their ports anymore if they match what the original
analyzer was using (because the old one's registrations will map
over).
Packet analyzer replacement doesn't quite work yet but will be fixed
in next commit.
Closes#3573.
Like traditional file analyzers, we now query Zeek's
`get_file_handle()` event for handles when a connection begins
analyzing an embedded file. That means that Spicy-side protocol
analyzers that are forwarding data into file analysis now need to call
Zeek's `Files::register_protocol()` and provide a callback for
computing file handles. If that's missing, Zeek will now issue a
warning. This aligns with the requirements Zeek's traditional protocol
analyzers. (If the EVT file defines a protocol analyzer to `replace`
an existing one, that one's `register_protocol()` will be consulted.)
Because Zeek's `get_file_handle()` event requires a current
connection, if a Spicy file analyzer isn't directly part of a
connection context (e.g., with nested files), we continue to use
hardcoded, built-in file handle. Scriptland won't be consulted in
that case, just like before.
Closes#3440.
Mainly, this should not crash. :-) The baseline output file also should remain
empty, since Zeekygen's staleness test should conclude that it's sufficiently
fresh.
While in Spicy code a hook priority is spelled `priority=4711` the
attribute is still called `&priority` (like in HILTI) and we rely on
exactly that name when e.g., extracting hook priorities for scheduling.
This change was introduced as part of
db98dc4193 and caused the default hook
priority for hooks defined in EVT files (intended to be -1000 to likely
schedule after e.g., hooks in the Spicy grammars) to be ignored. This
could then e.g., introduce issue when a `%done` hook would mutate state
exposed in an EVT hook (which now might not have seen the updated state
due to different scheduling).
Add localversion to the VersionDescription record and populate it
during version string parsing.
This change also modifies the version string syntax, removing the
deprecated dash (-) between beta|dev|rc and the commmit count; those
must now be separated by a period.
The test version strings were updated accordingly along with the
baseline.
The variant ended up conflicting with std::bind, which resulted in failures
on the btest invoking it. Change back to a single function that takes a
flow, and default it to a value in Exec.
The Python docker-compose executable in the docker_builder image is not
functional anymore (TypeError: kwargs_from_env() got an unexpected
keyword argument 'ssl_version').
docker-compose v1 has been sun-setted for quite some time. Do the
minimal to switch to docker compose v2.
@vpax reported surprising behavior when working with "void values".
While these are not exposed to script land, plumb the places he
pointed out are causing confusing behavior.
Closes#3640.
Since 81a9745fb3, the assert condition is
evaluated twice. This leads to unexpected behavior when cond has a side
effect like publishing a message or creating a log stream or filter.
Found while using the following in ad-hoc testing code and wondering
why two messages were published.
assert publish(Cluster::worker_topic, hello, "abc")
* origin/topic/awelzel/move-iso-9660-sig-to-policy:
signatures/iso-9660: Add \x01 suffix to CD001
test-all-policy: Do not load iso-9660.zeek
signatures: Move ISO 9660 signature to policy
Changing the default_file_bof_buffer_size has subtle impact on
MIME type detection and changed the zeek-testing baseline. Do
not load this new script via test-all-policy to avoid this.
The new test was mainly an aid to understand what is actually going on.
In short, if default_file_bof_buffer_size is larger than the file MIME
detection only runs when the buffer is full, or when the file is removed.
When a file transfer happens over multiple HTTP connections, only
some or one of the http.log entries will have a proper response MIME type.
PCAP extracted from 2009-M57-day11-18.trace.gz.
The previous "fix" caused significant performance degradation without
the signature ever having a chance to trigger. Moving it to policy
seems the best compromise, the alternative being outright removing it.
