In some cases, e.g. running zeek on short pcaps as opposed to continuous packet streams, network_time() may not equal the time that was used when generating the file object.
This results in the pe.log entry having a different timestamp than its corresponding files.log entry which is strange as they refer to the exact same file.
* origin/topic/robin/gh-3561-forward-to-udp:
Update docs.
Add explicit children life-cycle management method to analyzers.
Spicy: Support UDP in Spicy's `protocol_*` runtime functions.
Add method to analyzer to retrieve direct child by name.
Extend PIA's `FirstPacket` API.
Spicy: Prepare for supporting forwarding to protocols other than TCP.
So far the Spicy runtime supported forwarding data into other
analyzers only for TCP analyzers. This puts branching logic in place
that let the relevant runtime functions dispatch differently based on
the target transport-layer protocol. We don't implement anything else
than TCP yet; that will come next.
Along with the internal changes, this also updates the user-visible
runtime function to pass protocol information in. For now, this
likewise remains limited to TCP. The function signatures are chosen so
that they stay backwards-compatible to previous Spicy version. In
particular, they default to TCP where not otherwise specified.
IANA's IPv6 special-purpose address registry now has members that technically
meet the definition of not being globally reachable, but don't imply operating
locally. An example: https://datatracker.ietf.org/doc/draft-ietf-6man-sids/06/
This change just explains that distinction.
Add localversion to the VersionDescription record and populate it
during version string parsing.
This change also modifies the version string syntax, removing the
deprecated dash (-) between beta|dev|rc and the commmit count; those
must now be separated by a period.
The test version strings were updated accordingly along with the
baseline.
The regex used for validating version strings was updated to allow
for an optional localversion component. The piece of the version string
also needs to be removed before parsing a possible commit number.
Zeekygen implements its own make-style update logic to prevent
re-creation of files that have not changed. To fulfill this, we
currently encode the current time into spicyz generated .cc files.
This degrades ccache efficiency for built-in analyzers and also
for all .evt files compiled during testing. Switch SpicyModuleInfo
to return current time instead. This results in the re-generation
of documentation files unconditionally when running Zeekygen, but
that seems more acceptable IMO.
Generally wonder if Zeekygen should produce output unconditionally
and if we need to clobber prevention, compare with the content of
the existing file.
Closes#3619
* origin/topic/awelzel/move-iso-9660-sig-to-policy:
signatures/iso-9660: Add \x01 suffix to CD001
test-all-policy: Do not load iso-9660.zeek
signatures: Move ISO 9660 signature to policy
Changing the default_file_bof_buffer_size has subtle impact on
MIME type detection and changed the zeek-testing baseline. Do
not load this new script via test-all-policy to avoid this.
The new test was mainly an aid to understand what is actually going on.
In short, if default_file_bof_buffer_size is larger than the file MIME
detection only runs when the buffer is full, or when the file is removed.
When a file transfer happens over multiple HTTP connections, only
some or one of the http.log entries will have a proper response MIME type.
PCAP extracted from 2009-M57-day11-18.trace.gz.
The previous "fix" caused significant performance degradation without
the signature ever having a chance to trigger. Moving it to policy
seems the best compromise, the alternative being outright removing it.
A user reported being confused about the fuid association of subsequent
FTP commands when a data transfer has completed. It seems reasonable to
unset fuid upon logging a FTP command which had a fuid.
The current behavior results in the PORT or PASV commands after a RETR or STOR
to have the fuid of the prior file transfer. Similarly, any CWD or DEL commands
following a file transfer will unnecessarily be logged with the fuid of the
prior file transfer.
This tickles the baselines for the private testing PCAP a lot, primarily
because there data connections in that pcap are never established properly.
E.g, the fuids FzDzid1Dxm9srVKHXf and FEfYX73q5C6GEQZXX9 have been re-used
for multiple commands.
This may look like we're losing information, but the fuids vanishing
in the normal btests belong to a LIST command that isn't logged by
default into ftp.log. If it was, the fuid would be attached to it.
The Spicy analyzer is added as a child analyzer when enabled and the
WebSocket.cc logic dispatches between the BinPac and Spicy version.
It substantially slower when tested against a somewhat artificial
2.4GB PCAP. The first flamegraph indicates that the unmask() function
stands out with 35% of all samples, and above it shared_ptr samples.
* origin/topic/johanna/netcontrol-updates:
Netcontrol: add rule_added_policy
Netcontrol: more logging in catch-and-release
Netcontrol: allow supplying explicit name to Debug plugin
