Commit graph

5073 commits

Author SHA1 Message Date
Vern Paxson
50b1f6e013 updated list of BiFs for script optimization 2024-06-04 10:36:36 -07:00
Vern Paxson
b0d9a841f5 improved error cascade for invalid attributes 2024-06-04 10:36:36 -07:00
Tim Wojtulewicz
46ff48c29a Change all instruments to only handle doubles 2024-05-31 13:36:37 -07:00
Tim Wojtulewicz
e195d3d778 Fix some determinism issues with btests 2024-05-31 13:30:31 -07:00
Tim Wojtulewicz
d6e97ab306 Temporarily disable the scripts/base/frameworks/telemetry/internal-metrics btest 2024-05-31 13:30:31 -07:00
Tim Wojtulewicz
074a87b609 Fix the scripts.policy.frameworks.telemetry.prometheus btest to use the service discovery endpoint 2024-05-31 13:30:31 -07:00
Tim Wojtulewicz
a63ea5a04e Btest updates due to recent changes 2024-05-31 13:30:31 -07:00
Tim Wojtulewicz
017ee4509c Update telemetry log policy due to the fact that unit will not be filled in anymore 2024-05-31 13:30:31 -07:00
Tim Wojtulewicz
e93e4cc26d Add a services.json endpoint for Prometheus service discovery 2024-05-31 13:30:31 -07:00
Tim Wojtulewicz
4718e5cf00 Remove everything related to aggregation 2024-05-31 13:30:31 -07:00
Tim Wojtulewicz
d7b9924058 Update test baselines due to underscore changes 2024-05-31 13:30:31 -07:00
Tim Wojtulewicz
97a35011a7 Add necessary script-land changes 2024-05-31 13:30:31 -07:00
Christian Kreibich
a599fe0438 More precise error reporting for the disable_analyzer() BiF
This replaces generic reporter->Error() calls with the builtin-specific variety,
which gives better context in the resulting error messages (such as the script
and line causing it).

Includes corresponding baseline update in one affected btest.
2024-05-30 16:38:22 -07:00
Christian Kreibich
09b70879b0 Add btests for the lookup_connection_analyzer_id() BiF. 2024-05-30 16:38:09 -07:00
Vern Paxson
0e5bece385 "add" and "delete" are now expressions rather than statements 2024-05-29 12:40:06 -07:00
Vern Paxson
e84b60762a added a space when rendering some expressions so they're more readable 2024-05-29 12:40:05 -07:00
Johanna Amann
03b358f6d1 Merge branch 'files_pe_timestamp_sync' of https://github.com/mvhensbergen/zeek
* 'files_pe_timestamp_sync' of https://github.com/mvhensbergen/zeek:
  Don't hardcode values
  Add btest for timestamp check
  Copy timestamp from file object
2024-05-29 14:16:31 +01:00
Martin van Hensbergen
7f77075c43 Don't hardcode values 2024-05-29 14:00:04 +01:00
Martin van Hensbergen
e993f75ccb Add btest for timestamp check 2024-05-29 13:58:32 +01:00
Johanna Amann
34225e83ba Update TLS consts, mainly new named curves.
Add test for X25519Kyber768Draft00 (post-quantum key agreement)
2024-05-23 14:50:36 +01:00
Vern Paxson
74bf453d6d Fix for suppressing SMB logging of previously-logged files 2024-05-18 14:13:52 -07:00
Vern Paxson
39d2ba410e "-a zam" BTest baseline update reflecting recent Spicy baseline change 2024-05-15 17:29:06 -07:00
Robin Sommer
82be6425e6
Merge remote-tracking branch 'origin/topic/robin/gh-3561-forward-to-udp'
* origin/topic/robin/gh-3561-forward-to-udp:
  Update docs.
  Add explicit children life-cycle management method to analyzers.
  Spicy: Support UDP in Spicy's `protocol_*` runtime functions.
  Add method to analyzer to retrieve direct child by name.
  Extend PIA's `FirstPacket` API.
  Spicy: Prepare for supporting forwarding to protocols other than TCP.
2024-05-10 11:15:20 +02:00
Robin Sommer
93a424b28a
Spicy: Fix service reporting for replaced analyzers.
We accidentally applied analyzer mappings when looking up an
analyzer's name from scriptland.

Closes #3725.
2024-05-08 14:01:46 +02:00
Vern Paxson
a0888b7e36 make SSH analyzer robust to half-duplex connections 2024-05-07 11:40:47 -07:00
Tim Wojtulewicz
d463141ded Merge remote-tracking branch 'origin/topic/christian/fix-zeekygen-crash'
* origin/topic/christian/fix-zeekygen-crash:
  Avoid segfault when generating Zeekygen docs on Zeek-internal identifiers.
  Add btest for Zeekygen docs extraction on identifiers defined by the Zeek core.
2024-05-07 10:26:19 -07:00
Robin Sommer
a2ae9c4b02
Spicy: Support UDP in Spicy's protocol_* runtime functions.
This extends the ability to feed new payload back into Zeek's analyzer
pipeline from TCP to now also UDP.

Note: We don't extend this further to ICMP because the ICMP analyzer
cannot be dynamically instantiated (Zeek aborts when trying so). As
ICMP isn't very interesting from use-case perspective anyways, that
seems fine.

Closes #3561.
2024-05-07 18:19:46 +02:00
Robin Sommer
8ce3c877ff
Merge remote-tracking branch 'origin/topic/robin/gh-3573-replaces-cleanup'
* origin/topic/robin/gh-3573-replaces-cleanup:
  Fix packet analyzer replacement.
  Spicy: Wenn replacing an analyzer add a component mapping.
  Add component API to transparently remap one component to another one.
  Move enabled/disabled functionality from analyzers into `Component` base class API.
2024-05-07 09:45:25 +02:00
Robin Sommer
083c682878
Merge remote-tracking branch 'origin/topic/robin/gh-3440-file-handles'
* origin/topic/robin/gh-3440-file-handles:
  Spicy: Query Zeek scriptland for file handles.
2024-05-07 09:44:41 +02:00
Robin Sommer
383a77dab7
Merge remote-tracking branch 'origin/topic/robin/gh-3606-port-enum'
* origin/topic/robin/gh-3606-port-enum:
  Spicy: Cleanup some runtime code.
  Spicy: Map Spicy's `Protocol` enum to Zeek's `transport_proto`.
2024-05-07 09:42:04 +02:00
Tim Wojtulewicz
6821a41c4e Move the options from policy/tuning/defaults to actual Zeek defaults, deprecate that package 2024-05-06 11:13:04 -07:00
Robin Sommer
ccfeffaf2b
Fix packet analyzer replacement.
Also reworking the existing test, which was pretty odd/broken.
2024-05-06 09:45:11 +02:00
Robin Sommer
3a469b3619
Spicy: Wenn replacing an analyzer add a component mapping.
This uses the new API to replace components internally.

With these changes in place, replacing protocol analyzers now don't
need to register their ports anymore if they match what the original
analyzer was using (because the old one's registrations will map
over).

Packet analyzer replacement doesn't quite work yet but will be fixed
in next commit.

Closes #3573.
2024-05-06 09:45:11 +02:00
Robin Sommer
56b9a79a65
Spicy: Query Zeek scriptland for file handles.
Like traditional file analyzers, we now query Zeek's
`get_file_handle()` event for handles when a connection begins
analyzing an embedded file. That means that Spicy-side protocol
analyzers that are forwarding data into file analysis now need to call
Zeek's `Files::register_protocol()` and provide a callback for
computing file handles. If that's missing, Zeek will now issue a
warning. This aligns with the requirements Zeek's traditional protocol
analyzers. (If the EVT file defines a protocol analyzer to `replace`
an existing one, that one's `register_protocol()` will be consulted.)

Because Zeek's `get_file_handle()` event requires a current
connection, if a Spicy file analyzer isn't directly part of a
connection context (e.g., with nested files), we continue to use
hardcoded, built-in file handle. Scriptland won't be consulted in
that case, just like before.

Closes #3440.
2024-05-06 09:20:38 +02:00
Christian Kreibich
9d89b1156c Add btest for Zeekygen docs extraction on identifiers defined by the Zeek core.
Mainly, this should not crash. :-) The baseline output file also should remain
empty, since Zeekygen's staleness test should conclude that it's sufficiently
fresh.
2024-05-03 12:22:42 -07:00
Vern Paxson
1f9fa4304d refine Val "footprint" to equate long strings with multiple objects 2024-04-29 12:39:36 -07:00
Robin Sommer
5a74db30ef
Spicy: Map Spicy's Protocol enum to Zeek's transport_proto.
We now hardcode this specific enum mapping to make Zeek-side usage
more convenient.

Closes #3606.
2024-04-29 08:38:53 +02:00
Vern Paxson
033cf2d8dd factored BTest constants into globals to prevent premature ZAM optimization when testing 2024-04-27 19:40:39 -07:00
Vern Paxson
8e25181e1e BTest updates for changes due to use of ZAM BiF replacements 2024-04-25 09:15:13 -07:00
Vern Paxson
4b39e1f5f5 shift ZAM maintenance monitoring of new BiFs to a BTest so it automatically runs 2024-04-25 09:15:12 -07:00
Vern Paxson
c11c2830b1 performance speed-up for SMB base scripts 2024-04-25 09:15:12 -07:00
Benjamin Bannier
3c89a5492c Fix typo in hook name
While in Spicy code a hook priority is spelled `priority=4711` the
attribute is still called `&priority` (like in HILTI) and we rely on
exactly that name when e.g., extracting hook priorities for scheduling.

This change was introduced as part of
db98dc4193 and caused the default hook
priority for hooks defined in EVT files (intended to be -1000 to likely
schedule after e.g., hooks in the Spicy grammars) to be ignored. This
could then e.g., introduce issue when a `%done` hook would mutate state
exposed in an EVT hook (which now might not have seen the updated state
due to different scheduling).
2024-04-25 13:43:09 +02:00
Tim Wojtulewicz
bac6ad62b5 Allow SMB_TCP record to contain multiple protocol identifiers/headers 2024-04-22 15:55:25 -07:00
Tim Wojtulewicz
5aa18287f1 Merge remote-tracking branch 'origin/topic/timw/3456-when-unknown-capture'
* origin/topic/timw/3456-when-unknown-capture:
  Check for valid identifier in WhenStmt::Describe
2024-04-22 15:16:16 -07:00
Tim Wojtulewicz
d566961da5 Merge remote-tracking branch 'pbcullen/topic/pbcullen/remove_exclude'
* pbcullen/topic/pbcullen/remove_exclude:
  Update core.pcap.filter-warning baseline
  Add PacketFilter::remove_exclude function
2024-04-18 09:03:57 -07:00
Peter Cullen
f1941186eb Update core.pcap.filter-warning baseline
Only the line numbers changed.
2024-04-18 12:27:46 +00:00
Peter Cullen
187f93cb96 Parse and store localversion string
Add localversion to the VersionDescription record and populate it
during version string parsing.
This change also modifies the version string syntax, removing the
deprecated dash (-) between beta|dev|rc and the commmit count; those
must now be separated by a period.
The test version strings were updated accordingly along with the
baseline.
2024-04-17 14:17:22 -07:00
Peter Cullen
32289521f5 Update version string btests for localversion
Added some more version strings to be tested that include
a localversion component. Baseline updates, as well.
2024-04-17 14:17:10 -07:00
Tim Wojtulewicz
9abf8ea8b7 Check for valid identifier in WhenStmt::Describe 2024-04-10 16:20:36 -07:00
Vern Paxson
0538c68221 ZAM-specific BTest baseline changes for tweak to how ZAM bodies print 2024-03-19 10:59:49 -07:00