Commit graph

4835 commits

Author SHA1 Message Date
Arne Welzel
4f084b0b9a ssh: Fallback to client or server selected version for parsing
In half-duplex setups (or when client/server coalesce the SSH version
line with the KEX packet, get_version() would return UNK as version,
causing a protocol violation. Make this slightly more robust by using
and setting the version which either side had set to continue parsing.

For the special case of SSH-1.99, select SSH-2.0. We could try to peak
into the payload following the packet length field and check for
a KEX_INIT type byte to select SSH2 as a heuristic, but not sure how
to accomplish this.

Slight regression fix for 3769ed6c66
which started to require visibility for client and server version
rather than just the client's version.
2024-06-12 16:30:18 +02:00
Arne Welzel
a0e0b7161f ssh: Revert half-duplex robustness
This reverts part of commit a0888b7e36 due
to inhibiting analyzer violations when parsing non SSH traffic when
the &restofdata path is entered.

@J-Gras reported the analyzer not being disabled when sending HTTP
traffic on port 22.

This adds the verbose analyzer.log baselines such that future improvements
of these scenarios become visible.
2024-06-12 16:28:50 +02:00
Tim Wojtulewicz
c0f14bdc0b Change prometheus test to check for require jq 2024-06-06 08:53:48 -07:00
Tim Wojtulewicz
99e64aa113 Restore label_names field in MetricOpts record 2024-06-04 14:14:58 -07:00
Tim Wojtulewicz
433c257886 Move telmetry label names out of opts records, into main metric records 2024-06-04 14:14:58 -07:00
Tim Wojtulewicz
b1578d4ded Ensure the order of label values matches the label names 2024-06-04 14:14:58 -07:00
Tim Wojtulewicz
87717fed0a Remove prefix column from telemetry.log 2024-06-04 14:14:58 -07:00
Tim Wojtulewicz
00b24b043a Set running_under_test for scripts.base.frameworks.logging.telemetry test 2024-06-04 14:14:57 -07:00
Vern Paxson
0ee28866a1 script optimization baseline tweaks due to recent minor changes 2024-06-04 10:36:36 -07:00
Vern Paxson
50b1f6e013 updated list of BiFs for script optimization 2024-06-04 10:36:36 -07:00
Vern Paxson
b0d9a841f5 improved error cascade for invalid attributes 2024-06-04 10:36:36 -07:00
Tim Wojtulewicz
46ff48c29a Change all instruments to only handle doubles 2024-05-31 13:36:37 -07:00
Tim Wojtulewicz
e195d3d778 Fix some determinism issues with btests 2024-05-31 13:30:31 -07:00
Tim Wojtulewicz
d6e97ab306 Temporarily disable the scripts/base/frameworks/telemetry/internal-metrics btest 2024-05-31 13:30:31 -07:00
Tim Wojtulewicz
074a87b609 Fix the scripts.policy.frameworks.telemetry.prometheus btest to use the service discovery endpoint 2024-05-31 13:30:31 -07:00
Tim Wojtulewicz
a63ea5a04e Btest updates due to recent changes 2024-05-31 13:30:31 -07:00
Tim Wojtulewicz
017ee4509c Update telemetry log policy due to the fact that unit will not be filled in anymore 2024-05-31 13:30:31 -07:00
Tim Wojtulewicz
e93e4cc26d Add a services.json endpoint for Prometheus service discovery 2024-05-31 13:30:31 -07:00
Tim Wojtulewicz
4718e5cf00 Remove everything related to aggregation 2024-05-31 13:30:31 -07:00
Tim Wojtulewicz
d7b9924058 Update test baselines due to underscore changes 2024-05-31 13:30:31 -07:00
Tim Wojtulewicz
97a35011a7 Add necessary script-land changes 2024-05-31 13:30:31 -07:00
Christian Kreibich
a599fe0438 More precise error reporting for the disable_analyzer() BiF
This replaces generic reporter->Error() calls with the builtin-specific variety,
which gives better context in the resulting error messages (such as the script
and line causing it).

Includes corresponding baseline update in one affected btest.
2024-05-30 16:38:22 -07:00
Christian Kreibich
09b70879b0 Add btests for the lookup_connection_analyzer_id() BiF. 2024-05-30 16:38:09 -07:00
Vern Paxson
0e5bece385 "add" and "delete" are now expressions rather than statements 2024-05-29 12:40:06 -07:00
Vern Paxson
e84b60762a added a space when rendering some expressions so they're more readable 2024-05-29 12:40:05 -07:00
Johanna Amann
03b358f6d1 Merge branch 'files_pe_timestamp_sync' of https://github.com/mvhensbergen/zeek
* 'files_pe_timestamp_sync' of https://github.com/mvhensbergen/zeek:
  Don't hardcode values
  Add btest for timestamp check
  Copy timestamp from file object
2024-05-29 14:16:31 +01:00
Martin van Hensbergen
7f77075c43 Don't hardcode values 2024-05-29 14:00:04 +01:00
Martin van Hensbergen
e993f75ccb Add btest for timestamp check 2024-05-29 13:58:32 +01:00
Tim Wojtulewicz
ec4661f4b5 CI: Revert part of 2bde82ffa2 to fix coverage builds 2024-05-28 09:01:21 -07:00
Johanna Amann
34225e83ba Update TLS consts, mainly new named curves.
Add test for X25519Kyber768Draft00 (post-quantum key agreement)
2024-05-23 14:50:36 +01:00
Tim Wojtulewicz
0cd023b839 CI: Run coverage builds for PRs, but only upload on master 2024-05-21 15:11:44 -07:00
Tim Wojtulewicz
e8f504c0c0 Coverage: Ignore a few errors during generation 2024-05-21 15:11:44 -07:00
Tim Wojtulewicz
0e0852a876 Coverage: don't bother ignoring non-existent bro.dir files 2024-05-21 15:11:43 -07:00
Tim Wojtulewicz
179e4903f1 CI: Avoid divide by zero error when generating coverage files 2024-05-20 17:02:53 -07:00
Vern Paxson
74bf453d6d Fix for suppressing SMB logging of previously-logged files 2024-05-18 14:13:52 -07:00
Tim Wojtulewicz
87870f8345 Merge remote-tracking branch 'origin/topic/vern/zam-subnet-fix'
* origin/topic/vern/zam-subnet-fix:
  Fix for ZAM inlining of nested function calls with the same parameter names
  Fixed ZAM logic error in canonicalizing specialized min/max instructions
  Fixed order-of-evaluation bug in ZAM Subnet-To-Addr instruction
  "-a zam" BTest baseline update reflecting recent Spicy baseline change
2024-05-16 11:09:33 -07:00
Vern Paxson
39d2ba410e "-a zam" BTest baseline update reflecting recent Spicy baseline change 2024-05-15 17:29:06 -07:00
Tim Wojtulewicz
2bde82ffa2 CI: Use llvm-cov-18 on Cirrus for building coverage data 2024-05-15 13:58:33 -07:00
Robin Sommer
82be6425e6
Merge remote-tracking branch 'origin/topic/robin/gh-3561-forward-to-udp'
* origin/topic/robin/gh-3561-forward-to-udp:
  Update docs.
  Add explicit children life-cycle management method to analyzers.
  Spicy: Support UDP in Spicy's `protocol_*` runtime functions.
  Add method to analyzer to retrieve direct child by name.
  Extend PIA's `FirstPacket` API.
  Spicy: Prepare for supporting forwarding to protocols other than TCP.
2024-05-10 11:15:20 +02:00
Robin Sommer
93a424b28a
Spicy: Fix service reporting for replaced analyzers.
We accidentally applied analyzer mappings when looking up an
analyzer's name from scriptland.

Closes #3725.
2024-05-08 14:01:46 +02:00
Vern Paxson
a0888b7e36 make SSH analyzer robust to half-duplex connections 2024-05-07 11:40:47 -07:00
Tim Wojtulewicz
d463141ded Merge remote-tracking branch 'origin/topic/christian/fix-zeekygen-crash'
* origin/topic/christian/fix-zeekygen-crash:
  Avoid segfault when generating Zeekygen docs on Zeek-internal identifiers.
  Add btest for Zeekygen docs extraction on identifiers defined by the Zeek core.
2024-05-07 10:26:19 -07:00
Robin Sommer
a2ae9c4b02
Spicy: Support UDP in Spicy's protocol_* runtime functions.
This extends the ability to feed new payload back into Zeek's analyzer
pipeline from TCP to now also UDP.

Note: We don't extend this further to ICMP because the ICMP analyzer
cannot be dynamically instantiated (Zeek aborts when trying so). As
ICMP isn't very interesting from use-case perspective anyways, that
seems fine.

Closes #3561.
2024-05-07 18:19:46 +02:00
Robin Sommer
8ce3c877ff
Merge remote-tracking branch 'origin/topic/robin/gh-3573-replaces-cleanup'
* origin/topic/robin/gh-3573-replaces-cleanup:
  Fix packet analyzer replacement.
  Spicy: Wenn replacing an analyzer add a component mapping.
  Add component API to transparently remap one component to another one.
  Move enabled/disabled functionality from analyzers into `Component` base class API.
2024-05-07 09:45:25 +02:00
Robin Sommer
083c682878
Merge remote-tracking branch 'origin/topic/robin/gh-3440-file-handles'
* origin/topic/robin/gh-3440-file-handles:
  Spicy: Query Zeek scriptland for file handles.
2024-05-07 09:44:41 +02:00
Robin Sommer
383a77dab7
Merge remote-tracking branch 'origin/topic/robin/gh-3606-port-enum'
* origin/topic/robin/gh-3606-port-enum:
  Spicy: Cleanup some runtime code.
  Spicy: Map Spicy's `Protocol` enum to Zeek's `transport_proto`.
2024-05-07 09:42:04 +02:00
Tim Wojtulewicz
6821a41c4e Move the options from policy/tuning/defaults to actual Zeek defaults, deprecate that package 2024-05-06 11:13:04 -07:00
Robin Sommer
ccfeffaf2b
Fix packet analyzer replacement.
Also reworking the existing test, which was pretty odd/broken.
2024-05-06 09:45:11 +02:00
Robin Sommer
3a469b3619
Spicy: Wenn replacing an analyzer add a component mapping.
This uses the new API to replace components internally.

With these changes in place, replacing protocol analyzers now don't
need to register their ports anymore if they match what the original
analyzer was using (because the old one's registrations will map
over).

Packet analyzer replacement doesn't quite work yet but will be fixed
in next commit.

Closes #3573.
2024-05-06 09:45:11 +02:00
Robin Sommer
56b9a79a65
Spicy: Query Zeek scriptland for file handles.
Like traditional file analyzers, we now query Zeek's
`get_file_handle()` event for handles when a connection begins
analyzing an embedded file. That means that Spicy-side protocol
analyzers that are forwarding data into file analysis now need to call
Zeek's `Files::register_protocol()` and provide a callback for
computing file handles. If that's missing, Zeek will now issue a
warning. This aligns with the requirements Zeek's traditional protocol
analyzers. (If the EVT file defines a protocol analyzer to `replace`
an existing one, that one's `register_protocol()` will be consulted.)

Because Zeek's `get_file_handle()` event requires a current
connection, if a Spicy file analyzer isn't directly part of a
connection context (e.g., with nested files), we continue to use
hardcoded, built-in file handle. Scriptland won't be consulted in
that case, just like before.

Closes #3440.
2024-05-06 09:20:38 +02:00