Commit graph

18740 commits

Author SHA1 Message Date
Arne Welzel
a975d65d01 input/Manager: Ignore empty record types
Somewhere record types with zero fields get the optional attribute
apparently. The input/sqlite/basic test failed due to complaining
that ctx is optional. It isn't optional and when it has zero fields
we can just ignore it, too.

Also adds a input framework test with an explicit empty record type
2025-07-03 18:21:15 +02:00
Arne Welzel
8c7d732d4c external: Bump commit hashes for external suites
get_file_handle() may include c$id and perturbs their values when adding new
fields. I think that's reasonable, as files transferred in one VLAN should
be treated separate from files transferred in a different VLAN.
2025-07-03 18:19:47 +02:00
Arne Welzel
e221042f14 ip/vlan_fivetuple: Populate nested conn_id_context, not conn_id
This also enforces conn_id and conn_id_ctx types instead of being
able to handle any conn_id-like record.
2025-07-03 18:19:46 +02:00
Arne Welzel
b7a22a87c6 ConnKey: Extend DoPopulateConnIdVal() with ctx
This prepares the move where ConnKey implementations should fill out
ctx rather than filling conn_id directly. The API continues to receive
both, conn_id and ctx, as adding fields to `conn_id` is reasonable
use-case even if it's just for logging purposes.
2025-07-03 18:19:46 +02:00
Arne Welzel
112e3c1c03 btest: Update tests and baselines after adding ctx to conn_id 2025-07-03 18:19:46 +02:00
Arne Welzel
25fc4d5d98 init-bare: Add conn_id_ctx to conn_id
This nested record can be used to discriminate orig_h or resp_h being
observed in different "contexts". A context can be based on VLAN tags,
but any custom ConnKey implementation should populate the ctx field,
allowing to write context-aware Zeek scripts without needing to know
what the context really is.
2025-07-03 18:19:46 +02:00
Arne Welzel
5847a2d32e Merge remote-tracking branch 'origin/topic/awelzel/cluster-telemetry-follow-up'
* origin/topic/awelzel/cluster-telemetry-follow-up:
  Bump cluster test suite
  cluster/Telemetry: Cache CallExpr locations
  cluster/Telemetry: Avoid unneeded StringVal() construction
  Val: Switch TablePatternMatcher to std::string_view
  RE: Add MatchAll() and MatchSet() for std::string_view
  cluster/websocket: Fix and test for invalid X-Application-Name
  cluster/telemetry: Move topic_normalization redef to zeromq
2025-06-30 13:30:04 +02:00
Arne Welzel
42ed82824e Bump cluster test suite
Since zeek/zeek#4552, metrics are labeled with node, not endpoint,
adapt the tests for that.
2025-06-30 13:22:31 +02:00
Arne Welzel
73e55fa909 cluster/Telemetry: Cache CallExpr locations 2025-06-30 13:22:31 +02:00
Arne Welzel
be8b4747c4 cluster/Telemetry: Avoid unneeded StringVal() construction 2025-06-30 13:22:31 +02:00
Arne Welzel
dad5ccd622 Val: Switch TablePatternMatcher to std::string_view
...and add TableVal::LookupPattern(std::string_view sv).
2025-06-30 13:22:31 +02:00
Arne Welzel
dce51b99e5 RE: Add MatchAll() and MatchSet() for std::string_view 2025-06-30 13:22:31 +02:00
Arne Welzel
5c6a6d9427 cluster/websocket: Fix and test for invalid X-Application-Name 2025-06-30 13:22:31 +02:00
zeek-bot
e79ae5a1d9 Update doc submodule [nomail] [skip ci] 2025-06-27 00:19:29 +00:00
Johanna Amann
dbeadc65ba Merge remote-tracking branch 'origin/topic/johanna/gh-4547'
* origin/topic/johanna/gh-4547:
  Spicy SSL analyzer: make record layer version parsing more strict
2025-06-26 20:21:54 +01:00
Arne Welzel
222f34b4ac Bump cluster testuite for endpoint to node renaming 2025-06-26 19:22:09 +02:00
Arne Welzel
3cd6e1ca06 Merge remote-tracking branch 'origin/topic/vern/if-coverage'
* origin/topic/vern/if-coverage:
  extend script coverage profiling to track whether conditionals evaluate to true/false
2025-06-26 18:49:42 +02:00
Johanna Amann
74b847a122 Spicy SSL analyzer: make record layer version parsing more strict
The Spicy based SSL analyzer was, so far, more permissive with the
record layer versions that it would accept.

This change brings the parsing of record layer versions in line with the
binpac based analyzer. This behavioral difference was discovered due to
a test that changed with the recent dpd log changes.
2025-06-26 16:57:51 +01:00
Vern Paxson
0255b5d120 extend script coverage profiling to track whether conditionals evaluate to true/false 2025-06-26 16:10:50 +02:00
Arne Welzel
26f5166d7a cluster/telemetry: Move topic_normalization redef to zeromq 2025-06-26 15:22:11 +02:00
Arne Welzel
22958f7cdf Merge remote-tracking branch 'origin/topic/awelzel/1474-cluster-telemetry'
* origin/topic/awelzel/1474-cluster-telemetry:
  btest/cluster/telemetry: Add smoke testing for telemetry
  cluster/WebSocket: Fetch X-Application-Name header as app label
  cluster/WebSocket: Pass X-Application-Name to dispatcher
  broker/WebSocketShim: Add calls to Telemetry hooks
  cluster/WebSocket: Configure telemetry for WebSocket backends
  broker: Hook up generic cluster telemetry
  cluster: Introduce telemetry component

One bug fix removing static from a variable that shouldn't be static.
2025-06-26 14:54:01 +02:00
Johanna Amann
14e801a709 Merge remote-tracking branch 'origin/topic/johanna/gh-4598'
* origin/topic/johanna/gh-4598:
  Only pass session ticket data in ssl_session_ticket_handshake event
2025-06-26 09:59:16 +01:00
zeek-bot
9f16050d0a Update doc submodule [nomail] [skip ci] 2025-06-26 00:26:06 +00:00
Arne Welzel
771c37b6b2 Merge branch 'rename' of https://github.com/bhaskarbhar/zeek
* 'rename' of https://github.com/bhaskarbhar/zeek:
  Update zeek.bif
  Update init-bare.zeek
  Added Baseline
  Renamed
2025-06-25 19:27:13 +02:00
bhaskarbhar
f9c34f29c7
Update zeek.bif 2025-06-25 22:53:29 +05:30
bhaskarbhar
722381366b
Update init-bare.zeek 2025-06-25 22:51:43 +05:30
root
1fba346f5d Added Baseline 2025-06-25 21:22:21 +05:30
root
da89e7ee6e Renamed 2025-06-25 21:10:08 +05:30
Arne Welzel
0e1431eef4 btest/cluster/telemetry: Add smoke testing for telemetry 2025-06-25 17:13:01 +02:00
Arne Welzel
b28e5f261e cluster/WebSocket: Fetch X-Application-Name header as app label 2025-06-25 17:12:59 +02:00
Arne Welzel
2f7d5eaf2a cluster/WebSocket: Pass X-Application-Name to dispatcher
This is going to be used to add labels to telemetry if the
X-Application-Name header is set.
2025-06-25 17:07:29 +02:00
Arne Welzel
6a84237a95 broker/WebSocketShim: Add calls to Telemetry hooks
WebSocket clients with Broker do not use the normal Broker backend, so
we need to add the telemetry invocations explicitly.
2025-06-25 16:59:49 +02:00
Arne Welzel
376e6dba97 cluster/WebSocket: Configure telemetry for WebSocket backends 2025-06-25 16:59:49 +02:00
Arne Welzel
f011e7e667 broker: Hook up generic cluster telemetry 2025-06-25 16:59:49 +02:00
Arne Welzel
4c34274a6c cluster: Introduce telemetry component 2025-06-25 16:59:49 +02:00
Evan Typanski
d3593e0489 Merge remote-tracking branch 'origin/topic/etyp/remove-list'
* origin/topic/etyp/remove-list:
  Remove `list` from Zeek grammar
2025-06-25 10:36:08 -04:00
Arne Welzel
cd934c460b Merge remote-tracking branch 'origin/topic/christian/extensible-conntuples'
* origin/topic/christian/extensible-conntuples:
  btest/plugins: Add test for custom ConnKey factory
  NEWS updates for pluggable connection tuples.
  Add a VLAN-aware flow tuple implementation.
  Deprecate ConnTuple and related APIs.
  Deprecate the old Connection constructor and detail::ConnKey class.
  Switch to virtualized use of new zeek::ConnKey class tree
  Provide a connkey factory for Zeek's default five-tuples.
  Add IP-specific ConnKey implementation.
  Establish plugin infrastructure for ConnKey factories.
  Add new ConnKey abstraction.
2025-06-25 14:17:49 +02:00
Arne Welzel
e7b1b174f0 btest/plugins: Add test for custom ConnKey factory
This just counts DoInits() and adds that information to the conn_id
record, but without including it into the hash. Mostly for smoke
testing.
2025-06-25 13:19:26 +02:00
Christian Kreibich
a040f550f4 NEWS updates for pluggable connection tuples. 2025-06-25 13:19:26 +02:00
Christian Kreibich
29b0f844c0 Add a VLAN-aware flow tuple implementation.
This is a first "real" implementation of a custom tuple, adding additional
fields over the standard five-tuple.

Includes test cases.
2025-06-25 13:19:26 +02:00
Christian Kreibich
a5122b5032 Deprecate ConnTuple and related APIs.
Given IP-aware ConnKeys, ConnTuples aren't really required any more. ConnTuple
had two benefits:

- It preserved the original src/dst orientation from the packet headers it was
based on, which IPBasedConnKey now tracks and provides accessor methods for.

- In IPBasedAnalyzer::AnalyzePacket() its instance survived past the std:move()
of the key into NewConn(), which we sidestep by keeping the original src address
and port around until we need after the connection is obtained.
2025-06-25 13:19:26 +02:00
Christian Kreibich
7548dc9e96 Deprecate the old Connection constructor and detail::ConnKey class.
The new key-based Connection constructor replaces the former, and the new
ConnKey class tree replaces the latter.
2025-06-25 13:19:26 +02:00
Christian Kreibich
52d6228b06 Switch to virtualized use of new zeek::ConnKey class tree
This touches quite a few places, but each just swaps out existing
APIs and/or zeek::detail::ConnKey instances.
2025-06-25 13:19:26 +02:00
Christian Kreibich
b8f82ff659 Provide a connkey factory for Zeek's default five-tuples.
Since the base factory is pure virtual this is now the first full
implementation, but still a bit of a special case because it implements Zeek's
default behavior and doesn't add "custom" content to the tuple.
2025-06-25 13:18:08 +02:00
Christian Kreibich
5af8fc242a Add IP-specific ConnKey implementation.
The InitTuple() implementation here is a placeholder for a fuller one following
later, when we do away with the need for ConnTuple.
2025-06-25 13:18:08 +02:00
Christian Kreibich
0c64f6a7b9 Establish plugin infrastructure for ConnKey factories.
ConnKey factories are intermediaries that encapsulate the details of how to
instantiate ConnKeys, which codify the hash input for connection lookups.
2025-06-25 13:18:07 +02:00
Christian Kreibich
d19fdfd17c Add new ConnKey abstraction. 2025-06-25 13:18:07 +02:00
Arne Welzel
4b472f2771 Merge remote-tracking branch 'origin/topic/awelzel/telemetry-endpoint-to-node-rename'
* origin/topic/awelzel/telemetry-endpoint-to-node-rename:
  telemetry: Rename endpoint label to node label
2025-06-25 09:33:55 +02:00
Arne Welzel
eea194ddd8 telemetry: Rename endpoint label to node label
Using a label named "endpoint" is not intuitive and requires explaining to
users that it's really just the Cluster::node value. Change the label to
"node", so that we don't need to do the explaining.

This probably breaks some existing users of the Prometheus metrics, but after
looking more at metrics recently, "endpoint" really is a thorn in my eye.
2025-06-25 09:33:01 +02:00
zeek-bot
261582bcd5 Update doc submodule [nomail] [skip ci] 2025-06-25 00:17:53 +00:00