Commit graph

5571 commits

Author SHA1 Message Date
Arne Welzel
01fe022e07 cluster/serializer/broker: Do not send empty metadata vectors around
Event when there's no metadata attached to an event, we'd still use the
constructor passing an empty metadata vector, resulting in an on-the-wire
representation with an empty trailing vector.

Particularly visible when just snooping events via websocat. There also
seems to be some bug with the timestamp -1 handling.
2025-08-20 19:55:38 +02:00
Evan Typanski
59e84e06f6 Merge remote-tracking branch 'origin/topic/etyp/fix-optional-attr-errors'
* origin/topic/etyp/fix-optional-attr-errors:
  Fix parameter attributes pretending to be records
  Only allow `&optional` in records
2025-08-18 14:49:56 -04:00
Arne Welzel
2c9015d247 Merge branch 'master' of https://github.com/blightzero/zeek
* 'master' of https://github.com/blightzero/zeek:
  Changed behavior of var-extraction-uri.zeek from policy/protocol/http to extract only the URI parameter names. Do not include the path in the first parameter name. Only extract uri vars if parameters actually exist.
2025-08-18 13:14:34 +02:00
Arne Welzel
e04f725523 Merge remote-tracking branch 'amazing-pp/t/psql-login-no-role'
* amazing-pp/t/psql-login-no-role:
  Report PostgreSQL login success only after ReadyForQuery
2025-08-18 09:40:22 +02:00
Fupeng Zhao
e4e56789db
Report PostgreSQL login success only after ReadyForQuery
Previously, Zeek treated the receipt of `AuthenticationOk` as a
successful login. However, according to the PostgreSQL
Frontend/Backend Protocol, the startup phase is not complete until
the server sends `ReadyForQuery`. It is still possible for the server
to emit an `ErrorResponse` (e.g. ERRCODE_INVALID_AUTHORIZATION_SPECIFICATION)
after `AuthenticationOk` but before `ReadyForQuery`.

This change updates the PostgreSQL analyzer to defer reporting login
success until `ReadyForQuery` is observed. This prevents false
positives in cases where authentication succeeds but session startup
fails.
2025-08-18 10:59:44 +08:00
Arne Welzel
c0a863cba0 Merge remote-tracking branch 'origin/topic/vern/stmt-line-numbers'
* origin/topic/vern/stmt-line-numbers:
  maintenance updates for ZAM BiF-tracking
  fix line numbers associated with "if" and initialization statements
2025-08-17 17:29:09 +02:00
Arne Welzel
f57a1263d4 cluster/serializer/broker: Do not special case Broker::Data anymore
The previous approach ignored the fact that nested / inner values might
also be Broker::Data values. I'm not super sure about the validity of
the test, because it's essentially demonstrating any-nesting, but
it's not leading to extra Broker::Data encoding.
2025-08-17 16:56:20 +02:00
Vern Paxson
5b74b3d0ac maintenance updates for ZAM BiF-tracking 2025-08-16 14:10:17 -07:00
Tim Wojtulewicz
e64ec54172
Revert "Move BinPAC, bifcl, af_packet, and gen_zam submodules into main zeek repo" 2025-08-15 15:11:22 -07:00
Tim Wojtulewicz
a1f5f0a40e Fix some clang-tidy findings in generated BIF code 2025-08-15 13:57:27 -07:00
Evan Typanski
4445bc1daf Fix parameter attributes pretending to be records
Parameters relied on is_record for a couple of validations, but they are
not records and should not be treated as such. This way we can validate
&optional better.
2025-08-14 12:00:15 -04:00
Evan Typanski
4e5a56c5e0 Only allow &optional in records
There was some confusing behavior with &optional and locals, so this
should get rid of that by making it an error. However, there is a case
where function parameters are still allowed to have &optional - this is
because there are checks for &default in parameters as well.
2025-08-14 11:38:19 -04:00
Benjamin Grap
fafc0212a5 Changed behavior of var-extraction-uri.zeek from policy/protocol/http to extract only the URI parameter names. Do not include the path in the first parameter name. Only extract uri vars if parameters actually exist. 2025-08-13 22:45:20 +02:00
Tim Wojtulewicz
c8818d76bd Remove length limiting on string fields for HTTP 2025-08-12 17:31:29 -07:00
Tim Wojtulewicz
0ec2161b04 Add options to filter at the stream level as well as globally 2025-08-12 17:31:28 -07:00
Tim Wojtulewicz
339d46ae26 Add a weird that gets emitted when strings/containers are over the limits 2025-08-12 17:31:28 -07:00
Tim Wojtulewicz
837fde1a08 Add metrics to track string and container fields limited by length 2025-08-12 17:31:28 -07:00
Tim Wojtulewicz
e2e7ab28da Implement string- and container-length filtering at the log record level 2025-08-12 17:31:28 -07:00
Tim Wojtulewicz
d95affde4d Remove deprecations tagged for v8.1 2025-08-12 10:19:03 -07:00
Arne Welzel
b2a2ad7e10 smb2/read: Parse only 1 byte for data_offset, ignore reserved1
A user provided a SMB2 pcap with the reserved1 field of a ReadResponse
set to 1 instead of 0. This confused the padding computation due to
including this byte into the offset. Properly split data_offset and
reserved1 into individual byte fields.

Closes #4730
2025-08-08 16:12:20 +02:00
Arne Welzel
13f613eb1d Merge remote-tracking branch 'origin/topic/awelzel/4176-cluster-on-sub-unsub-hooks'
* origin/topic/awelzel/4176-cluster-on-sub-unsub-hooks:
  cluster: Add on_subscribe() and on_unsubscribe() hooks
2025-08-08 14:24:18 +02:00
Tim Wojtulewicz
54d67c3322 Merge remote-tracking branch 'origin/topic/timw/cleanup-warnings-from-plugin-btest-builds'
* origin/topic/timw/cleanup-warnings-from-plugin-btest-builds:
  Update zeek-aux to remove BRO_DIST from plugin skeleton
  cmake_minimum_required() should come before project()
2025-08-07 08:39:40 -07:00
Tim Wojtulewicz
162ecc022e Update zeek-aux to remove BRO_DIST from plugin skeleton 2025-08-07 08:39:08 -07:00
Tim Wojtulewicz
3c535ec215 cmake_minimum_required() should come before project() 2025-08-06 12:10:41 -07:00
Arne Welzel
f98508bbb0 btest/tap-analyzer: Update existing test and add new one for UpdateConnVal()
This also changes the output of connection UIDs from the tap analyzer to be
prefixed with C for easier correlation with other logs.

Relates to #4337 #4725 #4734 #4737
2025-08-06 17:22:59 +02:00
Johanna Amann
2f2f328a72 Merge remote-tracking branch 'origin/topic/johanna/analyzer-log-proto'
* origin/topic/johanna/analyzer-log-proto:
  Add proto to analyzer.log
2025-08-06 14:38:47 +01:00
Evan Typanski
22f77248f5 Merge remote-tracking branch 'origin/topic/etyp/fix-record-vec-type-conflict'
* origin/topic/etyp/fix-record-vec-type-conflict:
  Fix record coercion with compatible types
2025-08-06 09:10:19 -04:00
Arne Welzel
ac776b0aad btest/plugins/tap-analyzer: Update baseline
Relates to #4337 #4725 #4734
2025-08-06 14:17:42 +02:00
Johanna Amann
82266b1e78 Add proto to analyzer.log
The analyzer.log file was missing the protocol field to distinguish
tcp/udp connections.
2025-08-06 11:34:57 +01:00
Arne Welzel
b4925fbd16 TapAnalyzer: More verdict to action rename
Relates to #4725 #4337
2025-08-05 19:59:06 +02:00
Arne Welzel
4bc7f9532c IPBasedAnalyzer: Call TapPacket() when skipping
When skip_further_processing() is called, a TapAnalyzer should still see
the packets as skipped with SkipReason "skipping".
2025-08-05 19:47:04 +02:00
Arne Welzel
dc904b2216 SessionAdapter: Introduce TapAnalyzer for session adapter
This commit introduces a mechanism to attach light weight analyzers to
the root analyzer of sessions in order to tap into the packets delivered
to child analyzer.
2025-08-05 19:47:02 +02:00
Evan Typanski
006bef71b5 Fix record coercion with compatible types
Fixes #4722
2025-08-04 17:09:26 -04:00
Tim Wojtulewicz
6afeeca090 Start of 8.1.0 development 2025-08-04 08:26:29 -07:00
Arne Welzel
1a87ebab72 cluster: Add on_subscribe() and on_unsubscribe() hooks
Closes #4176
2025-08-01 14:06:19 +02:00
Tim Wojtulewicz
f2e155d7fa Merge remote-tracking branch 'origin/topic/timw/update-ct-ca-lists'
* origin/topic/timw/update-ct-ca-lists:
  Update CT/CA lists to versions from NSS 3.114
2025-07-31 14:32:21 -07:00
Tim Wojtulewicz
1daead9edd Update CT/CA lists to versions from NSS 3.114 2025-07-31 11:34:23 -07:00
Johanna Amann
136bdb43fd Merge remote-tracking branch 'origin/topic/johanna/gh-4694'
* origin/topic/johanna/gh-4694:
  Add tests for the deprecated-dpd-log.zeek policy script
  Move c$service_violation to deprecated-dpd-log.zeek
2025-07-31 16:11:00 +01:00
Arne Welzel
10e7f14f78 Merge remote-tracking branch 'origin/topic/awelzel/defer-more-stuff'
* origin/topic/awelzel/defer-more-stuff:
  RecordType: Ensure &default fields are always re-initialized
  Attr: Deprecate using &default and &optional together on record fields
  RecordType: Allow deferring &default=vector(), set(), table() fields
2025-07-30 10:35:56 +02:00
Arne Welzel
9eb94ee151 RecordType: Ensure &default fields are always re-initialized
This started working partly after the deferral logic introduced with
Zeek 6.0 so this finishes it :-)
2025-07-30 10:26:06 +02:00
Arne Welzel
473723cc47 Attr: Deprecate using &default and &optional together on record fields
If &default implies re-initialization of the field, using them together
doesn't make much sense.
2025-07-30 10:26:06 +02:00
Arne Welzel
23181e4811 RecordType: Allow deferring &default=vector(), set(), table() fields 2025-07-30 10:26:06 +02:00
Arne Welzel
d7fbd49d9e Merge remote-tracking branch 'origin/topic/vern/zam-record-fields-fixes'
* origin/topic/vern/zam-record-fields-fixes:
  fixes for specialized ZAM operations needing to check whether record fields exist
2025-07-30 10:08:21 +02:00
Johanna Amann
a90969800c Add tests for the deprecated-dpd-log.zeek policy script
This re-adds baselines for the old dpd.log to check functionality until
its removal in 8.1
2025-07-30 07:58:36 +01:00
Johanna Amann
8de178d923 Move c$service_violation to deprecated-dpd-log.zeek
This moves c$service_violation to the deprecated-dpd-log policy script.

This is the only script in the distribution that uses the field, and it
is unlikely to be used externally. It is also responsible for a
significant amount of memory use by itself.

This also restores the field being populated, which was broken in
GH-4362
2025-07-30 07:58:36 +01:00
Vern Paxson
47bf6af6a5 fixes for specialized ZAM operations needing to check whether record fields exist 2025-07-30 08:36:04 +02:00
Johanna Amann
a22b45c69e Merge remote-tracking branch 'origin/topic/johanna/gh-4202'
* origin/topic/johanna/gh-4202:
  Update NEWS for Conn::set_conn changes
  DNS-fuzzer: raise new_connection event
  Optimize Conn::set_conn to minimize operations
  Move Conn::set_conn() from connection_state_remove to new_connection
2025-07-29 21:01:51 +01:00
Arne Welzel
bda70067ec cluster: Move cluster::detail::Event to cluster::Event
This class is a parameter of virtual methods of the Backend API for users
to implement and also a parameter to the HookPublishEvent() API. Seems it
shouldn't be in detail and instead we should own it.

Alternatively, could mark the cluster APIs as not-stable-yet, but I
think we can move forward and make it non-detail for 8.0.
2025-07-29 18:13:59 +02:00
Arne Welzel
cd7836dda2 Merge remote-tracking branch 'origin/topic/awelzel/4431-zeromq-drop-policy-v2'
* origin/topic/awelzel/4431-zeromq-drop-policy-v2:
  cluster.bif: Improve Cluster::publish() docstring
  btest/cluster/zeromq: Add tests for overload behavior
  cluster/zeromq: Metric for msg errors
  cluster/zeromq: Drop events when overloaded
  cluster/zeromq: Comments and move lookups to InitPostScript()
  cluster/zeromq: Rework lambdas to member functions
  cluster/zeromq: Support local XPUB/XSUB hwm and buf configurability
  cluster/OnLoop: Support DontBlock and Force flags for queueing
  cluster/ThreadedBackend: Injectable OnLoopProcess instance
2025-07-29 11:38:49 +02:00
Arne Welzel
c8307487d1 btest/cluster/zeromq: Add tests for overload behavior
The overload-drop.zeek and overload-no-drop.zeek tests have proxy,
worker-1 and worker-2 publish to the manager topic. For the drop
case, we verify that both, the senders, but also the manager drops
events. For the no-drop test, the HWMs are set such that all events
are buffered.

The overload-worker-proxy-topic*.zeek tests are similar, but instead
of publishing to the manager topic, proxy, worker-1 and worker-2 publish
to the proxy and worker topics to overload each other. This had
previously resulted in lockups and these tests verify that this doesn't
happen anymore.
2025-07-29 11:23:53 +02:00