Commit graph

5593 commits

Author SHA1 Message Date
Tim Wojtulewicz
07804232e6 fixup! Add opcode/opcode_name to DNS log record 2025-09-30 12:30:52 -07:00
Tim Wojtulewicz
fb0fdf1dcf Update zeek-testing and zeek-testing-private commit hashes 2025-09-12 08:51:06 -07:00
Tim Wojtulewicz
d344f015e0 Add basic event for emitting DNS dynamic update data
This also changes the existing DU btest from checking to ensure a weird was
emitted to checking the output from the event.
2025-09-11 12:08:24 -07:00
Tim Wojtulewicz
fa6eb6c928 Add opcode/opcode_name to DNS log record 2025-09-11 12:08:24 -07:00
Arne Welzel
9e7537044f btest/core/suspend_processing: Add WebSocket example
Add a test/example forwarding all new_connection() events produced during
`zeek -r wikipedia.trace` as my_new_connection() to a WebSocket client.

This is mostly to demonstrate and verify usage of suspend_processing(),
websocket_client_added(), resume_processing(), Pcap::file_done(),
websocket_client_lost() and terminate() together.
2025-09-08 11:11:13 -07:00
Tim Wojtulewicz
7142b9efde Merge remote-tracking branch 'klemensya/svcparams'
* klemensya/svcparams:
  Parse SVCB/HTTPS SvcParams list
2025-09-03 15:36:03 -07:00
Tim Wojtulewicz
e3b22cd21f Merge remote-tracking branch 'origin/topic/vern/emphasize-IDPtrs'
* origin/topic/vern/emphasize-IDPtrs:
  fixup! fixup! shift much of the internal use of ID* identifier pointers over to IDPtr objects
  fixup! shift much of the internal use of ID* identifier pointers over to IDPtr objects
  fixup! shift much of the internal use of ID* identifier pointers over to IDPtr objects
  fixup! shift much of the internal use of ID* identifier pointers over to IDPtr objects
  annotate a number of BTests as unsuited for -O gen-C++ testing due to multiple Zeek runs
  BTest baseline updates for -O gen-C++ - all minor tweaks
  BTest updates for script optimization tracking of BiFs
  regression test for former ASAN issue with script optimization of lambdas
  shift much of the internal use of ID* identifier pointers over to IDPtr objects
  maintenance update for script optimization's knowledge of BiFs
  logger fix for interoperability with -O gen-C++ code
2025-09-03 15:33:43 -07:00
Evan Typanski
42f7a2d88c Merge remote-tracking branch 'origin/topic/vern/field-assign-parsing'
* origin/topic/vern/field-assign-parsing:
  tightened up parsing of $field=X record constructor expressions
2025-09-03 16:21:24 -04:00
Vern Paxson
b6b8484819 annotate a number of BTests as unsuited for -O gen-C++ testing due to multiple Zeek runs 2025-09-03 11:19:38 -07:00
Vern Paxson
77bb32ff78 BTest baseline updates for -O gen-C++ - all minor tweaks 2025-09-03 11:19:38 -07:00
Vern Paxson
1b4bc72802 BTest updates for script optimization tracking of BiFs 2025-09-03 11:19:38 -07:00
Vern Paxson
e9971e31d5 regression test for former ASAN issue with script optimization of lambdas 2025-09-03 11:19:38 -07:00
Benjamin Bannier
07dd2a1c6f Merge branch 'topic/bbannier/test-bittorrent-unnessary-sigs' 2025-09-03 14:51:16 +02:00
Klemens Nanni
397f7e5c0e Parse SVCB/HTTPS SvcParams list
Add full support for RFC 9460's SvcParams list.

Amend the existing `dns_svcb_rr` record by a vector of new
`dns_svcb_param` records containing aptly typed SvcParamKey and
SvcParamValue pairs.  Example output:

```
@load base/protocols/dns
event dns_HTTPS( c: connection , msg: dns_msg , ans: dns_answer , https: dns_svcb_rr ) {
	for (_, param in https$svc_params)
		print to_json(param);  # filter uninitialised values
}
```

```
$ dig https cloudflare-ech.com +short | tr [:space:] \\n
1
.
alpn="h3,h2"
ipv4hint=104.18.10.118,104.18.11.118
ech=AEX+DQBBHgAgACBGL2e9TiFwjK/w1Zg9AmRm7mgXHz3PjffP0mTFNMxmDQAEAAEAAQASY2xvdWRmbGFyZS1lY2guY29tAAA=
ipv6hint=2606:4700::6812:a76,2606:4700::6812:b76
```

```
{"key":1,"alpn":["h3","h2"]}
{"key":4,"hint":["104.18.10.118","104.18.11.118"]}
{"key":5,"ech":"AEX+DQBBHgAgACBGL2e9TiFwjK/w1Zg9AmRm7mgXHz3PjffP0mTFNMxmDQAEAAEAAQASY2xvdWRmbGFyZS1lY2guY29tAAA="}
{"key":6,"hint":["2606:4700::6812:a76","2606:4700::6812:b76"]}
```

Values with malformed data or belonging to invalid/reserved keys
are passed raw bytes in network order for script-level inspection.

Follow up to "Initial Support to DNS SVCB/HTTPS RR"
https://github.com/zeek/zeek/pull/1808
2025-09-03 09:53:55 +03:00
Tim Wojtulewicz
deeca84332 Merge remote-tracking branch 'origin/topic/timw/readability-container-contains'
* origin/topic/timw/readability-container-contains:
  Fix a few more random clang-tidy findings
  Use std::numbers::pi instead of hard-coded value
  Use std::scoped_lock instead of std::lock_guard
  Use .contains() instead of .find() or .count()
2025-09-02 11:49:29 -07:00
Tim Wojtulewicz
aa3940f16d Use std::numbers::pi instead of hard-coded value 2025-09-02 16:51:23 +00:00
Benjamin Bannier
4c9cbeb43f Remove unnecessary peer signature from test scripts.base.protocols.bittorrent.tracker 2025-09-02 10:53:38 +02:00
Vern Paxson
36862a18d6 tightened up parsing of $field=X record constructor expressions 2025-08-31 15:14:26 -07:00
Christian Kreibich
236d3a1ced Bump zeek-testing-cluster to pull in WebSocket TLS updates 2025-08-29 17:20:15 -07:00
Tim Wojtulewicz
d7db612b0f Merge remote-tracking branch 'origin/topic/bbannier/issue-3266'
* origin/topic/bbannier/issue-3266:
  Fix installation of symlink with `DESTDIR`
2025-08-26 08:59:34 -07:00
Benjamin Bannier
15604811cb Fix installation of symlink with DESTDIR
We install test data which we also make available under an alternative
path for backwards compatibility. The installation of this symlink did
not take `DESTDIR` installs like used by Zeek's packaging into account
which caused installations from packages to behave different from
installs from source.

This patch fixes the symlink to respect a possible `DESTDIR`.

Closes #3266.
2025-08-21 10:56:28 +02:00
Arne Welzel
9a7678f15a cluster/Backend: Fallback to current network time when current event has not timestamp
When a WebSocket client sends an event to Zeek without explicit network
timestamp metadata, Zeek would use -1.0 as a timestamp for any events
published while handling this event. Instead, it seems far more sensible
to use the current network time in that scenario.
2025-08-20 19:56:50 +02:00
Arne Welzel
01fe022e07 cluster/serializer/broker: Do not send empty metadata vectors around
Event when there's no metadata attached to an event, we'd still use the
constructor passing an empty metadata vector, resulting in an on-the-wire
representation with an empty trailing vector.

Particularly visible when just snooping events via websocat. There also
seems to be some bug with the timestamp -1 handling.
2025-08-20 19:55:38 +02:00
Evan Typanski
59e84e06f6 Merge remote-tracking branch 'origin/topic/etyp/fix-optional-attr-errors'
* origin/topic/etyp/fix-optional-attr-errors:
  Fix parameter attributes pretending to be records
  Only allow `&optional` in records
2025-08-18 14:49:56 -04:00
Arne Welzel
2c9015d247 Merge branch 'master' of https://github.com/blightzero/zeek
* 'master' of https://github.com/blightzero/zeek:
  Changed behavior of var-extraction-uri.zeek from policy/protocol/http to extract only the URI parameter names. Do not include the path in the first parameter name. Only extract uri vars if parameters actually exist.
2025-08-18 13:14:34 +02:00
Arne Welzel
e04f725523 Merge remote-tracking branch 'amazing-pp/t/psql-login-no-role'
* amazing-pp/t/psql-login-no-role:
  Report PostgreSQL login success only after ReadyForQuery
2025-08-18 09:40:22 +02:00
Fupeng Zhao
e4e56789db
Report PostgreSQL login success only after ReadyForQuery
Previously, Zeek treated the receipt of `AuthenticationOk` as a
successful login. However, according to the PostgreSQL
Frontend/Backend Protocol, the startup phase is not complete until
the server sends `ReadyForQuery`. It is still possible for the server
to emit an `ErrorResponse` (e.g. ERRCODE_INVALID_AUTHORIZATION_SPECIFICATION)
after `AuthenticationOk` but before `ReadyForQuery`.

This change updates the PostgreSQL analyzer to defer reporting login
success until `ReadyForQuery` is observed. This prevents false
positives in cases where authentication succeeds but session startup
fails.
2025-08-18 10:59:44 +08:00
Arne Welzel
c0a863cba0 Merge remote-tracking branch 'origin/topic/vern/stmt-line-numbers'
* origin/topic/vern/stmt-line-numbers:
  maintenance updates for ZAM BiF-tracking
  fix line numbers associated with "if" and initialization statements
2025-08-17 17:29:09 +02:00
Arne Welzel
f57a1263d4 cluster/serializer/broker: Do not special case Broker::Data anymore
The previous approach ignored the fact that nested / inner values might
also be Broker::Data values. I'm not super sure about the validity of
the test, because it's essentially demonstrating any-nesting, but
it's not leading to extra Broker::Data encoding.
2025-08-17 16:56:20 +02:00
Vern Paxson
5b74b3d0ac maintenance updates for ZAM BiF-tracking 2025-08-16 14:10:17 -07:00
Tim Wojtulewicz
e64ec54172
Revert "Move BinPAC, bifcl, af_packet, and gen_zam submodules into main zeek repo" 2025-08-15 15:11:22 -07:00
Tim Wojtulewicz
a1f5f0a40e Fix some clang-tidy findings in generated BIF code 2025-08-15 13:57:27 -07:00
Evan Typanski
4445bc1daf Fix parameter attributes pretending to be records
Parameters relied on is_record for a couple of validations, but they are
not records and should not be treated as such. This way we can validate
&optional better.
2025-08-14 12:00:15 -04:00
Evan Typanski
4e5a56c5e0 Only allow &optional in records
There was some confusing behavior with &optional and locals, so this
should get rid of that by making it an error. However, there is a case
where function parameters are still allowed to have &optional - this is
because there are checks for &default in parameters as well.
2025-08-14 11:38:19 -04:00
Benjamin Grap
fafc0212a5 Changed behavior of var-extraction-uri.zeek from policy/protocol/http to extract only the URI parameter names. Do not include the path in the first parameter name. Only extract uri vars if parameters actually exist. 2025-08-13 22:45:20 +02:00
Tim Wojtulewicz
c8818d76bd Remove length limiting on string fields for HTTP 2025-08-12 17:31:29 -07:00
Tim Wojtulewicz
0ec2161b04 Add options to filter at the stream level as well as globally 2025-08-12 17:31:28 -07:00
Tim Wojtulewicz
339d46ae26 Add a weird that gets emitted when strings/containers are over the limits 2025-08-12 17:31:28 -07:00
Tim Wojtulewicz
837fde1a08 Add metrics to track string and container fields limited by length 2025-08-12 17:31:28 -07:00
Tim Wojtulewicz
e2e7ab28da Implement string- and container-length filtering at the log record level 2025-08-12 17:31:28 -07:00
Tim Wojtulewicz
d95affde4d Remove deprecations tagged for v8.1 2025-08-12 10:19:03 -07:00
Arne Welzel
b2a2ad7e10 smb2/read: Parse only 1 byte for data_offset, ignore reserved1
A user provided a SMB2 pcap with the reserved1 field of a ReadResponse
set to 1 instead of 0. This confused the padding computation due to
including this byte into the offset. Properly split data_offset and
reserved1 into individual byte fields.

Closes #4730
2025-08-08 16:12:20 +02:00
Arne Welzel
13f613eb1d Merge remote-tracking branch 'origin/topic/awelzel/4176-cluster-on-sub-unsub-hooks'
* origin/topic/awelzel/4176-cluster-on-sub-unsub-hooks:
  cluster: Add on_subscribe() and on_unsubscribe() hooks
2025-08-08 14:24:18 +02:00
Tim Wojtulewicz
54d67c3322 Merge remote-tracking branch 'origin/topic/timw/cleanup-warnings-from-plugin-btest-builds'
* origin/topic/timw/cleanup-warnings-from-plugin-btest-builds:
  Update zeek-aux to remove BRO_DIST from plugin skeleton
  cmake_minimum_required() should come before project()
2025-08-07 08:39:40 -07:00
Tim Wojtulewicz
162ecc022e Update zeek-aux to remove BRO_DIST from plugin skeleton 2025-08-07 08:39:08 -07:00
Tim Wojtulewicz
3c535ec215 cmake_minimum_required() should come before project() 2025-08-06 12:10:41 -07:00
Arne Welzel
f98508bbb0 btest/tap-analyzer: Update existing test and add new one for UpdateConnVal()
This also changes the output of connection UIDs from the tap analyzer to be
prefixed with C for easier correlation with other logs.

Relates to #4337 #4725 #4734 #4737
2025-08-06 17:22:59 +02:00
Johanna Amann
2f2f328a72 Merge remote-tracking branch 'origin/topic/johanna/analyzer-log-proto'
* origin/topic/johanna/analyzer-log-proto:
  Add proto to analyzer.log
2025-08-06 14:38:47 +01:00
Evan Typanski
22f77248f5 Merge remote-tracking branch 'origin/topic/etyp/fix-record-vec-type-conflict'
* origin/topic/etyp/fix-record-vec-type-conflict:
  Fix record coercion with compatible types
2025-08-06 09:10:19 -04:00
Arne Welzel
ac776b0aad btest/plugins/tap-analyzer: Update baseline
Relates to #4337 #4725 #4734
2025-08-06 14:17:42 +02:00