Commit graph

1539 commits

Author SHA1 Message Date
Arne Welzel
0a06a77c69 btest/frameworks/cluster: Use generic cluster-layout.zeek 2025-05-20 20:30:01 +02:00
Arne Welzel
e114b0e371 btest/frameworks/config: Use generic cluster-layout.zeek 2025-05-20 20:30:01 +02:00
Jan Grashoefer
84cc4b890d Add STLS command to POP3 DPD signature 2025-05-14 16:37:25 +02:00
Tim Wojtulewicz
6f8924596f Merge remote-tracking branch 'origin/topic/johanna/fix-failed-service-logging'
* origin/topic/johanna/fix-failed-service-logging:
  Fix policy/protocols/conn/failed-service-logging.zeek
2025-05-07 10:29:54 -07:00
Tim Wojtulewicz
58ee8d3c5c Add Storage::is_connected BIF 2025-05-07 08:13:16 -07:00
Johanna Amann
f293d5a852 Fix policy/protocols/conn/failed-service-logging.zeek
In GH-4422 it was pointed out that the protocols/conn/failed-service-logging.zeek
policy script only works when
`DPD::track_removed_services_in_connection=T` is set.

This was caused by a logic error in the script. This commit fixes this
logic error and introduces an additional test that checks that
failed-service-logging works even when the option is not set to true.
2025-05-06 13:37:12 +01:00
Arne Welzel
0e327a0c12 testing/btest: Fix double commented @TEST- lines
sed -i 's/^# # @/# @/g'
2025-05-06 14:06:29 +02:00
Arne Welzel
fe89a521d1 QUIC: Use initial destination conn_id for decryption
Ensure the client side also uses the initial destination connection ID
for decryption purposes instead of the one from the current long header
packet. PCAP from local WiFi hotspot.
2025-05-05 14:34:11 +02:00
Arne Welzel
ae90524027 QUIC: Handle CRYPTO frames across multiple INITIAL packets
Instead of sending the accumulated CRYPTO frames after processing an
INITIAL packet, add logic to determine the total length of the TLS
Client or Server Hello (by peeking into the first 4 byte). Once all
CRYPTO frames have arrived, flush the reassembled data to the TLS
analyzer at once.
2025-05-05 14:34:11 +02:00
Kshitiz Bartariya
40935c31b1 Ignore case when matching prefix in http analyzer 2025-04-25 10:33:11 -07:00
Arne Welzel
86249db2a3 testing/btest/*test: Comment all @TEST lines 2025-04-17 16:30:23 +02:00
Arne Welzel
85b8c8866b testing/btest/*zeek: Comment all @TEST lines 2025-04-17 16:30:23 +02:00
Arne Welzel
ff58be2f36 ldap: Only switch into MS_KRB5 mode if responseToken exists
If the server doesn't include a responseToken within negTokenResp,
assume there won't be signing or sealing happening on the
connection. Don't switch into MS_KRB5 mode.

Closes #4275
2025-04-15 20:10:52 +02:00
Tim Wojtulewicz
cb1ef47a31 Add STORAGE_ prefixes for backends and serializers 2025-04-14 10:11:13 -07:00
Tim Wojtulewicz
88786a28a2 Add JSON storage serializer, use with existing backends/tests 2025-04-14 10:11:13 -07:00
Arne Welzel
2f6c31df89 btest/files/x509/files: Sort analyzers in baseline
This test is very sensitive to new enum values that perturb the final
order of the output. Listing the commits that modify that file in the
past clearly shows changes to the baseline happen when a new component
with an enum value is introduced.

    40f04d4ccf broker: Add WebSocketShim backend
    51836d08ae protocol: Add StreamEvent analyzer
    fdde1e9841 cluster/serializer: Add binary-serialization-format
    6fb73aa9da broker: Add shim plugin adding a backend component
    85ca59484b postgresql: Initial parser implementation
    efc2681152 WebSocket: Introduce new analyzer and log
    ...

That's just distraction. Point fix the test to sort the analyzers before
writing the final log line.

I've previously attempted this in a more complete fashion in PR #3365
(issue #3364), but seems this direct fix might just make more sense.
2025-04-10 15:02:44 +02:00
Tim Wojtulewicz
30d2642272 Merge remote-tracking branch 'origin/topic/timw/4281-unconditionally-forward-out-of-snap'
* origin/topic/timw/4281-unconditionally-forward-out-of-snap:
  Make SNAP analyzer use both OUI and protocol for forwarding
  Change packet analyzer identifiers to be 64-bit
2025-04-08 10:59:15 -07:00
Tim Wojtulewicz
32ae8f4eaa Make storage events take a tag for the backend instead of a string 2025-03-27 16:12:24 -07:00
Arne Welzel
c3c6ee5a2b telemetry: Run callbacks at collect time
Calling collect_metrics() from a script would not invoke metric
callbacks, resulting in most of the process metrics to be zero
when a Zeek process isn't scraped via Prometheus.

Fixes #4309
2025-03-26 12:07:27 +01:00
Tim Wojtulewicz
195b87b873 Make SNAP analyzer use both OUI and protocol for forwarding 2025-03-24 15:20:50 -07:00
Tim Wojtulewicz
855c530b64 Redis: Handle other errors from requests, fix KEY_EXISTS for put operations 2025-03-21 11:56:27 -07:00
Tim Wojtulewicz
3d7fcfb428 SQLite: handle existing keys when overwrite=F correctly 2025-03-21 11:56:27 -07:00
Tim Wojtulewicz
ba9cf1e4db Remove unnecessary type aliases from storage btests 2025-03-21 11:56:27 -07:00
Tim Wojtulewicz
d5ebaf476d Avoid thread-leak in scripts.base.frameworks.file-analysis.bifs.enable-disable btest
This btest uses the exit() BIF to shut down, which immediately calls
::exit() and kills Zeek without doing any shutdown. This will sometimes
leave the thread running the storage manager, which causes TSan to
complain about a thread leak. Switch to use the terminate() BIF instead
which cleanly shuts down all of Zeek.
2025-03-21 11:56:27 -07:00
Christian Kreibich
637baefd2c Merge branch 'topic/christian/is_valid_subnet'
* topic/christian/is_valid_subnet:
  Minor typo/markup fixes in NEWS.
  Add is_valid_subnet BiF
  Add btests for has_valid_octets() and normalize_mac()
2025-03-19 10:11:02 -07:00
Johanna Amann
7626039635 Merge remote-tracking branch 'origin/topic/johanna/ssh-server-banners-can-be-wild'
* origin/topic/johanna/ssh-server-banners-can-be-wild:
  SSH analyzer - tiny aesthetic fixes
  SSH: make banner parsing more robust
  SSH: split banner into client/server parts
2025-03-19 15:31:35 +00:00
Yacin Nadji
8b83c2995a Add is_valid_subnet BiF
Also includes consistency tweak for is_valid_ip() plus test cases.
2025-03-18 20:25:32 -07:00
Christian Kreibich
7ba9609992 Add btests for has_valid_octets() and normalize_mac()
These functions reside in base/utils/addrs.zeek and were not tested. We don't
use has_valid_octets() anywhere right now, normalize_mac() is used in the RADIUS
analyzer, and addr_to_uri() is used in the signatures/dpd.zeek tests.
2025-03-18 20:21:43 -07:00
Tim Wojtulewicz
d0741c8001 Allow sync methods to be called from when conditions, add related btest 2025-03-18 10:20:34 -07:00
Tim Wojtulewicz
b067a6e588 Redis: Fix sync erase, add btest for it 2025-03-18 10:20:34 -07:00
Tim Wojtulewicz
cc7b2dc890 Implement Storage::backend_opened and Storage::backend_lost events 2025-03-18 10:20:34 -07:00
Tim Wojtulewicz
a99a13dc4c SQLite: expand expiration test 2025-03-18 10:20:33 -07:00
Tim Wojtulewicz
cca1d4f988 Redis: Fix thread-contention issues with Expire(), add more tests 2025-03-18 10:20:33 -07:00
Tim Wojtulewicz
b81e876ec8 Change how redis-server is run during btests, removing redis.conf 2025-03-18 10:20:33 -07:00
Tim Wojtulewicz
9ed3e33f97 Completely rework return values from storage operations 2025-03-18 10:20:33 -07:00
Tim Wojtulewicz
e766af7322 Split sync/async handling into the BIF methods 2025-03-18 10:20:33 -07:00
Tim Wojtulewicz
c247de8ec3 Redis: Rework everything to only use async mode 2025-03-18 10:20:33 -07:00
Tim Wojtulewicz
a485b1d237 Make backend options a record, move actual options to be sub-records 2025-03-18 10:20:33 -07:00
Tim Wojtulewicz
28951dccf1 Split sync and async into separate script-land namespaces 2025-03-18 10:20:33 -07:00
Tim Wojtulewicz
42ad5bbf7d Add btest that uses a Redis backend in a cluster 2025-03-18 09:32:34 -07:00
Tim Wojtulewicz
f1a7376e0a Return generic result for get operations that includes error messages 2025-03-18 09:32:34 -07:00
Tim Wojtulewicz
4695060d75 Allow opening and closing backends to be async 2025-03-18 09:32:34 -07:00
Tim Wojtulewicz
ea87c773cd Redis: Support non-native expiration when reading traces 2025-03-18 09:32:34 -07:00
Tim Wojtulewicz
08bebaa426 Redis: Add btests for the redis backend 2025-03-18 09:32:34 -07:00
Tim Wojtulewicz
6289eb8e15 SQLite: Fix some issues with expiration, including in the btest 2025-03-18 09:32:34 -07:00
Tim Wojtulewicz
6bc5f70236 SQLite: Add additional btests, which also cover general storage functionality
- New erase/overwrite tests
- Change existing sqlite-basic test to use async
- Test passing bad keys to validate backend type checking
- New test for compound keys and values
2025-03-18 09:32:34 -07:00
Tim Wojtulewicz
ec49f5d550 SQLite: Handle automated expiration 2025-03-18 09:32:34 -07:00
Tim Wojtulewicz
9d1eef3fbc Add basic SQLite storage backend 2025-03-18 09:32:34 -07:00
Johanna Amann
6023c8b906 SSH: make banner parsing more robust
This change revamps SSH banner parsing.  The previous behavior was both
a bit too strict in some regards, and too permissive in other.

Specifically, clients are now required to send a line starting with
"SSH-" as the first line.  This is in line with the RFC, as well with
observed behavior. This also prevents the creation of `ssh.log` for
non-SSH traffic on port 22.

For the server side, we now accept text before the SSH banner. This
previously led to a protocol violation but is allowed by the spec.

New tests are added to cover these cases.
2025-03-18 16:19:33 +00:00
Johanna Amann
2daf692c95 Add two protocol mismatch testcases
These traces contain different protocols being used by originator/responder.

Traces from GH-4251
2025-03-04 15:38:20 +00:00