Commit graph

5567 commits

Author SHA1 Message Date
Arne Welzel
53b0f0ad64 Event: Deprecate default network timestamp metadata
This deprecates the Event constructor and the ``ts`` parameter of Enqueue()
Instead, versions are introduced that take a detail::MetadataVectorPtr which
can hold the network timestamp metadata and is meant to be allocated by the
caller instead of automatically during Enqueue() or within the Event
constructor.

This also introduces a BifConst ``EventMetadata::add_network_timestamp`` to
opt-in adding network timestamps to events globally. It's disabled by
default as there are not a lot of known use cases that need this.
2025-05-23 19:32:23 +02:00
Arne Welzel
cc7dc60c1e EventRegistry/zeek.bif/init-bare: Add event metadata infrastructure
Introduce a new EventMetadata module and members on EventMgr to register
event metadata types.
2025-05-23 19:31:58 +02:00
Tim Wojtulewicz
25f144381c SQLite: Fix typo in variable name causing pragmas not to retry on busy 2025-05-22 10:23:17 -07:00
Tim Wojtulewicz
25dd1a2702 Disable sqlite-cluster btest
This test is being flaky on some platforms and still having problems
with executing pragmas at startup. Disable it for now until it can be
fixed.
2025-05-21 15:42:29 -07:00
Tim Wojtulewicz
0c7ad126d6 Update baseline of sqlite-basic btest after recent changes 2025-05-21 11:01:54 -07:00
Tim Wojtulewicz
a58128a45c SQLite: Move integrity_check to pragma table 2025-05-21 09:38:27 -07:00
Tim Wojtulewicz
78dffb1d6f SQLite: Add backend option for pragma timeout 2025-05-21 09:38:27 -07:00
Tim Wojtulewicz
f0e7b78554 SQLite: Rename tuning_params to pragma_commands, move running pragmas to utility method 2025-05-21 09:38:27 -07:00
Tim Wojtulewicz
26cc6d4e7b SQLite: Add busy_timeout pragma to default options 2025-05-21 09:38:27 -07:00
Tim Wojtulewicz
e91421a8de Prefix sqlite-based btests with sqlite- to match redis tests 2025-05-21 09:38:27 -07:00
Tim Wojtulewicz
41bddae59f Add sqlite cluster storage btest 2025-05-21 09:38:27 -07:00
Arne Welzel
aabd1e3825 btest/coverage/test-all-policy-zeekygen: Load test-all-policy with zeekygen enabled
There should not be warnings produced. The default ZEEK_DISABLE_ZEEKYGEN=1
setting in the btest configuration hid some issues previously.
2025-05-21 11:49:12 +02:00
Arne Welzel
00eabb6cbb btest remaining: Use generic cluster-layout.zeek 2025-05-20 20:30:01 +02:00
Arne Welzel
9365f71965 btest/frameworks/logging: Use generic cluster-layout.zeek 2025-05-20 20:30:01 +02:00
Arne Welzel
6c00c05249 btest/files/x509: Use generic cluster-layout.zeek 2025-05-20 20:30:01 +02:00
Arne Welzel
d7b5955e5e btest/frameworks/notice: Use generic cluster-layout.zeek 2025-05-20 20:30:01 +02:00
Arne Welzel
2a3f2d0004 btest/policy: Use generic cluster-layout.zeek
Catch-all for cluster scripts in policy directory.
2025-05-20 20:30:01 +02:00
Arne Welzel
00a12a4cc5 btest/frameworks/intel: Use generic cluster-layout.zeek 2025-05-20 20:30:01 +02:00
Arne Welzel
4dec63936e btest/frameworks/sumstats: Use generic cluster-layout.zeek 2025-05-20 20:30:01 +02:00
Arne Welzel
0a06a77c69 btest/frameworks/cluster: Use generic cluster-layout.zeek 2025-05-20 20:30:01 +02:00
Arne Welzel
e114b0e371 btest/frameworks/config: Use generic cluster-layout.zeek 2025-05-20 20:30:01 +02:00
Arne Welzel
f9ff396acf btest/broker: Use generic cluster-layout.zeek 2025-05-20 20:30:01 +02:00
Arne Welzel
a15df5fc11 btest/cluster: Use generic cluster-layout.zeek 2025-05-20 20:30:01 +02:00
Arne Welzel
b3f53dc455 testing/btest/Files: Add generic broker/cluster-layout.zeek
This is a cluster-layout.zeek template that can be copied into a testing
directory if needed. The idea is that a developer sets environment
variables within their btest and the Cluster::nodes variable is
implicitly extended by appropriate nodes.

For example, using @TEST-PORT BROKER_LOGGER1_PORT will add an appropriate
logger-1 node to Cluster::nodes, based on the existence of the
BROKER_LOGGER1_PORT environment variable.
2025-05-20 20:30:01 +02:00
Arne Welzel
906b91dca8 Merge remote-tracking branch 'origin/topic/vern/http-sqli-replacement'
* origin/topic/vern/http-sqli-replacement:
  site/local: Switch to detect-sql-injection
  Add a revised script for detecting HTTP SQL injection, deprecate original
2025-05-20 16:26:13 +02:00
Arne Welzel
c687a24503 site/local: Switch to detect-sql-injection 2025-05-20 16:24:28 +02:00
Vern Paxson
dcd14f7a16 Add a revised script for detecting HTTP SQL injection, deprecate original 2025-05-20 16:24:20 +02:00
Christian Kreibich
fdecfba6b4 Merge branch 'smoot-improve-from_json' of github.com:/stevesmoot/zeek
* 'smoot-improve-from_json' of github.com:/stevesmoot/zeek:
  update baseline for zam
  Update src/zeek.bif
  Change from_json to return an error rather than print it.
2025-05-19 11:06:29 -07:00
Arne Welzel
eb15997cc3 Merge remote-tracking branch 'origin/topic/awelzel/event-trace-mgr-destructor-fclose'
* origin/topic/awelzel/event-trace-mgr-destructor-fclose:
  btest/core: Add event-trace test
  zeek-setup: Free event_trace_mgr after generating trace
  EventTraceMgr: Rename etm to event_trace_mgr
  EventTraceMgr: Move fclose() to destructor
2025-05-19 20:02:12 +02:00
Tim Wojtulewicz
456c1fa42c Remove unnecessary #includes in analyzer/packet analyzer/file analyzer headers 2025-05-19 10:25:05 -07:00
Arne Welzel
000cc50813 btest/core: Add event-trace test 2025-05-19 18:23:08 +02:00
Tim Wojtulewicz
e618d00326 Remove including <cinttypes> from util.h 2025-05-16 10:14:37 -07:00
Tim Wojtulewicz
c659592773 Reduce includes in plugin/Component.h 2025-05-16 10:14:37 -07:00
Tim Wojtulewicz
648f0f0623 Use quotes instead of <> for zeek includes 2025-05-16 10:14:36 -07:00
Jan Grashoefer
84cc4b890d Add STLS command to POP3 DPD signature 2025-05-14 16:37:25 +02:00
Arne Welzel
2255fa23b8 Merge remote-tracking branch 'origin/topic/vern/zam-aggr-change-in-loop'
* origin/topic/vern/zam-aggr-change-in-loop:
  fix for ZAM optimization when an aggregate is modified inside of a loop
2025-05-13 19:50:56 +02:00
Arne Welzel
6d2bd93f1f btest/cluster/websocket: Update tests for new event signature 2025-05-13 18:26:03 +02:00
Arne Welzel
a61aff010f cluster/websocket: Propagate code and reason to websocket_client_lost()
This allows to get visibility into the reason why ixwebsocket or the
client decided to disconnect.

Closed #4440
2025-05-13 18:26:03 +02:00
Arne Welzel
aaddeb19ad cluster/websocket: Support configurable ping interval
Primarily for testing purposes and maybe the hard-coded 5 seconds is too
aggressive for some deployments, so makes sense for it to be
configurable.
2025-05-13 18:26:03 +02:00
Vern Paxson
da689f1835 fix for ZAM optimization when an aggregate is modified inside of a loop 2025-05-09 15:01:55 -07:00
Tim Wojtulewicz
fd10dd015f Move options to redis backend options instead of module-level options 2025-05-07 15:38:58 -07:00
Tim Wojtulewicz
824b91216f Add operation_timeout and command_timeout storage backend options 2025-05-07 15:38:58 -07:00
Tim Wojtulewicz
6f8924596f Merge remote-tracking branch 'origin/topic/johanna/fix-failed-service-logging'
* origin/topic/johanna/fix-failed-service-logging:
  Fix policy/protocols/conn/failed-service-logging.zeek
2025-05-07 10:29:54 -07:00
Tim Wojtulewicz
8096388904 Update opt.ZAM-bif-tracking baseline 2025-05-07 09:12:56 -07:00
Arne Welzel
8089f5bed4 Merge remote-tracking branch 'origin/topic/awelzel/more-terminate-while-queueing-hardening'
* origin/topic/awelzel/more-terminate-while-queueing-hardening:
  btest/cluster/generic/publish-any: Apply Christian's fix from broker/publish-any
  wstest/terminate-while-queueing: Patch close_socket()
2025-05-07 17:24:04 +02:00
Arne Welzel
3ec3205074 btest/cluster/generic/publish-any: Apply Christian's fix from broker/publish-any 2025-05-07 17:18:01 +02:00
Tim Wojtulewicz
58ee8d3c5c Add Storage::is_connected BIF 2025-05-07 08:13:16 -07:00
Arne Welzel
82731992d9 wstest/terminate-while-queueing: Patch close_socket()
I believe there's a bug/usage issue in the websockets library
where during send(), EOF is detected and stored, but the receiving
thread is then discarding the last received frame. Avoid the bug
by replacing the close_socket() implementation of the websockets
library just for that test and leave detecting the EOF condition
to the receiving thread.
2025-05-07 16:33:54 +02:00
Arne Welzel
ca02316671 cluster/websocket: Stop and wait for reply thread during Terminate()
The terminate-while-queueing test added for #4428 failed spuriously
indicating that sometimes WebSocket clients receive code 1000 instead of 1001.
This happens if the ixwebsocket server is shutdown before the reply thread had a
chance to process queued close messages.

Fix by signaling and waiting for the dispatcher's reply thread to terminate
before returning from Terminate().
2025-05-07 12:45:01 +02:00
Steve Smoot
53f2f6ec57 update baseline for zam 2025-05-06 15:11:12 -07:00