Commit graph

18525 commits

Author SHA1 Message Date
Tim Wojtulewicz
460fe24a9a Fix clang-tidy cppcoreguidelines-macro-usage findings (macro functions) 2025-06-04 09:24:05 -07:00
Tim Wojtulewicz
ad99a6821e Fix clang-tidy cppcoreguidelines-macro-usage findings (macros as constants) 2025-06-04 09:24:05 -07:00
Tim Wojtulewicz
d6d56d330b script_opt: Add missing virtual destructor (cppcoreguidelines-virtual-class-destructor) 2025-06-04 09:24:05 -07:00
Johanna Amann
c392cda11c Merge remote-tracking branch 'origin/topic/johanna/parallelize-coverage-bare-mode-errors'
* origin/topic/johanna/parallelize-coverage-bare-mode-errors:
  Parallelize coverage/bare-mode-errors
2025-06-04 15:56:37 +01:00
Evan Typanski
14fa756d31 Merge remote-tracking branch 'origin/topic/etyp/fix-reenable-analyzer-log'
* origin/topic/etyp/fix-reenable-analyzer-log:
  Fix Spicy re-enable builtin analyzer debug message
2025-06-04 08:40:27 -04:00
Johanna Amann
c51998361c Parallelize coverage/bare-mode-errors
Currently, coverage/bare-mode-errors is one of the slowest tests in the
entire test suite. This is caused by the fact that it has to repeatedly
launch Zeek for every script that we ship. This is done sequentially.

This commit changes this test to use xargs to spawn 20 parallell
processes.
2025-06-04 13:18:05 +01:00
zeek-bot
fd98958b92 Update doc submodule [nomail] [skip ci] 2025-06-04 00:23:30 +00:00
Tim Wojtulewicz
0ef2710ca1 Add a missing #include needed by ZeekString on Windows 2025-06-03 15:19:22 -07:00
Tim Wojtulewicz
38ec080196 Update redis btest baselines 2025-06-03 13:37:03 -07:00
Tim Wojtulewicz
3ae9d8ba90 Merge remote-tracking branch 'origin/topic/timw/4350-redis-passwords'
* origin/topic/timw/4350-redis-passwords:
  Redis: Add support for sending AUTH commands during connection
  Redis: disconnect cleanly if INFO request fails
  Fix segfault if storage sync open_backend returns bad code
  Add ToStdString and ToStdStringView to ZeekString
2025-06-03 11:54:23 -07:00
Tim Wojtulewicz
f2aca331ec Redis: Add support for sending AUTH commands during connection 2025-06-03 11:38:38 -07:00
Tim Wojtulewicz
9f12208f57 Redis: disconnect cleanly if INFO request fails 2025-06-03 11:38:38 -07:00
Tim Wojtulewicz
0d18ce4e13 Fix segfault if storage sync open_backend returns bad code 2025-06-03 11:38:38 -07:00
Tim Wojtulewicz
d570486f36 Add ToStdString and ToStdStringView to ZeekString 2025-06-03 11:38:38 -07:00
Evan Typanski
23b244367b Fix Spicy re-enable builtin analyzer debug message 2025-06-03 11:02:30 -04:00
zeek-bot
f96ce6727f Update doc submodule [nomail] [skip ci] 2025-06-03 00:29:11 +00:00
Tim Wojtulewicz
f76a2437cd Merge remote-tracking branch 'origin/topic/timw/update-windows-ci'
* origin/topic/timw/update-windows-ci:
  Update libunistd submodule [nomail]
  Update Windows CI image to Visual Studio 2022
2025-06-02 14:53:01 -07:00
Tim Wojtulewicz
cd2514f9de Update libunistd submodule [nomail] 2025-06-02 14:52:05 -07:00
Tim Wojtulewicz
295efa7e83 Update Windows CI image to Visual Studio 2022 2025-06-02 14:52:05 -07:00
Tim Wojtulewicz
237c7da47a Merge remote-tracking branch 'origin/topic/timw/bump-opensuse-tumbleweed-docker-image'
* origin/topic/timw/bump-opensuse-tumbleweed-docker-image:
  CI: Force rebuild of opensuse tumbleweed docker image
2025-06-02 14:31:39 -07:00
Tim Wojtulewicz
aea614be56 Merge remote-tracking branch 'origin/topic/robin/gh-4501-eod-abort'
* origin/topic/robin/gh-4501-eod-abort:
  Bump Spicy to pull in fix.
2025-06-02 14:30:51 -07:00
Tim Wojtulewicz
d0f82d8e15 Merge remote-tracking branch 'origin/topic/awelzel/bump-zeekjs-0-17-1'
* origin/topic/awelzel/bump-zeekjs-0-17-1:
  Bump zeekjs to v0.17.1
2025-06-02 11:45:15 -07:00
Tim Wojtulewicz
f2b6fbe1a4 Merge remote-tracking branch 'origin/topic/bbannier/comment-fix'
* origin/topic/bbannier/comment-fix:
  Fix incorrectly copied comment [skip CI]
2025-06-02 11:44:05 -07:00
Tim Wojtulewicz
44a95a61b8 CI: Force rebuild of opensuse tumbleweed docker image
Something is off with the libhiredis package that's installed
as part of the current image, and 'zypper patch' is failing to
update it during the prepare step. Force a rebuild so the
updated package is part of the image to start with.
2025-06-02 10:40:39 -07:00
Arne Welzel
4d03f21031 Bump zeekjs to v0.17.1
b340648 version: 0.17.1
    ccd32e1 ci/docker: Use Fedora 42, retire Fedora 40
    6893431 Nodejs: Use argv.data() over &argv[0]
2025-06-02 19:05:18 +02:00
Arne Welzel
0a34b39e7a Merge remote-tracking branch 'origin/topic/awelzel/4177-4178-custom-event-metadata-part-2'
* origin/topic/awelzel/4177-4178-custom-event-metadata-part-2:
  Event: Bail on add_missing_remote_network_timestamp without add_network_timestamp
  btest/plugin: Test custom metadata publish
  NEWS: Add note about generic event metadata
  cluster: Remove deprecated Event constructor
  cluster: Remove some explicit timestamp handling
  broker/Manager: Fetch and forward all metadata from events
  Event/init-bare: Add add_missing_remote_network_timestamp logic
  cluster/Backend/DoProcessEvent: Use generic metadata, not just timestamps
  cluster/Event: Support moving args and metadata from event
  cluster/serializer/broker: Support generic metadata
  cluster/Event: Generic metadata support
  Event: Use -1.0 for undefined/unset timestamps
  cluster: Use shorter obj_desc versions
  Desc: Add obj_desc() / obj_desc_short() overloads for IntrusivePtr
2025-06-02 17:33:22 +02:00
Arne Welzel
8fc86bb4b6 Event: Bail on add_missing_remote_network_timestamp without add_network_timestamp 2025-06-02 17:31:36 +02:00
Arne Welzel
0ab53c75cd btest/plugin: Test custom metadata publish
Usage demo for plugin writers to add custom event metadata and access in
in Zeek scripts.
2025-06-02 17:31:36 +02:00
Arne Welzel
7db03a8c77 NEWS: Add note about generic event metadata 2025-06-02 17:31:36 +02:00
Arne Welzel
8e87dcbdb2 cluster: Remove deprecated Event constructor
It is now unused, ditch it. This wasn't available in an LTS release yet
and anyhow is in the detail namespace.
2025-06-02 17:31:36 +02:00
Arne Welzel
e3a83addce cluster: Remove some explicit timestamp handling
Backend::MakeClusterEvent() for now is the only place to add implicit
network timestamp metadata within the cluster component.
2025-06-02 17:31:36 +02:00
Arne Welzel
e1f70164e0 broker/Manager: Fetch and forward all metadata from events
Also use the generic metadata version for publishing, keep the
ts-based API for now, but only add timestamps when
EventMetadata::add_network_timestamp is T. I'm not sure what the
right way forward here is, maybe deprecating Broker's publish event
variations and funneling through cluster.
2025-06-02 17:31:36 +02:00
Arne Welzel
96f2d5d369 Event/init-bare: Add add_missing_remote_network_timestamp logic
Make defaulting to the local network timestamp for remote events opt-in.
2025-06-02 17:31:36 +02:00
Arne Welzel
b87109fcf1 cluster/Backend/DoProcessEvent: Use generic metadata, not just timestamps 2025-06-02 17:31:36 +02:00
Arne Welzel
4996ba88fb cluster/Event: Support moving args and metadata from event 2025-06-02 17:31:36 +02:00
Arne Welzel
46d4b5825b cluster/serializer/broker: Support generic metadata
Instead of handling just the network timestamp, support extraction of
the whole metadata vector that broker events hold.
2025-06-02 17:31:36 +02:00
Arne Welzel
71412f35b7 cluster/Event: Generic metadata support
Instead of a timestamp attribute, switch to holding a EventMetadataVectorPtr
like zeek::Event instances do. Keep the old constructor until the end of
the patch series.
2025-06-02 17:31:36 +02:00
Arne Welzel
7b4b1779bf Event: Use -1.0 for undefined/unset timestamps
This can happen if either there's no network timestamp associated with
an event, or there's currently no event being dispatched. Using 0.0
isn't great as it's the normal start timestamp before reading a network
packet. Using -1.0 gives the caller a chance to check and realize what's
going on.
2025-06-02 17:31:36 +02:00
Benjamin Bannier
0dae8b8d2d Fix incorrectly copied comment [skip CI] 2025-06-02 13:24:24 +02:00
Arne Welzel
31f51f7a87 Merge remote-tracking branch 'origin/topic/bbannier/coverity-fixes'
* origin/topic/bbannier/coverity-fixes:
  Prefer `std::move` over copy
2025-06-02 10:17:24 +02:00
Arne Welzel
e5bb6317fa Merge remote-tracking branch 'origin/topic/vern/CPP-maint.May25'
* origin/topic/vern/CPP-maint.May25:
  minor BTest maintenance updates for -O gen-C++
  fix for more robustly finding BTests to assess for -O gen-C++
  fix for -O gen-C++ dealing with type constants of unnamed compound types
2025-06-02 10:12:27 +02:00
Arne Welzel
41f04eda72 Merge remote-tracking branch 'origin/topic/awelzel/intel-indicator-hooks'
* origin/topic/awelzel/intel-indicator-hooks:
  intel/seen/manage-event-groups: Policy script for toggling intel event groups
  intel: Add indicator_inserted and indicator_removed hooks
2025-06-02 09:52:07 +02:00
Arne Welzel
0619fe2f4f intel/seen/manage-event-groups: Policy script for toggling intel event groups
Co-authored-by: Mohan Dhawan <mohan@corelight.com>
2025-06-02 09:51:14 +02:00
Arne Welzel
7eb849ddf4 intel: Add indicator_inserted and indicator_removed hooks
This change adds two new hooks to the Intel framework that can be used
to intercept added and removed indicators and their type.

These hooks are fairly low-level. One immediate use-case is to count the
number of indicators loaded per Intel::Type and enable and disable the
corresponding event groups of the intel/seen scripts.

I attempted to gauge the overhead and while it's definitely there, loading
a file with ~500k DOMAIN entries takes somewhere around ~0.5 seconds hooks
when populated via the min_data_store store mechanism. While that
doesn't sound great, it actually takes the manager on my system 2.5
seconds to serialize and Cluster::publish() the min_data_store alone
and its doing that serially for every active worker. Mostly to say that
the bigger overhead in that area on the manager doing redundant work
per worker.

Co-authored-by: Mohan Dhawan <mohan@corelight.com>
2025-06-02 09:50:48 +02:00
Benjamin Bannier
1760d99c49 Prefer std::move over copy 2025-06-02 08:45:32 +02:00
Vern Paxson
614eb8d343 minor BTest maintenance updates for -O gen-C++ 2025-05-31 12:52:44 -07:00
Vern Paxson
9117ccab12 fix for more robustly finding BTests to assess for -O gen-C++ 2025-05-31 12:50:14 -07:00
Vern Paxson
e165e64fa5 fix for -O gen-C++ dealing with type constants of unnamed compound types 2025-05-31 12:49:37 -07:00
zeek-bot
224519c11a Update doc submodule [nomail] [skip ci] 2025-05-31 00:26:58 +00:00
Tim Wojtulewicz
3282bbc429 Merge remote-tracking branch 'origin/topic/vern/ZAM-maint.May25'
* origin/topic/vern/ZAM-maint.May25:
  fix for crash when interpreting transformed ASTs that include multi-field record assignments/additions
  Remove unused ZAM compiler method
2025-05-30 13:07:01 -07:00