Commit graph

3508 commits

Author SHA1 Message Date
Tim Wojtulewicz
6ecb8f0f5f Merge remote-tracking branch 'origin/topic/timw/storage-serialization'
* origin/topic/timw/storage-serialization:
  Add STORAGE_ prefixes for backends and serializers
  Add versioning to JSON serializer
  Remove unnecessary includes in Val.h
  Move byte_buffer types from cluster and storage into util
  Remove unnecessary <array> and <memory> includes from util.h
  Mark storage classes as final where appropriate
  Add JSON storage serializer, use with existing backends/tests
  Make ValFromJSON return zeek::expected instead of a variant
  Ground work for pluggable storage serializers
2025-04-14 10:12:29 -07:00
Arne Welzel
faac36f4cd Merge remote-tracking branch 'origin/topic/awelzel/bump-zeekjs-0-17-0'
* origin/topic/awelzel/bump-zeekjs-0-17-0:
  Bump zeekjs to v0.17.0
2025-04-14 18:59:13 +02:00
Arne Welzel
c2e039f14d Merge remote-tracking branch 'origin/topic/awelzel/generic-metadata-pre-work'
* origin/topic/awelzel/generic-metadata-pre-work:
  cluster/Backend: Add name and lookup component tag
  cluster/Event: Hide members behind accessors
  cluster/PublishEvent:: Make event non-const
  broker/Manager: Re-use broker serializer for conversion
  EventMgr: Add Dispatch() with handler and args
  plugin/Manager: Fix MetaHookPre and MetaHookPost using HOOK_CALL_FUNCTION
2025-04-13 17:16:46 +02:00
Arne Welzel
f1ae944c9c Merge branch 'topic/jgras/fix-future-event-ts' of https://github.com/J-Gras/zeek
* 'topic/jgras/fix-future-event-ts' of https://github.com/J-Gras/zeek:
  Prevent event timestamps set to future
2025-04-11 15:17:11 +02:00
Robin Sommer
75b3bca7de
Bump Spicy. 2025-04-11 13:00:30 +02:00
Arne Welzel
3b478ddc0a Merge remote-tracking branch 'origin/topic/awelzel/make-files-x509-files-test-deterministic'
* origin/topic/awelzel/make-files-x509-files-test-deterministic:
  btest/files/x509/files: Sort analyzers in baseline
2025-04-10 15:47:52 +02:00
Tim Wojtulewicz
f74f5d2734 Merge remote-tracking branch 'origin/topic/timw/revert-unreachable'
* origin/topic/timw/revert-unreachable:
  Revert addition of std::unreachable
2025-04-09 11:12:05 -07:00
Arne Welzel
c0f2ba468d Merge remote-tracking branch 'origin/topic/awelzel/4312-paper-over-flakiness-through-sleep'
* origin/topic/awelzel/4312-paper-over-flakiness-through-sleep:
  btest/brokerstore-attr-persistence-clone: Add BTEST_BG_RUN_SLEEP=1
2025-04-09 18:11:42 +02:00
Tim Wojtulewicz
f3568d148f Merge branch 'topic/timw/coverity-fixes'
* topic/timw/coverity-fixes:
  Add move assignment operator to ZeekString, which already had a move constructor
  Don't initialize telemetry listener at all if ZEEKCTL_CHECK_CONFIG is set
  Avoid extra temporary strings in concatenation
  Use const auto references to avoid some copies
  Clean up initialization of TunnelEncapsulation to use default member values
  Remove some unnecessary std::move calls in broker::Manager
  Avoid divide-by-zero in CardinalityCounter::Size
  Minor fixes in POP3 analyzer based on flycheck warnings
  More std::move changes based on Coverity findings
  Fix a potential memory leak in Debug function
2025-04-08 11:31:10 -07:00
Tim Wojtulewicz
718dc19345 Merge remote-tracking branch 'origin/topic/bbannier/bump-spicy'
* origin/topic/bbannier/bump-spicy:
  Bump pre-commit hooks
  Bump auxil/spicy to latest development snapshot
2025-04-08 11:00:56 -07:00
Tim Wojtulewicz
290c813373 Updating CHANGES and VERSION. 2025-04-08 11:00:38 -07:00
Arne Welzel
94b1ce8c15 Merge remote-tracking branch 'origin/topic/awelzel/set-val-on-ids-for-enums'
* origin/topic/awelzel/set-val-on-ids-for-enums:
  broker/main: Adapt enum values to agree with comm.bif
  btest/plugin: Add plugin testing enum identifiers
  Var/add_type: Do not clone EnumType when declared for the first time
  type/id/zeekygen: Add EnumVal to enum identifiers
2025-04-04 18:37:06 +02:00
Arne Welzel
f0ccd5c7f8 Merge remote-tracking branch 'origin/topic/awelzel/4323-traverse-all-stmt-nil-fix'
* origin/topic/awelzel/4323-traverse-all-stmt-nil-fix:
  Traverse: Do not short-circuit traverse_all() if stmts is nullptr
2025-04-03 13:11:30 +02:00
Johanna Amann
37be65dfd0 Merge remote-tracking branch 'origin/topic/johanna/remove-bind-library-check'
* origin/topic/johanna/remove-bind-library-check:
  Remove unnecessary check for bind library.

Closes GH-432t log9
2025-04-03 10:53:21 +01:00
Tim Wojtulewicz
8d71420d09 Merge remote-tracking branch 'origin/topic/timw/enum-val-lookup-speedup'
* origin/topic/timw/enum-val-lookup-speedup:
  Some minor c++ modernization in EnumType methods
  Avoid O(n) search in EnumType::Lookup
2025-04-02 16:36:08 -07:00
Robin Sommer
28fd2a7a7a
Merge remote-tracking branch 'origin/topic/robin/spicy-typeinfo-switch'
* origin/topic/robin/spicy-typeinfo-switch:
  Spicy: Rework code for converting Spicy values to Zeek values.
2025-04-02 18:50:09 +02:00
Robin Sommer
af46322152
Merge remote-tracking branch 'origin/topic/robin/new-tuple'
* origin/topic/robin/new-tuple:
  Spicy: Port over to Spicy's new tuple representation.
2025-04-02 14:53:26 +02:00
Tim Wojtulewicz
b1cd502a96 Merge remote-tracking branch 'origin/topic/timw/add-ignore-docs-annotation'
* origin/topic/timw/add-ignore-docs-annotation:
  Add comment annotation to disable copying redef value into docs
2025-04-01 12:58:31 -07:00
Arne Welzel
363fbd9e28 Merge remote-tracking branch 'origin/topic/awelzel/fix-canon-name-builtin-plugins'
* origin/topic/awelzel/fix-canon-name-builtin-plugins:
  CI: Upgrade zeekctl task to Debian 12
  Bump cmake submodule
  CI: Add external plugins to builtin plugins task
2025-03-31 18:59:47 +02:00
Arne Welzel
14697ea6ba Merge remote-tracking branch 'origin/topic/neverlord/broker-logging'
* origin/topic/neverlord/broker-logging:
  Integrate review feedback
  Hook into Broker logs via its new API
2025-03-31 18:53:43 +02:00
Christian Kreibich
f068f0da4f Merge branch 'topic/christian/quic-history-u'
* topic/christian/quic-history-u:
  Add "U" to QUIC history docstrings and expand version string docs
2025-03-31 09:37:19 -07:00
Benjamin Bannier
06e41cec3a Merge remote-tracking branch 'origin/topic/bbannier/warning-arg-counting' 2025-03-31 15:08:18 +02:00
Arne Welzel
4f8a6a1dfc Merge remote-tracking branch 'origin/topic/awelzel/4177-4178-custom-event-metadata-prework'
* origin/topic/awelzel/4177-4178-custom-event-metadata-prework:
  Event/EventMgr: protected to private
  EventHandler: Deprecate Call(args, no_remote, ts), add Call(args)
  EventHandler: Header cleanup
  EventMgr: Do not cache current event attributes
  EventMgr: Drop src_val
  Event: Header cleanup
2025-03-31 14:22:17 +02:00
Tim Wojtulewicz
edbe8f0831 Merge remote-tracking branch 'origin/topic/timw/pass-tags-for-storage-events'
* origin/topic/timw/pass-tags-for-storage-events:
  Make storage events take a tag for the backend instead of a string
  Add move constructor to Tag class
2025-03-27 16:12:53 -07:00
Tim Wojtulewicz
ed47eedd6a Merge remote-tracking branch 'origin/topic/timw/storage-timeout-testing'
* origin/topic/timw/storage-timeout-testing:
  Fix handling of timeout conditions from storage backends
  Reformat plugin.storage btest to be more consistent with other storage tests
2025-03-27 14:10:39 -07:00
Christian Kreibich
656e88eaa8 Merge branch 'topic/christian/unexperimental-quic-histories'
* topic/christian/unexperimental-quic-histories:
  Remove "experimental" from the QUIC history field's comment string [skip ci]
2025-03-27 12:41:03 -07:00
Arne Welzel
acab4236e2 Merge remote-tracking branch 'origin/topic/awelzel/zeromq-no-fprintf-at-overload'
* origin/topic/awelzel/zeromq-no-fprintf-at-overload:
  cluster/zeromq: Improve XPUB stall behavior, add a metric
2025-03-26 14:24:02 +01:00
Arne Welzel
cd9b37e4a5 Merge remote-tracking branch 'origin/topic/awelzel/4309-telemetry-collect-calbacks'
* origin/topic/awelzel/4309-telemetry-collect-calbacks:
  telemetry: Run callbacks at collect time
2025-03-26 14:08:40 +01:00
Arne Welzel
33d7e5a7bf Merge remote-tracking branch 'origin/topic/awelzel/zeromq-unsubscription-visibility'
* origin/topic/awelzel/zeromq-unsubscription-visibility:
  cluster/OnLoop: Add metric for queue stalling instead of fprintf
  btest/cluster/websocket: ZeroMQ backend test
  cluster/OnLoop: Switch to condition variable
  cluster/zeromq: Fix node_topic() and nodeid_topic()
  cluster/websocket: Fix null deref at WebSocket server shutdown
  btest/cluster/websocket: Harden multi-client tests
  cluster/zeromq: Fix unsubscription visibility
2025-03-24 19:46:20 +01:00
Tim Wojtulewicz
bc38dbcc99 Merge remote-tracking branch 'origin/topic/timw/storage-framework-followup'
* origin/topic/timw/storage-framework-followup:
  Redis: Handle other errors from requests, fix KEY_EXISTS for put operations
  SQLite: handle existing keys when overwrite=F correctly
  Remove unnecessary type aliases from storage btests
  Avoid thread-leak in scripts.base.frameworks.file-analysis.bifs.enable-disable btest
  Fix data-race with calling DBG_LOG from a separate thread
  Fix data-race with ReturnCode objects in Sqlite::DoExpire
  Fix data race with calling run_state::network_time from a separate thread
  Add NEWS entry for Storage, reduce CHANGES spam
  Fix Coverity findings in the SQLite backend
  Remove inclusion of non-existent expected-lite path during dynamic plugin builds
  Squash code from OperationResultCallback into ResultCallback
  Add hiredis to generate-docs workflow to enable Redis backend
2025-03-21 11:57:17 -07:00
Tim Wojtulewicz
458bf762f1 Add NEWS entry for Storage, reduce CHANGES spam 2025-03-21 11:56:27 -07:00
Arne Welzel
0a6429d447 Merge remote-tracking branch 'origin/topic/awelzel/btest-bg-run-sleep-0'
* origin/topic/awelzel/btest-bg-run-sleep-0:
  testing/btest: Remove btest-bg-run sleep 1
2025-03-21 09:45:15 +01:00
Tim Wojtulewicz
0b590c575e Merge remote-tracking branch 'origin/topic/awelzel/allow-terminate-processing-from-non-main-thread'
* origin/topic/awelzel/allow-terminate-processing-from-non-main-thread:
  util: Fix terminate_processing() called from non-main thread
2025-03-19 12:47:48 -07:00
Tim Wojtulewicz
38484b6dc6 Merge remote-tracking branch 'origin/topic/vern/CPP-maint.Mar25'
* origin/topic/vern/CPP-maint.Mar25:
  minor baseline updates for -O gen-C++
  -O gen-C++ support for pattern-to-pattern comparison
  -O gen-C++ support for pattern vector comparisons
  -O gen-C++ support for type expressions
2025-03-19 12:46:52 -07:00
Tim Wojtulewicz
ba28dfe87e Merge remote-tracking branch 'origin/topic/timw/run-macos-ci-tests-as-root'
* origin/topic/timw/run-macos-ci-tests-as-root:
  CI: Run btests on macOS under sudo
2025-03-19 11:58:55 -07:00
Arne Welzel
f4d7243365 Merge remote-tracking branch 'origin/topic/awelzel/bump-zeekjs-0-16-0'
* origin/topic/awelzel/bump-zeekjs-0-16-0:
  Bump zeekjs to v0.16.0
2025-03-19 19:57:16 +01:00
Christian Kreibich
637baefd2c Merge branch 'topic/christian/is_valid_subnet'
* topic/christian/is_valid_subnet:
  Minor typo/markup fixes in NEWS.
  Add is_valid_subnet BiF
  Add btests for has_valid_octets() and normalize_mac()
2025-03-19 10:11:02 -07:00
Johanna Amann
7626039635 Merge remote-tracking branch 'origin/topic/johanna/ssh-server-banners-can-be-wild'
* origin/topic/johanna/ssh-server-banners-can-be-wild:
  SSH analyzer - tiny aesthetic fixes
  SSH: make banner parsing more robust
  SSH: split banner into client/server parts
2025-03-19 15:31:35 +00:00
Tim Wojtulewicz
cb5e3d0054 Merge remote-tracking branch 'origin/topic/timw/vntag-in-vlan'
* origin/topic/timw/vntag-in-vlan:
  Add analyzer registration from VLAN to VNTAG
2025-03-18 11:52:19 -07:00
Tim Wojtulewicz
d9af6d0270 Update src/3rdparty submodule [nomail] 2025-03-18 11:47:50 -07:00
Tim Wojtulewicz
e92f41e737 Updating CHANGES and VERSION. 2025-03-18 11:44:38 -07:00
Robin Sommer
3d25328204
Merge remote-tracking branch 'origin/topic/robin/gh-4250-vec-allocator'
* origin/topic/robin/gh-4250-vec-allocator:
  Spicy: Fix passing vectors to Zeek.
2025-03-18 09:08:50 +01:00
Tim Wojtulewicz
43f108bb71 Merge remote-tracking branch 'origin/topic/timw/ci-macos-sequoia'
* origin/topic/timw/ci-macos-sequoia:
  ci/init-external-repo.sh: Use regex to match macos cirrus task
  CI: Change macOS runner to Sequoia
2025-03-17 11:55:27 -07:00
Arne Welzel
1b655836be Merge remote-tracking branch 'origin/topic/awelzel/ci-generate-docs-run-pre-commit'
* origin/topic/awelzel/ci-generate-docs-run-pre-commit:
  github/generate-docs: Run pre-commit
  cluster/backends/zeromq: Fix rst link in docs
2025-03-12 13:11:18 +01:00
Arne Welzel
bb58148c64 Merge remote-tracking branch 'origin/topic/awelzel/4136-cluster-websocket-support'
* origin/topic/awelzel/4136-cluster-websocket-support:
  ci/opensuse-tumpleweed: Bust cache
  ci/macos/prepare: Install python@3 explicitly
  cluster/websocket: Implement WebSocket server
  cluster/websocket: Add IXWebsocket submodule
  ci/alpine: Install openssl package for testing
  ci: Install websockets from pip for all distros
  auxil/libunistd: Bump for ssize_t typedef
  auxil/broker: Bump to latest master version
  cluster/zeromq: Catch log_push.send() exception
  cluster/zeromq: Catch exceptions as const zmq::error_t&
  cluster/zeromq: No assert on inproc handling
  cluster/zeromq: Support configuring IO threads for proxy thread
  cluster/zeromq: Move variable lookups from DoInit() to DoInitPostScript()
  cluster/zeromq: Handle EINTR at shutdown
  cluster/zeromq: Queue one message at a time
  cluster/Backend: Queue a single message only
  cluster/zeromq: Adapt for OnLoopProcess changes
  cluster/ThreadedBackend: Switch to OnLoopProcess
  cluster/OnLoop: Introduce helper template class
  serializer/broker: Expose to_broker_event() and to_zeek_event()
2025-03-11 10:51:09 +01:00
Arne Welzel
db28ff04e2 Merge remote-tracking branch 'origin/topic/awelzel/javascript-separate-thread'
* origin/topic/awelzel/javascript-separate-thread:
  btest/javascript: Add file_sniff() and file_state_remove() test
  Bump zeekjs to v0.15.0
2025-03-10 19:42:36 +01:00
Tim Wojtulewicz
e8d91c8227 Merge remote-tracking branch 'origin/topic/timw/ci-macos-upgrade-pip'
* origin/topic/timw/ci-macos-upgrade-pip:
  CI: Unconditionally upgrade pip on macOS
2025-03-10 08:29:04 -07:00
Tim Wojtulewicz
8ed503314d Merge remote-tracking branch 'origin/topic/timw/fix-double-to-docs'
* origin/topic/timw/fix-double-to-docs:
  Fix return types in docs for double_to_count and double_to_int
2025-03-10 06:11:56 -07:00
Evan Typanski
c8f8bbaccb Merge remote-tracking branch 'origin/topic/etyp/print-type-const'
* origin/topic/etyp/print-type-const:
  Fix printing type constants like `double`
2025-03-07 13:14:22 -05:00
Tim Wojtulewicz
62a3119678 Merge remote-tracking branch 'origin/topic/timw/iwyu-clang-tidy'
* origin/topic/timw/iwyu-clang-tidy:
  Fix indentation in .clang-tidy
  Remove existing hand-written clang-tidy setup
  Add ability to enable iwyu and clang-tidy at configure time
2025-03-07 11:02:39 -07:00