Commit graph

18084 commits

Author SHA1 Message Date
Tim Wojtulewicz
e545fe8256 Ground work for pluggable storage serializers 2025-04-14 10:02:35 -07:00
Arne Welzel
faac36f4cd Merge remote-tracking branch 'origin/topic/awelzel/bump-zeekjs-0-17-0'
* origin/topic/awelzel/bump-zeekjs-0-17-0:
  Bump zeekjs to v0.17.0
2025-04-14 18:59:13 +02:00
Arne Welzel
ffaeeb6b12 Bump zeekjs to v0.17.0
fc005a5 Plugin: Drop unneeded zeekjs.bif.h
    cbf737f Plugin: Remove unneeded zeek::Args copy
    2129feb Plugin/Nodejs: Remove intermediate Event instance
    5de78b4 Ignore the typescript test.
    550a0f3 Force users to specify to Node that they want to run typescript.
    7a44aad Add a test for typescript support.
    cda2d69 Begin support for the Node's new experimental typescript support
2025-04-14 17:43:29 +02:00
Tim Wojtulewicz
c4d0273ffa Update bifcl submodule [nomail] 2025-04-14 08:11:51 -07:00
Arne Welzel
c2e039f14d Merge remote-tracking branch 'origin/topic/awelzel/generic-metadata-pre-work'
* origin/topic/awelzel/generic-metadata-pre-work:
  cluster/Backend: Add name and lookup component tag
  cluster/Event: Hide members behind accessors
  cluster/PublishEvent:: Make event non-const
  broker/Manager: Re-use broker serializer for conversion
  EventMgr: Add Dispatch() with handler and args
  plugin/Manager: Fix MetaHookPre and MetaHookPost using HOOK_CALL_FUNCTION
2025-04-13 17:16:46 +02:00
Arne Welzel
f1ae944c9c Merge branch 'topic/jgras/fix-future-event-ts' of https://github.com/J-Gras/zeek
* 'topic/jgras/fix-future-event-ts' of https://github.com/J-Gras/zeek:
  Prevent event timestamps set to future
2025-04-11 15:17:11 +02:00
Jan Grashoefer
3858a2920e Prevent event timestamps set to future
For scheduled events, the event timestamp is the intended timestamp. If
we force timer expiration, the timestamp might be in the future. Today,
this happens on shutdown. This change guarantees that event timestamps
are never set beyond network time.
2025-04-11 13:06:33 +02:00
Robin Sommer
75b3bca7de
Bump Spicy. 2025-04-11 13:00:30 +02:00
Robin Sommer
6bf6e695b5
Merge remote-tracking branch 'origin/topic/robin/gh-4301-with-spicy'
* origin/topic/robin/gh-4301-with-spicy:
  Fix `--with-spicy`.
2025-04-11 12:59:29 +02:00
Robin Sommer
a2f4588a15
Merge remote-tracking branch 'origin/topic/robin/gh-3522-spicy-docs-state'
* origin/topic/robin/gh-3522-spicy-docs-state:
  Spicy: Document lifetime semantics of Zeek analyzers created from Spicy.
2025-04-11 12:59:05 +02:00
Arne Welzel
3946856f06 cluster/Backend: Add name and lookup component tag
This adds two new accessors on Backend, Name() and Tag() that can
be used for introspection of a Backend instance.
2025-04-11 10:01:30 +02:00
Arne Welzel
214629e054 cluster/Event: Hide members behind accessors 2025-04-11 09:58:07 +02:00
Arne Welzel
90f94ff4f2 cluster/PublishEvent:: Make event non-const
We want to introduce a hook that can modify the cluster event instances, so
need to pass around a non-const version of it.
2025-04-11 09:58:07 +02:00
Arne Welzel
c4a48baeda broker/Manager: Re-use broker serializer for conversion 2025-04-11 09:58:07 +02:00
Arne Welzel
0e027fa4e3 EventMgr: Add Dispatch() with handler and args
Allow users to call event_mgr.Dispatch(handler, args) instead of
constructing the Event instance themselves. Deprecate the old API
and replace users.

There's a subtle change that net_done() may be propagated via
auto_publish() now, but that still needs opt-in from script land
and likely no one did that, or else they'd expected to have it
work anyhow.
2025-04-11 09:58:07 +02:00
Arne Welzel
6d97d5526a plugin/Manager: Fix MetaHookPre and MetaHookPost using HOOK_CALL_FUNCTION 2025-04-11 09:58:07 +02:00
Arne Welzel
3b478ddc0a Merge remote-tracking branch 'origin/topic/awelzel/make-files-x509-files-test-deterministic'
* origin/topic/awelzel/make-files-x509-files-test-deterministic:
  btest/files/x509/files: Sort analyzers in baseline
2025-04-10 15:47:52 +02:00
Arne Welzel
2f6c31df89 btest/files/x509/files: Sort analyzers in baseline
This test is very sensitive to new enum values that perturb the final
order of the output. Listing the commits that modify that file in the
past clearly shows changes to the baseline happen when a new component
with an enum value is introduced.

    40f04d4ccf broker: Add WebSocketShim backend
    51836d08ae protocol: Add StreamEvent analyzer
    fdde1e9841 cluster/serializer: Add binary-serialization-format
    6fb73aa9da broker: Add shim plugin adding a backend component
    85ca59484b postgresql: Initial parser implementation
    efc2681152 WebSocket: Introduce new analyzer and log
    ...

That's just distraction. Point fix the test to sort the analyzers before
writing the final log line.

I've previously attempted this in a more complete fashion in PR #3365
(issue #3364), but seems this direct fix might just make more sense.
2025-04-10 15:02:44 +02:00
Robin Sommer
9db73415cd
Spicy: Document lifetime semantics of Zeek analyzers created from Spicy.
Closes #3522.
2025-04-10 12:17:05 +02:00
Robin Sommer
ee674e3d80
Fix --with-spicy.
Closes #4301.
2025-04-10 11:03:49 +02:00
Tim Wojtulewicz
f74f5d2734 Merge remote-tracking branch 'origin/topic/timw/revert-unreachable'
* origin/topic/timw/revert-unreachable:
  Revert addition of std::unreachable
2025-04-09 11:12:05 -07:00
Arne Welzel
c0f2ba468d Merge remote-tracking branch 'origin/topic/awelzel/4312-paper-over-flakiness-through-sleep'
* origin/topic/awelzel/4312-paper-over-flakiness-through-sleep:
  btest/brokerstore-attr-persistence-clone: Add BTEST_BG_RUN_SLEEP=1
2025-04-09 18:11:42 +02:00
Arne Welzel
fa7ffaf8c1 btest/brokerstore-attr-persistence-clone: Add BTEST_BG_RUN_SLEEP=1
This test has become flaky after #4295. It hasn't been failing regularly
enough locally to make it approachable. Annotate with
BTEST_BG_RUN_SLEEP=1 to revert to pre #4295 behavior, hopefully
fixing the flakiness as a side-effect.
2025-04-09 18:10:35 +02:00
Tim Wojtulewicz
58fb91315e Revert addition of std::unreachable
After further testing with the fuzzer corpus, std::unreachable isn't
necessary here. It's fine to just let the default case break to fix
the Coverity warning.
2025-04-09 08:39:47 -07:00
Tim Wojtulewicz
f3568d148f Merge branch 'topic/timw/coverity-fixes'
* topic/timw/coverity-fixes:
  Add move assignment operator to ZeekString, which already had a move constructor
  Don't initialize telemetry listener at all if ZEEKCTL_CHECK_CONFIG is set
  Avoid extra temporary strings in concatenation
  Use const auto references to avoid some copies
  Clean up initialization of TunnelEncapsulation to use default member values
  Remove some unnecessary std::move calls in broker::Manager
  Avoid divide-by-zero in CardinalityCounter::Size
  Minor fixes in POP3 analyzer based on flycheck warnings
  More std::move changes based on Coverity findings
  Fix a potential memory leak in Debug function
2025-04-08 11:31:10 -07:00
Tim Wojtulewicz
637ea28653 Add move assignment operator to ZeekString, which already had a move constructor 2025-04-08 11:01:21 -07:00
Tim Wojtulewicz
4f0d18f45f Don't initialize telemetry listener at all if ZEEKCTL_CHECK_CONFIG is set 2025-04-08 11:01:21 -07:00
Tim Wojtulewicz
5c47533653 Avoid extra temporary strings in concatenation 2025-04-08 11:01:21 -07:00
Tim Wojtulewicz
f7495831a2 Use const auto references to avoid some copies 2025-04-08 11:01:21 -07:00
Tim Wojtulewicz
739efc79be Clean up initialization of TunnelEncapsulation to use default member values 2025-04-08 11:01:21 -07:00
Tim Wojtulewicz
9091fe5c99 Remove some unnecessary std::move calls in broker::Manager 2025-04-08 11:01:21 -07:00
Tim Wojtulewicz
bf82f8afc1 Avoid divide-by-zero in CardinalityCounter::Size 2025-04-08 11:01:21 -07:00
Tim Wojtulewicz
bde2dec685 Minor fixes in POP3 analyzer based on flycheck warnings
This also adds an implementation of unreachable() to use for default
cases where it shouldn't ever actually cause the default.
2025-04-08 11:01:21 -07:00
Tim Wojtulewicz
34ee136a3c More std::move changes based on Coverity findings 2025-04-08 11:01:21 -07:00
Tim Wojtulewicz
2390625732 Fix a potential memory leak in Debug function 2025-04-08 11:01:21 -07:00
Tim Wojtulewicz
718dc19345 Merge remote-tracking branch 'origin/topic/bbannier/bump-spicy'
* origin/topic/bbannier/bump-spicy:
  Bump pre-commit hooks
  Bump auxil/spicy to latest development snapshot
2025-04-08 11:00:56 -07:00
Tim Wojtulewicz
290c813373 Updating CHANGES and VERSION. 2025-04-08 11:00:38 -07:00
Tim Wojtulewicz
30d2642272 Merge remote-tracking branch 'origin/topic/timw/4281-unconditionally-forward-out-of-snap'
* origin/topic/timw/4281-unconditionally-forward-out-of-snap:
  Make SNAP analyzer use both OUI and protocol for forwarding
  Change packet analyzer identifiers to be 64-bit
2025-04-08 10:59:15 -07:00
Benjamin Bannier
8aa45c6476 Bump pre-commit hooks 2025-04-08 15:12:21 +02:00
Benjamin Bannier
eb55ea13b2 Bump auxil/spicy to latest development snapshot 2025-04-08 15:12:21 +02:00
zeek-bot
14d91efb71 Update doc submodule [nomail] [skip ci] 2025-04-05 00:14:59 +00:00
Arne Welzel
94b1ce8c15 Merge remote-tracking branch 'origin/topic/awelzel/set-val-on-ids-for-enums'
* origin/topic/awelzel/set-val-on-ids-for-enums:
  broker/main: Adapt enum values to agree with comm.bif
  btest/plugin: Add plugin testing enum identifiers
  Var/add_type: Do not clone EnumType when declared for the first time
  type/id/zeekygen: Add EnumVal to enum identifiers
2025-04-04 18:37:06 +02:00
Arne Welzel
6bc36e8cf8 broker/main: Adapt enum values to agree with comm.bif
Logic to detect this error already existed, but due to enum identifiers
not having a value set, it never triggered before.

Should probably backport this one.
2025-04-04 15:36:42 +02:00
Arne Welzel
cb610bdea2 btest/plugin: Add plugin testing enum identifiers
This plugin has a generic name, but for now just tests the API around
enum types and enum vals at InitPostScript() time.
2025-04-04 15:36:42 +02:00
Arne Welzel
ab87ba9786 Var/add_type: Do not clone EnumType when declared for the first time
EnumType receives the name into its constructor. Even for the  first declaration
the name is not empty and instead the same as the identifier's name. Due to that,
add_type() previously took the else path and created a shallow clone of the
initial type instead of using it. This lead to buggy behavior where enum value
identifiers declared within an enum's first body have a different TypePtr
associated than the one that is found via `zeek:🆔:find_type()`. It also
means that enum identifiers added via redef later would have a different
TypePtr than those in the initial pointer.
2025-04-04 15:36:42 +02:00
Arne Welzel
18597ea49c type/id/zeekygen: Add EnumVal to enum identifiers
Provide a direct way to go from a zeek🆔:ID value to EnumVal without
needing to go through the type.
2025-04-04 15:36:07 +02:00
Arne Welzel
f0ccd5c7f8 Merge remote-tracking branch 'origin/topic/awelzel/4323-traverse-all-stmt-nil-fix'
* origin/topic/awelzel/4323-traverse-all-stmt-nil-fix:
  Traverse: Do not short-circuit traverse_all() if stmts is nullptr
2025-04-03 13:11:30 +02:00
Arne Welzel
513eede34e Traverse: Do not short-circuit traverse_all() if stmts is nullptr
Since commit 0a813a53c7,
zeek::detail::stmts is set to a nullptr when before it was usually
an empty list of statements. This caused traverse_all() to be
short-circuited unless global statements were available.

Fixes #4323
2025-04-03 12:19:34 +02:00
Johanna Amann
37be65dfd0 Merge remote-tracking branch 'origin/topic/johanna/remove-bind-library-check'
* origin/topic/johanna/remove-bind-library-check:
  Remove unnecessary check for bind library.

Closes GH-432t log9
2025-04-03 10:53:21 +01:00
Tim Wojtulewicz
8d71420d09 Merge remote-tracking branch 'origin/topic/timw/enum-val-lookup-speedup'
* origin/topic/timw/enum-val-lookup-speedup:
  Some minor c++ modernization in EnumType methods
  Avoid O(n) search in EnumType::Lookup
2025-04-02 16:36:08 -07:00