Commit graph

18966 commits

Author SHA1 Message Date
Tim Wojtulewicz
743b9e27cc Merge remote-tracking branch 'origin/topic/timw/fix-irc-analyzer-event-types'
* origin/topic/timw/fix-irc-analyzer-event-types:
  Fix types passed to some of the IRC analyzer events
2025-07-29 07:19:36 -07:00
Tim Wojtulewicz
06ec03046d Merge remote-tracking branch 'origin/topic/timw/fix-ranges-debian-11-build-failure'
* origin/topic/timw/fix-ranges-debian-11-build-failure:
  Fix build failure with std::ranges on Debian 11
2025-07-29 07:19:11 -07:00
Arne Welzel
cd7836dda2 Merge remote-tracking branch 'origin/topic/awelzel/4431-zeromq-drop-policy-v2'
* origin/topic/awelzel/4431-zeromq-drop-policy-v2:
  cluster.bif: Improve Cluster::publish() docstring
  btest/cluster/zeromq: Add tests for overload behavior
  cluster/zeromq: Metric for msg errors
  cluster/zeromq: Drop events when overloaded
  cluster/zeromq: Comments and move lookups to InitPostScript()
  cluster/zeromq: Rework lambdas to member functions
  cluster/zeromq: Support local XPUB/XSUB hwm and buf configurability
  cluster/OnLoop: Support DontBlock and Force flags for queueing
  cluster/ThreadedBackend: Injectable OnLoopProcess instance
2025-07-29 11:38:49 +02:00
Arne Welzel
55ecd90928 cluster.bif: Improve Cluster::publish() docstring 2025-07-29 11:23:53 +02:00
Arne Welzel
c8307487d1 btest/cluster/zeromq: Add tests for overload behavior
The overload-drop.zeek and overload-no-drop.zeek tests have proxy,
worker-1 and worker-2 publish to the manager topic. For the drop
case, we verify that both, the senders, but also the manager drops
events. For the no-drop test, the HWMs are set such that all events
are buffered.

The overload-worker-proxy-topic*.zeek tests are similar, but instead
of publishing to the manager topic, proxy, worker-1 and worker-2 publish
to the proxy and worker topics to overload each other. This had
previously resulted in lockups and these tests verify that this doesn't
happen anymore.
2025-07-29 11:23:53 +02:00
Arne Welzel
d2bb86f8b4 cluster/zeromq: Metric for msg errors 2025-07-29 11:23:53 +02:00
Arne Welzel
073de9f5fd cluster/zeromq: Drop events when overloaded
When either the XPUB socket's hwm is reached, or the onloop queue is
full, drop the events. Users can set ths xpub_sndhwm and
onloop_queue_hwm to 0 to avoid these drops at the risk of unbounded
memory growth.
2025-07-29 11:23:53 +02:00
Arne Welzel
5de9296c77 cluster/zeromq: Comments and move lookups to InitPostScript() 2025-07-29 11:23:53 +02:00
Arne Welzel
85d5dda028 cluster/zeromq: Rework lambdas to member functions 2025-07-29 11:23:53 +02:00
Arne Welzel
5dc4586b70 cluster/zeromq: Support local XPUB/XSUB hwm and buf configurability 2025-07-29 11:23:53 +02:00
Arne Welzel
d79d4b1b2a cluster/OnLoop: Support DontBlock and Force flags for queueing
Also allow max_queue_size to be 0 for unlimited queueing.
2025-07-29 11:23:53 +02:00
Arne Welzel
12518e8256 Merge remote-tracking branch 'origin/topic/awelzel/expose-num-packets-unprocessed'
* origin/topic/awelzel/expose-num-packets-unprocessed:
  ConnStats: Expose num_packets_unprocessed
  packet_analysis/Manager: Rename GetUnprocessedCount() to PacketsUnprocessed()
2025-07-29 10:12:46 +02:00
Arne Welzel
96f9cc73c3 Merge remote-tracking branch 'origin/topic/awelzel/spicy-format-for-8.0'
* origin/topic/awelzel/spicy-format-for-8.0:
  Update .git-blame-ignore-revs
  analyzer/protocol: Reformat with spicy-format
  pre-commit-config: Bump spicy-format to 0.26.0
2025-07-29 10:05:33 +02:00
Arne Welzel
e535f4bab1 Update .git-blame-ignore-revs 2025-07-29 10:04:14 +02:00
Arne Welzel
d70bcd07b9 analyzer/protocol: Reformat with spicy-format 2025-07-29 10:01:32 +02:00
Arne Welzel
aa2afa3e9b pre-commit-config: Bump spicy-format to 0.26.0 2025-07-29 09:59:50 +02:00
Tim Wojtulewicz
d8e93f1b2d Fix types passed to some of the IRC analyzer events 2025-07-28 17:43:08 -07:00
zeek-bot
941ea4282b Update doc submodule [nomail] [skip ci] 2025-07-29 00:28:44 +00:00
Tim Wojtulewicz
6b4b3b41cd Fix build failure with std::ranges on Debian 11 2025-07-28 15:58:46 -07:00
Tim Wojtulewicz
73d56407b1 Merge remote-tracking branch 'origin/topic/awelzel/3935-dce-rpc-named-pipe-docs'
* origin/topic/awelzel/3935-dce-rpc-named-pipe-docs:
  dce-rpc: Make named_pipe filed docs extensive
2025-07-28 14:18:23 -07:00
Tim Wojtulewicz
22fc57a90a Merge remote-tracking branch 'origin/topic/johanna/gh-4656'
* origin/topic/johanna/gh-4656:
  Fix parsing of EDNS rcode
2025-07-28 14:16:24 -07:00
Tim Wojtulewicz
8063be111a Merge remote-tracking branch 'origin/topic/timw/more-irc-coverage'
* origin/topic/timw/more-irc-coverage:
  Expand coverage of IRC analyzer with more commands
  Add support for ircv3 tags
2025-07-28 14:15:47 -07:00
Tim Wojtulewicz
e2a702f1ab Expand coverage of IRC analyzer with more commands 2025-07-28 14:13:55 -07:00
Tim Wojtulewicz
a33f100880 Add support for ircv3 tags
This also replaces the irc-dcc-send.trace file in our pcap library with
a cleaner one that includes ircv3 tags.
2025-07-28 14:06:33 -07:00
Tim Wojtulewicz
8aeaed69c5 Merge remote-tracking branch 'origin/topic/timw/lazy-allocate-id-option-handlers'
* origin/topic/timw/lazy-allocate-id-option-handlers:
  Move ID::type in structure to fill memory padding
  Use sorted forward_list instead of multimap for ID option change handlers
2025-07-28 13:57:26 -07:00
Tim Wojtulewicz
47553c79ec Move ID::type in structure to fill memory padding 2025-07-28 13:41:51 -07:00
Tim Wojtulewicz
035b4a4a8e Use sorted forward_list instead of multimap for ID option change handlers 2025-07-28 13:41:51 -07:00
Tim Wojtulewicz
020dd1a848 Merge remote-tracking branch 'origin/topic/timw/cpp20-modernization'
* origin/topic/timw/cpp20-modernization:
  Remove intermediate cipher vectors in ssl-analyzer.pac
  Reduce the size of Func::Body quite a bit with some extra packing
  Switch to using std::ranges algorithms
  Enable modernize-std-numbers clang-tidy checker, fix findings
  Switch to using c++20 constraints instead of std::enable_if
2025-07-28 13:13:02 -07:00
Tim Wojtulewicz
d021845079 Remove intermediate cipher vectors in ssl-analyzer.pac 2025-07-28 13:03:25 -07:00
Tim Wojtulewicz
2cfa5e79fa Reduce the size of Func::Body quite a bit with some extra packing 2025-07-28 13:03:25 -07:00
Tim Wojtulewicz
72c79006ac Switch to using std::ranges algorithms 2025-07-28 13:03:25 -07:00
Tim Wojtulewicz
b4cbda4e02 Enable modernize-std-numbers clang-tidy checker, fix findings 2025-07-28 13:03:25 -07:00
Tim Wojtulewicz
414728cc71 Switch to using c++20 constraints instead of std::enable_if 2025-07-28 13:03:25 -07:00
Evan Typanski
c00314746a Merge branch 'topic/etyp/deprecate-record-ty-to-vec'
* topic/etyp/deprecate-record-ty-to-vec:
  Deprecate `record_type_to_vector`
2025-07-28 12:25:44 -04:00
Evan Typanski
7f93ff5ecd Deprecate record_type_to_vector
After #4682, record_fields is ordered so this is redundant. Furthermore,
it is incorrectly documented, as it says nested records are flattened.
2025-07-28 11:11:33 -04:00
Arne Welzel
120f061bcd ConnStats: Expose num_packets_unprocessed
Not sure it's the best place to put, but we don't have packet analysis stats
bif and also num_packets is already there, so seems reasonable to put the
num_packets_unprocessed into
2025-07-28 14:10:53 +02:00
Arne Welzel
fa476746bf packet_analysis/Manager: Rename GetUnprocessedCount() to PacketsUnprocessed()
Mostly to align with PacketsProcessed()
2025-07-28 13:41:32 +02:00
zeek-bot
7894d88461 Update doc submodule [nomail] [skip ci] 2025-07-26 00:28:44 +00:00
Tim Wojtulewicz
a09b4108c8 Merge remote-tracking branch 'origin/topic/timw/odesc-size-t'
* origin/topic/timw/odesc-size-t:
  Use size_t for byte sizes in SerializationFormat
  Add ODesc::Size() that returns size_t, deprecate ODesc::Len()
2025-07-25 09:14:01 -07:00
Tim Wojtulewicz
e1342ff72d Use size_t for byte sizes in SerializationFormat 2025-07-25 09:13:10 -07:00
Tim Wojtulewicz
7a5209855f Add ODesc::Size() that returns size_t, deprecate ODesc::Len() 2025-07-25 09:13:10 -07:00
Arne Welzel
3f810e038b dce-rpc: Make named_pipe filed docs extensive
Closes #3935
2025-07-25 11:12:38 +02:00
Arne Welzel
6e2a18ce4f Merge remote-tracking branch 'origin/topic/awelzel/conn-id-ctx-singleton'
* origin/topic/awelzel/conn-id-ctx-singleton:
  RecordType: Allow field init deferral of deferrable record constructors
  Conn: Use conn_id_ctx singleton
  Conn: Add InitPostScript() and conn_id_ctx singleton
  ID: Add conn_id_ctx
2025-07-25 10:59:07 +02:00
Arne Welzel
2cb26eda51 RecordType: Allow field init deferral of deferrable record constructors
The ctx: conn_id_ctx &default=conn_id_ctx() field was not optimized
as deferrable even though by default its an empty record and so deferring
initialization seems safe. Open-code the special record constructor
expression case in ExprFieldInit so that the ctx field is not default
initialized at record construction anymore. I am wondering a bit if the
same applies to &default=set() and &default=table().
2025-07-25 10:55:51 +02:00
zeek-bot
8a6aca6175 Update doc submodule [nomail] [skip ci] 2025-07-25 00:19:51 +00:00
Tim Wojtulewicz
90da71ef16 Merge remote-tracking branch 'origin/topic/timw/update-broker'
* origin/topic/timw/update-broker:
  Update broker submodule to pull in clang 20 fix [nomail]
2025-07-24 12:46:27 -07:00
Tim Wojtulewicz
73f2cadd88 Update broker submodule to pull in clang 20 fix [nomail] 2025-07-24 10:43:57 -07:00
Tim Wojtulewicz
9c845d9979 Merge remote-tracking branch 'origin/topic/timw/pid-start-time-metric'
* origin/topic/timw/pid-start-time-metric:
  Add process_start_time_seconds and process_pid metrics
2025-07-24 10:39:52 -07:00
Tim Wojtulewicz
d5db359772 Add process_start_time_seconds and process_pid metrics 2025-07-24 10:39:14 -07:00
Arne Welzel
906cec9adb Conn: Use conn_id_ctx singleton 2025-07-24 17:35:07 +02:00