Commit graph

18870 commits

Author SHA1 Message Date
Arne Welzel
d79d4b1b2a cluster/OnLoop: Support DontBlock and Force flags for queueing
Also allow max_queue_size to be 0 for unlimited queueing.
2025-07-29 11:23:53 +02:00
Arne Welzel
b4d2af23dd cluster/ThreadedBackend: Injectable OnLoopProcess instance
This allows injecting a custom onloop process to configure the
max_queue_size at instantiation time. Also allow access to the
instance directly and deprecate the QueueForProcessing() helper
2025-07-21 21:36:33 +02:00
zeek-bot
db018253fe Update doc submodule [nomail] [skip ci] 2025-07-19 00:21:36 +00:00
Tim Wojtulewicz
cb2e193452 Merge remote-tracking branch 'origin/topic/timw/storage-metrics'
* origin/topic/timw/storage-metrics:
  Add SQLite page_count and file_size metrics
  Add btests to cover storage metrics
  Add storage metrics for operations, expirations, data transferred
  Fix ordering of telemtry metrics when running under test
  Make RunPragma take an optional value parser to return data
  Make SQLite::Step take a callback function for parsing result data
2025-07-18 14:28:46 -07:00
Tim Wojtulewicz
d0a6d84237 Add SQLite page_count and file_size metrics 2025-07-18 14:28:04 -07:00
Tim Wojtulewicz
f73ac7089f Add btests to cover storage metrics 2025-07-18 14:28:04 -07:00
Tim Wojtulewicz
a0ffe7f748 Add storage metrics for operations, expirations, data transferred 2025-07-18 14:28:04 -07:00
Tim Wojtulewicz
cab0883254 Fix ordering of telemtry metrics when running under test 2025-07-18 14:28:04 -07:00
Tim Wojtulewicz
365e6cbc9e Make RunPragma take an optional value parser to return data 2025-07-18 14:28:04 -07:00
Tim Wojtulewicz
b44f7ca9ad Make SQLite::Step take a callback function for parsing result data 2025-07-18 14:28:04 -07:00
Benjamin Bannier
784c4537e6 Merge branch 'topic/bbannier/bump-spicy' 2025-07-18 13:32:45 +02:00
Benjamin Bannier
e470c3241d Bump auxil/spicy to latest development snapshot 2025-07-18 11:07:37 +02:00
Tim Wojtulewicz
1dc7d88efd Merge remote-tracking branch 'origin/topic/timw/update-libkqueue'
* origin/topic/timw/update-libkqueue:
  Update libkqueue submodule [nomail]
2025-07-17 12:59:10 -07:00
Tim Wojtulewicz
5773283e10 Update libkqueue submodule [nomail] 2025-07-17 10:38:05 -07:00
Tim Wojtulewicz
beb70e27b5 Merge remote-tracking branch 'origin/topic/timw/cpp20-starts-and-ends-with'
* origin/topic/timw/cpp20-starts-and-ends-with:
  Use std::string/string_view versions of starts_with/ends_with where appropriate
2025-07-17 09:09:40 -07:00
Tim Wojtulewicz
a1d121e5aa Use std::string/string_view versions of starts_with/ends_with where appropriate
The util:: versions of these methods remain as a thin wrapper around them so
they can be used with const char* arguments. Otherwise callers have to manually
make string_view objects from the input.
s Please enter the commit message for your changes. Lines starting
2025-07-17 09:08:54 -07:00
Tim Wojtulewicz
6218643347 Merge remote-tracking branch 'origin/topic/timw/hilti-nolint-enum'
* origin/topic/timw/hilti-nolint-enum:
  Add nolint for enum size for HILTI_RT_ENUM use
2025-07-17 08:40:58 -07:00
zeek-bot
79639499fb Update doc submodule [nomail] [skip ci] 2025-07-17 00:27:51 +00:00
Tim Wojtulewicz
49a7f64460 Add nolint for enum size for HILTI_RT_ENUM use 2025-07-16 09:49:57 -07:00
Robin Sommer
c94ce6b946
Merge remote-tracking branch 'origin/topic/robin/gh-4481-test-analyzer'
* origin/topic/robin/gh-4481-test-analyzer:
  Spicy: Fix missing include.
  Bump Spicy.
  Spicy: Add functions to check if Zeek provides an analyzer of a given name.
2025-07-16 17:47:08 +02:00
Arne Welzel
8f4470926f Merge remote-tracking branch 'origin/topic/awelzel/eml-extraction-v3'
* origin/topic/awelzel/eml-extraction-v3:
  NEWS: Add entry about SMTP::enable_rfc822_msg_file_analysis
  btest: Add tests for full email extraction
  SMTP: Adapt scripts for enable_rfc822_msg_file_analysis
  SMTP: Add missing Undelivered() call
  SMTP: Add enable_rfc822_msg_file_analysis
2025-07-16 12:38:29 +02:00
Arne Welzel
c69ed1adf7 NEWS: Add entry about SMTP::enable_rfc822_msg_file_analysis 2025-07-16 12:37:33 +02:00
cccs-jsjm
1b3b3892b5 btest: Add tests for full email extraction 2025-07-16 12:37:33 +02:00
Arne Welzel
4c60dfd6c5 SMTP: Adapt scripts for enable_rfc822_msg_file_analysis
Specifically, set a MIME part's parent_id to the rfc822_msg_fuid if it
is set and take into account the current rfc822_msg_fuid for describe_file()
to avoid fuid collisions of the top-level RFC822 message and the first
MIME part.
2025-07-16 12:37:33 +02:00
Arne Welzel
a42875d033 SMTP: Add missing Undelivered() call
Not that it's implement, but for consistency.
2025-07-16 11:27:00 +02:00
Arne Welzel
6f05fbf2ce SMTP: Add enable_rfc822_msg_file_analysis
Enabling this option will instantiate a new fa_file instance for every
top-level RFC 822 message in an SMTP transaction.
2025-07-16 11:26:49 +02:00
Robin Sommer
a840613441
Spicy: Fix missing include. 2025-07-16 09:34:22 +02:00
Robin Sommer
7575c35c68
Bump Spicy. 2025-07-16 09:34:22 +02:00
Christian Kreibich
fba319857b Merge branch 'topic/bbannier/named-ctr'
* topic/bbannier/named-ctr:
  Prefer explicit construction to coercion in record initialization
2025-07-15 17:38:04 -07:00
Christian Kreibich
907ddce581 Merge branch 'topic/christian/fix-debuglogger-stdsort'
* topic/christian/fix-debuglogger-stdsort:
  Add missing header to allow std::sort() on GCC 15.1
2025-07-15 17:37:02 -07:00
Christian Kreibich
1dc7d1f124 Add missing header to allow std::sort() on GCC 15.1 2025-07-15 16:38:52 -07:00
Tim Wojtulewicz
42a2a19f28 Merge remote-tracking branch 'origin/topic/bbannier/sqlite-no-stale-entries'
* origin/topic/bbannier/sqlite-no-stale-entries:
  Make PUT on SQLite backend implicitly overwrite expired entries
  Prevent SQLite storage backend from serving expired entries
2025-07-15 15:51:54 -07:00
Benjamin Bannier
16c40f4f3a Make PUT on SQLite backend implicitly overwrite expired entries
The backend does not serve expired but still present entries so to a
user they do not exist. When they put new data over such an entry their
expecation is that the value is overwritten, even if not explicitly
requested.
2025-07-15 15:50:43 -07:00
Benjamin Bannier
2f67539c0f Prevent SQLite storage backend from serving expired entries
The SQLite storage backend implements expiration by hand and garbage
collection is done in `DoExpire`. This previously relied exclusively on
gets not running within `Storage::expire_interval` of the put, otherwise
we would potentially serve expired entries.

With this patch we explictly check that entries are not expired before
serving them so that the SQLite backend should never serve expired
entries.
2025-07-15 15:47:31 -07:00
Tim Wojtulewicz
abac0b577c Merge remote-tracking branch 'origin/topic/timw/update-submodules'
* origin/topic/timw/update-submodules:
  Update package-manager submodule [nomail]
  Update binpac submodule [nomail]
2025-07-15 08:58:11 -07:00
Robin Sommer
7dc5a70dc8
Spicy: Add functions to check if Zeek provides an analyzer of a given name.
```
## Checks if there is a Zeek analyzer of a given name.
##
## analyzer: the Zeek-side name of the analyzer to check for
## if_enabled: if true, only checks for analyzers that are enabled
##
## Returns the type of the analyzer if it exists, or ``Undef`` if it does not.
public function has_analyzer(analyzer: string, if_enabled: bool = True): bool &cxxname="zeek::spicy::rt::has_analyzer";

## Differentiates between the types of analyzers Zeek provides.
public type AnalyzerType = enum { Protocol, File, Packet, };

## Returns the type of a Zeek analyzer of a given name.
##
## analyzer: the Zeek-side name of the analyzer to check
## if_enabled: if true, only checks for analyzers that are enabled
##
## Returns the type of the analyzer if it exists, or ``Undef`` if it does not.
public function analyzer_type(analyzer: string, if_enabled: bool = True): AnalyzerType &cxxname="zeek::spicy::rt::analyzer_type";

```

Closes #4481.
2025-07-15 14:22:27 +02:00
Tim Wojtulewicz
e55c37ae8b Update package-manager submodule [nomail] 2025-07-14 21:40:21 -07:00
Tim Wojtulewicz
4a7c681b99 Update binpac submodule [nomail] 2025-07-14 21:40:21 -07:00
zeek-bot
e0e62e28f6 Update doc submodule [nomail] [skip ci] 2025-07-15 00:30:17 +00:00
Tim Wojtulewicz
dc5d7c3fc9 Merge remote-tracking branch 'origin/topic/timw/std-filesystem'
* origin/topic/timw/std-filesystem:
  Fix some missing #includes resulting from removal of ghc::filesystem
  Remove ghc::filesystem submodule, switch to std::filesystem
2025-07-14 13:09:24 -07:00
Tim Wojtulewicz
55b427142d Merge remote-tracking branch 'origin/topic/johanna/connection-syn-packet-doc'
* origin/topic/johanna/connection-syn-packet-doc:
  Documentation: connection_SYN_packet
2025-07-14 11:26:13 -07:00
Johanna Amann
2fc75009f4 Documentation: connection_SYN_packet
Slightly clarify documentation related to the connection_SYN_packet
event.
2025-07-14 11:25:49 -07:00
Tim Wojtulewicz
1f87382302 Fix some missing #includes resulting from removal of ghc::filesystem 2025-07-14 11:23:54 -07:00
Tim Wojtulewicz
770bc0491e Remove ghc::filesystem submodule, switch to std::filesystem 2025-07-14 11:23:54 -07:00
Tim Wojtulewicz
a843521e78 Merge remote-tracking branch 'origin/topic/timw/cpp20-clang-tidy-updates'
* origin/topic/timw/cpp20-clang-tidy-updates:
  Use ranges::reverse_view to fix a few reverse ranged-for loops
  Disable a few new modernize clang-tidy checkers, enabled by C++20
2025-07-14 09:31:31 -07:00
Tim Wojtulewicz
8640f92b1f Use ranges::reverse_view to fix a few reverse ranged-for loops 2025-07-14 09:30:54 -07:00
Tim Wojtulewicz
832f67e91c Disable a few new modernize clang-tidy checkers, enabled by C++20 2025-07-14 09:30:54 -07:00
Arne Welzel
a014f48e0f Merge remote-tracking branch 'origin/topic/awelzel/4655-putty-org-to-chiark'
* origin/topic/awelzel/4655-putty-org-to-chiark:
  btest: Switch putty.org usage in tests to official URL
2025-07-14 09:31:20 +02:00
Arne Welzel
13f32a6b37 btest: Switch putty.org usage in tests to official URL
Closes #4655
2025-07-14 09:07:58 +02:00
Tim Wojtulewicz
448590270b Merge remote-tracking branch 'origin/topic/timw/move-util-deferred'
* origin/topic/timw/move-util-deferred:
  Move util::Deferred into util-types.h
2025-07-13 19:23:08 -07:00