Tim Wojtulewicz
938073dfed
Merge remote-tracking branch 'origin/topic/timw/4742-skip-storage-thread'
...
* origin/topic/timw/4742-skip-storage-thread:
Avoid starting up storage expiration thread with zero backends
Remove a couple of obsolete TODOs in storage manager
2025-09-17 08:44:55 -07:00
Tim Wojtulewicz
dd14e380a1
Force SQLite to be in thread-safe mode during CMake
...
This allows us to remove the need to check for thread-safe mode in the
various SQLite plugins. See https://www.sqlite.org/compile.html#threadsafe
for why `1` is a good choice here.
2025-09-16 12:19:09 -07:00
Tim Wojtulewicz
50581bfd3c
Avoid starting up storage expiration thread with zero backends
2025-09-15 14:18:03 -07:00
Tim Wojtulewicz
8bc849d425
Remove a couple of obsolete TODOs in storage manager
2025-09-15 14:17:41 -07:00
Tim Wojtulewicz
72c79006ac
Switch to using std::ranges algorithms
2025-07-28 13:03:25 -07:00
Tim Wojtulewicz
2da3b3a2a6
Ignore a couple of known-unused results reported by Coverity
2025-07-23 15:17:53 -07:00
Tim Wojtulewicz
22e78c3c24
Reset the value of a status variable in SQLite backend before using it in a loop
2025-07-23 15:17:53 -07:00
Tim Wojtulewicz
0b3b5e2ef2
Fix a potential memory leak reported by Coverity
2025-07-23 15:17:53 -07:00
Tim Wojtulewicz
205c72d26f
Add some additional std::moves reported by Coverity
2025-07-23 15:17:53 -07:00
Tim Wojtulewicz
4c58fb26c4
Fix uninitialized class member Coverity findings
2025-07-23 15:17:53 -07:00
Tim Wojtulewicz
146837fe73
Fix swapped storage metrics names
2025-07-23 13:14:46 -07:00
Tim Wojtulewicz
7e3ed2010d
Add flag to force synchronous mode when calling storage script-land functions
2025-07-23 13:14:34 -07:00
Tim Wojtulewicz
d0a6d84237
Add SQLite page_count and file_size metrics
2025-07-18 14:28:04 -07:00
Tim Wojtulewicz
a0ffe7f748
Add storage metrics for operations, expirations, data transferred
2025-07-18 14:28:04 -07:00
Tim Wojtulewicz
365e6cbc9e
Make RunPragma take an optional value parser to return data
2025-07-18 14:28:04 -07:00
Tim Wojtulewicz
b44f7ca9ad
Make SQLite::Step take a callback function for parsing result data
2025-07-18 14:28:04 -07:00
Tim Wojtulewicz
a1d121e5aa
Use std::string/string_view versions of starts_with/ends_with where appropriate
...
The util:: versions of these methods remain as a thin wrapper around them so
they can be used with const char* arguments. Otherwise callers have to manually
make string_view objects from the input.
s Please enter the commit message for your changes. Lines starting
2025-07-17 09:08:54 -07:00
Benjamin Bannier
16c40f4f3a
Make PUT on SQLite backend implicitly overwrite expired entries
...
The backend does not serve expired but still present entries so to a
user they do not exist. When they put new data over such an entry their
expecation is that the value is overwritten, even if not explicitly
requested.
2025-07-15 15:50:43 -07:00
Benjamin Bannier
2f67539c0f
Prevent SQLite storage backend from serving expired entries
...
The SQLite storage backend implements expiration by hand and garbage
collection is done in `DoExpire`. This previously relied exclusively on
gets not running within `Storage::expire_interval` of the put, otherwise
we would potentially serve expired entries.
With this patch we explictly check that entries are not expired before
serving them so that the SQLite backend should never serve expired
entries.
2025-07-15 15:47:31 -07:00
Tim Wojtulewicz
1f87382302
Fix some missing #includes resulting from removal of ghc::filesystem
2025-07-14 11:23:54 -07:00
Tim Wojtulewicz
770bc0491e
Remove ghc::filesystem submodule, switch to std::filesystem
2025-07-14 11:23:54 -07:00
Tim Wojtulewicz
cd356ce45d
Add zeek-namespaced alias for jthread
...
Apple Clang (as of version 17.0.0) does not support jthread unless
you enable experimental libcpp features by defining
_LIBCPP_ENABLE_EXPERIMENTAL or passing -fexperimental_library to
the compiler. Even if you don't do those, our 3rdparty jthread
library fails to build because of a bunch of namespace/naming
collisions. I moved our 3rdparty one to the `nonstd` namespace, and
am using an alias under __APPLE__ to avoid the collisions.
2025-07-10 14:07:07 -07:00
Tim Wojtulewicz
fd7259f436
SQLite: Reset expiration time on overwrite
2025-06-30 14:41:10 -07:00
Tim Wojtulewicz
d12d0d7a29
Don't pass local source/binary directory to zeek_add_plugin for Redis backend
2025-06-23 08:35:24 -07:00
Tim Wojtulewicz
ee319fc1c5
Fix clang-tidy modernize-use-nullptr findings
2025-06-06 11:43:06 -07:00
Tim Wojtulewicz
5930d2f944
Fix clang-tidy modernize-pass-by-value findings
2025-06-06 11:43:06 -07:00
Tim Wojtulewicz
be71196fa7
Add busy_timeout script-level option, override any busy_timeout pragma
2025-06-05 10:21:50 -07:00
Tim Wojtulewicz
0e5b3def84
Handle potential contention when running sqlite expiration
2025-06-05 10:21:50 -07:00
Tim Wojtulewicz
97a2ec379e
Use unique_ptr to avoid needing to call sqlite3_reset manually
2025-06-05 10:21:50 -07:00
Tim Wojtulewicz
f2aca331ec
Redis: Add support for sending AUTH commands during connection
2025-06-03 11:38:38 -07:00
Tim Wojtulewicz
9f12208f57
Redis: disconnect cleanly if INFO request fails
2025-06-03 11:38:38 -07:00
Tim Wojtulewicz
0d18ce4e13
Fix segfault if storage sync open_backend returns bad code
2025-06-03 11:38:38 -07:00
Tim Wojtulewicz
975f24bde6
Fix clang-tidy bugprone-suspicious-stringview-data-usage warnings
2025-05-27 11:58:27 -07:00
Tim Wojtulewicz
85e27afaa8
Redis: bump version of hiredis required
2025-05-23 12:13:13 -07:00
Tim Wojtulewicz
0fb4548ff0
Redis: return proper error if connection fails
2025-05-23 12:13:13 -07:00
Tim Wojtulewicz
8a27b894b6
SQLite: Add TODO note about possibly using sqlite3_busy_timeout
2025-05-22 10:23:17 -07:00
Tim Wojtulewicz
25f144381c
SQLite: Fix typo in variable name causing pragmas not to retry on busy
2025-05-22 10:23:17 -07:00
Tim Wojtulewicz
57d797e087
SQLite: Use tableval iteration instead of ToMap for pragmas
2025-05-22 10:22:40 -07:00
Tim Wojtulewicz
384ea046b3
SQLite: Fix logging/error messages around executing pragmas
2025-05-22 10:22:40 -07:00
Tim Wojtulewicz
a58128a45c
SQLite: Move integrity_check to pragma table
2025-05-21 09:38:27 -07:00
Tim Wojtulewicz
78dffb1d6f
SQLite: Add backend option for pragma timeout
2025-05-21 09:38:27 -07:00
Tim Wojtulewicz
f0e7b78554
SQLite: Rename tuning_params to pragma_commands, move running pragmas to utility method
2025-05-21 09:38:27 -07:00
Tim Wojtulewicz
53cb3c3681
SQLite: Retry pragma statements at startup to avoid contention
2025-05-21 09:38:27 -07:00
Tim Wojtulewicz
227d24b64d
SQLite: Check for locked database as well as busy databases
2025-05-21 09:38:27 -07:00
Tim Wojtulewicz
1bff4c9537
SQLite: Fix some string-sizing issues
2025-05-21 09:38:27 -07:00
Tim Wojtulewicz
ac064bd6af
SQLite: Run pragmas on connection before creating table
2025-05-21 09:38:27 -07:00
Tim Wojtulewicz
26cc6d4e7b
SQLite: Add busy_timeout pragma to default options
2025-05-21 09:38:27 -07:00
Tim Wojtulewicz
e618d00326
Remove including <cinttypes> from util.h
2025-05-16 10:14:37 -07:00
Tim Wojtulewicz
174bf09d77
Remove telemetry #includes from OpaqueVal.h
2025-05-16 10:14:37 -07:00
Tim Wojtulewicz
c659592773
Reduce includes in plugin/Component.h
2025-05-16 10:14:37 -07:00
