Tim Wojtulewicz
605973497f
Remove file-local expire_running variable
2025-03-18 10:20:34 -07:00
Tim Wojtulewicz
ebefb21c53
Pass network time down to Expire()
2025-03-18 10:20:34 -07:00
Tim Wojtulewicz
c7503654e8
Add IN_PROGRESS return code, handle for async backends
2025-03-18 10:20:34 -07:00
Tim Wojtulewicz
60aa987e06
Store sqlite3_stmts directly instead of looking up from a map
2025-03-18 10:20:34 -07:00
Tim Wojtulewicz
44c6e32ae7
Reduce code duplication in storage.bif
2025-03-18 10:20:34 -07:00
Tim Wojtulewicz
ad224d9a3b
Add OperationResult::MakeVal, use it to reduce some code duplication
2025-03-18 10:20:34 -07:00
Tim Wojtulewicz
99160f8fcd
Rearrange visibility of Backend methods, add DoPoll/DoExpire, add return comments
2025-03-18 10:20:34 -07:00
Tim Wojtulewicz
cc7b2dc890
Implement Storage::backend_opened and Storage::backend_lost events
2025-03-18 10:20:34 -07:00
Tim Wojtulewicz
a99a13dc4c
SQLite: expand expiration test
2025-03-18 10:20:33 -07:00
Tim Wojtulewicz
ac4aef2d94
SQLite: Handle other return values from sqlite3_step
2025-03-18 10:20:33 -07:00
Tim Wojtulewicz
cca1d4f988
Redis: Fix thread-contention issues with Expire(), add more tests
2025-03-18 10:20:33 -07:00
Tim Wojtulewicz
b81e876ec8
Change how redis-server is run during btests, removing redis.conf
2025-03-18 10:20:33 -07:00
Tim Wojtulewicz
9ed3e33f97
Completely rework return values from storage operations
2025-03-18 10:20:33 -07:00
Tim Wojtulewicz
8ddda016ff
Update some btests due to timing changes
2025-03-18 10:20:33 -07:00
Tim Wojtulewicz
e766af7322
Split sync/async handling into the BIF methods
2025-03-18 10:20:33 -07:00
Tim Wojtulewicz
c247de8ec3
Redis: Rework everything to only use async mode
2025-03-18 10:20:33 -07:00
Tim Wojtulewicz
40f60f26b3
Run expiration on a separate thread
2025-03-18 10:20:33 -07:00
Tim Wojtulewicz
cad48cebd4
Pass network-time-based expiration time to backends instead of an interval
2025-03-18 10:20:33 -07:00
Tim Wojtulewicz
a485b1d237
Make backend options a record, move actual options to be sub-records
2025-03-18 10:20:33 -07:00
Tim Wojtulewicz
64f3969434
Always register backend for expiration, check for open during loop
2025-03-18 10:20:33 -07:00
Tim Wojtulewicz
28951dccf1
Split sync and async into separate script-land namespaces
2025-03-18 10:20:33 -07:00
Tim Wojtulewicz
e8074c40d4
Remove Backend::SupportsAsync
2025-03-18 09:32:34 -07:00
Tim Wojtulewicz
42ad5bbf7d
Add btest that uses a Redis backend in a cluster
2025-03-18 09:32:34 -07:00
Tim Wojtulewicz
f1a7376e0a
Return generic result for get operations that includes error messages
2025-03-18 09:32:34 -07:00
Tim Wojtulewicz
4695060d75
Allow opening and closing backends to be async
2025-03-18 09:32:34 -07:00
Tim Wojtulewicz
ea87c773cd
Redis: Support non-native expiration when reading traces
2025-03-18 09:32:34 -07:00
Tim Wojtulewicz
08bebaa426
Redis: Add btests for the redis backend
2025-03-18 09:32:34 -07:00
Tim Wojtulewicz
52d94b781a
Redis: Force storage sync mode when reading pcaps, default to async mode
2025-03-18 09:32:34 -07:00
Tim Wojtulewicz
31e146b16d
Redis: Add new backend
2025-03-18 09:32:34 -07:00
Tim Wojtulewicz
6289eb8e15
SQLite: Fix some issues with expiration, including in the btest
2025-03-18 09:32:34 -07:00
Tim Wojtulewicz
6bc5f70236
SQLite: Add additional btests, which also cover general storage functionality
...
- New erase/overwrite tests
- Change existing sqlite-basic test to use async
- Test passing bad keys to validate backend type checking
- New test for compound keys and values
2025-03-18 09:32:34 -07:00
Tim Wojtulewicz
b2bcb19b22
SQLite: Add pragma integrity_check
2025-03-18 09:32:34 -07:00
Tim Wojtulewicz
3e8ff836aa
SQLite: Add tuning options to configuration
2025-03-18 09:32:34 -07:00
Tim Wojtulewicz
ec49f5d550
SQLite: Handle automated expiration
2025-03-18 09:32:34 -07:00
Tim Wojtulewicz
e95784db16
SQLite: Store/lookup prepared statements instead of recreating
2025-03-18 09:32:34 -07:00
Tim Wojtulewicz
9d1eef3fbc
Add basic SQLite storage backend
2025-03-18 09:32:34 -07:00
Tim Wojtulewicz
7ad6a05f5b
Add infrastructure for asynchronous storage operations
2025-03-18 09:32:34 -07:00
Tim Wojtulewicz
d07d27453a
Add infrastructure for automated expiration of storage entries
...
This is used for backends that don't support expiration natively.
2025-03-18 09:32:34 -07:00
Tim Wojtulewicz
8dee733a7d
Change args to Storage::put to be a record
...
The number of args being passed to the put() methods was getting to be
fairly long, with more on the horizon. Changing to a record means simplifying
things a little bit.
2025-03-18 09:32:34 -07:00
Tim Wojtulewicz
69d940533d
Pass key/value types for validation when opening backends
2025-03-18 09:32:34 -07:00
Tim Wojtulewicz
2ea0f3e70a
Lay out initial parts for the Storage framework
...
This includes a manager, component manager, BIF and script code, and
parts to support new storage backend plugins.
2025-03-18 09:32:34 -07:00
Tim Wojtulewicz
3d6e7c85b0
DebugLogger: add stream for storage
2025-03-18 09:32:34 -07:00
Tim Wojtulewicz
e2b9e81c53
plugin: Add component enum for storage backends
2025-03-18 09:32:34 -07:00
Tim Wojtulewicz
aeedd25cff
Add martinmoene/expected-lite as a submodule
2025-03-18 09:32:34 -07:00
Robin Sommer
3d25328204
Merge remote-tracking branch 'origin/topic/robin/gh-4250-vec-allocator'
...
* origin/topic/robin/gh-4250-vec-allocator:
Spicy: Fix passing vectors to Zeek.
2025-03-18 09:08:50 +01:00
Robin Sommer
993e1a6562
Merge remote-tracking branch 'origin/topic/robin/gh-zeek-4238-spicy-bump'
...
* origin/topic/robin/gh-zeek-4238-spicy-bump:
Bump `cmake` submodule.
Bump Spicy.
[Spicy] Add missing `to_string` overload.
2025-03-18 09:07:16 +01:00
zeek-bot
189fd0b071
Update doc submodule [nomail] [skip ci]
2025-03-18 00:14:00 +00:00
Tim Wojtulewicz
43f108bb71
Merge remote-tracking branch 'origin/topic/timw/ci-macos-sequoia'
...
* origin/topic/timw/ci-macos-sequoia:
ci/init-external-repo.sh: Use regex to match macos cirrus task
CI: Change macOS runner to Sequoia
2025-03-17 11:55:27 -07:00
Arne Welzel
919176873e
ci/init-external-repo.sh: Use regex to match macos cirrus task
2025-03-17 10:48:59 +01:00
Tim Wojtulewicz
46edc829cf
CI: Change macOS runner to Sequoia
2025-03-14 18:29:52 -07:00
