Tim Wojtulewicz
50581bfd3c
Avoid starting up storage expiration thread with zero backends
2025-09-15 14:18:03 -07:00
Tim Wojtulewicz
7e3ed2010d
Add flag to force synchronous mode when calling storage script-land functions
2025-07-23 13:14:34 -07:00
Tim Wojtulewicz
f73ac7089f
Add btests to cover storage metrics
2025-07-18 14:28:04 -07:00
Tim Wojtulewicz
a0ffe7f748
Add storage metrics for operations, expirations, data transferred
2025-07-18 14:28:04 -07:00
Benjamin Bannier
16c40f4f3a
Make PUT on SQLite backend implicitly overwrite expired entries
...
The backend does not serve expired but still present entries so to a
user they do not exist. When they put new data over such an entry their
expecation is that the value is overwritten, even if not explicitly
requested.
2025-07-15 15:50:43 -07:00
Benjamin Bannier
2f67539c0f
Prevent SQLite storage backend from serving expired entries
...
The SQLite storage backend implements expiration by hand and garbage
collection is done in `DoExpire`. This previously relied exclusively on
gets not running within `Storage::expire_interval` of the put, otherwise
we would potentially serve expired entries.
With this patch we explictly check that entries are not expired before
serving them so that the SQLite backend should never serve expired
entries.
2025-07-15 15:47:31 -07:00
Tim Wojtulewicz
fd7259f436
SQLite: Reset expiration time on overwrite
2025-06-30 14:41:10 -07:00
Christian Kreibich
8b39e59572
Btests: don't use -C in Zeek invocations that don't actually need it
2025-06-06 13:46:45 -07:00
Tim Wojtulewicz
fb165c5ef1
Add expiration to sqlite-cluster.btest
2025-06-05 10:21:50 -07:00
Tim Wojtulewicz
f2aca331ec
Redis: Add support for sending AUTH commands during connection
2025-06-03 11:38:38 -07:00
Vern Paxson
614eb8d343
minor BTest maintenance updates for -O gen-C++
2025-05-31 12:52:44 -07:00
Tim Wojtulewicz
0fb4548ff0
Redis: return proper error if connection fails
2025-05-23 12:13:13 -07:00
Tim Wojtulewicz
25f144381c
SQLite: Fix typo in variable name causing pragmas not to retry on busy
2025-05-22 10:23:17 -07:00
Tim Wojtulewicz
25dd1a2702
Disable sqlite-cluster btest
...
This test is being flaky on some platforms and still having problems
with executing pragmas at startup. Disable it for now until it can be
fixed.
2025-05-21 15:42:29 -07:00
Tim Wojtulewicz
e91421a8de
Prefix sqlite-based btests with sqlite- to match redis tests
2025-05-21 09:38:27 -07:00
Tim Wojtulewicz
41bddae59f
Add sqlite cluster storage btest
2025-05-21 09:38:27 -07:00
Arne Welzel
00eabb6cbb
btest remaining: Use generic cluster-layout.zeek
2025-05-20 20:30:01 +02:00
Tim Wojtulewicz
58ee8d3c5c
Add Storage::is_connected BIF
2025-05-07 08:13:16 -07:00
Arne Welzel
85b8c8866b
testing/btest/*zeek: Comment all @TEST lines
2025-04-17 16:30:23 +02:00
Tim Wojtulewicz
cb1ef47a31
Add STORAGE_ prefixes for backends and serializers
2025-04-14 10:11:13 -07:00
Tim Wojtulewicz
88786a28a2
Add JSON storage serializer, use with existing backends/tests
2025-04-14 10:11:13 -07:00
Tim Wojtulewicz
32ae8f4eaa
Make storage events take a tag for the backend instead of a string
2025-03-27 16:12:24 -07:00
Tim Wojtulewicz
855c530b64
Redis: Handle other errors from requests, fix KEY_EXISTS for put operations
2025-03-21 11:56:27 -07:00
Tim Wojtulewicz
3d7fcfb428
SQLite: handle existing keys when overwrite=F correctly
2025-03-21 11:56:27 -07:00
Tim Wojtulewicz
ba9cf1e4db
Remove unnecessary type aliases from storage btests
2025-03-21 11:56:27 -07:00
Tim Wojtulewicz
d0741c8001
Allow sync methods to be called from when conditions, add related btest
2025-03-18 10:20:34 -07:00
Tim Wojtulewicz
b067a6e588
Redis: Fix sync erase, add btest for it
2025-03-18 10:20:34 -07:00
Tim Wojtulewicz
cc7b2dc890
Implement Storage::backend_opened and Storage::backend_lost events
2025-03-18 10:20:34 -07:00
Tim Wojtulewicz
a99a13dc4c
SQLite: expand expiration test
2025-03-18 10:20:33 -07:00
Tim Wojtulewicz
cca1d4f988
Redis: Fix thread-contention issues with Expire(), add more tests
2025-03-18 10:20:33 -07:00
Tim Wojtulewicz
b81e876ec8
Change how redis-server is run during btests, removing redis.conf
2025-03-18 10:20:33 -07:00
Tim Wojtulewicz
9ed3e33f97
Completely rework return values from storage operations
2025-03-18 10:20:33 -07:00
Tim Wojtulewicz
e766af7322
Split sync/async handling into the BIF methods
2025-03-18 10:20:33 -07:00
Tim Wojtulewicz
c247de8ec3
Redis: Rework everything to only use async mode
2025-03-18 10:20:33 -07:00
Tim Wojtulewicz
a485b1d237
Make backend options a record, move actual options to be sub-records
2025-03-18 10:20:33 -07:00
Tim Wojtulewicz
28951dccf1
Split sync and async into separate script-land namespaces
2025-03-18 10:20:33 -07:00
Tim Wojtulewicz
42ad5bbf7d
Add btest that uses a Redis backend in a cluster
2025-03-18 09:32:34 -07:00
Tim Wojtulewicz
f1a7376e0a
Return generic result for get operations that includes error messages
2025-03-18 09:32:34 -07:00
Tim Wojtulewicz
4695060d75
Allow opening and closing backends to be async
2025-03-18 09:32:34 -07:00
Tim Wojtulewicz
ea87c773cd
Redis: Support non-native expiration when reading traces
2025-03-18 09:32:34 -07:00
Tim Wojtulewicz
08bebaa426
Redis: Add btests for the redis backend
2025-03-18 09:32:34 -07:00
Tim Wojtulewicz
6289eb8e15
SQLite: Fix some issues with expiration, including in the btest
2025-03-18 09:32:34 -07:00
Tim Wojtulewicz
6bc5f70236
SQLite: Add additional btests, which also cover general storage functionality
...
- New erase/overwrite tests
- Change existing sqlite-basic test to use async
- Test passing bad keys to validate backend type checking
- New test for compound keys and values
2025-03-18 09:32:34 -07:00
Tim Wojtulewicz
ec49f5d550
SQLite: Handle automated expiration
2025-03-18 09:32:34 -07:00
Tim Wojtulewicz
9d1eef3fbc
Add basic SQLite storage backend
2025-03-18 09:32:34 -07:00
