Tim Wojtulewicz
0b3b5e2ef2
Fix a potential memory leak reported by Coverity
2025-07-23 15:17:53 -07:00
Tim Wojtulewicz
205c72d26f
Add some additional std::moves reported by Coverity
2025-07-23 15:17:53 -07:00
Tim Wojtulewicz
d0a6d84237
Add SQLite page_count and file_size metrics
2025-07-18 14:28:04 -07:00
Tim Wojtulewicz
a0ffe7f748
Add storage metrics for operations, expirations, data transferred
2025-07-18 14:28:04 -07:00
Tim Wojtulewicz
365e6cbc9e
Make RunPragma take an optional value parser to return data
2025-07-18 14:28:04 -07:00
Tim Wojtulewicz
b44f7ca9ad
Make SQLite::Step take a callback function for parsing result data
2025-07-18 14:28:04 -07:00
Benjamin Bannier
16c40f4f3a
Make PUT on SQLite backend implicitly overwrite expired entries
...
The backend does not serve expired but still present entries so to a
user they do not exist. When they put new data over such an entry their
expecation is that the value is overwritten, even if not explicitly
requested.
2025-07-15 15:50:43 -07:00
Benjamin Bannier
2f67539c0f
Prevent SQLite storage backend from serving expired entries
...
The SQLite storage backend implements expiration by hand and garbage
collection is done in `DoExpire`. This previously relied exclusively on
gets not running within `Storage::expire_interval` of the put, otherwise
we would potentially serve expired entries.
With this patch we explictly check that entries are not expired before
serving them so that the SQLite backend should never serve expired
entries.
2025-07-15 15:47:31 -07:00
Tim Wojtulewicz
770bc0491e
Remove ghc::filesystem submodule, switch to std::filesystem
2025-07-14 11:23:54 -07:00
Tim Wojtulewicz
fd7259f436
SQLite: Reset expiration time on overwrite
2025-06-30 14:41:10 -07:00
Tim Wojtulewicz
ee319fc1c5
Fix clang-tidy modernize-use-nullptr findings
2025-06-06 11:43:06 -07:00
Tim Wojtulewicz
be71196fa7
Add busy_timeout script-level option, override any busy_timeout pragma
2025-06-05 10:21:50 -07:00
Tim Wojtulewicz
0e5b3def84
Handle potential contention when running sqlite expiration
2025-06-05 10:21:50 -07:00
Tim Wojtulewicz
97a2ec379e
Use unique_ptr to avoid needing to call sqlite3_reset manually
2025-06-05 10:21:50 -07:00
Tim Wojtulewicz
8a27b894b6
SQLite: Add TODO note about possibly using sqlite3_busy_timeout
2025-05-22 10:23:17 -07:00
Tim Wojtulewicz
25f144381c
SQLite: Fix typo in variable name causing pragmas not to retry on busy
2025-05-22 10:23:17 -07:00
Tim Wojtulewicz
57d797e087
SQLite: Use tableval iteration instead of ToMap for pragmas
2025-05-22 10:22:40 -07:00
Tim Wojtulewicz
384ea046b3
SQLite: Fix logging/error messages around executing pragmas
2025-05-22 10:22:40 -07:00
Tim Wojtulewicz
a58128a45c
SQLite: Move integrity_check to pragma table
2025-05-21 09:38:27 -07:00
Tim Wojtulewicz
78dffb1d6f
SQLite: Add backend option for pragma timeout
2025-05-21 09:38:27 -07:00
Tim Wojtulewicz
f0e7b78554
SQLite: Rename tuning_params to pragma_commands, move running pragmas to utility method
2025-05-21 09:38:27 -07:00
Tim Wojtulewicz
53cb3c3681
SQLite: Retry pragma statements at startup to avoid contention
2025-05-21 09:38:27 -07:00
Tim Wojtulewicz
227d24b64d
SQLite: Check for locked database as well as busy databases
2025-05-21 09:38:27 -07:00
Tim Wojtulewicz
1bff4c9537
SQLite: Fix some string-sizing issues
2025-05-21 09:38:27 -07:00
Tim Wojtulewicz
ac064bd6af
SQLite: Run pragmas on connection before creating table
2025-05-21 09:38:27 -07:00
Tim Wojtulewicz
26cc6d4e7b
SQLite: Add busy_timeout pragma to default options
2025-05-21 09:38:27 -07:00
Tim Wojtulewicz
88786a28a2
Add JSON storage serializer, use with existing backends/tests
2025-04-14 10:11:13 -07:00
Tim Wojtulewicz
201d4508e6
Make ValFromJSON return zeek::expected instead of a variant
2025-04-14 10:02:35 -07:00
Tim Wojtulewicz
32ae8f4eaa
Make storage events take a tag for the backend instead of a string
2025-03-27 16:12:24 -07:00
Tim Wojtulewicz
3d7fcfb428
SQLite: handle existing keys when overwrite=F correctly
2025-03-21 11:56:27 -07:00
Tim Wojtulewicz
a67e138d4e
Fix data-race with ReturnCode objects in Sqlite::DoExpire
2025-03-21 11:56:27 -07:00
Tim Wojtulewicz
0929cf507f
Fix Coverity findings in the SQLite backend
2025-03-21 11:56:27 -07:00
Tim Wojtulewicz
e5b06367f7
Squash code from OperationResultCallback into ResultCallback
2025-03-21 11:44:39 -07:00
Tim Wojtulewicz
e6f1eea1b7
Remove default argument for callbacks, reorder function arguments
2025-03-18 10:20:34 -07:00
Tim Wojtulewicz
ebefb21c53
Pass network time down to Expire()
2025-03-18 10:20:34 -07:00
Tim Wojtulewicz
60aa987e06
Store sqlite3_stmts directly instead of looking up from a map
2025-03-18 10:20:34 -07:00
Tim Wojtulewicz
99160f8fcd
Rearrange visibility of Backend methods, add DoPoll/DoExpire, add return comments
2025-03-18 10:20:34 -07:00
Tim Wojtulewicz
ac4aef2d94
SQLite: Handle other return values from sqlite3_step
2025-03-18 10:20:33 -07:00
Tim Wojtulewicz
9ed3e33f97
Completely rework return values from storage operations
2025-03-18 10:20:33 -07:00
Tim Wojtulewicz
cad48cebd4
Pass network-time-based expiration time to backends instead of an interval
2025-03-18 10:20:33 -07:00
Tim Wojtulewicz
a485b1d237
Make backend options a record, move actual options to be sub-records
2025-03-18 10:20:33 -07:00
Tim Wojtulewicz
4695060d75
Allow opening and closing backends to be async
2025-03-18 09:32:34 -07:00
Tim Wojtulewicz
6289eb8e15
SQLite: Fix some issues with expiration, including in the btest
2025-03-18 09:32:34 -07:00
Tim Wojtulewicz
b2bcb19b22
SQLite: Add pragma integrity_check
2025-03-18 09:32:34 -07:00
Tim Wojtulewicz
3e8ff836aa
SQLite: Add tuning options to configuration
2025-03-18 09:32:34 -07:00
Tim Wojtulewicz
ec49f5d550
SQLite: Handle automated expiration
2025-03-18 09:32:34 -07:00
Tim Wojtulewicz
e95784db16
SQLite: Store/lookup prepared statements instead of recreating
2025-03-18 09:32:34 -07:00
Tim Wojtulewicz
9d1eef3fbc
Add basic SQLite storage backend
2025-03-18 09:32:34 -07:00
