Mirror/zeek
1
0
Fork 0
mirror of https://github.com/zeek/zeek.git synced 2025-10-17 14:08:20 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 3

Add extract_limit_includes_missing option for file extraction

Browse source 
Setting this option to false does not count missing bytes in files towards the
extraction limits, and allows to extract data up to the desired limit,
even when partial files are written.

When missing bytes are encountered, files are now written as sparse
files.

Using this option requires the underlying storage and utilities to support
sparse files.

(cherry picked from commit afa6f3a0d3b8db1ec5b5e82d26225504c2891089)
This commit is contained in:
Johanna Amann 2023-09-06 11:00:05 +01:00 • committed by Tim Wojtulewicz
parent b2c40a22cb
commit f4d0fdcd5c
14 changed files with 151 additions and 18 deletions
Download patch file Download diff file Expand all files Collapse all files

17
scripts/base/files/extract/main.zeek
View file

@ -11,6 +11,15 @@ export {
## number of bytes). A value of zero means unlimited.
option default_limit = 0;
## This setting configures if the file extract limit is inclusive
## of missing bytes. By default, missing bytes do count towards the
## limit.
## Setting this option to false changes this behavior so that missing
## bytes no longer count towards these limits. Files with
## missing bytes are created as sparse files on disk. Their apparent size
## can exceed this file size limit.
option default_limit_includes_missing = T;
redef record Files::Info += {
## Local filename of extracted file.
extracted: string &optional &log;
@ -37,6 +46,14 @@ export {
## :zeek:see:`FileExtract::set_limit` is called to increase the
## limit. A value of zero means "no limit".
extract_limit: count &default=default_limit;
## By default, missing bytes in files count towards the extract file size.
## Missing bytes can, e.g., occur due to missed traffic, or offsets
## used when downloading files.
## Setting this option to false changes this behavior so that holes
## in files do no longer count towards these limits. Files with
## holes are created as sparse files on disk. Their apparent size
## can exceed this file size limit.
extract_limit_includes_missing: bool &default=default_limit_includes_missing;
};
## Sets the maximum allowed extracted file size.