Mirror/zeek
1
0
Fork 0
mirror of https://github.com/zeek/zeek.git synced 2025-10-02 06:38:20 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 1

Add extract_limit_includes_missing option for file extraction

Browse source 
Setting this option to false does not count missing bytes in files towards the
extraction limits, and allows to extract data up to the desired limit,
even when partial files are written.

When missing bytes are encountered, files are now written as sparse
files.

Using this option requires the underlying storage and utilities to support
sparse files.
This commit is contained in:
Johanna Amann 2023-09-06 11:00:05 +01:00 • committed by Tim Wojtulewicz
parent 9928f7efb7
commit e18edfa452
14 changed files with 132 additions and 11 deletions
Download patch file Download diff file Expand all files Collapse all files

21
scripts/base/files/extract/main.zeek
View file

@ -9,12 +9,17 @@ export {
## The default max size for extracted files (they won't exceed this
## number of bytes). A value of zero means unlimited.
##
## Note: Holes in files do not count towards these limits. Files with
## holes are created as sparse files on disk. This means that their
## apparent size can exceed this limit.
option default_limit = 0;
## This setting configures if the file extract limit is inclusive
## of missing bytes. By default, missing bytes do count towards the
## limit.
## Setting this option to false changes this behavior so that missing
## bytes no longer count towards these limits. Files with
## missing bytes are created as sparse files on disk. Their apparent size
## can exceed this file size limit.
option default_limit_includes_missing = T;
redef record Files::Info += {
## Local filename of extracted file.
extracted: string &optional &log;
@ -41,6 +46,14 @@ export {
## :zeek:see:`FileExtract::set_limit` is called to increase the
## limit. A value of zero means "no limit".
extract_limit: count &default=default_limit;
## By default, missing bytes in files count towards the extract file size.
## Missing bytes can, e.g., occur due to missed traffic, or offsets
## used when downloading files.
## Setting this option to false changes this behavior so that holes
## in files do no longer count towards these limits. Files with
## holes are created as sparse files on disk. Their apparent size
## can exceed this file size limit.
extract_limit_includes_missing: bool &default=default_limit_includes_missing;
};
## Sets the maximum allowed extracted file size.