Commit graph

18 commits

Author SHA1 Message Date
Johanna Amann
e18edfa452 Add extract_limit_includes_missing option for file extraction
Setting this option to false does not count missing bytes in files towards the
extraction limits, and allows to extract data up to the desired limit,
even when partial files are written.

When missing bytes are encountered, files are now written as sparse
files.

Using this option requires the underlying storage and utilities to support
sparse files.
2023-09-14 12:11:42 -07:00
Johanna Amann
9928f7efb7 File extraction: use fseek
In the past, we allocated a buffer with zeroes and wrote that with
fwrite. Now, instead we just fseek to the correct offset.

This changes the way in which the file extract limit is counted a bit;
skipped bytes do no longer count against the file size limit.

(cherry picked from commit 5071592e9b7105090a1d9de19689c499070749d4)
2023-09-14 12:11:37 -07:00
Tim Wojtulewicz
5934e143aa Revert "Add extract_limit_includes_missing option for file extraction"
This reverts commit f4d0fdcd5c.
2023-09-14 12:10:40 -07:00
Johanna Amann
f4d0fdcd5c Add extract_limit_includes_missing option for file extraction
Setting this option to false does not count missing bytes in files towards the
extraction limits, and allows to extract data up to the desired limit,
even when partial files are written.

When missing bytes are encountered, files are now written as sparse
files.

Using this option requires the underlying storage and utilities to support
sparse files.

(cherry picked from commit afa6f3a0d3b8db1ec5b5e82d26225504c2891089)
2023-09-12 12:00:36 -07:00
Jon Siwek
aebcb1415d GH-234: rename Broxygen to Zeexygen along with roles/directives
* All "Broxygen" usages have been replaced in
  code, documentation, filenames, etc.

* Sphinx roles/directives like ":bro:see" are now ":zeek:see"

* The "--broxygen" command-line option is now "--zeexygen"
2019-04-22 19:45:50 -07:00
Jon Siwek
a994be9eeb Merge remote-tracking branch 'origin/topic/seth/zeek_init'
* origin/topic/seth/zeek_init:
  Some more testing fixes.
  Update docs and tests for bro_(init|done) -> zeek_(init|done)
  Implement the zeek_init handler.
2019-04-19 11:24:29 -07:00
Seth Hall
8cefb9be42 Implement the zeek_init handler.
Implements the change and a test.
2019-04-14 08:37:35 -04:00
Daniel Thayer
18bd74454b Rename all scripts to have ".zeek" file extension 2019-04-11 21:12:40 -05:00
Daniel Thayer
9bfc01b705 Convert more redef-able constants to runtime options 2018-08-27 19:38:47 -05:00
Seth Hall
2d1f007186 Extend file extraction log.
- New fields: extracted_cutoff and extracted_size.
   These fields will be null if the file isn't extracted.

 - Extended the extraction test to test the files log too.
2017-01-25 01:16:46 -05:00
Daniel Thayer
23a5f83c94 Added missing README files for documentation
These are the one-line script package descriptions.
2016-10-10 22:55:50 -05:00
Jon Siwek
0b957cbe75 Include timestamp in default extracted file names.
And add a policy script to extract all files.

BIT-1335 #close
2015-03-13 14:25:30 -05:00
Daniel Thayer
72a4a90416 Add more script package README files
The text from these README files appears on the "Bro Script Packages"
page after building the documentation.
2013-10-23 16:36:14 -05:00
Daniel Thayer
b5af589246 Improvements to file analysis docs
Fixed reference to wrong field name.
Added documentation of a function arg.
Added a couple references to other parts of the documentation.
Explained how not specifying extraction filename results in automatic
filename generation.
Several other minor clarifications.
2013-10-11 16:31:53 -05:00
Daniel Thayer
7ddc670a02 Fix typos and formatting in the file analysis docs 2013-10-10 12:46:11 -05:00
Jon Siwek
17d0ecd388 File extraction tweaks.
- Default extraction limit of 100MB now provided via a tuning script
  loaded in local.bro so that command-line Bro is unlimited by default.

- Extraction directory is now created on request of file extraction
  rather than unconditionally in bro_init().
2013-08-23 11:57:07 -05:00
Jon Siwek
89ae4ffd05 Add options to limit extracted file sizes w/ 100MB default. 2013-08-22 16:37:58 -05:00
Seth Hall
df2841458d Large overhaul in name and appearance for file analysis. 2013-07-05 02:00:14 -04:00