Mirror/zeek
1
0
Fork 0
mirror of https://github.com/zeek/zeek.git synced 2025-10-03 15:18:20 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 2
8450 commits 386 branches 195 tags 257 MiB
Commit graph

27 commits

Author SHA1 Message Date
Jon Siwek
a994be9eeb Merge remote-tracking branch 'origin/topic/seth/zeek_init' 
* origin/topic/seth/zeek_init:
  Some more testing fixes.
  Update docs and tests for bro_(init|done) -> zeek_(init|done)
  Implement the zeek_init handler.
 2019-04-19 11:24:29 -07:00
Jon Siwek
1e57e3f026 Use .zeek file suffix in unit tests 2019-04-16 16:08:57 -07:00
Seth Hall
5db766bd88 Update docs and tests for bro_(init|done) -> zeek_(init|done) 2019-04-14 08:49:12 -04:00
Seth Hall
7cb6cf24a6 Functions for retrieving files by their id. 
There are two new script level functions to query and lookup files
from the core by their IDs.  These are adding feature parity for
similarly named functions for files.  The function prototypes are
as follows:

  Files::file_exists(fuid: string): bool
  Files::lookup_File(fuid: string): fa_file
 2018-01-09 12:16:17 -05:00
Robin Sommer
e21cca4ba8 Adding test with command line that used to trigger a crash. 2017-01-31 14:52:37 -08:00
Seth Hall
a97cd1f3a2 Fix a bug in the core files framework with handling the BOF buffer. 
- Any files where the total size was below the size of the
   default bof_buffer size couldn't have stream analyzers successfully
   attached because the bof_buffer never reached the full size
   and was never flushed.  This branch explicitly marks the buf_buffer
   as full and flushes it when the file is being removed.
 2015-02-05 09:09:08 -05:00
Robin Sommer
c9524757d2 Adding Files::register_for_mime_type() to associate a file analyzer 
with a MIME type.

Whenever that MIME is detected, Bro will now automatically activate
the analyzer. The interface mimics how well-known ports are defined
for protocol analyzers.

This isn't actually used by any existing file analyzer (because we
don't have any yet that target a specific file format), but there's a
test making sure it works.
 2014-07-21 16:31:22 +02:00
Jon Siwek
22bf3e1196 Increase UIDs to 96 bits w/ C/F prefix - BIT-1016 
- The bit-length is adjustable via redef'ing bits_per_uid.

- Prefix 'C' is used for connection UIDS (including IP tunnels) and
  'F' for files.
 2013-08-26 15:36:31 -05:00
Jon Siwek
939619889d File analysis fixes and test updates. 
- Several places were just using old variable names or not loading
  scripts correctly after they'd been renamed/moved.

- Revert/adjust a change in how HTTP file handles are generated that
  broke partial content responses.

- Turn some libmagic builtin checks back on; seems some are actually
  useful (e.g. text detection seems to be a builtin).  The rule going
  forward probably will be only to turn off a builtin if we confirm it
  causes issues.

- Removed some tests that are redundant or not necessary anymore because
  the generic file analysis tests cover them.

- A couple FTP tests still fail that I think need an actual solution via
  script changes.
 2013-07-25 16:51:16 -05:00
Seth Hall
58d133e764 Merge remote-tracking branch 'origin/master' into topic/seth/faf-updates 
Conflicts:
	scripts/base/frameworks/files/main.bro
	scripts/base/init-bare.bro
	scripts/base/protocols/ftp/file-analysis.bro
	scripts/base/protocols/http/file-analysis.bro
	scripts/base/protocols/irc/file-analysis.bro
	scripts/base/protocols/smtp/file-analysis.bro
	src/const.bif
	src/event.bif
	src/file_analysis/Analyzer.h
	src/file_analysis/file_analysis.bif
 2013-07-05 02:13:27 -04:00
Seth Hall
df2841458d Large overhaul in name and appearance for file analysis. 2013-07-05 02:00:14 -04:00
Jon Siwek
f2574636b6 Merge branch 'master' into topic/jsiwek/faf-cleanup 
Conflicts:
	scripts/base/protocols/ftp/file-analysis.bro
	scripts/base/protocols/http/file-analysis.bro
	scripts/base/protocols/irc/file-analysis.bro
	scripts/base/protocols/smtp/file-analysis.bro
	src/file_analysis/File.cc
	src/file_analysis/File.h
	src/file_analysis/Manager.cc
	src/file_analysis/Manager.h
	testing/btest/Baseline/scripts.base.frameworks.file-analysis.logging/file_analysis.log
	testing/btest/Baseline/scripts.base.protocols.ftp.ftp-extract/ftp-item-0.dat
	testing/btest/Baseline/scripts.base.protocols.ftp.ftp-extract/ftp-item-1.dat
	testing/btest/Baseline/scripts.base.protocols.ftp.ftp-extract/ftp-item-2.dat
	testing/btest/Baseline/scripts.base.protocols.ftp.ftp-extract/ftp-item-3.dat
	testing/btest/Baseline/scripts.base.protocols.ftp.ftp-extract/ftp-item-BTsa70Ua9x7-1.dat
	testing/btest/Baseline/scripts.base.protocols.ftp.ftp-extract/ftp-item-BTsa70Ua9x7.dat
	testing/btest/Baseline/scripts.base.protocols.ftp.ftp-extract/ftp-item-Rqjkzoroau4-0.dat
	testing/btest/Baseline/scripts.base.protocols.ftp.ftp-extract/ftp-item-Rqjkzoroau4.dat
	testing/btest/Baseline/scripts.base.protocols.ftp.ftp-extract/ftp-item-VLQvJybrm38-2.dat
	testing/btest/Baseline/scripts.base.protocols.ftp.ftp-extract/ftp-item-VLQvJybrm38.dat
	testing/btest/Baseline/scripts.base.protocols.ftp.ftp-extract/ftp-item-zrfwSs9K1yk-3.dat
	testing/btest/Baseline/scripts.base.protocols.ftp.ftp-extract/ftp-item-zrfwSs9K1yk.dat
	testing/btest/Baseline/scripts.base.protocols.ftp.ftp-extract/ftp.log
	testing/btest/Baseline/scripts.base.protocols.http.http-extract-files/http-item-BFymS6bFgT3-0.dat
	testing/btest/Baseline/scripts.base.protocols.http.http-extract-files/http-item-BFymS6bFgT3.dat
	testing/btest/Baseline/scripts.base.protocols.http.http-extract-files/http-item.dat
	testing/btest/Baseline/scripts.base.protocols.http.http-extract-files/http.log
	testing/btest/Baseline/scripts.base.protocols.irc.dcc-extract/irc-dcc-item-wqKMAamJVSb-0.dat
	testing/btest/Baseline/scripts.base.protocols.irc.dcc-extract/irc-dcc-item-wqKMAamJVSb.dat
	testing/btest/Baseline/scripts.base.protocols.irc.dcc-extract/irc-dcc-item.dat
	testing/btest/Baseline/scripts.base.protocols.irc.dcc-extract/irc.log
	testing/btest/Baseline/scripts.base.protocols.smtp.mime-extract/smtp-entity-0.dat
	testing/btest/Baseline/scripts.base.protocols.smtp.mime-extract/smtp-entity-1.dat
	testing/btest/Baseline/scripts.base.protocols.smtp.mime-extract/smtp-entity-Ltd7QO7jEv3-1.dat
	testing/btest/Baseline/scripts.base.protocols.smtp.mime-extract/smtp-entity-Ltd7QO7jEv3.dat
	testing/btest/Baseline/scripts.base.protocols.smtp.mime-extract/smtp-entity-cwR7l6Zctxb-0.dat
	testing/btest/Baseline/scripts.base.protocols.smtp.mime-extract/smtp-entity-cwR7l6Zctxb.dat
	testing/btest/Baseline/scripts.base.protocols.smtp.mime-extract/smtp_entities.log
	testing/btest/scripts/base/protocols/ftp/ftp-extract.bro
	testing/btest/scripts/base/protocols/http/http-extract-files.bro
	testing/btest/scripts/base/protocols/irc/dcc-extract.test
	testing/btest/scripts/base/protocols/smtp/mime-extract.test
 2013-06-07 15:44:36 -05:00
Jon Siwek
3cbef60f57 Fix HTTP multipart body file analysis. 
Each part now gets assigned a different file handle/id.
 2013-05-21 15:35:22 -05:00
Jon Siwek
16f924c2c0 Remove FileAnalysis::postpone_timeout. 
FileAnalysis::set_timeout_interval can now perform same function.
 2013-05-21 10:50:07 -05:00
Jon Siwek
0ef074594d Add input interface to forward data for file analysis. 
The new Input::add_analysis function is used to automatically forward
input data on to the file analysis framework.
 2013-05-21 10:29:22 -05:00
Robin Sommer
4ccd6d76fd Fixing tests. 
Part of this involves making the file-analysis tests independent of
specific hash values. I've done that only partially though.
 2013-05-17 18:14:35 -07:00
Jon Siwek
b8c98b8bf7 FileAnalysis: change terminology s/action/analyzer 2013-04-11 14:53:54 -05:00
Jon Siwek
d9321e2203 FileAnalysis: remove some file events. 
The file_new event now takes over the function of file_type, file_bof,
and file_bof_buffer.
 2013-04-10 14:34:23 -05:00
Jon Siwek
a2d9b47bcd FileAnalysis: finish switching hooks to events. 2013-04-10 11:13:43 -05:00
Jon Siwek
641154f8e8 FileAnalysis: checkpoint in middle of big reorganization. 
- FileAnalysis::Info is now just a record used for logging, the fa_file
  record type is defined in init-bare.bro as the analogue to a
  connection record.

- Starting to transfer policy hook triggers and analyzer results to
  events.
 2013-04-09 15:49:58 -05:00
Jon Siwek
7caa4aa45c FileAnalysis: unit test tweaks (portability, etc.) 2013-03-27 14:51:23 -05:00
Jon Siwek
00a1de3593 FileAnalysis: refactor unit tests to use a common script. 2013-03-22 17:27:16 -05:00
Jon Siwek
71f0e2d276 FileAnalysis: replace script-layer http file analysis. 
Other misc:

- Remove HTTP::MD5 notice.

- Add "last_active" field to FileAnalysis::Info record.

- Replace "conn_uids", "conn_ids" fields in FileAnalysis::Info record
  with just a "conns" fields containing full connection records.

- The http-methods unit test is failing now, but I think it will be
  fixed once I change the file handle callback mechanism to use events
  instead.
 2013-03-22 16:14:06 -05:00
Jon Siwek
7034785810 FileAnalysis: add logging, file_analysis.log. 2013-03-20 13:31:11 -05:00
Jon Siwek
1ef7465e30 FileAnalysis: add more unit tests. 2013-03-20 12:47:45 -05:00
Jon Siwek
661677d452 FileAnalysis: separating IRC/FTP data analyzers. 
It simplifies the file handle string callbacks.
 2013-03-20 11:12:06 -05:00
Jon Siwek
59ed5c75f1 FileAnalysis: add unit tests covering current protocol integration. 
And had to make various fixes/refinements after scrutinizing results.
 2013-03-19 15:50:05 -05:00