Robin Sommer
c9524757d2
Adding Files::register_for_mime_type() to associate a file analyzer
...
with a MIME type.
Whenever that MIME is detected, Bro will now automatically activate
the analyzer. The interface mimics how well-known ports are defined
for protocol analyzers.
This isn't actually used by any existing file analyzer (because we
don't have any yet that target a specific file format), but there's a
test making sure it works.
2014-07-21 16:31:22 +02:00
Jon Siwek
22bf3e1196
Increase UIDs to 96 bits w/ C/F prefix - BIT-1016
...
- The bit-length is adjustable via redef'ing bits_per_uid.
- Prefix 'C' is used for connection UIDS (including IP tunnels) and
'F' for files.
2013-08-26 15:36:31 -05:00
Jon Siwek
939619889d
File analysis fixes and test updates.
...
- Several places were just using old variable names or not loading
scripts correctly after they'd been renamed/moved.
- Revert/adjust a change in how HTTP file handles are generated that
broke partial content responses.
- Turn some libmagic builtin checks back on; seems some are actually
useful (e.g. text detection seems to be a builtin). The rule going
forward probably will be only to turn off a builtin if we confirm it
causes issues.
- Removed some tests that are redundant or not necessary anymore because
the generic file analysis tests cover them.
- A couple FTP tests still fail that I think need an actual solution via
script changes.
2013-07-25 16:51:16 -05:00
Seth Hall
58d133e764
Merge remote-tracking branch 'origin/master' into topic/seth/faf-updates
...
Conflicts:
scripts/base/frameworks/files/main.bro
scripts/base/init-bare.bro
scripts/base/protocols/ftp/file-analysis.bro
scripts/base/protocols/http/file-analysis.bro
scripts/base/protocols/irc/file-analysis.bro
scripts/base/protocols/smtp/file-analysis.bro
src/const.bif
src/event.bif
src/file_analysis/Analyzer.h
src/file_analysis/file_analysis.bif
2013-07-05 02:13:27 -04:00
Seth Hall
df2841458d
Large overhaul in name and appearance for file analysis.
2013-07-05 02:00:14 -04:00
Jon Siwek
f2574636b6
Merge branch 'master' into topic/jsiwek/faf-cleanup
...
Conflicts:
scripts/base/protocols/ftp/file-analysis.bro
scripts/base/protocols/http/file-analysis.bro
scripts/base/protocols/irc/file-analysis.bro
scripts/base/protocols/smtp/file-analysis.bro
src/file_analysis/File.cc
src/file_analysis/File.h
src/file_analysis/Manager.cc
src/file_analysis/Manager.h
testing/btest/Baseline/scripts.base.frameworks.file-analysis.logging/file_analysis.log
testing/btest/Baseline/scripts.base.protocols.ftp.ftp-extract/ftp-item-0.dat
testing/btest/Baseline/scripts.base.protocols.ftp.ftp-extract/ftp-item-1.dat
testing/btest/Baseline/scripts.base.protocols.ftp.ftp-extract/ftp-item-2.dat
testing/btest/Baseline/scripts.base.protocols.ftp.ftp-extract/ftp-item-3.dat
testing/btest/Baseline/scripts.base.protocols.ftp.ftp-extract/ftp-item-BTsa70Ua9x7-1.dat
testing/btest/Baseline/scripts.base.protocols.ftp.ftp-extract/ftp-item-BTsa70Ua9x7.dat
testing/btest/Baseline/scripts.base.protocols.ftp.ftp-extract/ftp-item-Rqjkzoroau4-0.dat
testing/btest/Baseline/scripts.base.protocols.ftp.ftp-extract/ftp-item-Rqjkzoroau4.dat
testing/btest/Baseline/scripts.base.protocols.ftp.ftp-extract/ftp-item-VLQvJybrm38-2.dat
testing/btest/Baseline/scripts.base.protocols.ftp.ftp-extract/ftp-item-VLQvJybrm38.dat
testing/btest/Baseline/scripts.base.protocols.ftp.ftp-extract/ftp-item-zrfwSs9K1yk-3.dat
testing/btest/Baseline/scripts.base.protocols.ftp.ftp-extract/ftp-item-zrfwSs9K1yk.dat
testing/btest/Baseline/scripts.base.protocols.ftp.ftp-extract/ftp.log
testing/btest/Baseline/scripts.base.protocols.http.http-extract-files/http-item-BFymS6bFgT3-0.dat
testing/btest/Baseline/scripts.base.protocols.http.http-extract-files/http-item-BFymS6bFgT3.dat
testing/btest/Baseline/scripts.base.protocols.http.http-extract-files/http-item.dat
testing/btest/Baseline/scripts.base.protocols.http.http-extract-files/http.log
testing/btest/Baseline/scripts.base.protocols.irc.dcc-extract/irc-dcc-item-wqKMAamJVSb-0.dat
testing/btest/Baseline/scripts.base.protocols.irc.dcc-extract/irc-dcc-item-wqKMAamJVSb.dat
testing/btest/Baseline/scripts.base.protocols.irc.dcc-extract/irc-dcc-item.dat
testing/btest/Baseline/scripts.base.protocols.irc.dcc-extract/irc.log
testing/btest/Baseline/scripts.base.protocols.smtp.mime-extract/smtp-entity-0.dat
testing/btest/Baseline/scripts.base.protocols.smtp.mime-extract/smtp-entity-1.dat
testing/btest/Baseline/scripts.base.protocols.smtp.mime-extract/smtp-entity-Ltd7QO7jEv3-1.dat
testing/btest/Baseline/scripts.base.protocols.smtp.mime-extract/smtp-entity-Ltd7QO7jEv3.dat
testing/btest/Baseline/scripts.base.protocols.smtp.mime-extract/smtp-entity-cwR7l6Zctxb-0.dat
testing/btest/Baseline/scripts.base.protocols.smtp.mime-extract/smtp-entity-cwR7l6Zctxb.dat
testing/btest/Baseline/scripts.base.protocols.smtp.mime-extract/smtp_entities.log
testing/btest/scripts/base/protocols/ftp/ftp-extract.bro
testing/btest/scripts/base/protocols/http/http-extract-files.bro
testing/btest/scripts/base/protocols/irc/dcc-extract.test
testing/btest/scripts/base/protocols/smtp/mime-extract.test
2013-06-07 15:44:36 -05:00
Jon Siwek
3cbef60f57
Fix HTTP multipart body file analysis.
...
Each part now gets assigned a different file handle/id.
2013-05-21 15:35:22 -05:00
Jon Siwek
16f924c2c0
Remove FileAnalysis::postpone_timeout.
...
FileAnalysis::set_timeout_interval can now perform same function.
2013-05-21 10:50:07 -05:00
Jon Siwek
0ef074594d
Add input interface to forward data for file analysis.
...
The new Input::add_analysis function is used to automatically forward
input data on to the file analysis framework.
2013-05-21 10:29:22 -05:00
Robin Sommer
4ccd6d76fd
Fixing tests.
...
Part of this involves making the file-analysis tests independent of
specific hash values. I've done that only partially though.
2013-05-17 18:14:35 -07:00
Jon Siwek
b8c98b8bf7
FileAnalysis: change terminology s/action/analyzer
2013-04-11 14:53:54 -05:00
Jon Siwek
d9321e2203
FileAnalysis: remove some file events.
...
The file_new event now takes over the function of file_type, file_bof,
and file_bof_buffer.
2013-04-10 14:34:23 -05:00
Jon Siwek
a2d9b47bcd
FileAnalysis: finish switching hooks to events.
2013-04-10 11:13:43 -05:00
Jon Siwek
641154f8e8
FileAnalysis: checkpoint in middle of big reorganization.
...
- FileAnalysis::Info is now just a record used for logging, the fa_file
record type is defined in init-bare.bro as the analogue to a
connection record.
- Starting to transfer policy hook triggers and analyzer results to
events.
2013-04-09 15:49:58 -05:00
Jon Siwek
7caa4aa45c
FileAnalysis: unit test tweaks (portability, etc.)
2013-03-27 14:51:23 -05:00
Jon Siwek
00a1de3593
FileAnalysis: refactor unit tests to use a common script.
2013-03-22 17:27:16 -05:00
Jon Siwek
71f0e2d276
FileAnalysis: replace script-layer http file analysis.
...
Other misc:
- Remove HTTP::MD5 notice.
- Add "last_active" field to FileAnalysis::Info record.
- Replace "conn_uids", "conn_ids" fields in FileAnalysis::Info record
with just a "conns" fields containing full connection records.
- The http-methods unit test is failing now, but I think it will be
fixed once I change the file handle callback mechanism to use events
instead.
2013-03-22 16:14:06 -05:00
Jon Siwek
7034785810
FileAnalysis: add logging, file_analysis.log.
2013-03-20 13:31:11 -05:00
Jon Siwek
1ef7465e30
FileAnalysis: add more unit tests.
2013-03-20 12:47:45 -05:00
Jon Siwek
661677d452
FileAnalysis: separating IRC/FTP data analyzers.
...
It simplifies the file handle string callbacks.
2013-03-20 11:12:06 -05:00
Jon Siwek
59ed5c75f1
FileAnalysis: add unit tests covering current protocol integration.
...
And had to make various fixes/refinements after scrutinizing results.
2013-03-19 15:50:05 -05:00
