Jon Siwek
f2574636b6
Merge branch 'master' into topic/jsiwek/faf-cleanup
...
Conflicts:
scripts/base/protocols/ftp/file-analysis.bro
scripts/base/protocols/http/file-analysis.bro
scripts/base/protocols/irc/file-analysis.bro
scripts/base/protocols/smtp/file-analysis.bro
src/file_analysis/File.cc
src/file_analysis/File.h
src/file_analysis/Manager.cc
src/file_analysis/Manager.h
testing/btest/Baseline/scripts.base.frameworks.file-analysis.logging/file_analysis.log
testing/btest/Baseline/scripts.base.protocols.ftp.ftp-extract/ftp-item-0.dat
testing/btest/Baseline/scripts.base.protocols.ftp.ftp-extract/ftp-item-1.dat
testing/btest/Baseline/scripts.base.protocols.ftp.ftp-extract/ftp-item-2.dat
testing/btest/Baseline/scripts.base.protocols.ftp.ftp-extract/ftp-item-3.dat
testing/btest/Baseline/scripts.base.protocols.ftp.ftp-extract/ftp-item-BTsa70Ua9x7-1.dat
testing/btest/Baseline/scripts.base.protocols.ftp.ftp-extract/ftp-item-BTsa70Ua9x7.dat
testing/btest/Baseline/scripts.base.protocols.ftp.ftp-extract/ftp-item-Rqjkzoroau4-0.dat
testing/btest/Baseline/scripts.base.protocols.ftp.ftp-extract/ftp-item-Rqjkzoroau4.dat
testing/btest/Baseline/scripts.base.protocols.ftp.ftp-extract/ftp-item-VLQvJybrm38-2.dat
testing/btest/Baseline/scripts.base.protocols.ftp.ftp-extract/ftp-item-VLQvJybrm38.dat
testing/btest/Baseline/scripts.base.protocols.ftp.ftp-extract/ftp-item-zrfwSs9K1yk-3.dat
testing/btest/Baseline/scripts.base.protocols.ftp.ftp-extract/ftp-item-zrfwSs9K1yk.dat
testing/btest/Baseline/scripts.base.protocols.ftp.ftp-extract/ftp.log
testing/btest/Baseline/scripts.base.protocols.http.http-extract-files/http-item-BFymS6bFgT3-0.dat
testing/btest/Baseline/scripts.base.protocols.http.http-extract-files/http-item-BFymS6bFgT3.dat
testing/btest/Baseline/scripts.base.protocols.http.http-extract-files/http-item.dat
testing/btest/Baseline/scripts.base.protocols.http.http-extract-files/http.log
testing/btest/Baseline/scripts.base.protocols.irc.dcc-extract/irc-dcc-item-wqKMAamJVSb-0.dat
testing/btest/Baseline/scripts.base.protocols.irc.dcc-extract/irc-dcc-item-wqKMAamJVSb.dat
testing/btest/Baseline/scripts.base.protocols.irc.dcc-extract/irc-dcc-item.dat
testing/btest/Baseline/scripts.base.protocols.irc.dcc-extract/irc.log
testing/btest/Baseline/scripts.base.protocols.smtp.mime-extract/smtp-entity-0.dat
testing/btest/Baseline/scripts.base.protocols.smtp.mime-extract/smtp-entity-1.dat
testing/btest/Baseline/scripts.base.protocols.smtp.mime-extract/smtp-entity-Ltd7QO7jEv3-1.dat
testing/btest/Baseline/scripts.base.protocols.smtp.mime-extract/smtp-entity-Ltd7QO7jEv3.dat
testing/btest/Baseline/scripts.base.protocols.smtp.mime-extract/smtp-entity-cwR7l6Zctxb-0.dat
testing/btest/Baseline/scripts.base.protocols.smtp.mime-extract/smtp-entity-cwR7l6Zctxb.dat
testing/btest/Baseline/scripts.base.protocols.smtp.mime-extract/smtp_entities.log
testing/btest/scripts/base/protocols/ftp/ftp-extract.bro
testing/btest/scripts/base/protocols/http/http-extract-files.bro
testing/btest/scripts/base/protocols/irc/dcc-extract.test
testing/btest/scripts/base/protocols/smtp/mime-extract.test
2013-06-07 15:44:36 -05:00
Jon Siwek
705a84d688
Improve tracking of HTTP file extraction (addresses #988 ).
...
http.log now has files taken from request and response bodies in
different fields for each, and can now track multiple files per body.
That is, the "extraction_file" field is now "extracted_request_files"
and "extracted_response_files".
2013-05-21 16:42:35 -05:00
Jon Siwek
28f51a9a22
Remove extraction counter in default file extraction scripts.
2013-05-21 11:12:00 -05:00
Robin Sommer
4ccd6d76fd
Fixing tests.
...
Part of this involves making the file-analysis tests independent of
specific hash values. I've done that only partially though.
2013-05-17 18:14:35 -07:00
Jon Siwek
037d582b0e
FileAnalysis: add custom libmagic database.
...
- It's derived from the magic database of libmagic 5.14, but with most
everything not related to mime types removed.
- The custom database is always used by default for mime detection, but
the more verbose file type detection will fall back on the default
libmagic installation's database. The result is: mime type strings
are now guaranteed to be consistent across platforms, but the verbose
file type descriptions are not.
- The custom database gets installed in $prefix/share/bro/magic, and
should even be extensible if files with new patterns are added inside
the directory.
- The search path for the mime magic database can be controlled via
BROMAGIC environment variable.
- Remove mime_desc field from ftp.log.
- Stop using the mime/file type canonifier with unit tests.
- libmagic >= 5.04 is now a requirement.
2013-04-12 11:58:19 -05:00
Jon Siwek
71f0e2d276
FileAnalysis: replace script-layer http file analysis.
...
Other misc:
- Remove HTTP::MD5 notice.
- Add "last_active" field to FileAnalysis::Info record.
- Replace "conn_uids", "conn_ids" fields in FileAnalysis::Info record
with just a "conns" fields containing full connection records.
- The http-methods unit test is failing now, but I think it will be
fixed once I change the file handle callback mechanism to use events
instead.
2013-03-22 16:14:06 -05:00
Robin Sommer
1fd0d7a607
Changing the start/end markers in logs to open/close now reflecting
...
wall clock.
Triggers lots of (simple) baseline updates.
2012-07-27 12:15:21 -07:00
Robin Sommer
5cfb8d65c3
Updating tests for the #start/#end change.
2012-07-19 22:28:55 -07:00
Robin Sommer
3ac4ff6b42
Updates for log format changes.
2011-12-19 09:09:32 -08:00
Robin Sommer
a9f0b10e2e
Updating baselines for recent commits.
2011-12-19 07:44:29 -08:00
Robin Sommer
3220bbce55
Merge remote branch 'origin/topic/jsiwek/log-escaping'
...
* origin/topic/jsiwek/log-escaping:
Add missing ascii writer options to log header.
Escape the ASCII log's set separator (addresses #712 )
Rewrite ODesc character escaping functionality. (addresses #681 )
Closes #712 .
2011-12-19 06:37:54 -08:00
Seth Hall
b12d2c768e
Tiny bugfix for http file extraction along with test.
2011-11-14 15:24:15 -05:00
