Commit graph

6667 commits

Author SHA1 Message Date
Jon Siwek
0c82b6aa14 Fix a couple nits reported by Coverity.
An unnecessary null pointer check and uninitialized scalar fields.
Don't expect these to be actual problems, but easy enough to fix in
order to silence Coverity.
2014-04-01 11:00:10 -05:00
Jon Siwek
a30caf6960 Fix potential mem leak in IP frag reassembly.
Reported by Coverity, but it's not a typical code path.
2014-04-01 10:29:41 -05:00
Jon Siwek
acc721c36c Fix mem leak and unchecked dynamic cast reported by Coverity. 2014-03-31 16:32:58 -05:00
Bernhard Amann
b2f8196dc2 fix potential memleak in x509 parser reported by coverity 2014-03-31 14:06:34 -07:00
Robin Sommer
a9bbff932b Updating submodule(s).
[nomail]
2014-03-31 12:01:16 -07:00
Robin Sommer
559fa595c7 Updating submodule(s).
[nomail]
2014-03-31 11:57:53 -07:00
Robin Sommer
f0eb144303 Updating submodule(s).
[nomail]
2014-03-30 22:51:26 +02:00
Robin Sommer
9efb549236 Merge remote-tracking branch 'origin/topic/jsiwek/file-signatures'
* origin/topic/jsiwek/file-signatures:
  File type detection changes and fix https.log {orig,resp}_fuids fields.
  Various minor changes related to file mime type detection.
  Refactor common MIME magic matching code.
  Replace libmagic w/ Bro signatures for file MIME type identification.

Conflicts:
	scripts/base/init-default.bro
	testing/btest/Baseline/coverage.bare-load-baseline/canonified_loaded_scripts.log
	testing/btest/Baseline/coverage.default-load-baseline/canonified_loaded_scripts.log

BIT-1143 #merged
2014-03-30 22:51:05 +02:00
Robin Sommer
4fb0288aa2 Updating submodule(s).
[nomail]
2014-03-30 22:21:58 +02:00
Robin Sommer
1289790f13 Merge remote-tracking branch 'origin/topic/bernhard/file-analysis-x509'
* origin/topic/bernhard/file-analysis-x509:
  x509 version always has to be incremented. Baselines already committed.
  Fix x509 certificate Version (the +1 was missing...).
  Add policy script to suppress non host-certificate logging in x509.log
  Forgot the preamble for the new leak test
  (hopefully) last change -> return real opaque vec instead of any_vec
  Fix dump-events - it cannot be used with ssl anymore, because openssl does not give the same string results in all versions.
  Finishing touches of the x509 file analyzer.
  Revert change to only log certificates once per hour.
  Change x509 log - now certificates are only logged once per hour.
  Fix circular reference problem and a few other small things.
  X509 file analyzer nearly done. Verification and most other policy scripts work fine now.
  Add verify functionality, including the ability to get the validated chain. This means that it is now possible to get information about the root-certificates that were used to secure a connection.
  Second try on the event interface.
  Backport crash fix that made it into master with the x509_extension backport from here.
  Make x509 certificates an opaque type
  rip out x509 code from ssl analyzer. Note that since at the moment the file analyzer does not yet re-populate the info record that means quite a lot of information is simply not available.
  parse out extension. One event for general extensions (just returns the openssl-parsed string-value), one event for basicconstraints (is a certificate a CA or not) and one event for subject-alternative-names (only DNS parts).
  Very basic file-analyzer for x509 certificates. Mostly ripped from the ssl-analyzer and the topic/bernhard/x509 branch.

BIT-1150 #merged
2014-03-30 22:16:36 +02:00
Robin Sommer
27b661cc24 Merge remote-tracking branch 'origin/topic/jsiwek/improve-type-checks'
* origin/topic/jsiwek/improve-type-checks:
  Add unit tests covering vector/set/table ctors/inits.
  Fix parsing of "local" named table constructors.
  Improve type checking of records, addresses BIT-1159.

BIT-1159 #merged
2014-03-30 20:28:07 +02:00
Robin Sommer
1292f1957d Updating submodule(s).
[nomail]
2014-03-30 20:22:23 +02:00
Robin Sommer
806851a21e Don't include locations into serialization when cloning values.
This should save some memory, see BIT-1161.
2014-03-30 20:20:06 +02:00
Robin Sommer
2f6146aa55 Merge remote-tracking branch 'origin/topic/jsiwek/faster-val-clone'
* origin/topic/jsiwek/faster-val-clone:
  Refactor SerializationFormat::EndWrite and ChunkedIO::Chunk mem mgmt.
  Improve SerializationFormat's write buffer growth strategy.

BIT-1161 #merged
2014-03-30 20:13:26 +02:00
Robin Sommer
a0d389bf4b Merge remote-tracking branch 'origin/topic/jsiwek/parse-only'
* origin/topic/jsiwek/parse-only:
  Add --parse-only option to exit after parsing scripts.

BIT-1169 #merged
2014-03-30 20:01:12 +02:00
Seth Hall
9d6c8769ea Quick fix to the ElasticSearch writer. 2014-03-28 08:37:37 -04:00
Jon Siwek
06b6dc8447 Add --parse-only option to exit after parsing scripts.
May be useful for syntax-checking tools.
2014-03-27 16:07:52 -05:00
Jon Siwek
bebedcd873 Merge branch 'master' into topic/jsiwek/file-signatures
Conflicts:
	scripts/policy/frameworks/files/detect-MHR.bro
2014-03-24 14:35:37 -05:00
Jon Siwek
bf3c3887fd Fix parsing of "local" named table constructors. 2014-03-20 16:47:20 -05:00
Jon Siwek
b1fd161274 Improve type checking of records, addresses BIT-1159. 2014-03-20 13:54:26 -05:00
Bernhard Amann
ae165fb262 x509 version always has to be incremented. Baselines already committed. 2014-03-19 21:50:02 -07:00
Jon Siwek
70131b5c84 Refactor SerializationFormat::EndWrite and ChunkedIO::Chunk mem mgmt.
SerializationFormat::EndWrite now transfers ownership of the buffer
to the caller instead of doing a memcpy.

ChunkedIO::Chunk is no longer a POD type, hopefully the ctor/dtor
make it easier to manage its associated memory.  It also now
tracks how to deallocate its buffer (i.e. delete vs. free).
2014-03-18 14:42:38 -05:00
Robin Sommer
bf6f21041c Updating submodule(s).
[nomail]
2014-03-17 09:18:01 -07:00
Jon Siwek
07a4a8d147 Merge branch 'master' into topic/jsiwek/faster-val-clone 2014-03-17 11:05:08 -05:00
Robin Sommer
13081c984c Merge remote-tracking branch 'origin/fastpath'
* origin/fastpath:
  update submodules
  Fix compile errror on freebsd - defines have to be moved up due to header dependencies.
2014-03-17 08:25:24 -07:00
Bernhard Amann
66ec267b2a update submodules 2014-03-17 00:11:27 -07:00
Bernhard Amann
636d25e526 Fix compile errror on freebsd - defines have to be moved up due to
header dependencies.
2014-03-16 22:22:44 -07:00
Jon Siwek
6595c7c75b Improve SerializationFormat's write buffer growth strategy. 2014-03-14 17:26:38 -05:00
Jon Siwek
dbe5dfb3c3 Derive results of DNS lookups from from input when in BRO_DNS_FAKE mode.
Addresses BIT-1134.
2014-03-14 12:01:09 -05:00
Robin Sommer
e8339d5c63 Merge remote-tracking branch 'origin/topic/bernhard/file-analysis-x509'
* origin/topic/bernhard/file-analysis-x509:
  Forgot the preamble for the new leak test
  (hopefully) last change -> return real opaque vec instead of any_vec
  Fix dump-events - it cannot be used with ssl anymore, because openssl does not give the same string results in all versions.
  Finishing touches of the x509 file analyzer.
  Revert change to only log certificates once per hour.
  Change x509 log - now certificates are only logged once per hour.
  Fix circular reference problem and a few other small things.
  X509 file analyzer nearly done. Verification and most other policy scripts work fine now.
  Add verify functionality, including the ability to get the validated chain. This means that it is now possible to get information about the root-certificates that were used to secure a connection.
  Second try on the event interface.
  Backport crash fix that made it into master with the x509_extension backport from here.
  Make x509 certificates an opaque type
  rip out x509 code from ssl analyzer. Note that since at the moment the file analyzer does not yet re-populate the info record that means quite a lot of information is simply not available.
  parse out extension. One event for general extensions (just returns the openssl-parsed string-value), one event for basicconstraints (is a certificate a CA or not) and one event for subject-alternative-names (only DNS parts).
  Very basic file-analyzer for x509 certificates. Mostly ripped from the ssl-analyzer and the topic/bernhard/x509 branch.
2014-03-14 09:53:07 -07:00
Robin Sommer
8b241947d6 Fixing a few cases of undefined behaviour introduced by recent
formatter work.

Thanks, Coverity!
2014-03-14 08:45:36 -07:00
Robin Sommer
17f9d0a47d Fixing compiler error.
Hopefully ...
2014-03-13 18:02:41 -07:00
Robin Sommer
00755f1e40 Fixing (very unlikely) double delete in HTTP analyzer when decapsulating
CONNECTs.

BIT-1149 #closed
2014-03-13 16:51:21 -07:00
Robin Sommer
ff261ea626 Merge remote-tracking branch 'origin/topic/seth/json-formatter'
* origin/topic/seth/json-formatter:
  Updating a couple of tests.
  Expanded support for modifying the timestamp format in the JSON formatter.
  Ascii input reader now supports all config options per-input stream.
  Added an option to the JSON formatter to use ISO 8601 for timestamps.
  Refactored formatters and updated the the writers a bit.

Includes some minor bugfixes and cleanup at various places, including
in old code.
2014-03-13 16:50:56 -07:00
Bernhard Amann
d42d9bbc3f (hopefully) last change -> return real opaque vec instead of any_vec 2014-03-13 16:10:59 -07:00
Bernhard Amann
3f52eeacda Fix dump-events - it cannot be used with ssl anymore, because openssl
does not give the same string results in all versions.

Add leak test for x509 verify and fix small leak (type).
2014-03-13 15:41:57 -07:00
Bernhard Amann
4da0718511 Finishing touches of the x509 file analyzer.
Mostly baseline updates and new tests.

addresses BIT-953, BIT-760, BIT-1150
2014-03-13 15:21:30 -07:00
Robin Sommer
4eb81de08e Refactoring code to reuse ApplyScheduledAnalyzers().
This is potentially changing the exact time when the
scheduled_analyzer_applied() event is executed, but that should be
fine afaict.
2014-03-13 15:00:19 -07:00
Robin Sommer
126bd298fe Merge remote-tracking branch 'origin/topic/jsiwek/dns_fake'
* origin/topic/jsiwek/dns_fake:
  Teach async DNS lookup builtin-functions about BRO_DNS_FAKE.
2014-03-13 14:27:44 -07:00
Robin Sommer
a90bf11e35 Merge remote-tracking branch 'origin/topic/jsiwek/flip-roles'
* origin/topic/jsiwek/flip-roles:
  Improve analysis of TCP SYN/SYN-ACK reversal situations.
  Fix bug in Connection::FlipRoles, addresses BIT-1148.

BIT-1148 #merged
2014-03-13 14:15:29 -07:00
Bernhard Amann
b0c3486fd6 Merge remote-tracking branch 'origin/master' into topic/bernhard/file-analysis-x509 2014-03-13 00:09:48 -07:00
Bernhard Amann
0d50b8b04f Change x509 log - now certificates are only logged once per hour.
Add parsing of several more types to SAN extension.

Make error messages of x509 file analyzer more useful.

Fix file ID generation.

You apparently have to be very careful which EndOfFile function of
the file analysis framework you call... otherwhise it might try
to close another file id. This took me quite a while to find.

addresses BIT-953, BIT-760, BIT-1150
2014-03-13 00:05:48 -07:00
Robin Sommer
04dc318806 Merge remote-tracking branch 'origin/fastpath'
* origin/fastpath:
  Improve DBG_LOG macro (perf. improvement for --enable-debug mode).
  Misc. documentation fixes.
2014-03-12 17:20:14 -07:00
Jon Siwek
302c063874 Improve DBG_LOG macro (perf. improvement for --enable-debug mode).
Many usages of this macro do some string formatting work inline in the
arguments that end up being unnecessary because the debug stream is
disabled.
2014-03-12 15:59:05 -05:00
Seth Hall
c591e4f57f Expanded support for modifying the timestamp format in the JSON formatter. 2014-03-12 10:01:59 -04:00
Seth Hall
6cd9358a71 Ascii input reader now supports all config options per-input stream. 2014-03-12 10:01:17 -04:00
Jon Siwek
066473b1f1 Improve analysis of TCP SYN/SYN-ACK reversal situations.
- Since it's just the handshake packets out of order, they're no
  longer treated as partial connections, which some protocol analyzers
  immediately refuse to look at.

- The TCP_Reassembler "is_orig" state failed to change, which led to
  protocol analyzers sometimes using the wrong value for that.

- Add a unit test which exercises the Connection::FlipRoles() code
  path (i.e. the SYN/SYN-ACK reversal situation).

Addresses BIT-1148.
2014-03-11 17:03:59 -05:00
Jon Siwek
063200c71a Merge branch 'master' into topic/jsiwek/flip-roles 2014-03-11 16:41:03 -05:00
Jon Siwek
da338c8ffe Teach async DNS lookup builtin-functions about BRO_DNS_FAKE.
And enable fake DNS mode for test suites.

Addresses BIT-1134.
2014-03-10 16:51:04 -05:00
Seth Hall
c9aaf9e753 Added an option to the JSON formatter to use ISO 8601 for timestamps.
- It's not *exactly* ISO 8601 which doesn't seem to support
   subseconds, but subseconds are very important to us and
   most things that support ISO8601 seem to also support subseconds
   in the way I'm implemented it.
2014-03-10 14:22:35 -04:00