Commit graph

473 commits

Author SHA1 Message Date
Robin Sommer
99c50251d4 Merge remote-tracking branch 'origin/topic/johanna/dtls'
* origin/topic/johanna/dtls:
  a few more small script-level fixes
  update test baselines
  add a simple leak test for dtls
  add signature for dtls client hello
  Make the plugin structure more... legal.
  Only force logging of SSL if it actually was the SSL analyzer that failed.
  DTLS working.
  Implement correct parsing of TLS record fragmentation.
  Make handshake analyzer flow-based. This means we can feed data to it in chunks, which makes dealing with fragmentation a little bit more convenient.
  When setting the SSL analyzer to fail, also stop processing data that already has been delivered to the analyzer, not just future data.
  First step for a DTLS analyzer.

BIT-1347 #merged

Conflicts:
	scripts/base/protocols/ssl/main.bro
	testing/btest/Baseline/plugins.hooks/output
2015-03-23 12:40:38 -07:00
Robin Sommer
88165ad72c Merge remote-tracking branch 'origin/topic/johanna/bit-1199'
* origin/topic/johanna/bit-1199:
  add a basic leak test for an unparseable enum
  Change the way the input framework deals with values it cannot convert into BroVals (especially enums)
  Make error message when encountering not existing enums better.

BIT-1199: #merged
2015-03-23 10:25:43 -07:00
Robin Sommer
9999bce142 Merge remote-tracking branch 'origin/topic/jsiwek/bit-788'
* origin/topic/jsiwek/bit-788:
  BIT-788: use DNS QR field to better identify flow direction.

BIT-788 #merged
2015-03-23 10:08:02 -07:00
Jon Siwek
0a7afab740 Add unit test to catch breaking changes to local.bro
BIT-1207 #close
2015-03-20 10:34:34 -05:00
Johanna Amann
c27848fc32 Change the way the input framework deals with values it cannot convert
into BroVals (especially enums)

Not we do not force an internal error anymore. Instead, we raise an
normal error and set an error flag that signals to the top-level
functions that the value could not be converted and should not be
propagated to the Bro core. This sadly makes the already messy code even
more messy - but since errors can happen in deeply nested data
structures, the alternative (catching the error at every possible
location and then trying to clean up there instead of recursively
deleting the data that cannot be used later) is much worse.

Addresses BIT-1199
2015-03-19 14:58:38 -07:00
Jon Siwek
8efaae96cd BIT-788: use DNS QR field to better identify flow direction. 2015-03-19 11:53:40 -05:00
Johanna Amann
28e6aa9561 Merge remote-tracking branch 'origin/master' into topic/johanna/dtls 2015-03-18 12:25:39 -07:00
Robin Sommer
468e7bbce2 Increasing a test timeout to not fail on slower machines. 2015-03-17 15:41:14 -07:00
Robin Sommer
b0e066d3e0 Merge remote-tracking branch 'origin/topic/johanna/cert-validation'
* origin/topic/johanna/cert-validation:
  add x509 canonifiers to test to not make it fail on differing openssl versions.
2015-03-17 15:29:47 -07:00
Johanna Amann
e291ccc14a add x509 canonifiers to test to not make it fail on differing openssl
versions.
2015-03-17 12:51:57 -07:00
Robin Sommer
1ec4243ea8 Merge remote-tracking branch 'origin/topic/jsiwek/bit-1077'
* origin/topic/jsiwek/bit-1077:
  BIT-1077: fix HTTP::log_server_header_names.

BIT-1077 #merged
2015-03-17 09:12:55 -07:00
Robin Sommer
0cfe431f15 Merge remote-tracking branch 'origin/topic/johanna/cert-validation'
* origin/topic/johanna/cert-validation:
  and still use the hash for notice suppression.
  add knob to revert to old validation behavior
  Update certificate validation script - new version will cache valid intermediate chains that it encounters on the wire and use those to try to validate chains that might be missing intermediate certificates.

BIT-1332 #merged
2015-03-17 09:09:54 -07:00
Jon Siwek
c09411bc8b BIT-1077: fix HTTP::log_server_header_names.
Before, it just re-logged fields from the client side.
2015-03-16 15:12:48 -05:00
Jon Siwek
0b957cbe75 Include timestamp in default extracted file names.
And add a policy script to extract all files.

BIT-1335 #close
2015-03-13 14:25:30 -05:00
Johanna Amann
991e4f5dc3 DTLS working.
The only thing that is missing is a signature to detect the protocol (it
has no well-known port).

Reassembly is kind of fidgety - at the moment we only support
re-assembling one simultaneous message per direction (which looking at
our test-traffic might not be a problem). And I am not quite sure if I
got all cases correct...

But - it works :)
2015-03-12 15:46:17 -07:00
Johanna Amann
ba27bb54d4 Implement correct parsing of TLS record fragmentation.
Finally. Our test-case is a >400kb certificate with 10,000 alternative
names. :)
2015-03-11 18:23:08 -07:00
Johanna Amann
144302d3e7 add knob to revert to old validation behavior 2015-03-09 12:53:29 -07:00
Johanna Amann
6ab5701ad0 Update certificate validation script - new version will cache valid
intermediate chains that it encounters on the wire and use those to try
to validate chains that might be missing intermediate certificates.

This vastly improves the number of certificates that Bro can validate.
The only drawback is that now validation behavior is not entirely
predictable anymore - the certificate of a server can fail to validate
when Bro just started up (due to the intermediate missing), and succeed
later, when the intermediate can be found in the cache.

Has been tested on big-ish clusters and should not introduce any
performance problems.
2015-03-09 12:46:33 -07:00
Johanna Amann
946f19fb9d Use our new features to send the CN and SAN fields of certificates to
the intel framework.
2015-03-03 17:15:24 -08:00
Johanna Amann
e48c6ccc4a Do not log common name by default (it is most interesting for scripts)
and add a test case.
2015-03-03 16:38:25 -08:00
Johanna Amann
897351f87e Extend the weak-keys policy file to also alert when encountering
ssl connections with old versions as well as unsafe cipher suites.

Also make the notice suppression handling of other ssl policy files
a tad more robust.
2015-02-25 13:57:04 -08:00
Robin Sommer
6350d330f3 Merge remote-tracking branch 'origin/fastpath'
* origin/fastpath:
  Crashing bug in WriterBackend when deserializing WriterInfo where config is present. Testcase crashes on unpatched versions of Bro.
  Fix wrong value test in WriterBackend. Found by Aaron Eppert (aeppert@gmail.com)
2015-02-23 16:33:58 -08:00
Robin Sommer
5dea09b7c1 Baseline updates for the addition of local_resp.
That patch is a strong contender for the smallest ever ratio of
lines-of-code-changed to lines-of-baselines-updated. :-)
2015-02-23 16:25:11 -08:00
Johanna Amann
ee290c3d7a Crashing bug in WriterBackend when deserializing WriterInfo where config
is present. Testcase crashes on unpatched versions of Bro.

Found by Aaron Eppert <aeppert@gmail.com>.

This (probably) fixes the crash issue with sqlite a few people have
reported on the mailing list in the past.
2015-02-23 13:54:44 -08:00
Jon Siwek
212368b245 Merge remote-tracking branch 'origin/topic/jsiwek/socks-authentication'
* origin/topic/jsiwek/socks-authentication:
  Refactor SOCKS5 user/pass authentication support.
  Update the SOCKS analyzer to support user/pass login.

BIT-1011 #merged
2015-02-13 09:15:50 -06:00
Seth Hall
9592f64225 Update the SOCKS analyzer to support user/pass login.
- This addresses BIT-1011
 - Add a new field to socks.log; "password".
 - Two new events; socks_login_userpass and socks_login_reply.
 - One new weird for unsupported authentication method.
 - A new test for authenticated socks traffic.
 - Credit to Nicolas Retrain for the initial patch.  Thanks!
2015-02-05 12:44:10 -05:00
Seth Hall
a97cd1f3a2 Fix a bug in the core files framework with handling the BOF buffer.
- Any files where the total size was below the size of the
   default bof_buffer size couldn't have stream analyzers successfully
   attached because the bof_buffer never reached the full size
   and was never flushed.  This branch explicitly marks the buf_buffer
   as full and flushes it when the file is being removed.
2015-02-05 09:09:08 -05:00
Jon Siwek
565ad360c6 Add x509 canonifier to a unit test. 2015-02-03 17:04:26 -06:00
Jon Siwek
23f04835c6 Deprecate split* family of BIFs.
These functions are now deprecated in favor of alternative versions that
return a vector of strings rather than a table of strings.

Deprecated functions:

- split: use split_string instead.
- split1: use split_string1 instead.
- split_all: use split_string_all instead.
- split_n: use split_string_n instead.
- cat_string_array: see join_string_vec instead.
- cat_string_array_n: see join_string_vec instead.
- join_string_array: see join_string_vec instead.
- sort_string_array: use sort instead instead.
- find_ip_addresses: use extract_ip_addresses instead.

Changed functions:

- has_valid_octets: uses a string_vec parameter instead of string_array.

Addresses BIT-924, BIT-757.
2015-01-21 15:34:42 -06:00
Robin Sommer
41ff1c4cd0 Merge remote-tracking branch 'origin/topic/robin/dnp3-merge-v4'
* origin/topic/robin/dnp3-merge-v4:
  add test trace in which DNP3 packets are over UDP; update test scripts and baseline results
  A bit more DNP3 tweaking.
  remove redundnt codes; find a way to use the analyzer function, such as Weird; fix a small bug in ProcessData function in DNP3.cc; passed the test
  Renameing the DNP3 TCP analyzer
  quickly fix another bug; adding missing field of the declaration of dnp3_request_application_header and dnp3_response_application_header
  Removing the debug printf in DNP3.cc
  fixed the bug of deciding the size of object 1 varition 1 in DNP3
  Fix some things in DNP3 UDP analyzer.
  changed a bug, but still not working
  modify DNP3.cc and DNP3.h to add DNP3_UDP_Analyzer; binpac unchanged

BIT-1231 #merged
2015-01-14 13:25:42 -08:00
Hui Lin
794273913f add test trace in which DNP3 packets are over UDP; update test scripts and baseline results 2015-01-07 15:04:22 -06:00
Jon Siwek
a3d78cc830 Revert "Workaround race condition in unified2 file module."
This reverts commit 1a03a95f35.
2015-01-05 14:51:58 -06:00
Jon Siwek
1a03a95f35 Workaround race condition in unified2 file module.
This makes the unit test pass consistently, but need to see about
fixing it in the unified2 file module directly.
2014-12-17 09:57:06 -06:00
Jon Siwek
edaf7edc11 Merge remote-tracking branch 'origin/topic/seth/files-reassembly-and-mime-updates' into topic/jsiwek/file-reassembly-merge
Conflicts:
	testing/btest/Baseline/scripts.policy.misc.dump-events/all-events.log
2014-12-15 10:33:09 -06:00
Johanna Amann
cd21b7f130 Fix x509 analyzer to correctly return ecdsa as the key_type for ecdsa certs.
Returned dsa so far.

Bug found by Michał Purzyński
2014-11-25 11:18:07 -08:00
Robin Sommer
9d3cfaddaa Merge branch 'master' of https://github.com/anthonykasza/bro
- I've changed/extended the URI record fields a bit:
        - path is always the full path including the full file name
        - if there's no path, the field still still be set set "/".
        - file_name is the full name including extenstion, and
          file_base and file_ext split it out.

    - Adding a test exercising a bunch of URLs.
2014-11-18 12:21:06 -08:00
Seth Hall
842dfd8b4a Merge remote-tracking branch 'origin/topic/seth/files-tracking' into topic/seth/files-reassembly-and-mime-updates
Conflicts:
	testing/btest/Baseline/scripts.base.frameworks.file-analysis.http.multipart/out
	testing/btest/Baseline/scripts.policy.misc.dump-events/all-events.log
2014-11-05 11:40:26 -05:00
Vlad Grigorescu
b259a41ef2 Merge remote-tracking branch 'origin/master' into topic/vladg/mysql
Conflicts:
	testing/btest/Baseline/coverage.bare-load-baseline/canonified_loaded_scripts.log
	testing/btest/Baseline/coverage.default-load-baseline/canonified_loaded_scripts.log
	testing/btest/Baseline/scripts.policy.misc.dump-events/all-events.log
2014-10-27 13:24:31 -04:00
Robin Sommer
fb56d3f0bb Fixing unstable test. 2014-10-24 13:40:00 -07:00
Christian Struck
d17b3746cf [ADD] added baseline for the new active-http test and added a test to check for the content-length 0 fix. 2014-10-22 16:04:04 -07:00
Christian Struck
0a59772043 [ADD] added baseline for the new exec test and added a test to check for the empty files fix. 2014-10-22 16:02:19 -07:00
Hui Lin
6e7a4a4fee Merge branch 'fastpath' of git://git.bro-ids.org/bro into fastpath 2014-09-26 14:48:58 -05:00
Hui Lin
f933899b17 adding a function in dnp3-analyzer.pac to translate the time stamp format 2014-09-26 14:47:51 -05:00
Seth Hall
cafd35e746 Updates the files event api and brings file reassembly up to master. 2014-09-26 00:40:37 -04:00
Jon Siwek
af9d31dcc1 Fix incorrect data delivery skips after gap in HTTP Content-Range.
The logic for determining whether a gap was entirely within a MIME
entity body was not asking the current entity, which may be better able
to answer that question if it was using the Content-Range header and
thus knows if the gap exceeds the length of the body that's still
expected.

Addresses BIT-1247
2014-09-11 14:53:47 -05:00
Jon Siwek
1e02d5d5b5 Fix file analysis placement of data after gap in HTTP Content-Range.
Addresses BIT-1248.
2014-09-11 12:25:43 -05:00
Jon Siwek
f1cef9d2a9 Fix issue w/ TCP reassembler not delivering some segments.
For example, if we have a connection between TCP "A" and TCP "B" and "A"
sends segments "1" and "2", but we don't see the first and then the next
acknowledgement from "B" is for everything up to, and including, "2",
the gap would be reported to include both segments instead of just the
first and then delivering the second.  Put generally: any segments that
weren't yet delivered because they're waiting for an earlier gap to be
filled would be dropped when an ACK comes in that includes the gap as
well as those pending segments.  (If a distinct ACK was seen for just
the gap, that situation would have worked).

Addresses BIT-1246.
2014-09-11 10:47:56 -05:00
Jon Siwek
f97f58e9db Raise http_entity_data in line with data arrival.
As opposed to delaying until a certain-sized-buffer fills, which is
problematic because then the event becomes out of sync with the "rest of
the world".  E.g. content_gap handlers being called sooner than
expected.

Addresses BIT-1240.
2014-09-10 13:20:47 -05:00
Robin Sommer
525816b03d Merge remote-tracking branch 'origin/topic/hui/dnp3-udp'
* origin/topic/hui/dnp3-udp:
  remove redundnt codes; find a way to use the analyzer function, such as Weird; fix a small bug in ProcessData function in DNP3.cc; passed the test
  Renameing the DNP3 TCP analyzer
  quickly fix another bug; adding missing field of the declaration of dnp3_request_application_header and dnp3_response_application_header
  Removing the debug printf in DNP3.cc
  fixed the bug of deciding the size of object 1 varition 1 in DNP3
  Fix some things in DNP3 UDP analyzer.
  changed a bug, but still not working
  modify DNP3.cc and DNP3.h to add DNP3_UDP_Analyzer; binpac unchanged
2014-09-07 21:09:53 -07:00
Johanna Amann
8f1cbb8b0a Fix ocsp reply validation - there were a few things that definitely were wrong.
Now the right signer certificate for the reply is looked up (and no longer assumed that it is the first one) and a few compares are fixed. Plus - there are more test cases that partially send certificates in the ocsp message and partially do not - and it seems to work fine in all cases.

Addresses BIT-1212
2014-09-04 12:22:55 -07:00