Commit graph

1690 commits

Author SHA1 Message Date
Jon Siwek
dcbd0819a6 Updates related to SSH analysis.
- Some scripts used wrong SSH module/namespace scoping on events.
- Fix outdated notice documentation related to SSH password guessing.
- Add a unit test for SSH pasword guessing notice.
2015-03-30 11:30:48 -05:00
Jon Siwek
97962d25f2 Fix outdated documentation unit tests. 2015-03-30 11:03:21 -05:00
Jon Siwek
fc190eb9f1 Add a canonifier to a unit test's output. 2015-03-30 10:23:52 -05:00
Johanna Amann
f79b5adc08 Merge remote-tracking branch 'origin/topic/vladg/ssh'
I replaced a few strcmps with either calls to std::str.compare
or with the == operator of BroString.

Also changed two of the input framework tests that did not pass
anymore after the merge. The new SSH analyzer no longer loads the
scripts that let network time run, hence those tests failed because
updates were not propagated from the threads (that took a while
to find.)

* origin/topic/vladg/ssh: (25 commits)
  SSH: Register analyzer for 22/tcp.
  SSH: Add 22/tcp to likely_server_ports
  SSH: Ignore encrypted packets by default.
  SSH: Fix some edge-cases which created BinPAC exceptions
  SSH: Add memleak btest
  SSH: Update baselines
  SSH: Added some more events for SSH2
  SSH: Intel framework integration (PUBKEY_HASH)
  Update baselines for new SSH analyzer.
  Update SSH policy scripts with new events.
  SSH: Add documentation
  Refactoring ssh-protocol.pac:
  SSH: Use the compression_algorithms const in another place.
  Some cleanup and refactoring on SSH main.bro.
  SSH: A bit of code cleanup.
  Move SSH constants to consts.pac
  SSH: Cleanup code style.
  SSH: Fix some memleaks.
  Refactored the SSH analyzer. Added supported for algorithm detection and more key exchange message types.
  Add host key support for SSH1.
  Add support for SSH1
  Move SSH analyzer to new plugin architecture.
  ...

Conflicts:
  scripts/base/protocols/ssh/main.bro
  testing/btest/Baseline/core.print-bpf-filters/output2
  testing/btest/Baseline/plugins.hooks/output

BIT-1344: #merged
2015-03-25 11:04:26 -07:00
Robin Sommer
d2998b6f64 Merge branch 'master' of git.bro.org:bro
Conflicts:
	CHANGES
	VERSION
2015-03-23 12:50:30 -07:00
Robin Sommer
99c50251d4 Merge remote-tracking branch 'origin/topic/johanna/dtls'
* origin/topic/johanna/dtls:
  a few more small script-level fixes
  update test baselines
  add a simple leak test for dtls
  add signature for dtls client hello
  Make the plugin structure more... legal.
  Only force logging of SSL if it actually was the SSL analyzer that failed.
  DTLS working.
  Implement correct parsing of TLS record fragmentation.
  Make handshake analyzer flow-based. This means we can feed data to it in chunks, which makes dealing with fragmentation a little bit more convenient.
  When setting the SSL analyzer to fail, also stop processing data that already has been delivered to the analyzer, not just future data.
  First step for a DTLS analyzer.

BIT-1347 #merged

Conflicts:
	scripts/base/protocols/ssl/main.bro
	testing/btest/Baseline/plugins.hooks/output
2015-03-23 12:40:38 -07:00
Jon Siwek
dbf58be0e5 Fix an example in quickstart docs.
BIT-1226 #close
2015-03-23 13:05:58 -05:00
Robin Sommer
1d49ec63f8 Merge remote-tracking branch 'origin/topic/jsiwek/bit-1324'
I added the $path to the create_stream() calls inside doc/ as well.

* origin/topic/jsiwek/bit-1324:
  Allow logging filters to inherit default path from stream.

BIT-1324: #merged
2015-03-23 10:35:30 -07:00
Robin Sommer
88165ad72c Merge remote-tracking branch 'origin/topic/johanna/bit-1199'
* origin/topic/johanna/bit-1199:
  add a basic leak test for an unparseable enum
  Change the way the input framework deals with values it cannot convert into BroVals (especially enums)
  Make error message when encountering not existing enums better.

BIT-1199: #merged
2015-03-23 10:25:43 -07:00
Robin Sommer
9999bce142 Merge remote-tracking branch 'origin/topic/jsiwek/bit-788'
* origin/topic/jsiwek/bit-788:
  BIT-788: use DNS QR field to better identify flow direction.

BIT-788 #merged
2015-03-23 10:08:02 -07:00
Robin Sommer
6c6873b668 Merge remote-tracking branch 'origin/topic/jsiwek/bit-342'
* origin/topic/jsiwek/bit-342:
  BIT-342: add "icmp_sent_payload" event.

BIT-342 #merged.
2015-03-23 10:04:07 -07:00
Jon Siwek
df60015333 Remove "unmatched_HTTP_reply" weird.
BIT-725 #close
2015-03-20 11:13:50 -05:00
Jon Siwek
0a7afab740 Add unit test to catch breaking changes to local.bro
BIT-1207 #close
2015-03-20 10:34:34 -05:00
Johanna Amann
8d4708300f fix failing sqlite leak test 2015-03-19 16:15:52 -07:00
Johanna Amann
1f33dd0c38 add a basic leak test for an unparseable enum 2015-03-19 15:59:49 -07:00
Johanna Amann
c27848fc32 Change the way the input framework deals with values it cannot convert
into BroVals (especially enums)

Not we do not force an internal error anymore. Instead, we raise an
normal error and set an error flag that signals to the top-level
functions that the value could not be converted and should not be
propagated to the Bro core. This sadly makes the already messy code even
more messy - but since errors can happen in deeply nested data
structures, the alternative (catching the error at every possible
location and then trying to clean up there instead of recursively
deleting the data that cannot be used later) is much worse.

Addresses BIT-1199
2015-03-19 14:58:38 -07:00
Jon Siwek
186e67ec1d Allow logging filters to inherit default path from stream.
This allows the path for the default filter to be specified explicitly
when creating a stream and reduces the need to rely on the default path
function to magically supply the path.

The default path function is now only used if, when a filter is added to
a stream, it has neither a path nor a path function already.

Adapted the existing Log::create_stream calls to explicitly specify a
path value.

Addresses BIT-1324
2015-03-19 14:49:55 -05:00
Jon Siwek
8efaae96cd BIT-788: use DNS QR field to better identify flow direction. 2015-03-19 11:53:40 -05:00
Seth Hall
3956df4407 Merge remote-tracking branch 'origin/topic/vladg/rrsig'
* origin/topic/vladg/rrsig:
  DNS: Log the type number for the DNS_RR_unknown_type weird.
2015-03-18 22:51:37 -04:00
Jon Siwek
981be3b670 BIT-342: add "icmp_sent_payload" event. 2015-03-18 16:16:24 -05:00
Johanna Amann
e180403e76 update test baselines 2015-03-18 12:56:02 -07:00
Johanna Amann
5f557849a6 add a simple leak test for dtls 2015-03-18 12:48:22 -07:00
Johanna Amann
28e6aa9561 Merge remote-tracking branch 'origin/master' into topic/johanna/dtls 2015-03-18 12:25:39 -07:00
Vlad Grigorescu
01e5de8234 DNS: Log the type number for the DNS_RR_unknown_type weird. 2015-03-18 13:31:12 -04:00
Vlad Grigorescu
29f78cf90f SSH: Add memleak btest 2015-03-18 13:04:44 -04:00
Vlad Grigorescu
be6188bf00 SSH: Update baselines 2015-03-18 13:02:33 -04:00
Robin Sommer
468e7bbce2 Increasing a test timeout to not fail on slower machines. 2015-03-17 15:41:14 -07:00
Robin Sommer
b0e066d3e0 Merge remote-tracking branch 'origin/topic/johanna/cert-validation'
* origin/topic/johanna/cert-validation:
  add x509 canonifiers to test to not make it fail on differing openssl versions.
2015-03-17 15:29:47 -07:00
Johanna Amann
e291ccc14a add x509 canonifiers to test to not make it fail on differing openssl
versions.
2015-03-17 12:51:57 -07:00
Vlad Grigorescu
092a78d14b Merge remote-tracking branch 'origin/master' into topic/vladg/ssh 2015-03-17 12:36:30 -04:00
Robin Sommer
1ec4243ea8 Merge remote-tracking branch 'origin/topic/jsiwek/bit-1077'
* origin/topic/jsiwek/bit-1077:
  BIT-1077: fix HTTP::log_server_header_names.

BIT-1077 #merged
2015-03-17 09:12:55 -07:00
Robin Sommer
0cfe431f15 Merge remote-tracking branch 'origin/topic/johanna/cert-validation'
* origin/topic/johanna/cert-validation:
  and still use the hash for notice suppression.
  add knob to revert to old validation behavior
  Update certificate validation script - new version will cache valid intermediate chains that it encounters on the wire and use those to try to validate chains that might be missing intermediate certificates.

BIT-1332 #merged
2015-03-17 09:09:54 -07:00
Jon Siwek
c09411bc8b BIT-1077: fix HTTP::log_server_header_names.
Before, it just re-logged fields from the client side.
2015-03-16 15:12:48 -05:00
Vlad Grigorescu
65d982acc1 Update baselines for new SSH analyzer. 2015-03-16 16:12:18 -04:00
Jon Siwek
0b957cbe75 Include timestamp in default extracted file names.
And add a policy script to extract all files.

BIT-1335 #close
2015-03-13 14:25:30 -05:00
Jon Siwek
6fbceb6a98 Identify GRE tunnels as Tunnel::GRE, not Tunnel::IP.
BIT-1311 #close
2015-03-13 13:03:58 -05:00
Johanna Amann
991e4f5dc3 DTLS working.
The only thing that is missing is a signature to detect the protocol (it
has no well-known port).

Reassembly is kind of fidgety - at the moment we only support
re-assembling one simultaneous message per direction (which looking at
our test-traffic might not be a problem). And I am not quite sure if I
got all cases correct...

But - it works :)
2015-03-12 15:46:17 -07:00
Jon Siwek
c56df225b0 Fix Broker leak tests.
Forgot to update Broker module names when they changed.
2015-03-12 16:17:34 -05:00
Johanna Amann
ba27bb54d4 Implement correct parsing of TLS record fragmentation.
Finally. Our test-case is a >400kb certificate with 10,000 alternative
names. :)
2015-03-11 18:23:08 -07:00
Johanna Amann
038fbf9b9e First step for a DTLS analyzer.
This commit mostly does a lot of refactoring of the current SSL
analyzer, which is split into several parts.

The handshake protocol is completely taken out of the SSL analyzer and
was refactored into its own analyzer (called tls-handshake-analyzer).
This will also (finally) make it possible to deal with TLS record
fragmentation.

Apart from that, the parts of the SSL analyzer that are common to DTLS
were split into their own pac files. Both the SSL analyzer and the (very
basic, mostly nonfunctional) DTLS analyzer use their own pac files and
those shared pac files.

All SSL tests still pass after refactoring so I hope I did not break
anything too badly.

At the moment, we have two different modules in one directory and I
guess the way I am doing this might be an abuse of the system. It seems
to work though...
2015-03-11 15:07:13 -07:00
Johanna Amann
144302d3e7 add knob to revert to old validation behavior 2015-03-09 12:53:29 -07:00
Johanna Amann
6ab5701ad0 Update certificate validation script - new version will cache valid
intermediate chains that it encounters on the wire and use those to try
to validate chains that might be missing intermediate certificates.

This vastly improves the number of certificates that Bro can validate.
The only drawback is that now validation behavior is not entirely
predictable anymore - the certificate of a server can fail to validate
when Bro just started up (due to the intermediate missing), and succeed
later, when the intermediate can be found in the cache.

Has been tested on big-ish clusters and should not introduce any
performance problems.
2015-03-09 12:46:33 -07:00
Jon Siwek
9e53722b57 Rename comm/ directories to broker/ 2015-03-05 17:02:25 -06:00
Jon Siwek
fa08083a92 Rename broker-related namespaces.
c++ namespace "comm" -> bro_broker
script module "Comm" -> BrokerComm
script module "Store" -> BrokerStore
2015-03-05 16:20:51 -06:00
Jon Siwek
25a4d0ebed Improve remote logging via broker.
Only send fields with the &log attribute.
2015-03-05 14:07:06 -06:00
Jon Siwek
4f918f1f98 Improve some broker communication unit tests.
For testing data store queries, when statements may not work well if
time stops advancing e.g. due to lack of input sources, so try to
workaround by reading a trace file in unit test.
2015-03-05 11:46:55 -06:00
Robin Sommer
e5adc768cc Merge branch 'stats-bytes-recvd' of https://github.com/msmiley/bro 2015-03-04 13:16:19 -08:00
Johanna Amann
946f19fb9d Use our new features to send the CN and SAN fields of certificates to
the intel framework.
2015-03-03 17:15:24 -08:00
Johanna Amann
e48c6ccc4a Do not log common name by default (it is most interesting for scripts)
and add a test case.
2015-03-03 16:38:25 -08:00
Robin Sommer
6fa03abdbc Merge remote-tracking branch 'origin/topic/gilbert/plugin-api-tweak'
* origin/topic/gilbert/plugin-api-tweak:
  Updating plugin.hooks baseline so that test succeeds
  Revert spacing change that shouldn't have been included with the previous changeset ... should fix all of the plugin tests save hooks, which needs to be updated.
  More small fixes
  Small fixes
  Incremental
  Re-updating plugin.hooks test to include new argument output (after merge).
  Fixing logic errors in HandlePluginResult
  Updating tests and tweaking HookArgument to include Frame support.
  Incremental commit: implementing a wrapper for the Val class.
  Reverting change to const status of network_time.  Also, see FIXME: in Func.cc / HandlePluginResult ...
  Tweaks to result handling to make things a little more sane.
  Plugin API: minor change (adding parent frame) to support calling methods from hook.  Also declare network time update argument to be const because good practice.

BIT-1270 #merged

Conflicts:
	testing/btest/Baseline/plugins.hooks/output
2015-03-02 18:17:52 -08:00