Commit graph

1234 commits

Author SHA1 Message Date
Johanna Amann
00204ab8a6 introduce &weaken attribute, which basically only prevents
the describe function for types to descend into record fields that
are marked with it.

With this, we can actually load the pacf scripts without crashing Bro
when running tests :)
2015-04-13 16:05:55 -07:00
Johanna Amann
21b78b7d92 add really simple log output plugin for openflow. 2015-04-13 12:55:38 -07:00
Johanna Amann
46058d0b02 a few small fixes to openflow
*rename module from Openflow to OpenFlow
*add match_conn function to convert conn_id to openflow match
*add a few things back into the openflow records like... table_id
*and - a test
2015-04-10 11:21:55 -07:00
Johanna Amann
70f6635cb1 do not load pacf by default. There is some recursive record
definition in there Bro does not like too much...
2015-04-10 11:19:09 -07:00
Johanna Amann
94c67dc030 Merge remote-tracking branch 'origin/topic/robin/pacf' into topic/johanna/openflow 2015-04-07 17:28:38 -07:00
Johanna Amann
883da516ee move pacf skeleton away to be able to replace it with old proposal of
Robin.
2015-04-07 17:27:50 -07:00
Johanna Amann
dbc51371cb Rewrite big parts of the Openflow framework.
The API now does not follow the openflow specification quite as closely,
however I think it is much more usable. Furthermore, the Ryu plugin was
basically completely rewritten and is now more usable for general flow
manipulation.

This also adds a debug mode that just outputs the json fragments that
would be sent to ryu. At the moment, Ryu still assumes that every
request that it receives succeeds - it is not possible to get an error
message from the controller. Instead, one has to check if a flow was
added by doing a second REST request. Which seems unnecessary, and also
requires complete json parsing functionality. Hence we are not doing
that at the moment.

The alternative would be to use an external script for the actual
add-and-check-operation.
2015-04-07 15:37:46 -07:00
Johanna Amann
4195a0066a move the json stuff into utils - I guess we will need functionality like
this not just for openflow at some point of time.
2015-04-07 10:43:36 -07:00
Johanna Amann
fe5408e676 as a first step, restructure things a bit and remove the site scripts
users have to suply
2015-04-06 15:08:08 -07:00
Johanna Amann
4c68d37175 Merge branch 'master' into topic/johanna/openflow 2015-04-06 14:17:54 -07:00
Jon Siwek
20a8ec53ca BIT-1366: improve checksum offloading warning. 2015-04-02 10:12:39 -05:00
Jon Siwek
32ae94de9a Merge remote-tracking branch 'origin/topic/seth/rdp'
* origin/topic/seth/rdp: (31 commits)
  Improved transition into SSL/TLS from RDP.
  Fixes tests in RDP branch.
  add a special case to the X509 code that deals with RDP certificates.
  A few more changes to handling encryption in RDP.
  Adds some comments and fixes a broxygen warning.
  Fixes another optional part of an RDP unit.
  Support RDP negotiation requests optionally and support zero length cookies.
  Changed UTF-16 to UTF-8 conversion to be more lenient.
  Fixed an issue with parse failure on an optional field.
  Removing a stray printf from RDP analyzer.
  Another big RDP update.
  New script to add a field to rdp.log when the connection is upgraded to SSL.
  Huge updates to the RDP analyzer from Josh Liburdi.
  FreeRDP test trace showing SSL encryption -- RDP analyzer does not currently handle this and SSL analyzer does not identify it either
  Wireshark test trace for native encryption -- generates a binpac error
  Delete RDP-004.pcap
  Delete nla_win7_win2k8r2.pcap
  Update dpd.sig
  Fixed typo
  Added check for connection existence
  ...

BIT-1340 #merged
2015-03-30 17:10:04 -05:00
Jon Siwek
dcbd0819a6 Updates related to SSH analysis.
- Some scripts used wrong SSH module/namespace scoping on events.
- Fix outdated notice documentation related to SSH password guessing.
- Add a unit test for SSH pasword guessing notice.
2015-03-30 11:30:48 -05:00
Johanna Amann
b9e2b7bef9 Log::write in signature framework was missing ts
(Andrew Benson/Michel Laterman)

BIT-1354 #close
2015-03-25 12:01:09 -07:00
Johanna Amann
f79b5adc08 Merge remote-tracking branch 'origin/topic/vladg/ssh'
I replaced a few strcmps with either calls to std::str.compare
or with the == operator of BroString.

Also changed two of the input framework tests that did not pass
anymore after the merge. The new SSH analyzer no longer loads the
scripts that let network time run, hence those tests failed because
updates were not propagated from the threads (that took a while
to find.)

* origin/topic/vladg/ssh: (25 commits)
  SSH: Register analyzer for 22/tcp.
  SSH: Add 22/tcp to likely_server_ports
  SSH: Ignore encrypted packets by default.
  SSH: Fix some edge-cases which created BinPAC exceptions
  SSH: Add memleak btest
  SSH: Update baselines
  SSH: Added some more events for SSH2
  SSH: Intel framework integration (PUBKEY_HASH)
  Update baselines for new SSH analyzer.
  Update SSH policy scripts with new events.
  SSH: Add documentation
  Refactoring ssh-protocol.pac:
  SSH: Use the compression_algorithms const in another place.
  Some cleanup and refactoring on SSH main.bro.
  SSH: A bit of code cleanup.
  Move SSH constants to consts.pac
  SSH: Cleanup code style.
  SSH: Fix some memleaks.
  Refactored the SSH analyzer. Added supported for algorithm detection and more key exchange message types.
  Add host key support for SSH1.
  Add support for SSH1
  Move SSH analyzer to new plugin architecture.
  ...

Conflicts:
  scripts/base/protocols/ssh/main.bro
  testing/btest/Baseline/core.print-bpf-filters/output2
  testing/btest/Baseline/plugins.hooks/output

BIT-1344: #merged
2015-03-25 11:04:26 -07:00
Vlad Grigorescu
75d7f3414f SSH: Register analyzer for 22/tcp. 2015-03-25 12:37:04 -04:00
Vlad Grigorescu
e6aee5e159 SSH: Add 22/tcp to likely_server_ports 2015-03-24 22:34:45 -04:00
Robin Sommer
99c50251d4 Merge remote-tracking branch 'origin/topic/johanna/dtls'
* origin/topic/johanna/dtls:
  a few more small script-level fixes
  update test baselines
  add a simple leak test for dtls
  add signature for dtls client hello
  Make the plugin structure more... legal.
  Only force logging of SSL if it actually was the SSL analyzer that failed.
  DTLS working.
  Implement correct parsing of TLS record fragmentation.
  Make handshake analyzer flow-based. This means we can feed data to it in chunks, which makes dealing with fragmentation a little bit more convenient.
  When setting the SSL analyzer to fail, also stop processing data that already has been delivered to the analyzer, not just future data.
  First step for a DTLS analyzer.

BIT-1347 #merged

Conflicts:
	scripts/base/protocols/ssl/main.bro
	testing/btest/Baseline/plugins.hooks/output
2015-03-23 12:40:38 -07:00
Robin Sommer
1d49ec63f8 Merge remote-tracking branch 'origin/topic/jsiwek/bit-1324'
I added the $path to the create_stream() calls inside doc/ as well.

* origin/topic/jsiwek/bit-1324:
  Allow logging filters to inherit default path from stream.

BIT-1324: #merged
2015-03-23 10:35:30 -07:00
Robin Sommer
9999bce142 Merge remote-tracking branch 'origin/topic/jsiwek/bit-788'
* origin/topic/jsiwek/bit-788:
  BIT-788: use DNS QR field to better identify flow direction.

BIT-788 #merged
2015-03-23 10:08:02 -07:00
Vlad Grigorescu
a119247dd3 SSH: Ignore encrypted packets by default. 2015-03-19 21:43:44 -04:00
Jon Siwek
186e67ec1d Allow logging filters to inherit default path from stream.
This allows the path for the default filter to be specified explicitly
when creating a stream and reduces the need to rely on the default path
function to magically supply the path.

The default path function is now only used if, when a filter is added to
a stream, it has neither a path nor a path function already.

Adapted the existing Log::create_stream calls to explicitly specify a
path value.

Addresses BIT-1324
2015-03-19 14:49:55 -05:00
Jon Siwek
4c00729104 Tune parameters related to TCP initial window.
Increase default values of "tcp_max_above_hole_without_any_acks" and
"tcp_max_initial_window" from 4096 to 16384 bytes.

BIT-1255 #close
2015-03-19 13:23:55 -05:00
Jon Siwek
8efaae96cd BIT-788: use DNS QR field to better identify flow direction. 2015-03-19 11:53:40 -05:00
Johanna Amann
443106dbdb a few more small script-level fixes
Sorry, forgot to commit these.
2015-03-18 13:26:46 -07:00
Johanna Amann
28e6aa9561 Merge remote-tracking branch 'origin/master' into topic/johanna/dtls 2015-03-18 12:25:39 -07:00
Johanna Amann
58ed2eb9ae add signature for dtls client hello 2015-03-18 11:58:46 -07:00
Vlad Grigorescu
61c94d1809 SSH: Added some more events for SSH2 2015-03-18 12:52:46 -04:00
Vlad Grigorescu
092a78d14b Merge remote-tracking branch 'origin/master' into topic/vladg/ssh 2015-03-17 12:36:30 -04:00
Vlad Grigorescu
0cffee7694 SSH: Intel framework integration (PUBKEY_HASH) 2015-03-17 12:33:09 -04:00
Vlad Grigorescu
8218461d35 Update SSH policy scripts with new events. 2015-03-16 13:50:43 -04:00
Vlad Grigorescu
370f4f2179 SSH: Add documentation 2015-03-16 13:32:13 -04:00
Jon Siwek
0b957cbe75 Include timestamp in default extracted file names.
And add a policy script to extract all files.

BIT-1335 #close
2015-03-13 14:25:30 -05:00
Johanna Amann
88beb31270 Only force logging of SSL if it actually was the SSL analyzer that
failed.
2015-03-12 16:10:26 -07:00
Johanna Amann
991e4f5dc3 DTLS working.
The only thing that is missing is a signature to detect the protocol (it
has no well-known port).

Reassembly is kind of fidgety - at the moment we only support
re-assembling one simultaneous message per direction (which looking at
our test-traffic might not be a problem). And I am not quite sure if I
got all cases correct...

But - it works :)
2015-03-12 15:46:17 -07:00
Johanna Amann
038fbf9b9e First step for a DTLS analyzer.
This commit mostly does a lot of refactoring of the current SSL
analyzer, which is split into several parts.

The handshake protocol is completely taken out of the SSL analyzer and
was refactored into its own analyzer (called tls-handshake-analyzer).
This will also (finally) make it possible to deal with TLS record
fragmentation.

Apart from that, the parts of the SSL analyzer that are common to DTLS
were split into their own pac files. Both the SSL analyzer and the (very
basic, mostly nonfunctional) DTLS analyzer use their own pac files and
those shared pac files.

All SSL tests still pass after refactoring so I hope I did not break
anything too badly.

At the moment, we have two different modules in one directory and I
guess the way I am doing this might be an abuse of the system. It seems
to work though...
2015-03-11 15:07:13 -07:00
Vlad Grigorescu
3ad6b3004b SSH: Use the compression_algorithms const in another place. 2015-03-10 11:57:12 -04:00
Vlad Grigorescu
d9b4693240 Some cleanup and refactoring on SSH main.bro.
Specifically, an overhaul of how the algorithm negotiation is
calculated, to simplify a lot of the code.
2015-03-09 16:04:35 -04:00
Jon Siwek
9e53722b57 Rename comm/ directories to broker/ 2015-03-05 17:02:25 -06:00
Jon Siwek
fa08083a92 Rename broker-related namespaces.
c++ namespace "comm" -> bro_broker
script module "Comm" -> BrokerComm
script module "Store" -> BrokerStore
2015-03-05 16:20:51 -06:00
Seth Hall
ffdf2a46d7 Fixes tests in RDP branch.
- Re-enable MySQL.  It had accidentally been disabled.
2015-03-05 16:08:18 -05:00
Seth Hall
4737b235b6 Merge remote-tracking branch 'origin/master' into topic/seth/rdp 2015-03-05 14:38:34 -05:00
Seth Hall
276e072e6e A few more changes to handling encryption in RDP. 2015-03-05 13:38:54 -05:00
Seth Hall
b92a68e2bd Adds some comments and fixes a broxygen warning. 2015-03-05 11:37:37 -05:00
Seth Hall
f45e057779 Another big RDP update.
- New fields for certificate type, number of certificates,
   if certificates are permanent on the server, and the selected
   security protocol.
 - Fixed some issues with X.509 certificate handling over RDP
   (the event handler wasn't sufficiently constrained).
 - Better detection of and transition into encrypted mode.  No more
   binpac parse failures from the test traces anymore!
 - Some event name clean up and new events.
 - X.509 Certificate chains are now handled correctly (was only grabbing
   a single certificate).
2015-03-05 01:15:12 -05:00
Robin Sommer
e5adc768cc Merge branch 'stats-bytes-recvd' of https://github.com/msmiley/bro 2015-03-04 13:16:19 -08:00
Seth Hall
d361deb975 Merge remote-tracking branch 'origin/master' into topic/seth/rdp 2015-03-04 13:12:45 -05:00
Seth Hall
bbedb73a45 Huge updates to the RDP analyzer from Josh Liburdi.
- More data pulled into scriptland.
  - Logs expanded with client screen resolution and desired color depth.
  - Values in UTF-16 on the wire are converted to UTF-8 before being
    sent to scriptland.
  - If the RDP turns into SSL records, we now pass data that appears
    to be SSL to the PIA analyzer.
  - If RDP uses native encryption with X.509 certs we pass those
    certs to the files framework and the base scripts pass them forward
    to the X.509 analyzer.
  - Lots of cleanup and adjustment to fit the documented protocol
    a bit better.
  - Cleaned up the DPD signatures.
  - Moved to flowunit instead of datagram.
  - Added tests.
2015-03-04 13:12:03 -05:00
Johanna Amann
e48c6ccc4a Do not log common name by default (it is most interesting for scripts)
and add a test case.
2015-03-03 16:38:25 -08:00
Johanna Amann
252d57fd2c extract most specific common name from certificates 2015-03-03 16:09:54 -08:00