Commit graph

1194 commits

Author SHA1 Message Date
Jon Siwek
dcbd0819a6 Updates related to SSH analysis.
- Some scripts used wrong SSH module/namespace scoping on events.
- Fix outdated notice documentation related to SSH password guessing.
- Add a unit test for SSH pasword guessing notice.
2015-03-30 11:30:48 -05:00
Johanna Amann
b9e2b7bef9 Log::write in signature framework was missing ts
(Andrew Benson/Michel Laterman)

BIT-1354 #close
2015-03-25 12:01:09 -07:00
Johanna Amann
f79b5adc08 Merge remote-tracking branch 'origin/topic/vladg/ssh'
I replaced a few strcmps with either calls to std::str.compare
or with the == operator of BroString.

Also changed two of the input framework tests that did not pass
anymore after the merge. The new SSH analyzer no longer loads the
scripts that let network time run, hence those tests failed because
updates were not propagated from the threads (that took a while
to find.)

* origin/topic/vladg/ssh: (25 commits)
  SSH: Register analyzer for 22/tcp.
  SSH: Add 22/tcp to likely_server_ports
  SSH: Ignore encrypted packets by default.
  SSH: Fix some edge-cases which created BinPAC exceptions
  SSH: Add memleak btest
  SSH: Update baselines
  SSH: Added some more events for SSH2
  SSH: Intel framework integration (PUBKEY_HASH)
  Update baselines for new SSH analyzer.
  Update SSH policy scripts with new events.
  SSH: Add documentation
  Refactoring ssh-protocol.pac:
  SSH: Use the compression_algorithms const in another place.
  Some cleanup and refactoring on SSH main.bro.
  SSH: A bit of code cleanup.
  Move SSH constants to consts.pac
  SSH: Cleanup code style.
  SSH: Fix some memleaks.
  Refactored the SSH analyzer. Added supported for algorithm detection and more key exchange message types.
  Add host key support for SSH1.
  Add support for SSH1
  Move SSH analyzer to new plugin architecture.
  ...

Conflicts:
  scripts/base/protocols/ssh/main.bro
  testing/btest/Baseline/core.print-bpf-filters/output2
  testing/btest/Baseline/plugins.hooks/output

BIT-1344: #merged
2015-03-25 11:04:26 -07:00
Vlad Grigorescu
75d7f3414f SSH: Register analyzer for 22/tcp. 2015-03-25 12:37:04 -04:00
Vlad Grigorescu
e6aee5e159 SSH: Add 22/tcp to likely_server_ports 2015-03-24 22:34:45 -04:00
Robin Sommer
99c50251d4 Merge remote-tracking branch 'origin/topic/johanna/dtls'
* origin/topic/johanna/dtls:
  a few more small script-level fixes
  update test baselines
  add a simple leak test for dtls
  add signature for dtls client hello
  Make the plugin structure more... legal.
  Only force logging of SSL if it actually was the SSL analyzer that failed.
  DTLS working.
  Implement correct parsing of TLS record fragmentation.
  Make handshake analyzer flow-based. This means we can feed data to it in chunks, which makes dealing with fragmentation a little bit more convenient.
  When setting the SSL analyzer to fail, also stop processing data that already has been delivered to the analyzer, not just future data.
  First step for a DTLS analyzer.

BIT-1347 #merged

Conflicts:
	scripts/base/protocols/ssl/main.bro
	testing/btest/Baseline/plugins.hooks/output
2015-03-23 12:40:38 -07:00
Robin Sommer
1d49ec63f8 Merge remote-tracking branch 'origin/topic/jsiwek/bit-1324'
I added the $path to the create_stream() calls inside doc/ as well.

* origin/topic/jsiwek/bit-1324:
  Allow logging filters to inherit default path from stream.

BIT-1324: #merged
2015-03-23 10:35:30 -07:00
Robin Sommer
9999bce142 Merge remote-tracking branch 'origin/topic/jsiwek/bit-788'
* origin/topic/jsiwek/bit-788:
  BIT-788: use DNS QR field to better identify flow direction.

BIT-788 #merged
2015-03-23 10:08:02 -07:00
Vlad Grigorescu
a119247dd3 SSH: Ignore encrypted packets by default. 2015-03-19 21:43:44 -04:00
Jon Siwek
186e67ec1d Allow logging filters to inherit default path from stream.
This allows the path for the default filter to be specified explicitly
when creating a stream and reduces the need to rely on the default path
function to magically supply the path.

The default path function is now only used if, when a filter is added to
a stream, it has neither a path nor a path function already.

Adapted the existing Log::create_stream calls to explicitly specify a
path value.

Addresses BIT-1324
2015-03-19 14:49:55 -05:00
Jon Siwek
4c00729104 Tune parameters related to TCP initial window.
Increase default values of "tcp_max_above_hole_without_any_acks" and
"tcp_max_initial_window" from 4096 to 16384 bytes.

BIT-1255 #close
2015-03-19 13:23:55 -05:00
Jon Siwek
8efaae96cd BIT-788: use DNS QR field to better identify flow direction. 2015-03-19 11:53:40 -05:00
Johanna Amann
443106dbdb a few more small script-level fixes
Sorry, forgot to commit these.
2015-03-18 13:26:46 -07:00
Johanna Amann
28e6aa9561 Merge remote-tracking branch 'origin/master' into topic/johanna/dtls 2015-03-18 12:25:39 -07:00
Johanna Amann
58ed2eb9ae add signature for dtls client hello 2015-03-18 11:58:46 -07:00
Vlad Grigorescu
61c94d1809 SSH: Added some more events for SSH2 2015-03-18 12:52:46 -04:00
Vlad Grigorescu
092a78d14b Merge remote-tracking branch 'origin/master' into topic/vladg/ssh 2015-03-17 12:36:30 -04:00
Vlad Grigorescu
0cffee7694 SSH: Intel framework integration (PUBKEY_HASH) 2015-03-17 12:33:09 -04:00
Vlad Grigorescu
8218461d35 Update SSH policy scripts with new events. 2015-03-16 13:50:43 -04:00
Vlad Grigorescu
370f4f2179 SSH: Add documentation 2015-03-16 13:32:13 -04:00
Jon Siwek
0b957cbe75 Include timestamp in default extracted file names.
And add a policy script to extract all files.

BIT-1335 #close
2015-03-13 14:25:30 -05:00
Johanna Amann
88beb31270 Only force logging of SSL if it actually was the SSL analyzer that
failed.
2015-03-12 16:10:26 -07:00
Johanna Amann
991e4f5dc3 DTLS working.
The only thing that is missing is a signature to detect the protocol (it
has no well-known port).

Reassembly is kind of fidgety - at the moment we only support
re-assembling one simultaneous message per direction (which looking at
our test-traffic might not be a problem). And I am not quite sure if I
got all cases correct...

But - it works :)
2015-03-12 15:46:17 -07:00
Johanna Amann
038fbf9b9e First step for a DTLS analyzer.
This commit mostly does a lot of refactoring of the current SSL
analyzer, which is split into several parts.

The handshake protocol is completely taken out of the SSL analyzer and
was refactored into its own analyzer (called tls-handshake-analyzer).
This will also (finally) make it possible to deal with TLS record
fragmentation.

Apart from that, the parts of the SSL analyzer that are common to DTLS
were split into their own pac files. Both the SSL analyzer and the (very
basic, mostly nonfunctional) DTLS analyzer use their own pac files and
those shared pac files.

All SSL tests still pass after refactoring so I hope I did not break
anything too badly.

At the moment, we have two different modules in one directory and I
guess the way I am doing this might be an abuse of the system. It seems
to work though...
2015-03-11 15:07:13 -07:00
Vlad Grigorescu
3ad6b3004b SSH: Use the compression_algorithms const in another place. 2015-03-10 11:57:12 -04:00
Vlad Grigorescu
d9b4693240 Some cleanup and refactoring on SSH main.bro.
Specifically, an overhaul of how the algorithm negotiation is
calculated, to simplify a lot of the code.
2015-03-09 16:04:35 -04:00
Jon Siwek
9e53722b57 Rename comm/ directories to broker/ 2015-03-05 17:02:25 -06:00
Jon Siwek
fa08083a92 Rename broker-related namespaces.
c++ namespace "comm" -> bro_broker
script module "Comm" -> BrokerComm
script module "Store" -> BrokerStore
2015-03-05 16:20:51 -06:00
Robin Sommer
e5adc768cc Merge branch 'stats-bytes-recvd' of https://github.com/msmiley/bro 2015-03-04 13:16:19 -08:00
Johanna Amann
e48c6ccc4a Do not log common name by default (it is most interesting for scripts)
and add a test case.
2015-03-03 16:38:25 -08:00
Johanna Amann
252d57fd2c extract most specific common name from certificates 2015-03-03 16:09:54 -08:00
Robin Sommer
dfc88094ab Merge remote-tracking branch 'origin/topic/jsiwek/broker'
* origin/topic/jsiwek/broker: (34 commits)
  Update broker submodule.
  Update broker submodule.
  broker integration: add missing baselines for doc tests
  broker integration: add prof.log statistics
  broker integration: add high-level usage documentation
  broker integration: add API documentation (broxygen/doxygen)
  broker integration: fix memory leak, add leak tests
  Update broker submodule.
  Improve comm tests.
  Fix gcc compile warnings.
  broker integration: fix unit tests to work when broker is not enabled.
  Add --enable-c++11 configure flag.
  broker integration: add (un)publish/(un)advertise functions
  broker integration: add knobs to set auto publish/advertise behavior
  broker integration: move listen port for unit tests to a btest variable
  broker integration: add events for incoming connection status updates
  broker integration: adapt to change in expiration_time
  Update coverage unit test baselines.
  broker integration: add Comm::enable function
  broker integration: process debug/diagnostic reports from broker
  ...

Conflicts:
	cmake
	testing/btest/Baseline/plugins.hooks/output
2015-03-02 17:10:15 -08:00
Mike Smiley
3877b3e34b add bytes recvd to Stats and stats.bro
use libpcap packet hdr.len to count bytes
2015-02-23 21:27:28 -05:00
Mike Smiley
a1d49e791e add local_resp to Conn Info
allow user to differentiate between local -> local and local -> remote
connections
2015-02-18 20:41:40 -05:00
Jon Siwek
b06d82cced broker integration: add API documentation (broxygen/doxygen)
Also changed asynchronous data store query code a bit; trying to make
memory management and handling of corner cases a bit clearer (former
maybe could still be better, but I need to lookup queries by memory
address to associate response cookies to them, and so wrapping pointers
kind of just gets in the way).
2015-02-17 10:50:57 -06:00
Jon Siwek
e95116ba85 Merge branch 'master' into topic/jsiwek/broker 2015-02-16 10:00:17 -06:00
Jon Siwek
212368b245 Merge remote-tracking branch 'origin/topic/jsiwek/socks-authentication'
* origin/topic/jsiwek/socks-authentication:
  Refactor SOCKS5 user/pass authentication support.
  Update the SOCKS analyzer to support user/pass login.

BIT-1011 #merged
2015-02-13 09:15:50 -06:00
Jon Siwek
961fd06cad Refactor SOCKS5 user/pass authentication support.
- Rename event "socks_login_userpass" to "socks_login_userpass_request"
- Rename event "socks_login_reply" to "socks_login_userpass_reply"
- Split unsupported authN weird into 2 types: method vs. version

Addresses BIT-1011
2015-02-12 17:06:38 -06:00
Jon Siwek
ebc9407a2b broker integration: add knobs to set auto publish/advertise behavior 2015-02-09 16:26:31 -06:00
Robin Sommer
23b9705a7b Fixing analyzer tag types for some Files::* functions. 2015-02-08 18:23:22 -08:00
Vlad Grigorescu
fc721d2d25 Merge remote-tracking branch 'origin/master' into topic/vladg/ssh 2015-02-06 18:58:38 -05:00
Seth Hall
9592f64225 Update the SOCKS analyzer to support user/pass login.
- This addresses BIT-1011
 - Add a new field to socks.log; "password".
 - Two new events; socks_login_userpass and socks_login_reply.
 - One new weird for unsupported authentication method.
 - A new test for authenticated socks traffic.
 - Credit to Nicolas Retrain for the initial patch.  Thanks!
2015-02-05 12:44:10 -05:00
Jon Siwek
6b115c6999 Merge branch 'master' into topic/jsiwek/broker 2015-02-02 11:45:21 -06:00
Jon Siwek
9875f5d3eb broker integration: add distributed data store api
But haven't done the full gamut of testing on it yet.
2015-01-30 14:39:16 -06:00
Jon Siwek
d2ea87735a broker integration: add bifs to inspect/manipulate broker data
i.e. script-layer functions to convert between bro values and broker
values; mostly for use w/ Bro's data store interface (coming soon).
2015-01-29 10:42:48 -06:00
Jon Siwek
23f04835c6 Deprecate split* family of BIFs.
These functions are now deprecated in favor of alternative versions that
return a vector of strings rather than a table of strings.

Deprecated functions:

- split: use split_string instead.
- split1: use split_string1 instead.
- split_all: use split_string_all instead.
- split_n: use split_string_n instead.
- cat_string_array: see join_string_vec instead.
- cat_string_array_n: see join_string_vec instead.
- join_string_array: see join_string_vec instead.
- sort_string_array: use sort instead instead.
- find_ip_addresses: use extract_ip_addresses instead.

Changed functions:

- has_valid_octets: uses a string_vec parameter instead of string_array.

Addresses BIT-924, BIT-757.
2015-01-21 15:34:42 -06:00
Jon Siwek
7e563b7275 broker integration: add remote events 2015-01-15 15:45:08 -06:00
Robin Sommer
c67c7c6196 Merge remote-tracking branch 'origin/fastpath'
* origin/fastpath:
  small changes to ec curve names in a newer draft
2015-01-15 09:44:41 -08:00
Robin Sommer
0024881f3d Merge remote-tracking branch 'origin/topic/vladg/mysql'
* origin/topic/vladg/mysql:
  Updating MySQL with Robin's suggestions:

BIT-1285 #merged
2015-01-14 14:21:00 -08:00
Robin Sommer
41ff1c4cd0 Merge remote-tracking branch 'origin/topic/robin/dnp3-merge-v4'
* origin/topic/robin/dnp3-merge-v4:
  add test trace in which DNP3 packets are over UDP; update test scripts and baseline results
  A bit more DNP3 tweaking.
  remove redundnt codes; find a way to use the analyzer function, such as Weird; fix a small bug in ProcessData function in DNP3.cc; passed the test
  Renameing the DNP3 TCP analyzer
  quickly fix another bug; adding missing field of the declaration of dnp3_request_application_header and dnp3_response_application_header
  Removing the debug printf in DNP3.cc
  fixed the bug of deciding the size of object 1 varition 1 in DNP3
  Fix some things in DNP3 UDP analyzer.
  changed a bug, but still not working
  modify DNP3.cc and DNP3.h to add DNP3_UDP_Analyzer; binpac unchanged

BIT-1231 #merged
2015-01-14 13:25:42 -08:00