Commit graph

2138 commits

Author SHA1 Message Date
Johanna Amann
1fb7f5121e Merge remote-tracking branch 'origin/master' into topic/johanna/openflow 2015-04-20 11:27:24 -07:00
Vlad Grigorescu
d0e4d17f31 Tweak the PE OS versions based on real-world traffic. 2015-04-20 12:49:42 -04:00
Vlad Grigorescu
928f870f58 Update pe/main.bro to user register_for_mime_types, ensuring it will also work with the upcoming Files framework changes. 2015-04-20 11:54:34 -04:00
Seth Hall
ed375167c8 File API updates complete.
Addresses BIT-1368.
2015-04-20 10:46:48 -04:00
Seth Hall
038e4c24f6 Merge remote-tracking branch 'origin/topic/jsiwek/bit-1368' into topic/seth/more-file-type-ident-fixes
Conflicts:
	src/file_analysis/File.cc
	testing/btest/Baseline/plugins.hooks/output
2015-04-20 09:36:40 -04:00
Seth Hall
faabe8a5e3 Fixes for file type identification.
- Backed out eTag changes.  The real world is more complicated
   than just using eTags to identify the same file.
 - A bit of code simplication in the http base scripts.
 - Test updates (more existing small problems were identified!).
 -
2015-04-20 09:34:09 -04:00
Vlad Grigorescu
9683d8419f Update baselines. 2015-04-19 23:09:19 -04:00
Vlad Grigorescu
d5ea6b8457 Add SIP btests. 2015-04-19 22:25:37 -04:00
Vlad Grigorescu
71230fec81 Update baselines. 2015-04-19 21:16:35 -04:00
Vlad Grigorescu
0199ac5ece Add a btest for the PE analyzer. 2015-04-19 20:27:24 -04:00
Vlad Grigorescu
93b84463f5 Add a PE memleak test, and fix a memleak. 2015-04-19 20:22:42 -04:00
Vlad Grigorescu
5cf8001cb0 Add Kerberos memleak btest. 2015-04-17 21:44:59 -04:00
Vlad Grigorescu
569f43d625 Add Kerberos analyzer btest. 2015-04-17 21:28:22 -04:00
Vlad Grigorescu
3d13e404b1 Update baselines for Kerberos analyzer. 2015-04-17 21:27:48 -04:00
Robin Sommer
d6dda8e984 Merge remote-tracking branch 'origin/topic/johanna/bit-985'
* origin/topic/johanna/bit-985:
  add seeking functionality to raw reader.

BIT-985 #merged
2015-04-17 13:13:45 -07:00
Johanna Amann
cbba73ab12 add seeking functionality to raw reader.
one can now add an option "offset" to the config map. Positive offsets
are interpreted to be from the beginning of the file, negative from the
end of the file (-1 is end of file).

Only works for raw reader in streaming or manual mode. Does not work
with executables.

Addresses BIT-985
2015-04-17 11:19:12 -07:00
Johanna Amann
b44b725d59 Wrap threshold stuff up - fix two small bugs and update baselines. 2015-04-17 09:59:34 -07:00
Johanna Amann
86c54c3dbd Add high level api for thresholding that holds lists of thresholds
and raises an event for each threshold exactly once.
2015-04-17 06:57:58 -07:00
Johanna Amann
f7edf70882 Allow setting packet and byte thresholds for connections.
This extends the ConnSize analyzer to be able to raise events when each
direction of a connection crosses a certain amount of bytes or packets.

Thresholds are set using
set_conn_bytes_threshold(c$id, [num-bytes], [direction]);
and
set_conn_packets_threshold(c$id, [num-packets], [direction]);
respectively.

They raise the event
event conn_bytes_threshold_crossed(c: connection, threshold: count, is_orig: bool)
and
event conn_packets_threshold_crossed(c: connection, threshold: count, is_orig: bool)
respectively.

Current thresholds can be examined using
get_conn_bytes_threshold and get_conn_packets_threshold

Currently only one threshold can be set per connection.

This also fixes a bug where child packet analyzers of the TCP analyzer
where not found using FindChild.
2015-04-17 06:57:51 -07:00
Robin Sommer
7225221c2a In bifs, change ODesc objects to have RAW_STYLE.
This prevents double-escaping with the logging changes of BIT-1333.
2015-04-16 21:50:50 -07:00
Johanna Amann
e21238d454 add a few more flow_mod options and the option to check via a predicate
if a module wants to be responsible for a certain rule...
2015-04-16 15:44:46 -07:00
Johanna Amann
d876c044df Add high level api for thresholding that holds lists of thresholds
and raises an event for each threshold exactly once.
2015-04-16 15:36:48 -07:00
Johanna Amann
9290718bd6 Allow setting packet and byte thresholds for connections.
This extends the ConnSize analyzer to be able to raise events when each
direction of a connection crosses a certain amount of bytes or packets.

Thresholds are set using
set_conn_bytes_threshold(c$id, [num-bytes], [direction]);
and
set_conn_packets_threshold(c$id, [num-packets], [direction]);
respectively.

They raise the event
event conn_bytes_threshold_crossed(c: connection, threshold: count, is_orig: bool)
and
event conn_packets_threshold_crossed(c: connection, threshold: count, is_orig: bool)
respectively.

Current thresholds can be examined using
get_conn_bytes_threshold and get_conn_packets_threshold

Currently only one threshold can be set per connection.

This also fixes a bug where child packet analyzers of the TCP analyzer
where not found using FindChild.
2015-04-16 13:02:08 -07:00
Robin Sommer
a7b7397f02 Merge branch 'master' of git.bro.org:bro 2015-04-16 08:14:10 -07:00
Jon Siwek
ceebad38a4 Fix the to_count function to use strtoull versus strtoll. 2015-04-16 09:56:18 -05:00
Robin Sommer
7344052b50 Changing what's escaped when printing.
With this patch the model is:

    - "print" cleans the data so that non-printable characters get
      escaped. This is not necessarily reversible.

    - to print in a reversible way, one can go through
      escape_string(); this escapes backslashes as well to make the
      decoding non-ambigious.

    - Logging always escapes similar to escape_string(), making it
      reversible.

Compared to master, we also change the escaping as follows:

    - We now only escape with "\xXX", no more "^X" or "\0". Exception:
      backslashes.

    - We escape backlashes as "\\".

    - There's no "alternative" output style anymore, i.e., fmt() '%A'
      qualifier is gone.

Baselines in testing/btest are updated, external tests not yet.

Addresses BIT-1333.
2015-04-15 16:59:50 -07:00
Johanna Amann
236a895418 Update mozilla CA list.
This also patches a few tests to contain certificates that were removed.
Furthermore, we include the old CA file with the external tests and load
it automatically. Those traces are kind of old now, more and more of the
CAs in them are no longer valid and it does not really make sense to
update them on each change...
2015-04-14 16:40:41 -07:00
Johanna Amann
7d7578146f Add basic OpenFlow plugin for Pacf.
This also changes a few types in pacf and adds a few needed bits and
pieces to the OpenFlow framework.

And - it even has a testcase...
2015-04-14 15:24:22 -07:00
Johanna Amann
c42fbdab12 move openflow to use subnets instead of addr for matches. 2015-04-14 15:00:49 -07:00
Johanna Amann
0e7ebffacf add bif function to test if a subnet revers to v4 or v6.
If there already was a way to check this, I completely missed it...
2015-04-14 14:51:52 -07:00
Johanna Amann
fd07b0bee9 mainly add a small test to the pacf framework that uses the debug
plugin for shunt / drop rule additions...
2015-04-13 16:45:31 -07:00
Johanna Amann
21b78b7d92 add really simple log output plugin for openflow. 2015-04-13 12:55:38 -07:00
Johanna Amann
fe9ff46cc5 Merge remote-tracking branch 'origin/topic/jsiwek/bit-1367'
Bit-1367 #close
2015-04-11 11:01:18 -07:00
Jon Siwek
a55ce01ef3 API changes to file analysis mime type detection.
Removed "file_mime_type" and "file_mime_types" event, replacing them
with a new event called "file_metadata_inferred".  It has a record
argument of type "inferred_file_metadata", which contains the mime type
information that the earlier events used to supply.  The idea here is
that future extensions to the record with new metadata will be less
likely to break user code than the alternatives (adding new events or
new event parameters).

Addresses BIT-1368.
2015-04-10 16:31:29 -05:00
Johanna Amann
46058d0b02 a few small fixes to openflow
*rename module from Openflow to OpenFlow
*add match_conn function to convert conn_id to openflow match
*add a few things back into the openflow records like... table_id
*and - a test
2015-04-10 11:21:55 -07:00
Seth Hall
49926ad7bf Merge remote-tracking branch 'origin/master' into topic/seth/more-file-type-ident-fixes 2015-04-09 23:58:52 -04:00
Robin Sommer
a0bb139f22 Sorting test output for stability. 2015-04-09 15:22:59 -07:00
Robin Sommer
ea7bc11aa1 Merge remote-tracking branch 'origin/topic/jsiwek/bit-844'
BIT-844 #merged

* origin/topic/jsiwek/bit-844:
  Remove stale signature benchmarking code (-L command-line option).
  BIT-844: fix UDP payload signatures to match packet-wise
2015-04-09 14:52:44 -07:00
Robin Sommer
a4edc0df08 Fixing input readers' component type.
A reader is a reader, not a writer.
2015-04-09 12:07:17 -07:00
Seth Hall
89d66af792 Fix an issue with packet loss in http file reporting.
The HTTP analyzer was propogating Gaps to the files framework even
in the case of a packet drop occurring immediately after the headers
are completed in an HTTP response when the response content length
was declared to be zero (no file started, so no loss).

Includes passing test.
2015-04-08 13:39:42 -04:00
Jon Siwek
56a7bf7936 BIT-844: fix UDP payload signatures to match packet-wise 2015-04-06 15:22:26 -05:00
Jon Siwek
57501c6069 BIT-1367: improve coercion of anonymous records in set constructor.
Error messages for set constructors that fail the type check may also be
more verbose than before and point out specifically the suspect types.
2015-04-06 12:14:59 -05:00
Jon Siwek
1a42296389 Increase some unit test timeouts. 2015-04-06 10:17:52 -05:00
Jon Siwek
32ae94de9a Merge remote-tracking branch 'origin/topic/seth/rdp'
* origin/topic/seth/rdp: (31 commits)
  Improved transition into SSL/TLS from RDP.
  Fixes tests in RDP branch.
  add a special case to the X509 code that deals with RDP certificates.
  A few more changes to handling encryption in RDP.
  Adds some comments and fixes a broxygen warning.
  Fixes another optional part of an RDP unit.
  Support RDP negotiation requests optionally and support zero length cookies.
  Changed UTF-16 to UTF-8 conversion to be more lenient.
  Fixed an issue with parse failure on an optional field.
  Removing a stray printf from RDP analyzer.
  Another big RDP update.
  New script to add a field to rdp.log when the connection is upgraded to SSL.
  Huge updates to the RDP analyzer from Josh Liburdi.
  FreeRDP test trace showing SSL encryption -- RDP analyzer does not currently handle this and SSL analyzer does not identify it either
  Wireshark test trace for native encryption -- generates a binpac error
  Delete RDP-004.pcap
  Delete nla_win7_win2k8r2.pcap
  Update dpd.sig
  Fixed typo
  Added check for connection existence
  ...

BIT-1340 #merged
2015-03-30 17:10:04 -05:00
Jon Siwek
d837d4a0be Increase timeout duration in some broker tests. 2015-03-30 12:02:48 -05:00
Jon Siwek
dcbd0819a6 Updates related to SSH analysis.
- Some scripts used wrong SSH module/namespace scoping on events.
- Fix outdated notice documentation related to SSH password guessing.
- Add a unit test for SSH pasword guessing notice.
2015-03-30 11:30:48 -05:00
Jon Siwek
97962d25f2 Fix outdated documentation unit tests. 2015-03-30 11:03:21 -05:00
Jon Siwek
fc190eb9f1 Add a canonifier to a unit test's output. 2015-03-30 10:23:52 -05:00
Johanna Amann
f79b5adc08 Merge remote-tracking branch 'origin/topic/vladg/ssh'
I replaced a few strcmps with either calls to std::str.compare
or with the == operator of BroString.

Also changed two of the input framework tests that did not pass
anymore after the merge. The new SSH analyzer no longer loads the
scripts that let network time run, hence those tests failed because
updates were not propagated from the threads (that took a while
to find.)

* origin/topic/vladg/ssh: (25 commits)
  SSH: Register analyzer for 22/tcp.
  SSH: Add 22/tcp to likely_server_ports
  SSH: Ignore encrypted packets by default.
  SSH: Fix some edge-cases which created BinPAC exceptions
  SSH: Add memleak btest
  SSH: Update baselines
  SSH: Added some more events for SSH2
  SSH: Intel framework integration (PUBKEY_HASH)
  Update baselines for new SSH analyzer.
  Update SSH policy scripts with new events.
  SSH: Add documentation
  Refactoring ssh-protocol.pac:
  SSH: Use the compression_algorithms const in another place.
  Some cleanup and refactoring on SSH main.bro.
  SSH: A bit of code cleanup.
  Move SSH constants to consts.pac
  SSH: Cleanup code style.
  SSH: Fix some memleaks.
  Refactored the SSH analyzer. Added supported for algorithm detection and more key exchange message types.
  Add host key support for SSH1.
  Add support for SSH1
  Move SSH analyzer to new plugin architecture.
  ...

Conflicts:
  scripts/base/protocols/ssh/main.bro
  testing/btest/Baseline/core.print-bpf-filters/output2
  testing/btest/Baseline/plugins.hooks/output

BIT-1344: #merged
2015-03-25 11:04:26 -07:00
Robin Sommer
d2998b6f64 Merge branch 'master' of git.bro.org:bro
Conflicts:
	CHANGES
	VERSION
2015-03-23 12:50:30 -07:00