Commit graph

1002 commits

Author SHA1 Message Date
Robin Sommer
7e7002329c Merge remote-tracking branch 'origin/topic/seth/smb' into topic/robin/smb-merge 2016-07-07 12:07:53 -07:00
Seth Hall
d1b1a560c0 Add a DCE-RPC test. 2016-07-07 14:17:25 -04:00
Robin Sommer
781fdfa5ba Merge remote-tracking branch 'origin/topic/seth/smb' into topic/robin/smb-merge 2016-07-07 10:41:31 -07:00
Robin Sommer
721693425f Escape the empty indicator in logs if it occurs literally as a field's
actual content.

Addresses BIT-931.
2016-07-05 16:34:24 -07:00
Seth Hall
514dfc3479 Merge remote-tracking branch 'origin/master' into topic/seth/smb
# Conflicts:
#	testing/btest/Baseline/plugins.hooks/output
#	testing/btest/Baseline/scripts.policy.misc.dump-events/all-events.log
#	testing/btest/Baseline/scripts.policy.misc.dump-events/smtp-events.log
2016-06-29 09:43:31 -04:00
Jan Grashoefer
df5d9adfb4 Fixed insertion of nested subnets.
When inserting, existance of the given subnet is checked using exact
matching instead of longest prefix matching. Before, inserting a subnet
would have updated the subnet item, which is the longest prefix of the
inserted subnet, if present.
2016-06-22 21:14:06 +02:00
Johanna Amann
6361a0d658 Merge remote-tracking branch 'origin/master' into topic/johanna/netcontrol-improvements 2016-06-21 12:42:06 -07:00
Seth Hall
8eb67a0a6c Update tests to match move of smb base scripts into policy/ 2016-06-20 16:22:57 -04:00
Johanna Amann
efab728876 Merge branch 'topic/jgras/input-default' of https://github.com/J-Gras/bro
* 'topic/jgras/input-default' of https://github.com/J-Gras/bro:
  Fixed &default values ignored by input framework.

BIT-1623 #merged
2016-06-15 17:32:50 -07:00
Jan Grashoefer
8d853872a4 Fixed &default values ignored by input framework.
Reading into a record, the input framework will no longer reset a
&default value in case there is no value to read.
2016-06-16 01:02:51 +02:00
Jan Grashoefer
5d340e669c Added expiration for intelligence items.
Expiration of intelligence items can be configured using
Intel::item_expiration. Expiration can be handled using the
Intel::item_expired hook.
2016-06-15 19:29:48 +02:00
Jan Grashoefer
7e0b5b4ba4 Merge branch 'master' into topic/jgras/intel-update 2016-06-15 19:26:26 +02:00
Seth Hall
90399db32d Additional test specifically for the HTTP filename handling. 2016-06-15 01:56:07 -04:00
Seth Hall
56a24bdef6 Merge remote-tracking branch 'origin/master' into topic/seth/smb
# Conflicts:
#	scripts/site/local.bro
2016-06-14 15:35:05 -04:00
Robin Sommer
58dea28504 Merge remote-tracking branch 'origin/topic/johanna/windows-newlines'
(Added a double-check that line isn't completely empty.)

BIT-1198 #merged

* origin/topic/johanna/windows-newlines:
  Ascii Input: Accept dos/windows newlines.
2016-06-06 18:14:21 -07:00
Johanna Amann
e8418ad5b0 Ascii Input: Accept dos/windows newlines.
The ascii reader now accepts \r\n newlines without complaining.
Furthermore, the reader was slightly rewritten in a more c++11-y way,
removing all raw pointers from the class.

Addresses BIT-1198
2016-06-01 11:21:44 -07:00
Johanna Amann
990836e868 NetControl: slightly update catch and release logging
Re-drops now contain the location of the original drop.
2016-05-31 11:52:42 -07:00
Robin Sommer
d195f1b047 Fixing FTP cwd getting overlue long.
Now storing them compressed.
2016-05-29 08:52:47 -07:00
Johanna Amann
2f74825785 NetControl: fix several small logging issues
forgotten messages are only logged on the manager (or standalone host)
now. Logs are not written by default anymore when Bro encounters traffic
that should have been blocked.
2016-05-27 08:51:06 -07:00
Johanna Amann
296c5611ec NetControl: more catch and release logging and cluster fix 2016-05-26 16:53:42 -07:00
Johanna Amann
9f7b23b179 NetControl: rewrite catch and release and small fixes.
This commit rewrites catch and release, fixing issues with it and making
it fully cluster capable. A dedicated netcontrol_catch_release.log is
also added.

This is not quite done yet; a few more log messages are missing. There
should hopefully not be many big issues left.
2016-05-23 16:16:21 -07:00
Robin Sommer
d86bf15dbf Do not use scientific notations when printing doubles in logs.
Closes BIT-1558.
2016-05-23 16:07:22 -07:00
Seth Hall
e2fb7591f4 Merge remote-tracking branch 'origin/master' into topic/seth/smb 2016-05-20 14:28:39 -04:00
Johanna Amann
6779325520 NetControl: find_rules_subnet works in cluster mode
This introduces two new events, NetControl::rule_new and
NetControl::rule_destroyed, which are raised when rules are first added
and then deleted from the internal state tracking.
2016-05-20 11:11:44 -07:00
Johanna Amann
52d694f3bd Merge remote-tracking branch 'origin/master' into topic/johanna/netcontrol-improvements 2016-05-19 16:17:07 -07:00
Robin Sommer
85213e6b55 Merge remote-tracking branch 'origin/topic/johanna/fix-analyzer-addition'
* origin/topic/johanna/fix-analyzer-addition:
  DTLS: Use magix constant from rfc5389 for stun detection.
  DTLS: Fix binpac bug with DTLSv1.2 client hellos
  Forgot to remove debug output.
  DTLS: Fix interaction with STUN
  Fix the way that child analyzers are added.

BIT-1604 #merged
2016-05-18 07:41:18 -07:00
Johanna Amann
39bdc397a0 DTLS: Fix interaction with STUN
Now the DTLS analyzer cleanly skips all STUN messages; no warnings
should be logged to dpd.log and parsing should work flawlessly with
intermixed STUN messages.
2016-05-17 16:36:46 -07:00
Johanna Amann
d091e9ca89 Fix duplication of new_connection_contents event
Addresses BIT-1602
2016-05-17 10:32:13 -07:00
Seth Hall
b28801ce95 Add unrolling separator & field name map to logging framework.
- When a log record is being "unrolled" (sub-records flattened
   out into a single record), it's now possible to choose the
   character/string to separate the outer name from the inner
   name.  This can be used to work around the problems
   with ElasticSearch 2.0 not supporting dots "." in field names.
   This value can be provided per-filter as well as a global
   default value.
 - Log fields can be renamed by providing a table per-filter
   (or a global default) to rename fields for any log writer.
   The name translation is performed after unrolling so the
   value in the field name table must match whatever is being
   used to separate field names.

   For example if the unrolling separator was set to "*":
	redef Log::default_unrolling_sep = "*";

   The field name map would need to reflect it:
	redef Log::default_field_name_map = {
		["id*orig_h"] = "src",
		["id*orig_p"] = "src_port",
		["id*resp_h"] = "dst",
		["id*resp_p"] = "dst_port",
	};
2016-05-16 12:28:45 -04:00
Jan Grashoefer
859eb5eac7 Merge branch 'master' into topic/jgras/intel-update 2016-05-11 18:59:58 +02:00
Daniel Thayer
d91dd8d9a8 Fix Bro and unit tests when broker is not enabled
When Bro was compiled with broker disabled, then some Bro scripts
were referencing functions and types that were not defined.  Fixed
by adding @ifdefs to several scripts.  Removed one @ifdef because
it was causing several unit tests to fail.

Also fixed the @TEST-REQUIRES check in tests that rely on broker so
that such tests are skipped when broker is disabled.
2016-05-10 06:24:35 -05:00
Johanna Amann
86836c0bb8 Merge remote-tracking branch 'origin/master' into topic/johanna/netcontrol-improvements 2016-05-02 11:23:36 -07:00
Johanna Amann
6905984ee7 Merge remote-tracking branch 'origin/master' into topic/johanna/xmpp-starttls 2016-04-29 12:56:12 -07:00
Robin Sommer
befad8f825 Merge remote-tracking branch 'origin/topic/dnthayer/ticket1449'
BIT-1449 #merged

* origin/topic/dnthayer/ticket1449:
  Rename broker BIF wrapper functions in a few more places
  Sync the core/leaks/broker/data.bro test with broker/data.bro
  Add missing tests for broker data BIFs
  Code cleanup for some broker tests
  Add script wrapper functions for broker data BIFs
  Add script wrapper functions for broker BIFs
2016-04-28 10:24:39 -07:00
Robin Sommer
2eeddac401 Merge remote-tracking branch 'origin/topic/johanna/imap-starttls'
BIT-1574 #merged

* origin/topic/johanna/imap-starttls:
  IMAP: add c++11 header file that gcc complains about.
  IMAP: documentation and test updates
  also generate an event when starttls is encounterd for imap.
  Add support of getting server capabilities to IMAP parser.
  Basic IMAP StartTLS analyzer.
2016-04-28 10:11:43 -07:00
Johanna Amann
3a70289e91 ARP: remove unnecessary variables and add testcase
BIT-1573 #close
2016-04-27 06:51:04 -07:00
Daniel Thayer
f44bb4d9b8 Add script wrapper functions for broker BIFs
Also renamed the "print" function to "send_print" and the "event"
function to "send_event" because Bro shows a syntax error when a
Bro script function is named "event" or "print".
2016-04-26 18:10:30 -05:00
Johanna Amann
124126cabc Merge remote-tracking branch 'origin/master' into topic/johanna/imap-starttls 2016-04-26 12:48:53 -07:00
Johanna Amann
e9a87566ef Fix parsing of x509 pre-y2k dates
There was a bug in the new parsing code, introduced in
708ede22c6 which parses validity times
incorrectly if they are before the year 2000. What happens in this case
is that the 2-digit year will be interpreted to be in the 21st century
(1999 will be parsed as 2099, e.g.).
2016-04-26 12:30:28 -07:00
Johanna Amann
3669b6aa9c Merge remote-tracking branch 'origin/master' into topic/johanna/imap-starttls 2016-04-26 10:52:16 -07:00
Johanna Amann
a705b2c08d Add DNS tests for huge TLL and CAA 2016-04-25 15:43:20 -07:00
Johanna Amann
e7ec537ed5 Merge remote-tracking branch 'origin/topic/dnthayer/broker-namespace'
* origin/topic/dnthayer/broker-namespace:
  Split the broker main.bro into two scripts
  Rename the BrokerStore namespace to Broker
  Rename the BrokerComm namespace to Broker

BIT-1563 #merged
2016-04-22 16:45:09 -07:00
Seth Hall
bfc06f7e17 Merge remote-tracking branch 'origin/master' into topic/seth/smb 2016-04-13 01:57:07 -04:00
Seth Hall
adcc978f14 Add a file entropy test. 2016-04-13 00:44:02 -04:00
Martin van Hensbergen
849875e8be Analyzer and bro script for RFB protocol (VNC)
This analyzer parses the Remote Frame Buffer
protocol, usually referred to as the 'VNC protocol'.

It supports several dialects (3.3, 3.7, 3.8) and
also handles the Apple Remote Desktop variant.

It will log such facts as client/server versions,
authentication method used, authentication result,
height, width and name of the shared screen.

It also includes two testcases.

Todo: Apple Remote Desktop seems to have some
bytes prepended to the screen name. This is
not interepreted correctly.
2016-04-11 10:35:00 +02:00
Seth Hall
e2dfaf8a5b Fixes to SMB file handling and logging.
There were some cases where the log would be missing a field
or data wouldn't get sent to file analysis.  At least some of
this is fixed now and I get confused a bit less when I look
at the logs now.

Also, I made the default handling "FILE" so that things like
FILE_UNKNOWN wouldn't show up in the logs so regularly.  It's
technically correct that way, but it doesn't look good and it's
correct as FILE often enough that it make sense to make it the
default I think.
2016-04-01 09:43:13 -04:00
Seth Hall
5721db4be7 Lots of cleanup and improvement to DCE/RPC analyzer.
- It works with DCE/RPC over SMB1+2 now.
   - Using named pipes in 1+2 and the transaction cmd in SMB1.
 - Base scripts based on work by Josh Liburdi.
 - New dce_rpc.log.  Feedback on how to make this log more compact
   and useful would be appreciated.
2016-04-01 09:38:52 -04:00
Daniel Thayer
9f5c820c7b Rename the BrokerComm namespace to Broker 2016-03-30 14:31:25 -05:00
Jan Grashoefer
2ebac70782 Added remove function to intel-framework. 2016-03-30 20:03:07 +02:00
Johanna Amann
34ad4cf638 NetControl: add rule exists as state besides added and failure.
Rules that are already existing will, by default, not automatically be
timed out by NetControl.
2016-03-24 15:06:07 -07:00