Merge remote-tracking branch 'origin/master' into topic/bernhard/input

CHANGES

@@ -1,4 +1,13 @@
 
+2.0-beta-177 | 2012-01-05 15:01:07 -0800
+
+  * Replace the --snaplen/-l command line option with a
+    scripting-layer option called "snaplen" (which can also be
+    redefined on the command line, e.g. `bro -i eth0 snaplen=65535`).
+
+  * Reduce snaplen default from 65535 to old default of 8192. Fixes
+    #720. (Jon Siwek)
+
 2.0-beta-174 | 2012-01-04 12:47:10 -0800
 
   * SSL improvements. (Seth Hall)

NEWS

@@ -0,0 +1,51 @@
+
+Release Notes
+=============
+
+This document summarizes the most important changes in the current Bro
+release. For a complete list of changes, see the ``CHANGES`` file.
+
+Bro 2.0
+-------
+
+As the version number jump suggests, Bro 2.0 is a major upgrade and
+lots of things have changed. We have assembled a separate upprade
+guide with the most important changes compared to Bro 1.5 at
+http://www.bro-ids.org/documentation/upgrade.bro.html. You can find
+the offline version of that document in ``doc/upgrade.rst.``.
+
+Compared to the earlier 2.0 Beta version, the major changes in the
+final release are:
+
+    * The default scripts now come with complete reference
+      documentation. See
+      http://www.bro-ids.org/documentation/index.html.
+
+    * libz and libmagic are now required dependencies.
+
+    * Reduced snaplen default from 65535 to old default of 8192. The
+      large value was introducing performance problems on many
+      systems.
+
+    * Replaced the --snaplen/-l command line option with a
+      scripting-layer option called "snaplen". The new option can also
+      be redefined on the command line, e.g. ``bro -i eth0
+      snaplen=65535``.
+
+    * Reintroduced the BRO_LOG_SUFFIX environment that the ASCII
+      logger now respects to add a suffix to the log files it creates.
+
+    * The ASCII logs now include further header information, and 
+      fields set to an empty value are now logged as ``(empty)`` by
+      default (instead of ``-``, which is already used for fields that
+      are not set at all).
+
+    * Some NOTICES were renamed, and the signatures of some SSL events
+      have changed.
+
+    * Many smaller bug fixes, portability improvements, and general
+      polishing.
+
+
+
+

README

@@ -4,13 +4,15 @@ Bro Network Security Monitor
 
 Bro is a powerful framework for network analysis and security
 monitoring. Please see the INSTALL file for installation instructions
-and pointers for getting started. For more documentation, research
+and pointers for getting started. NEWS contains releases notes for the
-publications, and community contact information, see Bro's home page:
+current version, and CHANGES has the complete history of changes.
+Please see COPYING for licensing information.
+
+For more documentation, research publications, and community contact
+information, please see Bro's home page:
 
     http://www.bro-ids.org
 
-Please see COPYING for licensing information.
-
 On behalf of the Bro Development Team,
 
 Vern Paxson & Robin Sommer,

VERSION

@@ -1 +1 @@
-2.0-beta-174
+2.0-beta-177

doc/faq.rst

@@ -28,6 +28,23 @@ Here are some pointers to more information:
   Lothar Braun et. al evaluates packet capture performance on
   commodity hardware
 
+Are there any gotchas regarding interface configuration for live capture?  Or why might I be seeing abnormally large packets much greater than interface MTU?
+-------------------------------------------------------------------------------------------------------------------------------------------------------------
+
+Some NICs offload the reassembly of traffic into "superpackets" so that
+fewer packets are then passed up the stack (e.g. "TCP segmentation
+offload", or "generic segmentation offload").  The result is that the
+capturing application will observe packets much larger than the MTU size
+of the interface they were captured from and may also interfere with the
+maximum packet capture length, ``snaplen``, so it's a good idea to disable
+an interface's offloading features.
+
+You can use the ``ethtool`` program on Linux to view and disable
+offloading features of an interface.  See this page for more explicit
+directions:
+
+http://securityonion.blogspot.com/2011/10/when-is-full-packet-capture-not-full.html
+
 What does an error message like ``internal error: NB-DNS error`` mean?
 ---------------------------------------------------------------------------------------------------------------------------------
 

doc/upgrade.rst

@@ -168,10 +168,6 @@ New Default Settings
   are loaded. See ``PacketFilter::all_packets`` for how to revert to old
   behavior.
 
-- By default, Bro now sets a libpcap snaplen of 65535. Depending on
-  the OS, this may have performance implications and you can use the
-  ``--snaplen`` option to change the value.
-  
 API Changes
 -----------
 

scripts/base/init-bare.bro

@@ -1505,6 +1505,9 @@ const skip_http_data = F &redef;
 ## UDP tunnels. See also: udp_tunnel_port, policy/udp-tunnel.bro.
 const parse_udp_tunnels = F &redef;
 
+## Number of bytes per packet to capture from live interfaces.
+const snaplen = 8192 &redef;
+
 # Load the logging framework here because it uses fairly deep integration with 
 # BiFs and script-land defined types.
 @load base/frameworks/logging

src/main.cc

@@ -99,7 +99,7 @@ extern char version[];
 char* command_line_policy = 0;
 vector<string> params;
 char* proc_status_file = 0;
-int snaplen = 65535;	// really want "capture entire packet"
+int snaplen = 0;	// this gets set from the scripting-layer's value
 
 int FLAGS_use_binpac = false;
 
@@ -147,7 +147,6 @@ void usage()
 	fprintf(stderr, "    -g|--dump-config               | dump current config into .state dir\n");
 	fprintf(stderr, "    -h|--help|-?                   | command line help\n");
 	fprintf(stderr, "    -i|--iface <interface>         | read from given interface\n");
-	fprintf(stderr, "    -l|--snaplen <snaplen>         | number of bytes per packet to capture from interfaces (default 65535)\n");
 	fprintf(stderr, "    -p|--prefix <prefix>           | add given prefix to policy file resolution\n");
 	fprintf(stderr, "    -r|--readfile <readfile>       | read from given tcpdump file\n");
 	fprintf(stderr, "    -y|--flowfile <file>[=<ident>] | read from given flow file\n");
@@ -374,7 +373,6 @@ int main(int argc, char** argv)
 		{"filter",		required_argument,	0,	'f'},
 		{"help",		no_argument,		0,	'h'},
 		{"iface",		required_argument,	0,	'i'},
-		{"snaplen",		required_argument,	0,	'l'},
 		{"doc-scripts",		no_argument,		0,	'Z'},
 		{"prefix",		required_argument,	0,	'p'},
 		{"readfile",		required_argument,	0,	'r'},
@@ -483,10 +481,6 @@ int main(int argc, char** argv)
 			interfaces.append(optarg);
 			break;
 
-		case 'l':
-			snaplen = atoi(optarg);
-			break;
-
 		case 'p':
 			prefixes.append(optarg);
 			break;
@@ -837,6 +831,8 @@ int main(int argc, char** argv)
 			}
 		}
 
+	snaplen = internal_val("snaplen")->AsCount();
+
 	// Initialize the secondary path, if it's needed.
 	secondary_path = new SecondaryPath();