Merge remote-tracking branch 'origin/master' into topic/seth/files-tracking

Conflicts: scripts/base/frameworks/files/main.bro src/file_analysis/File.cc testing/btest/Baseline/scripts.base.frameworks.file-analysis.actions.data_event/out
2025-10-02 14:48:21 +00:00 · 2014-09-23 13:05:39 -04:00 · 2014-09-23 13:05:39 -04:00 · 42b2d56279
commit 42b2d56279
parent 8d72cd20a8 dae80fc119
486 changed files with 106378 additions and 85985 deletions
--- a/.gitmodules
+++ b/.gitmodules
@ -19,3 +19,6 @@
 [submodule "src/3rdparty"]
 	path = src/3rdparty
 	url = git://git.bro.org/bro-3rdparty
 [submodule "aux/plugins"]
 	path = aux/plugins
 	url = git://git.bro.org/bro-plugins
--- a/411
+++ b/411
@ -1,4 +1,325 @@
 2.3-180 | 2014-09-22 12:52:41 -0500
  * BIT-1259: Fix issue w/ duplicate TCP reassembly deliveries.
    (Jon Siwek)
 2.3-178 | 2014-09-18 14:29:46 -0500
  * BIT-1256: Fix file analysis events from coming after bro_done().
    (Jon Siwek)
 2.3-177 | 2014-09-17 09:41:27 -0500
  * Documentation fixes. (Chris Mavrakis)
 2.3-174 | 2014-09-17 09:37:09 -0500
  * Fixed some "make doc" warnings caused by reST formatting
    (Daniel Thayer).
 2.3-172 | 2014-09-15 13:38:52 -0500
  * Remove unneeded allocations for HTTP messages. (Jon Siwek)
 2.3-171 | 2014-09-15 11:14:57 -0500
  * Fix a compile error on systems without pcap-int.h. (Jon Siwek)
 2.3-170 | 2014-09-12 19:28:01 -0700
  * Fix incorrect data delivery skips after gap in HTTP Content-Range.
    Addresses BIT-1247. (Jon Siwek)
  * Fix file analysis placement of data after gap in HTTP
    Content-Range. Addresses BIT-1248. (Jon Siwek)
  * Fix issue w/ TCP reassembler not delivering some segments.
    Addresses BIT-1246. (Jon Siwek)
  * Fix MIME entity file data/gap ordering and raise http_entity_data
    in line with data arrival. Addresses BIT-1240. (Jon Siwek)
  * Implement file ID caching for MIME_Mail. (Jon Siwek)
  * Fix a compile error. (Jon Siwek)
 2.3-161 | 2014-09-09 12:35:38 -0500
  * Bugfixes and test updates/additions. (Robin Sommer)
  * Interface tweaks and docs for PktSrc/PktDumper. (Robin Sommer)
  * Moving PCAP-related bifs to iosource/pcap.bif. (Robin Sommer)
  * Moving some of the BPF filtering code into base class.
    This will allow packet sources that don't support BPF natively to
    emulate the filtering via libpcap. (Robin Sommer)
  * Removing FlowSrc. (Robin Sommer)
  * Removing remaining pieces of the 2ndary path, and left-over
    files of packet sorter. (Robin Sommer)
  * A bunch of infrastructure work to move IOSource, IOSourceRegistry
    (now iosource::Manager) and PktSrc/PktDumper code into iosource/,
    and over to a plugin structure. (Robin Sommer)
 2.3-137 | 2014-09-08 19:01:13 -0500
  * Fix Broxygen's rendering of opaque types. (Jon Siwek)
 2.3-136 | 2014-09-07 20:50:46 -0700
  * Change more http links to https. (Johanna Amann)
 2.3-134 | 2014-09-04 16:16:36 -0700
  * Fixed a number of issues with OCSP reply validation. Addresses
    BIT-1212. (Johanna Amann)
  * Fix null pointer dereference in OCSP verification code in case no
    certificate is sent as part as the ocsp reply. Addresses BIT-1212.
    (Johanna Amann)
 2.3-131 | 2014-09-04 16:10:32 -0700
  * Make links in documentation templates protocol relative. (Johanna
    Amann)
 2.3-129 | 2014-09-02 17:21:21 -0700
  * Simplify a conditional with equivalent branches. (Jon Siwek)
  * Change EDNS parsing code to use rdlength more cautiously. (Jon
    Siwek)
  * Fix a memory leak when bind() fails due to EADDRINUSE. (Jon Siwek)
  * Fix possible buffer over-read in DNS TSIG parsing. (Jon Siwek)
 2.3-124 | 2014-08-26 09:24:19 -0500
  * Better documentation for sub_bytes (Jimmy Jones)
  * BIT-1234: Fix build on systems that already have ntohll/htonll
    (Jon Siwek)
 2.3-121 | 2014-08-22 15:22:15 -0700
  * Detect functions that try to bind variables from an outer scope
    and raise an error saying that's not supported. Addresses
    BIT-1233. (Jon Siwek)
 2.3-116 | 2014-08-21 16:04:13 -0500
  * Adding plugin testing to Makefile's test-all. (Robin Sommer)
  * Converting log writers and input readers to plugins.
    DataSeries and ElasticSearch plugins have moved to the new
    bro-plugins repository, which is now a git submodule in the
    aux/plugins directory. (Robin Sommer)
 2.3-98 | 2014-08-19 11:03:46 -0500
  * Silence some doc-related warnings when using `bro -e`.
    Closes BIT-1232. (Jon Siwek)
  * Fix possible null ptr derefs reported by Coverity. (Jon Siwek)
 2.3-96 | 2014-08-01 14:35:01 -0700
  * Small change to DHCP documentation. In server->client messages the
    host name may differ from the one requested by the client.
    (Johanna Amann)
  * Split DHCP log writing from record creation. This allows users to
    customize dhcp.log by changing the record in their own dhcp_ack
    event. (Johanna Amann)
  * Update PATH so that documentation btests can find bro-cut. (Daniel
    Thayer)
  * Remove gawk from list of optional packages in documentation.
    (Daniel Thayer)
  * Fix for redefining built-in constants. (Robin Sommer)
 2.3-86 | 2014-07-31 14:19:58 -0700
  * Fix for redefining built-in constants. (Robin Sommer)
  * Adding missing check that a plugin's API version matches what Bro
    defines. (Robin Sommer)
  * Adding NEWS entry for plugins. (Robin Sommer)
 2.3-83 | 2014-07-30 16:26:11 -0500
  * Minor adjustments to plugin code/docs. (Jon Siwek)
  * Dynamic plugin support. (Rpbin Sommer)
    Bro now supports extending core functionality, like protocol and
    file analysis, dynamically with external plugins in the form of
    shared libraries. See doc/devel/plugins.rst for an overview of the
    main functionality. Changes coming with this:
        - Replacing the old Plugin macro magic with a new API.
        - The plugin API changed to generally use std::strings instead
          of const char*.
        - There are a number of invocations of PLUGIN_HOOK_
          {VOID,WITH_RESULT} across the code base, which allow plugins
          to hook into the processing at those locations.
        - A few new accessor methods to various classes to allow
          plugins to get to that information.
        - network_time cannot be just assigned to anymore, there's now
          function net_update_time() for that.
        - Redoing how builtin variables are initialized, so that it
          works for plugins as well. No more init_net_var(), but
          instead bifcl-generated code that registers them.
        - Various changes for adjusting to the now dynamic generation
          of analyzer instances.
    - same_type() gets an optional extra argument allowing record type
      comparision to ignore if field names don't match. (Robin Sommer)
    - Further unify file analysis API with the protocol analyzer API
      (assigning IDs to analyzers; adding Init()/Done() methods;
      adding subtypes). (Robin Sommer)
    - A new command line option -Q that prints some basic execution
      time stats. (Robin Sommer)
    - Add support to the file analysis for activating analyzers by
      MIME type. (Robin Sommer)
          - File::register_for_mime_type(tag: Analyzer::Tag, mt:
            string): Associates a file analyzer with a MIME type.
          - File::add_analyzers_for_mime_type(f: fa_file, mtype:
            string): Activates all analyzers registered for a MIME
            type for the file.
          - The default file_new() handler calls
            File::add_analyzers_for_mime_type() with the file's MIME
            type.
 2.3-20 | 2014-07-22 17:41:02 -0700
  * Updating submodule(s).
 2.3-19 | 2014-07-22 17:29:19 -0700
  * Implement bytestring_to_coils() in Modbus analyzer so that coils
    gets passed to the corresponding events. (Hui Lin)
  * Add length field to ModbusHeaders. (Hui Lin)
 2.3-12 | 2014-07-10 19:17:37 -0500
  * Include yield of vectors in Broxygen's type descriptions.
    Addresses BIT-1217. (Jon Siwek)
 2.3-11 | 2014-07-10 14:49:27 -0700
  * Fixing DataSeries output. It was using a now illegal value as its
    default compression level. (Robin Sommer)
 2.3-7 | 2014-06-26 17:35:18 -0700
  * Extending "make test-all" to include aux/bro-aux. (Robin Sommer)
 2.3-6 | 2014-06-26 17:24:10 -0700
  * DataSeries compilation issue fixed. (mlaterman)
  * Fix a reference counting bug in ListVal ctor. (Jon Siwek)
 2.3-3 | 2014-06-26 15:41:04 -0500
  * Support tilde expansion when Bro tries to find its own path. (Jon
    Siwek)
 2.3-2 | 2014-06-23 16:54:15 -0500
  * Remove references to line numbers in tutorial text. (Daniel Thayer)
 2.3 | 2014-06-16 09:48:25 -0500
  * Release 2.3.
 2.3-beta-33 | 2014-06-12 11:59:28 -0500
  * Documentation improvements/fixes. (Daniel Thayer)
 2.3-beta-24 | 2014-06-11 15:35:31 -0500
  * Fix SMTP state tracking when server response is missing.
    (Robin Sommer)
 2.3-beta-22 | 2014-06-11 12:31:38 -0500
  * Fix doc/test that broke due to a Bro script change. (Jon Siwek)
  * Remove unused --with-libmagic configure option. (Jon Siwek)
 2.3-beta-20 | 2014-06-10 18:16:51 -0700
  * Fix use-after-free in some cases of reassigning a table index.
    Addresses BIT-1202. (Jon Siwek)
 2.3-beta-18 | 2014-06-06 13:11:50 -0700
  * Add two more SSL events, one triggered for each handshake message
    and one triggered for the tls change cipherspec message. (Bernhard
    Amann)
  * Small SSL bug fix. In case SSL::disable_analyzer_after_detection
    was set to false, the ssl_established event would fire after each
    data packet once the session is established. (Bernhard Amann)
 2.3-beta-16 | 2014-06-06 13:05:44 -0700
  * Re-activate notice suppression for expiring certificates.
    (Bernhard Amann)
 2.3-beta-14 | 2014-06-05 14:43:33 -0700
  * Add new TLS extension type numbers from IANA (Bernhard Amann)
  * Switch to double hashing for Bloomfilters for better performance.
    (Matthias Vallentin)
  * Bugfix to use full digest length instead of just one byte for
    Bloomfilter's universal hash function. Addresses BIT-1140.
    (Matthias Vallentin)
  * Make buffer for X509 certificate subjects larger. Addresses
    BIT-1195 (Bernhard Amann)
 2.3-beta-5 | 2014-05-29 15:34:42 -0500
  * Fix misc/load-balancing.bro's reference to
    PacketFilter::sampling_filter (Jon Siwek)
 2.3-beta-4 | 2014-05-28 14:55:24 -0500
  * Fix potential mem leak in remote function/event unserialization.
    (Jon Siwek)
  * Fix reference counting bug in table coercion expressions (Jon Siwek)
  * Fix an "unused value" warning. (Jon Siwek)
  * Remove a duplicate unit test baseline dir. (Jon Siwek)
 2.3-beta | 2014-05-19 16:36:50 -0500
  * Release 2.3-beta
@ -37,7 +358,7 @@
    and the load balancing FAQ on the website. (Daniel Thayer)
  * Update some doc tests and line numbers (Daniel Thayer)
-    
+
 2.2-457 | 2014-05-16 14:38:31 -0700
  * New script policy/protocols/ssl/validate-ocsp.bro that adds OSCP
@ -52,7 +373,7 @@
    Amann)
  * Improved Heartbleed attack/scan detection. (Bernhard Amann)
-    
+
  * Let TLS analyzer fail better when no longer in sync with the data
    stream. (Bernhard Amann)
@ -89,12 +410,12 @@
    *Undelivered methods now use a uint64 in place of an int for the
    relative sequence space offset parameter.
-    Addresses BIT-348. 
+    Addresses BIT-348.
  * Fixing compiler warnings. (Robin Sommer)
-    
+
  * Update SNMP analyzer's DeliverPacket method signature. (Jon Siwek)
-    
+
 2.2-417 | 2014-05-07 10:59:22 -0500
  * Change handling of atypical OpenSSL error case in x509 verification. (Jon Siwek)
@ -134,7 +455,7 @@
 2.2-397 | 2014-05-01 20:29:20 -0700
  * Fix reference counting for lookup_ID() usages. (Jon Siwek)
-    
+
 2.2-395 | 2014-05-01 20:25:48 -0700
  * Fix missing "irc-dcc-data" service field from IRC DCC connections.
@ -147,7 +468,7 @@
    Siwek)
  * Improve file analysis manager shutdown/cleanup. (Jon Siwek)
-    
+
 2.2-388 | 2014-04-24 18:38:07 -0700
  * Fix decoding of MIME quoted-printable. (Mareq)
@ -160,11 +481,11 @@
 2.2-381 | 2014-04-24 17:08:45 -0700
  * Add Java version to software framework. (Brian Little)
-    
+
 2.2-379 | 2014-04-24 17:06:21 -0700
  * Remove unused Val::attribs member. (Jon Siwek)
-    
+
 2.2-377 | 2014-04-24 16:57:54 -0700
  * A larger set of SSL improvements and extensions. Addresses
@ -177,7 +498,7 @@
          server_name, alpn, and ec-curves.
        - Adds support for the heartbeat events.
-          
+
        - Add Heartbleed detector script.
        - Adds basic support for OCSP stapling.
@ -188,7 +509,7 @@
 2.2-353 | 2014-04-24 16:12:30 -0700
  * Adapt HTTP partial content to cache file analysis IDs. (Jon Siwek)
-    
+
  * Adapt SSL analyzer to generate file analysis handles itself. (Jon
    Siwek)
@ -204,11 +525,11 @@
    Siwek)
  * Refactor file analysis file ID lookup. (Jon Siwek)
-    
+
 2.2-344 | 2014-04-22 20:13:30 -0700
  * Refactor various hex escaping code. (Jon Siwek)
-    
+
 2.2-341 | 2014-04-17 18:01:41 -0500
  * Fix duplicate DNS log entries. (Robin Sommer)
@ -216,9 +537,9 @@
 2.2-341 | 2014-04-17 18:01:01 -0500
  * Refactor initialization of ASCII log writer options. (Jon Siwek)
-    
+
  * Fix a memory leak in ASCII log writer. (Jon Siwek)
-    
+
 2.2-338 | 2014-04-17 17:48:17 -0500
  * Disable input/logging threads setting their names on every
@ -230,7 +551,7 @@
 2.2-335 | 2014-04-10 15:04:57 -0700
  * Small logic fix for main SSL script. (Bernhard Amann)
-    
+
  * Update DPD signatures for detecting TLS 1.2. (Bernhard Amann)
  * Remove unused data member of SMTP_Analyzer to silence a Coverity
@ -254,7 +575,7 @@
 2.2-317 | 2014-04-03 10:51:31 -0400
-  * Add a uid field to the signatures.log.  Addresses BIT-1171 
+  * Add a uid field to the signatures.log.  Addresses BIT-1171
    (Anthony Verez)
 2.2-315 | 2014-04-01 16:50:01 -0700
@ -265,7 +586,7 @@
 2.2-313 | 2014-04-01 16:40:19 -0700
  * Fix a couple nits reported by Coverity.(Jon Siwek)
-    
+
  * Fix potential memory leak in IP frag reassembly reported by
    Coverity. (Jon Siwek)
@ -355,14 +676,14 @@
 2.2-271 | 2014-03-30 20:25:17 +0200
  * Add unit tests covering vector/set/table ctors/inits. (Jon Siwek)
-    
+
  * Fix parsing of "local" named table constructors. (Jon Siwek)
  * Improve type checking of records. Addresses BIT-1159. (Jon Siwek)
-    
+
 2.2-267 | 2014-03-30 20:21:43 +0200
-  * Improve documentation of Bro clusters. Addresses BIT-1160. 
+  * Improve documentation of Bro clusters. Addresses BIT-1160.
    (Daniel Thayer)
 2.2-263 | 2014-03-30 20:19:05 +0200
@ -389,7 +710,7 @@
 2.2-254 | 2014-03-30 19:55:22 +0200
  * Update instructions on how to build Bro docs. (Daniel Thayer)
-    
+
 2.2-251 | 2014-03-28 08:37:37 -0400
  * Quick fix to the ElasticSearch writer. (Seth Hall)
@ -419,7 +740,7 @@
    formatter work.
  * Fixing compiler error. (Robin Sommer)
-    
+
  * Fixing (very unlikely) double delete in HTTP analyzer when
    decapsulating CONNECTs. (Robin Sommer)
@ -445,7 +766,7 @@
    Addresses BIT-1134. (Jon Siwek)
  * Enable fake DNS mode for test suites.
-    
+
  * Improve analysis of TCP SYN/SYN-ACK reversal situations. (Jon
    Siwek)
@ -477,7 +798,7 @@
    (Jon Siwek)
  * Silences some documentation warnings from Sphinx. (Jon Siwek)
-    
+
 2.2-215 | 2014-03-10 11:10:15 -0700
  * Fix non-deterministic logging of unmatched DNS msgs. Addresses
@ -515,7 +836,7 @@
    HTTP traffic. (Seth Hall)
  * Fixing removal of support analyzers. (Robin Sommer)
-    
+
 2.2-199 | 2014-03-03 16:34:20 -0800
  * Allow iterating over bif functions with result type vector of any.
@ -530,11 +851,11 @@
 2.2-194 | 2014-02-28 14:50:53 -0800
  * Remove packet sorter. Addresses BIT-700. (Bernhard Amann)
-    
+
 2.2-192 | 2014-02-28 09:46:43 -0800
  * Update Mozilla root bundle. (Bernhard Amann)
-    
+
 2.2-190 | 2014-02-27 07:34:44 -0800
  * Adjust timings of a few leak tests. (Bernhard Amann)
@ -564,7 +885,7 @@
 2.2-177 | 2014-02-20 17:27:46 -0800
  * Update to libmagic version 5.17. Addresses BIT-1136. (Jon Siwek)
-    
+
 2.2-174 | 2014-02-14 12:07:04 -0800
  * Support for MPLS over VLAN. (Chris Kanich)
@ -593,7 +914,7 @@
    defined, but we see it being actively used. (Bernhard Amann)
  * Test baseline updates for DNS change. (Robin Sommer)
-    
+
 2.2-158 | 2014-02-09 23:45:39 -0500
  * Change dns.log to include only standard DNS queries. (Jon Siwek)
@ -669,7 +990,7 @@
 2.2-128 | 2014-01-30 15:58:47 -0800
  * Add leak test for Exec module. (Bernhard Amann)
-    
+
  * Fix file_over_new_connection event to trigger when entire file is
    missed. (Jon Siwek)
@ -682,7 +1003,7 @@
    "detect_filtered_trace". (Jon Siwek)
  * Improve TCP FIN retransmission handling. (Jon Siwek)
-    
+
 2.2-120 | 2014-01-28 10:25:23 -0800
  * Fix and extend x509_extension() event, which now actually returns
@ -700,10 +1021,10 @@
 2.2-115 | 2014-01-22 12:11:18 -0800
  * Add unit tests for new Bro Manual docs. (Jon Siwek)
-    
+
  * New content for the "Using Bro" section of the manual. (Rafael
    Bonilla/Jon Siwek)
-    
+
 2.2-105 | 2014-01-20 12:16:48 -0800
  * Support GRE tunnel decapsulation, including enhanced GRE headers.
@ -712,7 +1033,7 @@
    Addresses BIT-867. (Jon Siwek)
  * Simplify FragReassembler memory management. (Jon Siwek)
-    
+
 2.2-102 | 2014-01-20 12:00:29 -0800
  * Include file information (MIME type and description) into notice
@ -721,7 +1042,7 @@
 2.2-100 | 2014-01-20 11:54:58 -0800
  * Fix caching of recently validated SSL certifcates. (Justin Azoff)
-    
+
 2.2-98 | 2014-01-20 11:50:32 -0800
  * For notice suppresion, instead of storing the entire notice in
@ -749,7 +1070,7 @@
    the city database instead of just the former. (Jon Siwek)
  * Broxygen init fixes. Addresses BIT-1110. (Jon Siwek)
-    
+
    - Don't check mtime of bro binary if BRO_DISABLE_BROXYGEN env var set.
    - Fix failure to locate bro binary if invoking from a relative
@ -762,7 +1083,7 @@
    len field being set to zero. (Seth Hall)
  * Canonify output of a unit test. (Jon Siwek)
-    
+
  * A set of documentation updates. (Daniel Thayer)
      - Fix typo in Bro 2.2 NEWS on string indexing.
@ -809,9 +1130,9 @@
    (Jon Siwek)
  * Close signature files after done parsing. (Jon Siwek)
-    
+
  * Fix unlikely null ptr deref in broxygen::Manager. (Jon Siwek)
-    
+
  * FreeBSD build fix addendum: unintended variable shadowing. (Jon
    Siwek)
@ -830,7 +1151,7 @@
    were cleaned up. Addresses BIT-1103. (Bernhard Amann)
  * Minor Broxygen improvements. Addresses BIT-1098. (Jon Siwek)
-    
+
 2.2-51 | 2013-12-05 07:53:37 -0800
  * Improve a unit test involving 'when' conditionals. (Jon Siwek)
@ -855,7 +1176,7 @@
 2.2-44 | 2013-12-04 12:41:51 -0800
  * Fix string slice notation. Addresses BIT-1097. (Jon Siwek)
-    
+
    Slice ranges were not correctly determined for negative indices
    and also off by one in general (included one more element at the
    end of the substring than what actually matched the index range).
@ -910,7 +1231,7 @@
 2.2-11 | 2013-12-03 10:56:28 -0800
  * Unit test for broccoli vector support. (Jon Siwek)
-    
+
  * Changed ordering of Bro type tag enum, which was out of sync. (Jon
    Siwek)
@ -922,14 +1243,14 @@
 2.2-6 | 2013-11-15 07:05:15 -0800
  * Make "install-example-configs" target use DESTDIR. (Jon Siwek)
-    
+
 2.2-5 | 2013-11-11 13:47:54 -0800
  * Fix the irc_reply event for certain server message types. (Seth
    Hall)
  * Fixed Segmentation fault in SQLite Writer. (Jon Crussell)
-    
+
 2.2 | 2013-11-07 10:25:50 -0800
  * Release 2.2.
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@ -1,5 +1,9 @@
 project(Bro C CXX)
 # When changing the minimum version here, also adapt
 # aux/bro-aux/plugin-support/skeleton/CMakeLists.txt
 cmake_minimum_required(VERSION 2.6.3 FATAL_ERROR)
 include(cmake/CommonCMakeConfig.cmake)
 ########################################################################
@ -16,12 +20,18 @@ endif ()
 get_filename_component(BRO_SCRIPT_INSTALL_PATH ${BRO_SCRIPT_INSTALL_PATH}
    ABSOLUTE)
 set(BRO_PLUGIN_INSTALL_PATH ${BRO_ROOT_DIR}/lib/bro/plugins CACHE STRING "Installation path for plugins" FORCE)
 configure_file(bro-path-dev.in ${CMAKE_CURRENT_BINARY_DIR}/bro-path-dev)
 file(WRITE ${CMAKE_CURRENT_BINARY_DIR}/bro-path-dev.sh
     "export BROPATH=`${CMAKE_CURRENT_BINARY_DIR}/bro-path-dev`\n"
     "export BRO_PLUGIN_PATH=\"${CMAKE_CURRENT_BINARY_DIR}/src:${BRO_PLUGIN_INSTALL_PATH}\"\n"
     "export PATH=\"${CMAKE_CURRENT_BINARY_DIR}/src\":$PATH\n")
 file(WRITE ${CMAKE_CURRENT_BINARY_DIR}/bro-path-dev.csh
     "setenv BROPATH `${CMAKE_CURRENT_BINARY_DIR}/bro-path-dev`\n"
     "setenv BRO_PLUGIN_PATH \"${CMAKE_CURRENT_BINARY_DIR}/src:${BRO_PLUGIN_INSTALL_PATH}\"\n"
     "setenv PATH \"${CMAKE_CURRENT_BINARY_DIR}/src\":$PATH\n")
 file(STRINGS "${CMAKE_CURRENT_SOURCE_DIR}/VERSION" VERSION LIMIT_COUNT 1)
@ -117,33 +127,6 @@ if (GOOGLEPERFTOOLS_FOUND)
    endif ()
 endif ()
 set(USE_DATASERIES false)
 find_package(Lintel)
 find_package(DataSeries)
 find_package(LibXML2)
 if (NOT DISABLE_DATASERIES AND
    LINTEL_FOUND AND DATASERIES_FOUND AND LIBXML2_FOUND)
    set(USE_DATASERIES true)
 	include_directories(BEFORE ${Lintel_INCLUDE_DIR})
 	include_directories(BEFORE ${DataSeries_INCLUDE_DIR})
 	include_directories(BEFORE ${LibXML2_INCLUDE_DIR})
 	list(APPEND OPTLIBS ${Lintel_LIBRARIES})
 	list(APPEND OPTLIBS ${DataSeries_LIBRARIES})
 	list(APPEND OPTLIBS ${LibXML2_LIBRARIES})
 endif()
 set(USE_ELASTICSEARCH false)
 set(USE_CURL false)
 find_package(LibCURL)
 if (NOT DISABLE_ELASTICSEARCH AND LIBCURL_FOUND)
  set(USE_ELASTICSEARCH true)
  set(USE_CURL true)
  include_directories(BEFORE ${LibCURL_INCLUDE_DIR})
  list(APPEND OPTLIBS ${LibCURL_LIBRARIES})
 endif()
 if (ENABLE_PERFTOOLS_DEBUG OR ENABLE_PERFTOOLS)
    # Just a no op to prevent CMake from complaining about manually-specified
    # ENABLE_PERFTOOLS_DEBUG or ENABLE_PERFTOOLS not being used if google
@ -165,6 +148,8 @@ set(brodeps
 include(TestBigEndian)
 test_big_endian(WORDS_BIGENDIAN)
 include(CheckSymbolExists)
 check_symbol_exists(htonll arpa/inet.h HAVE_BYTEORDER_64)
 include(OSSpecific)
 include(CheckTypes)
@ -174,6 +159,10 @@ include(MiscTests)
 include(PCAPTests)
 include(OpenSSLTests)
 include(CheckNameserCompat)
 include(GetArchitecture)
 # Tell the plugin code that we're building as part of the main tree.
 set(BRO_PLUGIN_INTERNAL_BUILD true CACHE INTERNAL "" FORCE)
 configure_file(${CMAKE_CURRENT_SOURCE_DIR}/config.h.in
               ${CMAKE_CURRENT_BINARY_DIR}/config.h)
@ -238,10 +227,6 @@ message(
    "\n        tcmalloc:  ${USE_PERFTOOLS_TCMALLOC}"
    "\n       debugging:  ${USE_PERFTOOLS_DEBUG}"
    "\njemalloc:          ${ENABLE_JEMALLOC}"
    "\ncURL:              ${USE_CURL}"
    "\n"
    "\nDataSeries:        ${USE_DATASERIES}"
    "\nElasticSearch:     ${USE_ELASTICSEARCH}"
    "\n"
    "\n================================================================\n"
 )
--- a/2
+++ b/2
@ -55,6 +55,8 @@ test:
 test-all: test
 	test -d aux/broctl && ( cd aux/broctl && make test )
 	test -d aux/btest  && ( cd aux/btest && make test )
 	test -d aux/bro-aux && ( cd aux/bro-aux && make test )
 	test -d aux/plugins && ( cd aux/plugins && make test-all )
 configured:
 	@test -d $(BUILD) || ( echo "Error: No build/ directory found. Did you run configure?" && exit 1 )
--- a/26
+++ b/26
@ -4,6 +4,32 @@ release. For an exhaustive list of changes, see the ``CHANGES`` file
 (note that submodules, such as BroControl and Broccoli, come with
 their own ``CHANGES``.)
 Bro 2.4 (in progress)
 =====================
 Dependencies
 ------------
 New Functionality
 -----------------
 - Bro now has support for external plugins that can extend its core
  functionality, like protocol/file analysis, via shared libraries.
  Plugins can be developed and distributed externally, and will be
  pulled in dynamically at startup. Currently, a plugin can provide
  custom protocol analyzers, file analyzers, log writers[TODO], input
  readers[TODO], packet sources[TODO], and new built-in functions. A
  plugin can furthermore hook into Bro's processing a number of places
  to add custom logic.
  See https://www.bro.org/sphinx-git/devel/plugins.html for more
  information on writing plugins.
 Changed Functionality
 ---------------------
 - bro-cut has been rewritten in C, and is hence much faster.
 Bro 2.3
 =======
--- a/2
+++ b/2
@ -1 +1 @@
-2.3-beta
+2.3-180
--- a/aux/binpac
+++ b/aux/binpac
@ -1 +1 @@
-Subproject commit ec1e052afd5a8cd3d1d2cbb28fcd688018e379a5
+Subproject commit 3a4684801aafa0558383199e9abd711650b53af9
--- a/aux/bro-aux
+++ b/aux/bro-aux
@ -1 +1 @@
-Subproject commit 5721df4f5f6fa84de6257cca6582a28e45831786
+Subproject commit 9ea20c3905bd3fd5109849c474a2f2b4ed008357
--- a/aux/broccoli
+++ b/aux/broccoli
@ -1 +1 @@
-Subproject commit c2f5dd2cb7876158fdf9721aebd22567db840db1
+Subproject commit 33d0ed4a54a6ecf08a0b5fe18831aa413b437066
--- a/aux/broctl
+++ b/aux/broctl
@ -1 +1 @@
-Subproject commit ca3b12421269fce655028c05939e0ff2ee82ca2d
+Subproject commit 2f808bc8541378b1a4953cca02c58c43945d154f
--- a/aux/btest
+++ b/aux/btest
@ -1 +1 @@
-Subproject commit 4da1bd24038d4977e655f2b210f34e37f0b73b78
+Subproject commit 1efa4d10f943351efea96def68e598b053fd217a
--- a/aux/plugins
+++ b/aux/plugins
@ -0,0 +1 @@
 Subproject commit 23055b473c689a79da12b2825d8388f71f28c709
--- a/2
+++ b/2
@ -1 +1 @@
-Subproject commit 0f301aa08a970150195a2ea5b3ed43d2d98b35b3
+Subproject commit 03de0cc467d2334dcb851eddd843d59fef217909
--- a/config.h.in
+++ b/config.h.in
@ -129,6 +129,9 @@
 /* whether words are stored with the most significant byte first */
 #cmakedefine WORDS_BIGENDIAN
 /* whether htonll/ntohll is defined in <arpa/inet.h> */
 #cmakedefine HAVE_BYTEORDER_64
 /* ultrix can't hack const */
 #cmakedefine NEED_ULTRIX_CONST_HACK
 #ifdef NEED_ULTRIX_CONST_HACK
@ -209,3 +212,14 @@
 /* Common IPv6 extension structure */
 #cmakedefine HAVE_IP6_EXT
 /* String with host architecture (e.g., "linux-x86_64") */
 #define HOST_ARCHITECTURE "@HOST_ARCHITECTURE@"
 /* String with extension of dynamic libraries (e.g., ".so") */
 #define DYNAMIC_PLUGIN_SUFFIX "@CMAKE_SHARED_MODULE_SUFFIX@"
 /* True if we're building outside of the main Bro source code tree. */
 #ifndef BRO_PLUGIN_INTERNAL_BUILD
 #define BRO_PLUGIN_INTERNAL_BUILD @BRO_PLUGIN_INTERNAL_BUILD@
 #endif
--- a/25
+++ b/25
@ -39,8 +39,6 @@ Usage: $0 [OPTION]... [VAR=VALUE]...
    --disable-auxtools     don't build or install auxiliary tools
    --disable-perftools    don't try to build with Google Perftools
    --disable-python       don't try to build python bindings for broccoli
    --disable-dataseries   don't use the optional DataSeries log writer
    --disable-elasticsearch don't use the optional ElasticSearch log writer
  Required Packages in Non-Standard Locations:
    --with-openssl=PATH    path to OpenSSL install root
@ -50,7 +48,6 @@ Usage: $0 [OPTION]... [VAR=VALUE]...
    --with-flex=PATH       path to flex executable
    --with-bison=PATH      path to bison executable
    --with-perl=PATH       path to perl executable
    --with-libmagic=PATH   path to libmagic install root
  Optional Packages in Non-Standard Locations:
    --with-geoip=PATH      path to the libGeoIP install root
@ -63,9 +60,6 @@ Usage: $0 [OPTION]... [VAR=VALUE]...
    --with-ruby-lib=PATH   path to ruby library
    --with-ruby-inc=PATH   path to ruby headers
    --with-swig=PATH       path to SWIG executable
    --with-dataseries=PATH path to DataSeries and Lintel libraries
    --with-xml2=PATH       path to libxml2 installation (for DataSeries)
    --with-curl=PATH       path to libcurl install root (for ElasticSearch)
  Packaging Options (for developers):
    --binary-package       toggle special logic for binary packaging
@ -184,12 +178,6 @@ while [ $# -ne 0 ]; do
        --enable-ruby)
            append_cache_entry DISABLE_RUBY_BINDINGS     BOOL   false
            ;;
        --disable-dataseries)
            append_cache_entry DISABLE_DATASERIES     BOOL   true
            ;;
        --disable-elasticsearch)
            append_cache_entry DISABLE_ELASTICSEARCH     BOOL   true
            ;;
        --with-openssl=*)
            append_cache_entry OpenSSL_ROOT_DIR PATH $optarg
            ;;
@ -211,9 +199,6 @@ while [ $# -ne 0 ]; do
        --with-perl=*)
            append_cache_entry PERL_EXECUTABLE PATH $optarg
            ;;
        --with-libmagic=*)
            append_cache_entry LibMagic_ROOT_DIR PATH $optarg
            ;;
        --with-geoip=*)
            append_cache_entry LibGeoIP_ROOT_DIR PATH $optarg
            ;;
@ -247,16 +232,6 @@ while [ $# -ne 0 ]; do
        --with-swig=*)
            append_cache_entry SWIG_EXECUTABLE      PATH    $optarg
            ;;
        --with-dataseries=*)
            append_cache_entry DataSeries_ROOT_DIR PATH $optarg
            append_cache_entry Lintel_ROOT_DIR PATH $optarg
            ;;
        --with-xml2=*)
            append_cache_entry LibXML2_ROOT_DIR PATH $optarg
            ;;
        --with-curl=*)
            append_cache_entry LibCURL_ROOT_DIR PATH $optarg
            ;;
        --binary-package)
            append_cache_entry BINARY_PACKAGING_MODE BOOL true
            ;;
--- a/doc/_templates/layout.html
+++ b/doc/_templates/layout.html
@ -10,7 +10,7 @@
 {% endblock %}
 {% block header %}
-<iframe src="http://www.bro.org/frames/header-no-logo.html" width="100%" height="100px" frameborder="0" marginheight="0" scrolling="no" marginwidth="0">
+<iframe src="//www.bro.org/frames/header-no-logo.html" width="100%" height="100px" frameborder="0" marginheight="0" scrolling="no" marginwidth="0">
 </iframe>
 {% endblock %}
@ -108,6 +108,6 @@
 {% endblock %}
 {% block footer %}
-<iframe src="http://www.bro.org/frames/footer.html" width="100%" height="420px" frameborder="0" marginheight="0" scrolling="no" marginwidth="0">
+<iframe src="//www.bro.org/frames/footer.html" width="100%" height="420px" frameborder="0" marginheight="0" scrolling="no" marginwidth="0">
 </iframe>
 {% endblock %}
--- a/doc/conf.py.in
+++ b/doc/conf.py.in
@ -21,7 +21,7 @@ sys.path.insert(0, os.path.abspath('sphinx_input/ext'))
 # ----- Begin of BTest configuration. -----
 btest = os.path.abspath("@CMAKE_SOURCE_DIR@/aux/btest")
-brocut = os.path.abspath("@CMAKE_SOURCE_DIR@/aux/bro-aux/bro-cut")
+brocut = os.path.abspath("@CMAKE_SOURCE_DIR@/build/aux/bro-aux/bro-cut")
 bro = os.path.abspath("@CMAKE_SOURCE_DIR@/build/src")
 os.environ["PATH"] += (":%s:%s/sphinx:%s:%s" % (btest, btest, bro, brocut))
--- a/doc/devel/plugins.rst
+++ b/doc/devel/plugins.rst
@ -0,0 +1,436 @@
 ===================
 Writing Bro Plugins
 ===================
 Bro is internally moving to a plugin structure that enables extending
 the system dynamically, without modifying the core code base. That way
 custom code remains self-contained and can be maintained, compiled,
 and installed independently. Currently, plugins can add the following
 functionality to Bro:
    - Bro scripts.
    - Builtin functions/events/types for the scripting language.
    - Protocol analyzers.
    - File analyzers.
    - Packet sources and packet dumpers. TODO: Not yet.
    - Logging framework backends. TODO: Not yet.
    - Input framework readers. TODO: Not yet.
 A plugin's functionality is available to the user just as if Bro had
 the corresponding code built-in. Indeed, internally many of Bro's
 pieces are structured as plugins as well, they are just statically
 compiled into the binary rather than loaded dynamically at runtime.
 Quick Start
 ===========
 Writing a basic plugin is quite straight-forward as long as one
 follows a few conventions. In the following we walk a simple example
 plugin that adds a new built-in function (bif) to Bro: we'll add 
 ``rot13(s: string) : string``, a function that rotates every character
 in a string by 13 places.
 Generally, a plugin comes in the form of a directory following a
 certain structure. To get started, Bro's distribution provides a
 helper script ``aux/bro-aux/plugin-support/init-plugin`` that creates
 a skeleton plugin that can then be customized. Let's use that::
    # mkdir rot13-plugin
    # cd rot13-plugin
    # init-plugin Demo Rot13
 As you can see the script takes two arguments. The first is a
 namespace the plugin will live in, and the second a descriptive name
 for the plugin itself. Bro uses the combination of the two to identify
 a plugin. The namespace serves to avoid naming conflicts between
 plugins written by independent developers; pick, e.g., the name of
 your organisation. The namespace ``Bro`` is reserved for functionality
 distributed by the Bro Project. In our example, the plugin will be
 called ``Demo::Rot13``.
 The ``init-plugin`` script puts a number of files in place. The full
 layout is described later. For now, all we need is
 ``src/functions.bif``. It's initially empty, but we'll add our new bif
 there as follows::
    # cat scripts/functions.bif
    module CaesarCipher;
    function rot13%(s: string%) : string
        %{
        char* rot13 = copy_string(s->CheckString());
        for ( char* p = rot13; *p; p++ )
            {
            char b = islower(*p) ? 'a' : 'A';
            *p  = (*p - b + 13) % 26 + b;
            }
        return new StringVal(new BroString(1, rot13, strlen(rot13)));
        %}
 The syntax of this file is just like any other ``*.bif`` file; we
 won't go into it here.
 Now we can already compile our plugin, we just need to tell the
 Makefile put in place by ``init-plugin`` where the Bro source tree is
 located (Bro needs to have been built there first)::
    # make BRO=/path/to/bro/dist
    [... cmake output ...]
 Now our ``rot13-plugin`` directory has everything that it needs
 for Bro to recognize it as a dynamic plugin. Once we point Bro to it,
 it will pull it in automatically, as we can check with the ``-N``
 option::
    # export BRO_PLUGIN_PATH=/path/to/rot13-plugin
    # bro -N
    [...]
    Plugin: Demo::Rot13 - <Insert brief description of plugin> (dynamic, version 1)
    [...]
 That looks quite good, except for the dummy description that we should
 replace with something nicer so that users will know what our plugin
 is about.  We do this by editing the ``config.description`` line in
 ``src/Plugin.cc``, like this::
    [...]
    plugin::Configuration Configure()
        {
        plugin::Configuration config;
        config.name = "Demo::Rot13";
        config.description = "Caesar cipher rotating a string's characters by 13 places.";
        config.version.major = 1;
        config.version.minor = 0;
        return config;
        }
    [...]
    # make
    [...]
    # bro -N | grep Rot13
    Plugin: Demo::Rot13 - Caesar cipher rotating a string's characters by 13 places. (dynamic, version 1)
 Better. Bro can also show us what exactly the plugin provides with the
 more verbose option ``-NN``::
    # bro -NN
    [...]
    Plugin: Demo::Rot13 - Caesar cipher rotating a string's characters by 13 places. (dynamic, version 1)
        [Function] CaesarCipher::rot13
    [...]
 There's our function. Now let's use it::
    # bro -e 'print CaesarCipher::rot13("Hello")'
    Uryyb
 It works. We next install the plugin along with Bro itself, so that it
 will find it directly without needing the ``BRO_PLUGIN_PATH``
 environment variable. If we first unset the variable, the function
 will no longer be available::
    # unset BRO_PLUGIN_PATH
    # bro -e 'print CaesarCipher::rot13("Hello")'
    error in <command line>, line 1: unknown identifier CaesarCipher::rot13, at or near "CaesarCipher::rot13"
 Once we install it, it works again::
    # make install
    # bro -e 'print CaesarCipher::rot13("Hello")'
    Uryyb
 The installed version went into
 ``<bro-install-prefix>/lib/bro/plugins/Demo_Rot13``.
 We can distribute the plugin in either source or binary form by using
 the Makefile's ``sdist`` and ``bdist`` target, respectively. Both
 create corrsponding tarballs::
    # make sdist
    [...]
    Source distribution in build/sdist/Demo_Rot13.tar.gz
    # make bdist
    [...]
    Binary distribution in build/Demo_Rot13-darwin-x86_64.tar.gz
 The source archive will contain everything in the plugin directory
 except any generated files. The binary archive will contain anything
 needed to install and run the plugin, i.e., just what ``make install``
 puts into place as well. As the binary distribution is
 platform-dependent, its name includes the OS and architecture the
 plugin was built on.
 Plugin Directory Layout
 =======================
 A plugin's directory needs to follow a set of conventions so that Bro
 (1) recognizes it as a plugin, and (2) knows what to load.  While
 ``init-plugin`` takes care of most of this, the following is the full
 story. We'll use ``<base>`` to represent a plugin's top-level
 directory.
 ``<base>/__bro_plugin__``
    A file that marks a directory as containing a Bro plugin. The file
    must exist, and its content must consist of a single line with the
    qualified name of the plugin (e.g., "Demo::Rot13").
 ``<base>/lib/<plugin-name>-<os>-<arch>.so``
    The shared library containing the plugin's compiled code. Bro will
    load this in dynamically at run-time if OS and architecture match
    the current platform.
 ``scripts/``
    A directory with the plugin's custom Bro scripts. When the plugin
    gets activated, this directory will be automatically added to
    ``BROPATH``, so that any scripts/modules inside can be
    "@load"ed.
 ``scripts``/__load__.bro
    A Bro script that will be loaded immediately when the plugin gets
    activated. See below for more information on activating plugins.
 ``lib/bif/``
    Directory with auto-generated Bro scripts that declare the plugin's
    bif elements. The files here are produced by ``bifcl``.
 By convention, a plugin should put its custom scripts into sub folders
 of ``scripts/``, i.e., ``scripts/<script-namespace>/<script>.bro`` to
 avoid conflicts. As usual, you can then put a ``__load__.bro`` in
 there as well so that, e.g., ``@load Demo/Rot13`` could load a whole
 module in the form of multiple individual scripts.
 Note that in addition to the paths above, the ``init-plugin`` helper
 puts some more files and directories in place that help with
 development and installation (e.g., ``CMakeLists.txt``, ``Makefile``,
 and source code in ``src/``). However, all these do not have a special
 meaning for Bro at runtime and aren't necessary for a plugin to
 function.
 ``init-plugin``
 ===============
 ``init-plugin`` puts a basic plugin structure in place that follows
 the above layout and augments it with a CMake build and installation
 system. Plugins with this structure can be used both directly out of
 their source directory (after ``make`` and setting Bro's
 ``BRO_PLUGIN_PATH``), and when installed alongside Bro (after ``make
 install``).
 ``make install`` copies over the ``lib`` and ``scripts`` directories,
 as well as the ``__bro_plugin__`` magic file and the ``README`` (which
 you should customize). One can add further CMake ``install`` rules to
 install additional files if needed.
 ``init-plugin`` will never overwrite existing files, so it's safe to
 rerun in an existing plugin directory; it only put files in place that
 don't exist yet. That also provides a convenient way to revert a file
 back to what ``init-plugin`` created originally: just delete it and
 rerun.
 Activating a Plugin
 ===================
 A plugin needs to be *activated* to make it available to the user.
 Activating a plugin will:
    1. Load the dynamic module
    2. Make any bif items available
    3. Add the ``scripts/`` directory to ``BROPATH``
    4. Load ``scripts/__load__.bro``
 By default, Bro will automatically activate all dynamic plugins found
 in its search path ``BRO_PLUGIN_PATH``. However, in bare mode (``bro
 -b``), no dynamic plugins will be activated by default; instead the
 user can selectively enable individual plugins in scriptland using the
 ``@load-plugin <qualified-plugin-name>`` directive (e.g.,
 ``@load-plugin Demo::Rot13``). Alternatively, one can activate a
 plugin from the command-line by specifying its full name
 (``Demo::Rot13``), or set the environment variable
 ``BRO_PLUGIN_ACTIVATE`` to a list of comma(!)-separated names of
 plugins to unconditionally activate, even in bare mode.
 ``bro -N`` shows activated plugins separately from found but not yet
 activated plugins. Note that plugins compiled statically into Bro are
 always activated, and hence show up as such even in bare mode.
 Plugin Component
 ================
 The following gives additional information about providing individual
 types of functionality via plugins. Note that a single plugin can
 provide more than one type. For example, a plugin could provide
 multiple protocol analyzers at once; or both a logging backend and
 input reader at the same time.
 We now walk briefly through the specifics of providing a specific type
 of functionality (a *component*) through a plugin. We'll focus on
 their interfaces to the plugin system, rather than specifics on
 writing the corresponding logic (usually the best way to get going on
 that is to start with an existing plugin providing a corresponding
 component and adapt that). We'll also point out how the CMake
 infrastructure put in place by the ``init-plugin`` helper script ties
 the various pieces together.
 Bro Scripts
 -----------
 Scripts are easy: just put them into ``scripts/``, as described above.
 The CMake infrastructure will automatically install them, as well
 include them into the source and binary plugin distributions.
 Builtin Language Elements
 -------------------------
 Functions
    TODO
 Events
    TODO
 Types
    TODO
 Protocol Analyzers
 ------------------
 TODO.
 File Analyzers
 --------------
 TODO.
 Logging Writer
 --------------
 Not yet available as plugins.
 Input Reader
 ------------
 Not yet available as plugins.
 Packet Sources
 --------------
 Not yet available as plugins.
 Packet Dumpers
 --------------
 Not yet available as plugins.
 Hooks
 =====
 TODO.
 Testing Plugins
 ===============
 A plugin should come with a test suite to exercise its functionality.
 The ``init-plugin`` script puts in place a basic </btest/README> setup
 to start with. Initially, it comes with a single test that just checks
 that Bro loads the plugin correctly. It won't have a baseline yet, so
 let's get that in place::
    # cd tests
    # btest -d
    [  0%] plugin.loading ... failed
    % 'btest-diff output' failed unexpectedly (exit code 100)
    % cat .diag
    == File ===============================
    Demo::Rot13 - Caesar cipher rotating a string's characters by 13 places. (dynamic, version 1.0)
        [Function] CaesarCipher::rot13
    == Error ===============================
    test-diff: no baseline found.
    =======================================
    # btest -U
    all 1 tests successful
    # cd ..
    # make test
    make -C tests
    make[1]: Entering directory `tests'
    all 1 tests successful
    make[1]: Leaving directory `tests'
 Now let's add a custom test that ensures that our bif works
 correctly::
    # cd tests
    # cat >plugin/rot13.bro
    # @TEST-EXEC: bro %INPUT >output
    # @TEST-EXEC: btest-diff output
    event bro_init()
        {
        print CaesarCipher::rot13("Hello");
        }
 Check the output::
    # btest -d plugin/rot13.bro
    [  0%] plugin.rot13 ... failed
    % 'btest-diff output' failed unexpectedly (exit code 100)
    % cat .diag
    == File ===============================
    Uryyb
    == Error ===============================
    test-diff: no baseline found.
    =======================================
    % cat .stderr
    1 of 1 test failed
 Install the baseline::
    # btest -U plugin/rot13.bro
    all 1 tests successful
 Run the test-suite::
    # btest
    all 2 tests successful
 Debugging Plugins
 =================
 Plugins can use Bro's standard debug logger by using the
 ``PLUGIN_DBG_LOG(<plugin>, <args>)`` macro (defined in
 ``DebugLogger.h``), where ``<plugin>`` is the ``Plugin`` instance and
 ``<args>`` are printf-style arguments, just as with Bro's standard
 debuggging macros.
 At runtime, one then activates a plugin's debugging output with ``-B
 plugin-<name>``, where ``<name>`` is the name of the plugin as
 returned by its ``Configure()`` method, yet with the
 namespace-separator ``::`` replaced with a simple dash. Example: If
 the plugin is called ``Bro::Demo``, use ``-B plugin-Bro-Demo``. As
 usual, the debugging output will be recorded to ``debug.log`` if Bro's
 compiled in debug mode.
 Documenting Plugins
 ===================
 ..todo::
    Integrate all this with Broxygen.
--- a/doc/frameworks/logging-dataseries.rst
+++ b/doc/frameworks/logging-dataseries.rst
@ -1,186 +0,0 @@
 =============================
 Binary Output with DataSeries
 =============================
 .. rst-class:: opening
   Bro's default ASCII log format is not exactly the most efficient
   way for storing and searching large volumes of data. An an
   alternative, Bro comes with experimental support for `DataSeries
   <http://www.hpl.hp.com/techreports/2009/HPL-2009-323.html>`_
   output, an efficient binary format for recording structured bulk
   data. DataSeries is developed and maintained at HP Labs.
 .. contents::
 Installing DataSeries
 ---------------------
 To use DataSeries, its libraries must be available at compile-time,
 along with the supporting *Lintel* package. Generally, both are
 distributed on `HP Labs' web site
 <http://tesla.hpl.hp.com/opensource/>`_. Currently, however, you need
 to use recent development versions for both packages, which you can
 download from github like this::
    git clone http://github.com/dataseries/Lintel
    git clone http://github.com/dataseries/DataSeries
 To build and install the two into ``<prefix>``, do::
    ( cd Lintel     && mkdir build && cd build && cmake -DCMAKE_INSTALL_PREFIX=<prefix> .. && make && make install )
    ( cd DataSeries && mkdir build && cd build && cmake -DCMAKE_INSTALL_PREFIX=<prefix> .. && make && make install )
 Please refer to the packages' documentation for more information about
 the installation process. In particular, there's more information on
 required and optional `dependencies for Lintel
 <https://raw.github.com/dataseries/Lintel/master/doc/dependencies.txt>`_
 and `dependencies for DataSeries
 <https://raw.github.com/dataseries/DataSeries/master/doc/dependencies.txt>`_.
 For users on RedHat-style systems, you'll need the following::
    yum install libxml2-devel boost-devel
 Compiling Bro with DataSeries Support
 -------------------------------------
 Once you have installed DataSeries, Bro's ``configure`` should pick it
 up automatically as long as it finds it in a standard system location.
 Alternatively, you can specify the DataSeries installation prefix
 manually with ``--with-dataseries=<prefix>``. Keep an eye on
 ``configure``'s summary output, if it looks like the following, Bro
 found DataSeries and will compile in the support::
    # ./configure --with-dataseries=/usr/local
    [...]
    ====================|  Bro Build Summary  |=====================
    [...]
    DataSeries:        true
    [...]
    ================================================================
 Activating DataSeries
 ---------------------
 The direct way to use DataSeries is to switch *all* log files over to
 the binary format. To do that, just add ``redef
 Log::default_writer=Log::WRITER_DATASERIES;`` to your ``local.bro``.
 For testing, you can also just pass that on the command line::
    bro -r trace.pcap Log::default_writer=Log::WRITER_DATASERIES
 With that, Bro will now write all its output into DataSeries files
 ``*.ds``. You can inspect these using DataSeries's set of command line
 tools, which its installation process installs into ``<prefix>/bin``.
 For example, to convert a file back into an ASCII representation::
    $ ds2txt conn.log
    [... We skip a bunch of metadata here ...]
    ts uid id.orig_h id.orig_p id.resp_h id.resp_p proto service duration orig_bytes resp_bytes conn_state local_orig missed_bytes history orig_pkts orig_ip_bytes resp_pkts resp_ip_bytes
    1300475167.096535 CRCC5OdDlXe 141.142.220.202 5353 224.0.0.251 5353 udp dns 0.000000 0 0 S0 F 0 D 1 73 0 0
    1300475167.097012 o7XBsfvo3U1 fe80::217:f2ff:fed7:cf65 5353 ff02::fb 5353 udp  0.000000 0 0 S0 F 0 D 1 199 0 0
    1300475167.099816 pXPi1kPMgxb 141.142.220.50 5353 224.0.0.251 5353 udp  0.000000 0 0 S0 F 0 D 1 179 0 0
    1300475168.853899 R7sOc16woCj 141.142.220.118 43927 141.142.2.2 53 udp dns 0.000435 38 89 SF F 0 Dd 1 66 1 117
    1300475168.854378 Z6dfHVmt0X7 141.142.220.118 37676 141.142.2.2 53 udp dns 0.000420 52 99 SF F 0 Dd 1 80 1 127
    1300475168.854837 k6T92WxgNAh 141.142.220.118 40526 141.142.2.2 53 udp dns 0.000392 38 183 SF F 0 Dd 1 66 1 211
    [...]
 (``--skip-all`` suppresses the metadata.)
 Note that the ASCII conversion is *not* equivalent to Bro's default
 output format.
 You can also switch only individual files over to DataSeries by adding
 code like this to your ``local.bro``:
 .. code:: bro
    event bro_init()
        {
        local f = Log::get_filter(Conn::LOG, "default"); # Get default filter for connection log.
        f$writer = Log::WRITER_DATASERIES;               # Change writer type.
        Log::add_filter(Conn::LOG, f);                   # Replace filter with adapted version.
        }
 Bro's DataSeries writer comes with a few tuning options, see
 :doc:`/scripts/base/frameworks/logging/writers/dataseries.bro`.
 Working with DataSeries
 =======================
 Here are a few examples of using DataSeries command line tools to work
 with the output files.
 * Printing CSV::
    $ ds2txt --csv conn.log
    ts,uid,id.orig_h,id.orig_p,id.resp_h,id.resp_p,proto,service,duration,orig_bytes,resp_bytes,conn_state,local_orig,missed_bytes,history,orig_pkts,orig_ip_bytes,resp_pkts,resp_ip_bytes
    1258790493.773208,ZTtgbHvf4s3,192.168.1.104,137,192.168.1.255,137,udp,dns,3.748891,350,0,S0,F,0,D,7,546,0,0
    1258790451.402091,pOY6Rw7lhUd,192.168.1.106,138,192.168.1.255,138,udp,,0.000000,0,0,S0,F,0,D,1,229,0,0
    1258790493.787448,pn5IiEslca9,192.168.1.104,138,192.168.1.255,138,udp,,2.243339,348,0,S0,F,0,D,2,404,0,0
    1258790615.268111,D9slyIu3hFj,192.168.1.106,137,192.168.1.255,137,udp,dns,3.764626,350,0,S0,F,0,D,7,546,0,0
    [...]
  Add ``--separator=X`` to set a different separator.
 * Extracting a subset of columns::
    $ ds2txt --select '*' ts,id.resp_h,id.resp_p --skip-all conn.log
    1258790493.773208 192.168.1.255 137
    1258790451.402091 192.168.1.255 138
    1258790493.787448 192.168.1.255 138
    1258790615.268111 192.168.1.255 137
    1258790615.289842 192.168.1.255 138
    [...]
 * Filtering rows::
    $ ds2txt --where '*' 'duration > 5 && id.resp_p > 1024' --skip-all  conn.ds
    1258790631.532888 V8mV5WLITu5 192.168.1.105 55890 239.255.255.250 1900 udp  15.004568 798 0 S0 F 0 D 6 966 0 0
    1258792413.439596 tMcWVWQptvd 192.168.1.105 55890 239.255.255.250 1900 udp  15.004581 798 0 S0 F 0 D 6 966 0 0
    1258794195.346127 cQwQMRdBrKa 192.168.1.105 55890 239.255.255.250 1900 udp  15.005071 798 0 S0 F 0 D 6 966 0 0
    1258795977.253200 i8TEjhWd2W8 192.168.1.105 55890 239.255.255.250 1900 udp  15.004824 798 0 S0 F 0 D 6 966 0 0
    1258797759.160217 MsLsBA8Ia49 192.168.1.105 55890 239.255.255.250 1900 udp  15.005078 798 0 S0 F 0 D 6 966 0 0
    1258799541.068452 TsOxRWJRGwf 192.168.1.105 55890 239.255.255.250 1900 udp  15.004082 798 0 S0 F 0 D 6 966 0 0
    [...]
 * Calculate some statistics:
    Mean/stddev/min/max over a column::
        $ dsstatgroupby '*' basic duration from conn.ds
        # Begin DSStatGroupByModule
        # processed 2159 rows, where clause eliminated 0 rows
        # count(*), mean(duration), stddev, min, max
        2159, 42.7938, 1858.34, 0, 86370
        [...]
    Quantiles of total connection volume::
        $ dsstatgroupby '*' quantile 'orig_bytes + resp_bytes' from conn.ds
        [...]
        2159 data points, mean 24616 +- 343295 [0,1.26615e+07]
        quantiles about every 216 data points:
        10%: 0, 124, 317, 348, 350, 350, 601, 798, 1469
        tails: 90%: 1469, 95%: 7302, 99%: 242629, 99.5%: 1226262
        [...]
 The ``man`` pages for these tools show further options, and their
 ``-h`` option gives some more information (either can be a bit cryptic
 unfortunately though).
 Deficiencies
 ------------
 Due to limitations of the DataSeries format, one cannot inspect its
 files before they have been fully written. In other words, when using
 DataSeries, it's currently not possible to inspect the live log
 files inside the spool directory before they are rotated to their
 final location. It seems that this could be fixed with some effort,
 and we will work with DataSeries development team on that if the
 format gains traction among Bro users.
 Likewise, we're considering writing custom command line tools for
 interacting with DataSeries files, making that a bit more convenient
 than what the standard utilities provide.
--- a/doc/frameworks/logging-elasticsearch.rst
+++ b/doc/frameworks/logging-elasticsearch.rst
@ -1,89 +0,0 @@
 =========================================
 Indexed Logging Output with ElasticSearch
 =========================================
 .. rst-class:: opening
   Bro's default ASCII log format is not exactly the most efficient
   way for searching large volumes of data. ElasticSearch
   is a new data storage technology for dealing with tons of data.
   It's also a search engine built on top of Apache's Lucene
   project. It scales very well, both for distributed indexing and 
   distributed searching.
 .. contents::
 Warning
 -------
 This writer plugin is still in testing and is not yet recommended for
 production use!  The approach to how logs are handled in the plugin is "fire
 and forget" at this time, there is no error handling if the server fails to
 respond successfully to the insertion request.
 Installing ElasticSearch
 ------------------------
 Download the latest version from: http://www.elasticsearch.org/download/.
 Once extracted, start ElasticSearch with::
 # ./bin/elasticsearch
 For more detailed information, refer to the ElasticSearch installation
 documentation: http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/setup.html
 Compiling Bro with ElasticSearch Support
 ----------------------------------------
 First, ensure that you have libcurl installed then run configure::
    # ./configure
    [...]
    ====================|  Bro Build Summary  |=====================
    [...]
    cURL:              true
    [...]
    ElasticSearch:     true
    [...]
    ================================================================
 Activating ElasticSearch
 ------------------------
 The easiest way to enable ElasticSearch output is to load the
 tuning/logs-to-elasticsearch.bro script.  If you are using BroControl,
 the following line in local.bro will enable it:
 .. console::
    @load tuning/logs-to-elasticsearch
 With that, Bro will now write most of its logs into ElasticSearch in addition
 to maintaining the Ascii logs like it would do by default.  That script has
 some tunable options for choosing which logs to send to ElasticSearch, refer
 to the autogenerated script documentation for those options.
 There is an interface being written specifically to integrate with the data
 that Bro outputs into ElasticSearch named Brownian.  It can be found here::
    https://github.com/grigorescu/Brownian
 Tuning
 ------
 A common problem encountered with ElasticSearch is too many files being held
 open.  The ElasticSearch website has some suggestions on how to increase the
 open file limit.
  - http://www.elasticsearch.org/tutorials/too-many-open-files/
 TODO
 ----
 Lots.
 - Perform multicast discovery for server.
 - Better error detection.
 - Better defaults (don't index loaded-plugins, for instance).
 - 
--- a/doc/frameworks/logging.rst
+++ b/doc/frameworks/logging.rst
@ -38,7 +38,7 @@ Bro's logging interface is built around three main abstractions:
    Writers
        A writer defines the actual output format for the information
        being logged. At the moment, Bro comes with only one type of
-        writer, which produces tab separated ASCII files. In the 
+        writer, which produces tab separated ASCII files. In the
        future we will add further writers, like for binary output and
        direct logging into a database.
@ -98,7 +98,7 @@ Note the fields that are set for the filter:
    ``include``
        A set limiting the fields to the ones given. The names
        correspond to those in the :bro:type:`Conn::Info` record, with
-        sub-records unrolled by concatenating fields (separated with 
+        sub-records unrolled by concatenating fields (separated with
        dots).
 Using the code above, you will now get a new log file ``origs.log``
@ -155,7 +155,7 @@ that returns the desired path:
        {
        local filter: Log::Filter = [$name="conn-split", $path_func=split_log, $include=set("ts", "id.orig_h")];
        Log::add_filter(Conn::LOG, filter);
-        }   
+        }
 Running this will now produce two files, ``local.log`` and
 ``remote.log``, with the corresponding entries. One could extend this
@ -263,7 +263,7 @@ specific destination exceeds a certain duration:
 .. code:: bro
    redef enum Notice::Type += {
-        ## Indicates that a connection remained established longer 
+        ## Indicates that a connection remained established longer
        ## than 5 minutes.
        Long_Conn_Found
    };
@ -271,8 +271,8 @@ specific destination exceeds a certain duration:
    event Conn::log_conn(rec: Conn::Info)
        {
        if ( rec$duration > 5mins )
-            NOTICE([$note=Long_Conn_Found, 
+            NOTICE([$note=Long_Conn_Found,
-                    $msg=fmt("unusually long conn to %s", rec$id$resp_h), 
+                    $msg=fmt("unusually long conn to %s", rec$id$resp_h),
                    $id=rec$id]);
        }
@ -335,11 +335,11 @@ example for the ``Foo`` module:
        # Define a hook event. By convention, this is called
        # "log_<stream>".
        global log_foo: event(rec: Info);
-        
+
    }
    # This event should be handled at a higher priority so that when
-    # users modify your stream later and they do it at priority 0, 
+    # users modify your stream later and they do it at priority 0,
    # their code runs after this.
    event bro_init() &priority=5
        {
@ -356,7 +356,7 @@ it easily accessible across event handlers:
        foo: Info &optional;
        }
-Now you can use the :bro:id:`Log::write` method to output log records and 
+Now you can use the :bro:id:`Log::write` method to output log records and
 save the logged ``Foo::Info`` record into the connection record:
 .. code:: bro
@ -380,11 +380,11 @@ uncommon to need to delete that data before the end of the connection.
 Other Writers
 -------------
-Bro supports the following output formats other than ASCII:
+Bro supports the following built-in output formats other than ASCII:
 .. toctree::
   :maxdepth: 1
   logging-dataseries
   logging-elasticsearch
   logging-input-sqlite
 Further formats are available as external plugins.
--- a/doc/index.rst
+++ b/doc/index.rst
@ -31,6 +31,7 @@ Using Bro Section
   httpmonitor/index.rst
   broids/index.rst
   mimestats/index.rst
   scripting/index.rst
 ..
@ -40,7 +41,6 @@ Reference Section
 .. toctree::
   :maxdepth: 2
   scripting/index.rst
   frameworks/index.rst
   script-reference/index.rst
   components/index.rst
--- a/doc/install/install.rst
+++ b/doc/install/install.rst
@ -91,7 +91,6 @@ build time:
    * LibGeoIP (for geolocating IP addresses)
    * sendmail (enables Bro and BroControl to send mail)
    * gawk (enables all features of bro-cut)
    * curl (used by a Bro script that implements active HTTP)
    * gperftools (tcmalloc is used to improve memory and CPU usage)
    * ipsumdump (for trace-summary; http://www.cs.ucla.edu/~kohler/ipsumdump)
@ -181,7 +180,7 @@ automatically.  Finally, use ``make install-aux`` to install some of
 the other programs that are in the ``aux/bro-aux`` directory.
 OpenBSD users, please see our `FAQ
-<http://www.bro.org/documentation/faq.html>`_ if you are having
+<//www.bro.org/documentation/faq.html>`_ if you are having
 problems installing Bro.
 Finally, if you want to build the Bro documentation (not required, because
--- a/doc/logs/index.rst
+++ b/doc/logs/index.rst
@ -162,8 +162,8 @@ tools like ``awk`` allow you to indicate the log file as a command
 line option, bro-cut only takes input through redirection such as
 ``|`` and ``<``.  There are a couple of ways to direct log file data
 into ``bro-cut``, each dependent upon the type of log file you're
-processing.  A caveat of its use, however, is that the 8 lines of
+processing.  A caveat of its use, however, is that all of the
-header data must be present.
+header lines must be present.
 .. note::
@ -177,8 +177,8 @@ moving the current log file into a directory with format
 ``YYYY-MM-DD`` and gzip compressing the file with a file format that
 includes the log file type and time range of the file.  In the case of
 processing a compressed log file you simply adjust your command line
-tools to use the complementary ``z*`` versions of commands such as cat
+tools to use the complementary ``z*`` versions of commands such as ``cat``
-(``zcat``), ``grep`` (``zgrep``), and ``head`` (``zhead``).
+(``zcat``) or ``grep`` (``zgrep``).
 Working with Timestamps
 -----------------------
--- a/doc/quickstart/index.rst
+++ b/doc/quickstart/index.rst
@ -1,5 +1,5 @@
-.. _FAQ: http://www.bro.org/documentation/faq.html
+.. _FAQ: //www.bro.org/documentation/faq.html
 .. _quickstart:
@ -234,7 +234,7 @@ is valid before installing it and then restarting the Bro instance:
 .. console::
   [BroControl] > check
-   bro is ok.
+   bro scripts are ok.
   [BroControl] > install
   removing old policies in /usr/local/bro/spool/policy/site ... done.
   removing old policies in /usr/local/bro/spool/policy/auto ... done.
@ -250,15 +250,15 @@ is valid before installing it and then restarting the Bro instance:
 Now that the SSL notice is ignored, let's look at how to send an email on
 the SSH notice.  The notice framework has a similar option called
-``emailed_types``, but that can't differentiate between SSH servers and we
+``emailed_types``, but using that would generate email for all SSH servers and
-only want email for logins to certain ones.  Then we come to the ``PolicyItem``
+we only want email for logins to certain ones.  There is a ``policy`` hook
-record and ``policy`` set and realize that those are actually what get used
+that is actually what is used to implement the simple functionality of
-to implement the simple functionality of ``ignored_types`` and
+``ignored_types`` and
 ``emailed_types``, but it's extensible such that the condition and action taken
 on notices can be user-defined.
-In ``local.bro``, let's add a new ``PolicyItem`` record to the ``policy`` set
+In ``local.bro``, let's define a new ``policy`` hook handler body
-that only takes the email action for SSH logins to a defined set of servers:
+that takes the email action for SSH logins only for a defined set of servers:
 .. code:: bro
@ -276,9 +276,9 @@ that only takes the email action for SSH logins to a defined set of servers:
 You'll just have to trust the syntax for now, but what we've done is
 first declare our own variable to hold a set of watched addresses,
-``watched_servers``; then added a record to the policy that will generate
+``watched_servers``; then added a hook handler body to the policy that will
-an email on the condition that the predicate function evaluates to true, which
+generate an email whenever the notice type is an SSH login and the responding
-is whenever the notice type is an SSH login and the responding host stored
+host stored
 inside the ``Info`` record's connection field is in the set of watched servers.
 .. note:: Record field member access is done with the '$' character
--- a/doc/scripting/data_struct_table_declaration.bro
+++ b/doc/scripting/data_struct_table_declaration.bro
@ -1,13 +1,19 @@
 event bro_init()
    {
    # Declaration of the table.
    local ssl_services: table[string] of port;
-    
+
    # Initialize the table.
    ssl_services = table(["SSH"] = 22/tcp, ["HTTPS"] = 443/tcp);
    # Insert one key-yield pair into the table.
    ssl_services["IMAPS"] = 993/tcp;
    # Check if the key "SMTPS" is not in the table.
    if ( "SMTPS" !in ssl_services )
        ssl_services["SMTPS"] = 587/tcp;
    # Iterate over each key in the table.
    for ( k in ssl_services )
        print fmt("Service Name:  %s - Common Port: %s", k, ssl_services[k]);
    }
--- a/doc/scripting/framework_logging_factorial_02.bro
+++ b/doc/scripting/framework_logging_factorial_02.bro
@ -1,8 +1,10 @@
 module Factor;
 export {
    # Append the value LOG to the Log::ID enumerable.
    redef enum Log::ID += { LOG };
    # Define a new type called Factor::Info.
    type Info: record {
        num:           count &log;
        factorial_num: count &log;
@ -20,6 +22,7 @@ function factorial(n: count): count
 event bro_init()
    {
    # Create the logging stream.
    Log::create_stream(LOG, [$columns=Info]);
    }
--- a/doc/scripting/framework_notice_shortcuts_01.bro
+++ b/doc/scripting/framework_notice_shortcuts_01.bro
@ -2,7 +2,6 @@
@load base/protocols/ssh/
 redef Notice::emailed_types += {
-    SSH::Interesting_Hostname_Login,
+    SSH::Interesting_Hostname_Login
    SSH::Login
 };
--- a/doc/scripting/framework_notice_shortcuts_02.bro
+++ b/doc/scripting/framework_notice_shortcuts_02.bro
@ -3,5 +3,4 @@
 redef Notice::type_suppression_intervals += {
    [SSH::Interesting_Hostname_Login] = 1day,
    [SSH::Login] = 12hrs,
 };
--- a/doc/scripting/index.rst
+++ b/doc/scripting/index.rst
@ -54,7 +54,8 @@ script and much more in following sections.
 .. btest-include:: ${BRO_SRC_ROOT}/scripts/policy/frameworks/files/detect-MHR.bro
   :lines: 4-6
-Lines 3 to 5 of the script process the ``__load__.bro`` script in the
+The first part of the script consists of ``@load`` directives which 
 process the ``__load__.bro`` script in the
 respective directories being loaded.  The ``@load`` directives are
 often considered good practice or even just good manners when writing
 Bro scripts to make sure they can be used on their own. While it's unlikely that in a
@ -78,29 +79,37 @@ of the :bro:id:`NOTICE` function to generate notices of type
 ``TeamCymruMalwareHashRegistry::Match`` as done in the next section.  Notices
 allow Bro to generate some kind of extra notification beyond its
 default log types.  Often times, this extra notification comes in the
-form of an email generated and sent to a preconfigured address, but can be altered
+form of an email generated and sent to a preconfigured address, but can
-depending on the needs of the deployment.  The export section is finished off with
+be altered depending on the needs of the deployment.  The export section
-the definition of two constants that list the kind of files we want to match against and
+is finished off with the definition of a few constants that list the kind
-the minimum percentage of detection threshold in which we are interested.
+of files we want to match against and the minimum percentage of
 detection threshold in which we are interested.
-Up until this point, the script has merely done some basic setup.  With the next section,
+Up until this point, the script has merely done some basic setup.  With
-the script starts to define instructions to take in a given event.
+the next section, the script starts to define instructions to take in
 a given event.
 .. btest-include:: ${BRO_SRC_ROOT}/scripts/policy/frameworks/files/detect-MHR.bro
   :lines: 38-71
 The workhorse of the script is contained in the event handler for
 ``file_hash``.  The :bro:see:`file_hash` event allows scripts to access
-the information associated with a file for which Bro's file analysis framework has
+the information associated with a file for which Bro's file analysis
-generated a hash.  The event handler is passed the file itself as ``f``, the type of digest
+framework has generated a hash.  The event handler is passed the
-algorithm used as ``kind`` and the hash generated as ``hash``.
+file itself as ``f``, the type of digest algorithm used as ``kind``
 and the hash generated as ``hash``.
-On line 3, an ``if`` statement is used to check for the correct type of hash, in this case
+In the ``file_hash`` event handler, there is an ``if`` statement that is used
-a SHA1 hash.  It also checks for a mime type we've defined as being of interest as defined in the
+to check for the correct type of hash, in this case
-constant ``match_file_types``.  The comparison is made against the expression ``f$mime_type``, which uses
+a SHA1 hash.  It also checks for a mime type we've defined as
-the ``$`` dereference operator to check the value ``mime_type`` inside the variable ``f``.  Once both
+being of interest as defined in the constant ``match_file_types``.
-values resolve to true, a local variable is defined to hold a string comprised of the SHA1 hash concatenated
+The comparison is made against the expression ``f$mime_type``, which uses
-with ``.malware.hash.cymru.com``; this value will be the domain queried in the malware hash registry.
+the ``$`` dereference operator to check the value ``mime_type``
 inside the variable ``f``.  If the entire expression evaluates to true,
 then a helper function is called to do the rest of the work.  In that
 function, a local variable is defined to hold a string comprised of
 the SHA1 hash concatenated with ``.malware.hash.cymru.com``; this
 value will be the domain queried in the malware hash registry.
 The rest of the script is contained within a ``when`` block.  In
 short, a ``when`` block is used when Bro needs to perform asynchronous
@ -111,24 +120,28 @@ this event continues and upon receipt of the values returned by
 :bro:id:`lookup_hostname_txt`, the ``when`` block is executed.  The
 ``when`` block splits the string returned into a portion for the date on which
 the malware was first detected and the detection rate by splitting on an text space
-and storing the values returned in a local table variable.  In line 12, if the table
+and storing the values returned in a local table variable.
-returned by ``split1`` has two entries, indicating a successful split, we store the detection
+In the ``do_mhr_lookup`` function, if the table
-date in ``mhr_first_detected`` and the rate in ``mhr_detect_rate`` on lines 14 and 15 respectively
+returned by ``split1`` has two entries, indicating a successful split, we
 store the detection
 date in ``mhr_first_detected`` and the rate in ``mhr_detect_rate``
 using the appropriate conversion functions.  From this point on, Bro knows it has seen a file
 transmitted which has a hash that has been seen by the Team Cymru Malware Hash Registry, the rest
 of the script is dedicated to producing a notice.
-On line 17, the detection time is processed into a string representation and stored in
+The detection time is processed into a string representation and stored in
-``readable_first_detected``.  The script then compares the detection rate against the
+``readable_first_detected``.  The script then compares the detection rate
-``notice_threshold`` that was defined earlier.  If the detection rate is high enough, the script
+against the ``notice_threshold`` that was defined earlier.  If the
-creates a concise description of the notice on line 22, a possible URL to check the sample against
+detection rate is high enough, the script creates a concise description
-``virustotal.com``'s database, and makes the call to :bro:id:`NOTICE` to hand the relevant information
+of the notice and stores it in the ``message`` variable.  It also
-off to the Notice framework.
+creates a possible URL to check the sample against
 ``virustotal.com``'s database, and makes the call to :bro:id:`NOTICE`
 to hand the relevant information off to the Notice framework.
-In approximately 25 lines of code, Bro provides an amazing
+In approximately a few dozen lines of code, Bro provides an amazing
 utility that would be incredibly difficult to implement and deploy
-with other products.  In truth, claiming that Bro does this in 25
+with other products.  In truth, claiming that Bro does this in such a small
-lines is a misdirection; there is a truly massive number of things
+number of lines is a misdirection; there is a truly massive number of things
 going on behind-the-scenes in Bro, but it is the inclusion of the
 scripting language that gives analysts access to those underlying
 layers in a succinct and well defined manner.
@ -180,7 +193,7 @@ event definition used by Bro.  As Bro detects DNS requests being
 issued by an originator, it issues this event and any number of
 scripts then have access to the data Bro passes along with the event.
 In this example, Bro passes not only the message, the query, query
-type and query class for the DNS request, but also a then record used
+type and query class for the DNS request, but also a record used
 for the connection itself.
 The Connection Record Data Type
@ -210,8 +223,7 @@ into the connection record data type will be
 :bro:id:`connection_state_remove` .  As detailed in the in-line
 documentation, Bro generates this event just before it decides to
 remove this event from memory, effectively forgetting about it.  Let's
-take a look at a simple script, stored as
+take a look at a simple example script, that will output the connection record
 ``connection_record_01.bro``, that will output the connection record
 for a single connection.
 .. btest-include:: ${DOC_ROOT}/scripting/connection_record_01.bro
@ -243,12 +255,12 @@ of reference for accessing data in a script.
 Bro makes extensive use of nested data structures to store state and
 information gleaned from the analysis of a connection as a complete
 unit.  To break down this collection of information, you will have to
-make use of use Bro's field delimiter ``$``.  For example, the
+make use of Bro's field delimiter ``$``.  For example, the
 originating host is referenced by ``c$id$orig_h`` which if given a
 narrative relates to ``orig_h`` which is a member of ``id`` which is
 a member of the data structure referred to as ``c`` that was passed
-into the event handler." Given that the responder port
+into the event handler. Given that the responder port
-(``c$id$resp_p``) is ``53/tcp``, it's likely that Bro's base HTTP scripts
+``c$id$resp_p`` is ``53/tcp``, it's likely that Bro's base HTTP scripts
 can further populate the connection record.  Let's load the
 ``base/protocols/http`` scripts and check the output of our script. 
@ -276,7 +288,7 @@ As mentioned above, including the appropriate ``@load`` statements is
 not only good practice, but can also help to indicate which
 functionalities are being used in a script.  Take a second to run the
 script without the ``-b`` flag and check the output when all of Bro's
-functionality is applied to the tracefile.  
+functionality is applied to the trace file.  
 Data Types and Data Structures
 ==============================
@ -384,9 +396,12 @@ which it was declared.  Local variables tend to be used for values
 that are only needed within a specific scope and once the processing
 of a script passes beyond that scope and no longer used, the variable
 is deleted. Bro maintains names of locals separately from globally
-visible ones, an example of which is illustrated below.  The script
+visible ones, an example of which is illustrated below.
-executes the event handler :bro:id:`bro_init` which in turn calls the
+
-function ``add_two(i: count)`` with an argument of ``10``.  Once Bro
+.. btest-include:: ${DOC_ROOT}/scripting/data_type_local.bro
 The script executes the event handler :bro:id:`bro_init` which in turn calls
 the function ``add_two(i: count)`` with an argument of ``10``.  Once Bro
 enters the ``add_two`` function, it provisions a locally scoped
 variable called ``added_two`` to hold the value of ``i+2``, in this
 case, ``12``.  The ``add_two`` function then prints the value of the
@ -398,8 +413,6 @@ processing the ``bro_init`` function, the variable called ``test`` is
 no longer in scope and, since there exist no other references to the
 value ``12``, the value is also deleted.  
 .. btest-include:: ${DOC_ROOT}/scripting/data_type_local.bro
 Data Structures
 ---------------
@ -506,20 +519,7 @@ Tables
 A table in Bro is a mapping of a key to a value or yield.  While the
 values don't have to be unique, each key in the table must be unique
-to preserve a one-to-one mapping of keys to values.  In the example
+to preserve a one-to-one mapping of keys to values.
 below, we've compiled a table of SSL-enabled services and their common
 ports.  The explicit declaration and constructor for the table on
 lines 5 and 7 lay out the data types of the keys (strings) and the
 data types of the yields (ports) and then fill in some sample key and
 yield pairs.  Line 8 shows how to use a table accessor to insert one
 key-yield pair into the table.  When using the ``in`` operator on a table,
 you are effectively working with the keys of the table.  In the case
 of an ``if`` statement, the ``in`` operator will check for membership among
 the set of keys and return a true or false value.  As seen on line 10,
 we are checking if ``SMTPS`` is not in the set of keys for the
 ssl_services table and if the condition holds true, we add the
 key-yield pair to the table.  Line 13 shows the use of a ``for`` statement
 to iterate over each key currently in the table.  
 .. btest-include:: ${DOC_ROOT}/scripting/data_struct_table_declaration.bro
@ -527,6 +527,21 @@ to iterate over each key currently in the table.
    @TEST-EXEC: btest-rst-cmd bro ${DOC_ROOT}/scripting/data_struct_table_declaration.bro
 In this example,
 we've compiled a table of SSL-enabled services and their common
 ports.  The explicit declaration and constructor for the table are on
 two different lines and lay out the data types of the keys (strings) and the
 data types of the yields (ports) and then fill in some sample key and
 yield pairs.  You can also use a table accessor to insert one
 key-yield pair into the table.  When using the ``in``
 operator on a table, you are effectively working with the keys of the table.
 In the case of an ``if`` statement, the ``in`` operator will check for
 membership among the set of keys and return a true or false value.
 The example shows how to check if ``SMTPS`` is not in the set
 of keys for the ``ssl_services`` table and if the condition holds true,
 we add the key-yield pair to the table.  Finally, the example shows how
 to use a ``for`` statement to iterate over each key currently in the table.  
 Simple examples aside, tables can become extremely complex as the keys
 and values for the table become more intricate.  Tables can have keys
 comprised of multiple data types and even a series of elements called
@ -535,9 +550,15 @@ Bro implies a cost in complexity for the person writing the scripts
 but pays off in effectiveness given the power of Bro as a network
 security platform.
-The script below shows a sample table of strings indexed by two
+.. btest-include:: ${DOC_ROOT}/scripting/data_struct_table_complex.bro
 .. btest:: data_struct_table_complex
    @TEST-EXEC: btest-rst-cmd bro -b ${DOC_ROOT}/scripting/data_struct_table_complex.bro
 This script shows a sample table of strings indexed by two
 strings, a count, and a final string.  With a tuple acting as an
-aggregate key, the order is the important as a change in order would
+aggregate key, the order is important as a change in order would
 result in a new key.  Here, we're using the table to track the
 director, studio, year or release, and lead actor in a series of
 samurai flicks.  It's important to note that in the case of the ``for``
@ -546,14 +567,9 @@ iterate over, say, the directors; we have to iterate with the exact
 format as the keys themselves.  In this case, we need squared brackets
 surrounding four temporary variables to act as a collection for our
 iteration.  While this is a contrived example, we could easily have
-had keys containing IP addresses (``addr``), ports (``port``) and even a ``string``
+had keys containing IP addresses (``addr``), ports (``port``) and even
-calculated as the result of a reverse hostname lookup.
+a ``string`` calculated as the result of a reverse hostname lookup.
 .. btest-include:: ${DOC_ROOT}/scripting/data_struct_table_complex.bro
 .. btest:: data_struct_table_complex
    @TEST-EXEC: btest-rst-cmd bro -b ${DOC_ROOT}/scripting/data_struct_table_complex.bro
 Vectors
 ~~~~~~~
@ -657,7 +673,7 @@ using a 20 bit subnet mask.
 Because this is a script that doesn't use any kind of network
 analysis, we can handle the event :bro:id:`bro_init` which is always
-generated by Bro's core upon startup.  On lines six and seven, two
+generated by Bro's core upon startup.  In the example script, two
 locally scoped vectors are created to hold our lists of subnets and IP
 addresses respectively.  Then, using a set of nested ``for`` loops, we
 iterate over every subnet and every IP address and use an ``if``
@ -714,7 +730,7 @@ Bro supports ``usec``, ``msec``, ``sec``, ``min``, ``hr``, or ``day`` which repr
 microseconds, milliseconds, seconds, minutes, hours, and days
 respectively.  In fact, the interval data type allows for a surprising
 amount of variation in its definitions.  There can be a space between
-the numeric constant or they can crammed together like a temporal
+the numeric constant or they can be crammed together like a temporal
 portmanteau.  The time unit can be either singular or plural.  All of
 this adds up to to the fact that both ``42hrs`` and ``42 hr`` are
 perfectly valid and logically equivalent in Bro.  The point, however,
@ -760,7 +776,7 @@ string against which it will be tested to be on the right.
 In the sample above, two local variables are declared to hold our
 sample sentence and regular expression.  Our regular expression in
 this case will return true if the string contains either the word
-``quick`` or the word ``fox``. The ``if`` statement on line six uses
+``quick`` or the word ``fox``. The ``if`` statement in the script uses
 embedded matching and the ``in`` operator to check for the existence
 of the pattern within the string.  If the statement resolves to true,
 :bro:id:`split` is called to break the string into separate pieces.
@ -768,8 +784,8 @@ of the pattern within the string.  If the statement resolves to true,
 table of strings indexed by a count.  Each element of the table will
 be the segments before and after any matches against the pattern but
 excluding the actual matches.  In this case, our pattern matches
-twice, and results in a table with three entries.  Lines 11 through 13
+twice, and results in a table with three entries.  The ``print`` statements
-print the contents of the table in order.  
+in the script will print the contents of the table in order.  
 .. btest:: data_type_pattern
@ -780,7 +796,7 @@ inequality operators through the ``==`` and ``!=`` operators
 respectively. When used in this manner however, the string must match
 entirely to resolve to true.  For example, the script below uses two
 ternary conditional statements to illustrate the use of the ``==``
-operators with patterns.  On lines 8 and 11 the output is altered based
+operator with patterns.  The output is altered based
 on the result of the comparison between the pattern and the string.  
 .. btest-include:: ${DOC_ROOT}/scripting/data_type_pattern_02.bro
@ -803,7 +819,7 @@ with the ``typedef`` and ``struct`` keywords, Bro allows you to cobble
 together new data types to suit the needs of your situation.
 When combined with the ``type`` keyword, ``record`` can generate a
-composite type.  We have, in fact, already encountered a a complex
+composite type.  We have, in fact, already encountered a complex
 example of the ``record`` data type in the earlier sections, the
 :bro:type:`connection` record passed to many events. Another one,
 :bro:type:`Conn::Info`, which corresponds to the fields logged into
@ -915,11 +931,7 @@ through a contrived example of simply logging the digits 1 through 10
 and their corresponding factorial to the default ASCII log writer. 
 It's always best to work through the problem once, simulating the
 desired output with ``print`` and ``fmt`` before attempting to dive
-into the Logging Framework.  Below is a script that defines a
+into the Logging Framework.
 factorial function to recursively calculate the factorial of a
 unsigned integer passed as an argument to the function.  Using
 ``print`` and  :bro:id:`fmt` we can ensure that Bro can perform these
 calculations correctly as well get an idea of the answers ourselves.
 .. btest-include:: ${DOC_ROOT}/scripting/framework_logging_factorial_01.bro
@ -927,19 +939,28 @@ calculations correctly as well get an idea of the answers ourselves.
   @TEST-EXEC: btest-rst-cmd bro ${DOC_ROOT}/scripting/framework_logging_factorial_01.bro
 This script defines a factorial function to recursively calculate the
 factorial of a unsigned integer passed as an argument to the function.  Using
 ``print`` and  :bro:id:`fmt` we can ensure that Bro can perform these
 calculations correctly as well get an idea of the answers ourselves.
 The output of the script aligns with what we expect so now it's time
-to integrate the Logging Framework.  As mentioned above we have to
+to integrate the Logging Framework.
-perform a few steps before we can issue the :bro:id:`Log::write` 
+
-method and produce a logfile.  As we are working within a namespace
+.. btest-include:: ${DOC_ROOT}/scripting/framework_logging_factorial_02.bro
-and informing an outside entity of workings and data internal to the
+
-namespace, we use an ``export`` block.  First we need to inform Bro
+As mentioned above we have to perform a few steps before we can
 issue the :bro:id:`Log::write` method and produce a logfile.
 As we are working within a namespace and informing an outside
 entity of workings and data internal to the namespace, we use
 an ``export`` block.  First we need to inform Bro
 that we are going to be adding another Log Stream by adding a value to
-the :bro:type:`Log::ID` enumerable.  In line 6 of the script, we append the
+the :bro:type:`Log::ID` enumerable.  In this script, we append the
 value ``LOG`` to the ``Log::ID`` enumerable, however due to this being in
 an export block the value appended to ``Log::ID`` is actually
 ``Factor::Log``.  Next, we need to define the name and value pairs
-that make up the data of our logs and dictate its format.  Lines 8
+that make up the data of our logs and dictate its format.  This script
-through 11 define a new datatype called an ``Info`` record (actually,
+defines a new record datatype called ``Info`` (actually,
 ``Factor::Info``) with two fields, both unsigned integers. Each of the
 fields in the ``Factor::Log`` record type include the ``&log``
 attribute, indicating that these fields should be passed to the
@ -947,16 +968,14 @@ Logging Framework when ``Log::write`` is called.  Were there to be
 any name value pairs without the ``&log`` attribute, those fields
 would simply be ignored during logging but remain available for the
 lifespan of the variable.  The next step is to create the logging
-stream with :bro:id:`Log::create_stream` which takes a Log::ID and a
+stream with :bro:id:`Log::create_stream` which takes a ``Log::ID`` and a
-record as its arguments.  In this example, on line 25, we call the
+record as its arguments.  In this example, we call the
 ``Log::create_stream`` method and pass ``Factor::LOG`` and the
 ``Factor::Info`` record as arguments. From here on out, if we issue
 the ``Log::write`` command with the correct ``Log::ID`` and a properly
 formatted ``Factor::Info`` record, a log entry will be generated.  
-.. btest-include:: ${DOC_ROOT}/scripting/framework_logging_factorial_02.bro
+Now, if we run this script, instead of generating
 Now, if we run the new version of the script, instead of generating
 logging information to stdout, no output is created.  Instead the
 output is all in ``factor.log``, properly formatted and organized.
@ -995,13 +1014,13 @@ remaining logs to factor.log.
   :lines: 38-62
   :linenos:
-To dynamically alter the file in which a stream writes its logs a
+To dynamically alter the file in which a stream writes its logs, a
-filter can specify function returns a string to be used as the
+filter can specify a function that returns a string to be used as the
 filename for the current call to ``Log::write``. The definition for
 this function has to take as its parameters a ``Log::ID`` called id, a
 string called ``path`` and the appropriate record type for the logs called
-``rec``.  You can see the definition of ``mod5`` used in this example on
+``rec``.  You can see the definition of ``mod5`` used in this example
-line one conforms to that requirement.  The function simply returns
+conforms to that requirement.  The function simply returns
 ``factor-mod5`` if the factorial is divisible evenly by 5, otherwise, it
 returns ``factor-non5``.  In the additional ``bro_init`` event
 handler, we define a locally scoped ``Log::Filter`` and assign it a
@ -1074,7 +1093,8 @@ make a call to :bro:id:`NOTICE` supplying it with an appropriate
 :bro:type:`Notice::Info` record.  Often times the call to ``NOTICE``
 includes just the ``Notice::Type``, and a concise message.  There are
 however, significantly more options available when raising notices as
-seen in the table below.  The only field in the table below whose
+seen in the definition of :bro:type:`Notice::Info`.  The only field in
 ``Notice::Info`` whose
 attributes make it a required field is the ``note`` field.  Still,
 good manners are always important and including a concise message in
 ``$msg`` and, where necessary, the contents of the connection record
@ -1086,57 +1106,6 @@ that are commonly included, ``$identifier`` and ``$suppress_for`` are
 built around the automated suppression feature of the Notice Framework
 which we will cover shortly.  
 .. todo::
    Once the link to ``Notice::Info`` work I think we should take out
    the table. That's too easy to get out of date.
 +---------------------+------------------------------------------------------------------+----------------+----------------------------------------+
 | Field               | Type                                                             | Attributes     | Use                                    |
 +=====================+==================================================================+================+========================================+
 | ts                  | time                                                             | &log &optional | The time of the notice                 |
 +---------------------+------------------------------------------------------------------+----------------+----------------------------------------+
 | uid                 | string                                                           | &log &optional | A unique connection ID                 |
 +---------------------+------------------------------------------------------------------+----------------+----------------------------------------+
 | id                  | conn_id                                                          | &log &optional | A 4-tuple to identify endpoints        |
 +---------------------+------------------------------------------------------------------+----------------+----------------------------------------+
 | conn                | connection                                                       | &optional      | Shorthand for the uid and id           |
 +---------------------+------------------------------------------------------------------+----------------+----------------------------------------+
 | iconn               | icmp_conn                                                        | &optional      | Shorthand for the uid and id           |
 +---------------------+------------------------------------------------------------------+----------------+----------------------------------------+
 | proto               | transport_proto                                                  | &log &optional | Transport protocol                     |
 +---------------------+------------------------------------------------------------------+----------------+----------------------------------------+
 | note                | Notice::Type                                                     | &log           | The Notice::Type of the notice         |
 +---------------------+------------------------------------------------------------------+----------------+----------------------------------------+
 | msg                 | string                                                           | &log &optional | Human readable message                 |
 +---------------------+------------------------------------------------------------------+----------------+----------------------------------------+
 | sub                 | string                                                           | &log &optional | Human readable message                 |
 +---------------------+------------------------------------------------------------------+----------------+----------------------------------------+
 | src                 | addr                                                             | &log &optional | Source address if no conn_id           |
 +---------------------+------------------------------------------------------------------+----------------+----------------------------------------+
 | dst                 | addr                                                             | &log &optional | Destination addr if no conn_id         |
 +---------------------+------------------------------------------------------------------+----------------+----------------------------------------+
 | p                   | port                                                             | &log &optional | Port if no conn_id                     |
 +---------------------+------------------------------------------------------------------+----------------+----------------------------------------+
 | n                   | count                                                            | &log &optional | Count or status code                   |
 +---------------------+------------------------------------------------------------------+----------------+----------------------------------------+
 | src_peer            | event_peer                                                       | &log &optional | Peer that raised the notice            |
 +---------------------+------------------------------------------------------------------+----------------+----------------------------------------+
 | peer_descr          | string                                                           | &log &optional | Text description of the src_peer       |
 +---------------------+------------------------------------------------------------------+----------------+----------------------------------------+
 | actions             | set[Notice::Action]                                              | &log &optional | Actions applied to the notice          |
 +---------------------+------------------------------------------------------------------+----------------+----------------------------------------+
 | policy_items        | set[count]                                                       | &log &optional | Policy items that have been applied    |
 +---------------------+------------------------------------------------------------------+----------------+----------------------------------------+
 | email_body_sections | vector                                                           | &optional       | Body of the email for email notices.  |
 +---------------------+------------------------------------------------------------------+----------------+----------------------------------------+
 | email_delay_tokens  | set[string]                                                      | &optional      | Delay functionality for email notices. |
 +---------------------+------------------------------------------------------------------+----------------+----------------------------------------+
 | identifier          | string                                                           | &optional      | A unique string identifier             |
 +---------------------+------------------------------------------------------------------+----------------+----------------------------------------+
 | suppress_for        | interval                                                         | &log &optional | Length of time to suppress a notice.   |
 +---------------------+------------------------------------------------------------------+----------------+----------------------------------------+
 One of the default policy scripts raises a notice when an SSH login
 has been heuristically detected and the originating hostname is one
 that would raise suspicion.  Effectively, the script attempts to
@ -1153,15 +1122,15 @@ possible while staying concise.
 While much of the script relates to the actual detection, the parts
 specific to the Notice Framework are actually quite interesting in
-themselves.  On line 18 the script's ``export`` block adds the value
+themselves.  The script's ``export`` block adds the value
 ``SSH::Interesting_Hostname_Login`` to the enumerable constant
 ``Notice::Type`` to indicate to the Bro core that a new type of notice
 is being defined.  The script then calls ``NOTICE`` and defines the
 ``$note``, ``$msg``, ``$sub`` and ``$conn`` fields of the
-:bro:type:`Notice::Info` record.  Line 42 also includes a ternary if
+:bro:type:`Notice::Info` record.  There are two ternary if
-statement that modifies the ``$msg`` text depending on whether the
+statements that modify the ``$msg`` text depending on whether the
 host is a local address and whether it is the client or the server.
-This use of :bro:id:`fmt` and a ternary operators is a concise way to
+This use of :bro:id:`fmt` and ternary operators is a concise way to
 lend readability to the notices that are generated without the need
 for branching ``if`` statements that each raise a specific notice.
@ -1222,7 +1191,7 @@ from the connection relative to the behavior that has been observed by
 Bro.  
 .. btest-include:: ${BRO_SRC_ROOT}/scripts/policy/protocols/ssl/expiring-certs.bro
-   :lines: 60-63
+   :lines: 64-68
 In the :doc:`/scripts/policy/protocols/ssl/expiring-certs.bro` script
 which identifies when SSL certificates are set to expire and raises
@ -1302,9 +1271,9 @@ in the call to ``NOTICE``.
 .. btest-include:: ${DOC_ROOT}/scripting/framework_notice_shortcuts_01.bro
-The Notice Policy shortcut above adds the ``Notice::Types`` of
+The Notice Policy shortcut above adds the ``Notice::Type`` of
-SSH::Interesting_Hostname_Login and SSH::Login to the
+``SSH::Interesting_Hostname_Login`` to the
-Notice::emailed_types set while the shortcut below alters the length
+``Notice::emailed_types`` set while the shortcut below alters the length
 of time for which those notices will be suppressed.
 .. btest-include:: ${DOC_ROOT}/scripting/framework_notice_shortcuts_02.bro
--- a/scripts/base/frameworks/files/main.bro
+++ b/scripts/base/frameworks/files/main.bro
@ -56,7 +56,7 @@ export {
 		## local file path which was read, or some other input source.
 		source: string &log &optional;
-		## A value to represent the depth of this file in relation 
+		## A value to represent the depth of this file in relation
 		## to its source.  In SMTP, it is the depth of the MIME
 		## attachment on the message.  In HTTP, it is the depth of the
 		## request within the TCP connection.
@ -73,7 +73,7 @@ export {
 		mime_type: string &log &optional;
 		## A filename for the file if one is available from the source
-		## for the file.  These will frequently come from 
+		## for the file.  These will frequently come from
 		## "Content-Disposition" headers in network protocols.
 		filename: string &log &optional;
@ -177,10 +177,19 @@ export {
 	## Returns: true if the analyzer will be added, or false if analysis
 	##          for the file isn't currently active or the *args*
 	##          were invalid for the analyzer type.
-	global add_analyzer: function(f: fa_file, 
+	global add_analyzer: function(f: fa_file,
-	                              tag: Files::Tag, 
+	                              tag: Files::Tag,
 	                              args: AnalyzerArgs &default=AnalyzerArgs()): bool;
 	## Adds all analyzers associated with a give MIME type to the analysis of
 	## a file.  Note that analyzers added via MIME types cannot take further
 	## arguments.
 	##
 	## f: the file.
 	##
 	## mtype: the MIME type; it will be compared case-insensitive.
 	global add_analyzers_for_mime_type: function(f: fa_file, mtype: string);
 	## Removes an analyzer from the analysis of a given file.
 	##
 	## f: the file.
@ -224,7 +233,7 @@ export {
 		## A callback to generate a file handle on demand when
 		## one is needed by the core.
 		get_file_handle: function(c: connection, is_orig: bool): string;
-		
+
 		## A callback to "describe" a file.  In the case of an HTTP
 		## transfer the most obvious description would be the URL.
 		## It's like an extremely compressed version of the normal log.
@ -235,7 +244,7 @@ export {
 	## Register callbacks for protocols that work with the Files framework.
 	## The callbacks must uniquely identify a file and each protocol can 
 	## only have a single callback registered for it.
-	## 
+	##
 	## tag: Tag for the protocol analyzer having a callback being registered.
 	##
 	## reg: A :bro:see:`Files::ProtoRegistration` record.
@ -253,6 +262,42 @@ export {
 	## callback: Function to execute when the given file analyzer is being added.
 	global register_analyzer_add_callback: function(tag: Files::Tag, callback: function(f: fa_file, args: AnalyzerArgs));
 	## Registers a set of MIME types for an analyzer. If a future connection on one of
 	## these types is seen, the analyzer will be automatically assigned to parsing it.
 	## The function *adds* to all MIME types already registered, it doesn't replace
 	## them.
 	##
 	## tag: The tag of the analyzer.
 	##
 	## mts: The set of MIME types, each in the form "foo/bar" (case-insensitive).
 	##
 	## Returns: True if the MIME types were successfully registered.
 	global register_for_mime_types: function(tag: Analyzer::Tag, mts: set[string]) : bool;
 	## Registers a MIME type for an analyzer. If a future file with this type is seen,
 	## the analyzer will be automatically assigned to parsing it. The function *adds*
 	## to all MIME types already registered, it doesn't replace them.
 	##
 	## tag: The tag of the analyzer.
 	##
 	## mt: The MIME type in the form "foo/bar" (case-insensitive).
 	##
 	## Returns: True if the MIME type was successfully registered.
 	global register_for_mime_type: function(tag: Analyzer::Tag, mt: string) : bool;
 	## Returns a set of all MIME types currently registered for a specific analyzer.
 	##
 	## tag: The tag of the analyzer.
 	##
 	## Returns: The set of MIME types.
 	global registered_mime_types: function(tag: Analyzer::Tag) : set[string];
 	## Returns a table of all MIME-type-to-analyzer mappings currently registered.
 	##
 	## Returns: A table mapping each analyzer to the set of MIME types
 	##          registered for it.
 	global all_registered_mime_types: function() : table[Analyzer::Tag] of set[string];
 	## Event that can be handled to access the Info record as it is sent on
 	## to the logging framework.
 	global log_files: event(rec: Info);
@ -265,6 +310,9 @@ redef record fa_file += {
 # Store the callbacks for protocol analyzers that have files.
 global registered_protocols: table[Analyzer::Tag] of ProtoRegistration = table();
 # Store the MIME type to analyzer mappings.
 global mime_types: table[Analyzer::Tag] of set[string];
 global analyzer_add_callbacks: table[Files::Tag] of function(f: fa_file, args: AnalyzerArgs) = table();
 event bro_init() &priority=5
@ -287,13 +335,13 @@ function set_info(f: fa_file)
 		f$info$source = f$source;
 	f$info$duration = f$last_active - f$info$ts;
 	f$info$seen_bytes = f$seen_bytes;
-	if ( f?$total_bytes ) 
+	if ( f?$total_bytes )
 		f$info$total_bytes = f$total_bytes;
 	f$info$missing_bytes = f$missing_bytes;
 	f$info$overflow_bytes = f$overflow_bytes;
 	if ( f?$is_orig )
 		f$info$is_orig = f$is_orig;
-	if ( f?$mime_type ) 
+	if ( f?$mime_type )
 		f$info$mime_type = f$mime_type;
 	}
@ -332,6 +380,15 @@ function add_analyzer(f: fa_file, tag: Files::Tag, args: AnalyzerArgs): bool
 	return T;
 	}
 function add_analyzers_for_mime_type(f: fa_file, mtype: string)
 	{
 	local dummy_args: AnalyzerArgs;
 	local analyzers = __add_analyzers_for_mime_type(f$id, mtype, dummy_args);
 	for ( tag in analyzers )
 		add f$info$analyzers[Files::analyzer_name(tag)];
 	}
 function register_analyzer_add_callback(tag: Files::Tag, callback: function(f: fa_file, args: AnalyzerArgs))
 	{
 	analyzer_add_callbacks[tag] = callback;
@ -356,6 +413,9 @@ event file_new(f: fa_file) &priority=10
 	{
 	set_info(f);
 	if ( f?$mime_type )
 		add_analyzers_for_mime_type(f, f$mime_type);
 	if ( enable_reassembler )
 		{
 		Files::enable_reassembly(f);
@ -405,6 +465,41 @@ function register_protocol(tag: Analyzer::Tag, reg: ProtoRegistration): bool
 	return result;
 	}
 function register_for_mime_types(tag: Analyzer::Tag, mime_types: set[string]) : bool
 	{
 	local rc = T;
 	for ( mt in mime_types )
 		{
 		if ( ! register_for_mime_type(tag, mt) )
 			rc = F;
 		}
 	return rc;
 	}
 function register_for_mime_type(tag: Analyzer::Tag, mt: string) : bool
 	{
 	if ( ! __register_for_mime_type(tag, mt) )
 		return F;
 	if ( tag !in mime_types )
 		mime_types[tag] = set();
 	add mime_types[tag][mt];
 	return T;
 	}
 function registered_mime_types(tag: Analyzer::Tag) : set[string]
 	{
 	return tag in mime_types ? mime_types[tag] : set();
 	}
 function all_registered_mime_types(): table[Analyzer::Tag] of set[string]
 	{
 	return mime_types;
 	}
 function describe(f: fa_file): string
 	{
 	local tag = Analyzer::get_tag(f$source);
--- a/scripts/base/frameworks/input/main.bro
+++ b/scripts/base/frameworks/input/main.bro
@ -4,6 +4,17 @@
 module Input;
 export {
 	type Event: enum {
 		EVENT_NEW = 0,
 		EVENT_CHANGED = 1,
 		EVENT_REMOVED = 2,
 	};
 	type Mode: enum {
 		MANUAL = 0,
 		REREAD = 1,
 		STREAM = 2
 	};
 	## The default input reader used. Defaults to `READER_ASCII`.
 	const default_reader = READER_ASCII &redef;
--- a/scripts/base/frameworks/logging/load.bro
+++ b/scripts/base/frameworks/logging/load.bro
@ -1,7 +1,5 @@
@load ./main
@load ./postprocessors
@load ./writers/ascii
@load ./writers/dataseries
@load ./writers/sqlite
@load ./writers/elasticsearch
@load ./writers/none
--- a/scripts/base/frameworks/logging/main.bro
+++ b/scripts/base/frameworks/logging/main.bro
@ -5,9 +5,15 @@
 module Log;
 # Log::ID and Log::Writer are defined in types.bif due to circular dependencies.
 export {
 	## Type that defines an ID unique to each log stream. Scripts creating new log
 	## streams need to redef this enum to add their own specific log ID. The log ID
 	## implicitly determines the default name of the generated log file.
 	type Log::ID: enum {
 		## Dummy place-holder.
 		UNKNOWN
 	};
 	## If true, local logging is by default enabled for all filters.
 	const enable_local_logging = T &redef;
@ -27,13 +33,13 @@ export {
 	const set_separator = "," &redef;
 	## String to use for empty fields. This should be different from
-        ## *unset_field* to make the output unambiguous. 
+	## *unset_field* to make the output unambiguous.
 	## Can be overwritten by individual writers.
 	const empty_field = "(empty)" &redef;
 	## String to use for an unset &optional field.
 	## Can be overwritten by individual writers.
-	const unset_field = "-" &redef;	
+	const unset_field = "-" &redef;
 	## Type defining the content of a logging stream.
 	type Stream: record {
--- a/scripts/base/frameworks/logging/writers/ascii.bro
+++ b/scripts/base/frameworks/logging/writers/ascii.bro
@ -26,20 +26,20 @@ export {
 	## This option is also available as a per-filter ``$config`` option.
 	const use_json = F &redef;
-	## Format of timestamps when writing out JSON. By default, the JSON formatter will
+	## Format of timestamps when writing out JSON. By default, the JSON
-	## use double values for timestamps which represent the number of seconds from the
+	## formatter will use double values for timestamps which represent the
-	## UNIX epoch.
+	## number of seconds from the UNIX epoch.
 	const json_timestamps: JSON::TimestampFormat = JSON::TS_EPOCH &redef;
 	## If true, include lines with log meta information such as column names
 	## with types, the values of ASCII logging options that are in use, and
 	## the time when the file was opened and closed (the latter at the end).
-        ##
+	##
 	## If writing in JSON format, this is implicitly disabled.
 	const include_meta = T &redef;
 	## Prefix for lines with meta information.
-        ##
+	##
 	## This option is also available as a per-filter ``$config`` option.
 	const meta_prefix = "#" &redef;
--- a/scripts/base/frameworks/logging/writers/dataseries.bro
+++ b/scripts/base/frameworks/logging/writers/dataseries.bro
@ -1,60 +0,0 @@
 ##! Interface for the DataSeries log writer.
 module LogDataSeries;
 export {
 	## Compression to use with the DS output file.  Options are:
 	##
 	## 'none' -- No compression.
 	## 'lzf'  -- LZF compression (very quick, but leads to larger output files).
 	## 'lzo'  -- LZO compression (very fast decompression times).
 	## 'gz'   -- GZIP compression (slower than LZF, but also produces smaller output).
 	## 'bz2'  -- BZIP2 compression (slower than GZIP, but also produces smaller output).
 	const compression = "gz" &redef;
 	## The extent buffer size.
 	## Larger values here lead to better compression and more efficient writes,
 	## but also increase the lag between the time events are received and
 	## the time they are actually written to disk.
 	const extent_size = 65536 &redef;
 	## Should we dump the XML schema we use for this DS file to disk?
 	## If yes, the XML schema shares the name of the logfile, but has
 	## an XML ending.
 	const dump_schema = F &redef;
 	## How many threads should DataSeries spawn to perform compression?
 	## Note that this dictates the number of threads per log stream.  If
 	## you're using a lot of streams, you may want to keep this number
 	## relatively small.
 	##
 	## Default value is 1, which will spawn one thread / stream.
 	##
 	## Maximum is 128, minimum is 1.
 	const num_threads = 1 &redef;
 	## Should time be stored as an integer or a double?
 	## Storing time as a double leads to possible precision issues and
 	## can (significantly) increase the size of the resulting DS log.
 	## That said, timestamps stored in double form are consistent
 	## with the rest of Bro, including the standard ASCII log. Hence, we
 	## use them by default.
 	const use_integer_for_time = F &redef;
 }
 # Default function to postprocess a rotated DataSeries log file. It moves the
 # rotated file to a new name that includes a timestamp with the opening time,
 # and then runs the writer's default postprocessor command on it.
 function default_rotation_postprocessor_func(info: Log::RotationInfo) : bool
 	{
 	# Move file to name including both opening and closing time.
 	local dst = fmt("%s.%s.ds", info$path,
 			strftime(Log::default_rotation_date_format, info$open));
 	system(fmt("/bin/mv %s %s", info$fname, dst));
 	# Run default postprocessor.
 	return Log::run_rotation_postprocessor_cmd(info, dst);
 	}
 redef Log::default_rotation_postprocessors += { [Log::WRITER_DATASERIES] = default_rotation_postprocessor_func };
--- a/scripts/base/frameworks/logging/writers/elasticsearch.bro
+++ b/scripts/base/frameworks/logging/writers/elasticsearch.bro
@ -1,48 +0,0 @@
 ##! Log writer for sending logs to an ElasticSearch server.
 ##!
 ##! Note: This module is in testing and is not yet considered stable!
 ##!
 ##! There is one known memory issue.  If your elasticsearch server is
 ##! running slowly and taking too long to return from bulk insert
 ##! requests, the message queue to the writer thread will continue
 ##! growing larger and larger giving the appearance of a memory leak.
 module LogElasticSearch;
 export {
 	## Name of the ES cluster.
 	const cluster_name = "elasticsearch" &redef;
 	## ES server.
 	const server_host = "127.0.0.1" &redef;
 	## ES port.
 	const server_port = 9200 &redef;
 	## Name of the ES index.
 	const index_prefix = "bro" &redef;
 	## The ES type prefix comes before the name of the related log.
 	## e.g. prefix = "bro\_" would create types of bro_dns, bro_software, etc.
 	const type_prefix = "" &redef;
 	## The time before an ElasticSearch transfer will timeout. Note that
 	## the fractional part of the timeout will be ignored. In particular,
 	## time specifications less than a second result in a timeout value of
 	## 0, which means "no timeout."
 	const transfer_timeout = 2secs;
 	## The batch size is the number of messages that will be queued up before
 	## they are sent to be bulk indexed.
 	const max_batch_size = 1000 &redef;
 	## The maximum amount of wall-clock time that is allowed to pass without
 	## finishing a bulk log send.  This represents the maximum delay you
 	## would like to have with your logs before they are sent to ElasticSearch.
 	const max_batch_interval = 1min &redef;
 	## The maximum byte size for a buffered JSON string to send to the bulk
 	## insert API.
 	const max_byte_size = 1024 * 1024 &redef;
 }
--- a/scripts/base/frameworks/notice/main.bro
+++ b/scripts/base/frameworks/notice/main.bro
@ -20,7 +20,8 @@ export {
 	## category along with the specific notice separating words with
 	## underscores and using leading capitals on each word except for
 	## abbreviations which are kept in all capitals.  For example,
-	## SSH::Login is for heuristically guessed successful SSH logins.
+	## SSH::Password_Guessing is for hosts that have crossed a threshold of
 	## heuristically determined failed SSH logins.
 	type Type: enum {
 		## Notice reporting a count of how often a notice occurred.
 		Tally,
--- a/scripts/base/frameworks/signatures/main.bro
+++ b/scripts/base/frameworks/signatures/main.bro
@ -71,7 +71,7 @@ export {
 		## to be logged has occurred.
 		ts:         time         &log;
 		## A unique identifier of the connection which triggered the
-		## signature match event
+		## signature match event.
 		uid:        string       &log &optional;
 		## The host which triggered the signature match event.
 		src_addr:   addr         &log &optional;
--- a/scripts/base/init-bare.bro
+++ b/scripts/base/init-bare.bro
@ -75,6 +75,13 @@ type addr_vec: vector of addr;
 ##    directly and then remove this alias.
 type table_string_of_string: table[string] of string;
 ## A set of file analyzer tags.
 ##
 ## .. todo:: We need this type definition only for declaring builtin functions
 ##    via ``bifcl``. We should extend ``bifcl`` to understand composite types
 ##    directly and then remove this alias.
 type files_tag_set: set[Files::Tag];
 ## A structure indicating a MIME type and strength of a match against
 ## file magic signatures.
 ##
@ -2479,8 +2486,7 @@ type http_message_stat: record {
 	header_length: count;
 };
-## Maximum number of HTTP entity data delivered to events. The amount of data
+## Maximum number of HTTP entity data delivered to events.
 ## can be limited for better performance, zero disables truncation.
 ##
 ## .. bro:see:: http_entity_data skip_http_entity_data skip_http_data
 global http_entity_data_delivery_size = 1500 &redef;
@ -2732,6 +2738,7 @@ type ModbusRegisters: vector of count;
 type ModbusHeaders: record {
 	tid:           count;
 	pid:           count;
 	len:           count;
 	uid:           count;
 	function_code: count;
 };
@ -3357,9 +3364,6 @@ const global_hash_seed: string = "" &redef;
 ## The maximum is currently 128 bits.
 const bits_per_uid: count = 96 &redef;
 # Load BiFs defined by plugins.
@load base/bif/plugins
 # Load these frameworks here because they use fairly deep integration with
 # BiFs and script-land defined types.
@load base/frameworks/logging
@ -3368,3 +3372,7 @@ const bits_per_uid: count = 96 &redef;
@load base/frameworks/files
@load base/bif
 # Load BiFs defined by plugins.
@load base/bif/plugins
--- a/scripts/base/protocols/dhcp/main.bro
+++ b/scripts/base/protocols/dhcp/main.bro
@ -47,13 +47,13 @@ redef record connection += {
 const ports = { 67/udp, 68/udp };
 redef likely_server_ports += { 67/udp };
-event bro_init()
+event bro_init() &priority=5
 	{
 	Log::create_stream(DHCP::LOG, [$columns=Info, $ev=log_dhcp]);
 	Analyzer::register_for_ports(Analyzer::ANALYZER_DHCP, ports);
 	}
-event dhcp_ack(c: connection, msg: dhcp_msg, mask: addr, router: dhcp_router_list, lease: interval, serv_addr: addr, host_name: string)
+event dhcp_ack(c: connection, msg: dhcp_msg, mask: addr, router: dhcp_router_list, lease: interval, serv_addr: addr, host_name: string) &priority=5
 	{
 	local info: Info;
 	info$ts          = network_time();
@ -71,6 +71,9 @@ event dhcp_ack(c: connection, msg: dhcp_msg, mask: addr, router: dhcp_router_lis
 		info$assigned_ip = c$id$orig_h;
 	c$dhcp = info;
 	}
 event dhcp_ack(c: connection, msg: dhcp_msg, mask: addr, router: dhcp_router_list, lease: interval, serv_addr: addr, host_name: string) &priority=-5
 	{
 	Log::write(DHCP::LOG, c$dhcp);
 	}
--- a/scripts/base/protocols/smtp/main.bro
+++ b/scripts/base/protocols/smtp/main.bro
@ -58,31 +58,31 @@ export {
 		## Indicates if client activity has been seen, but not yet logged.
 		has_client_activity:  bool            &default=F;
 	};
-	
+
 	type State: record {
 		helo:                     string    &optional;
 		## Count the number of individual messages transmitted during
 		## this SMTP session.  Note, this is not the number of
 		## recipients, but the number of message bodies transferred.
 		messages_transferred:     count     &default=0;
-		
+
 		pending_messages:         set[Info] &optional;
 	};
-	
+
 	## Direction to capture the full "Received from" path.
 	##    REMOTE_HOSTS - only capture the path until an internal host is found.
 	##    LOCAL_HOSTS - only capture the path until the external host is discovered.
 	##    ALL_HOSTS - always capture the entire path.
 	##    NO_HOSTS - never capture the path.
 	const mail_path_capture = ALL_HOSTS &redef;
-	
+
 	## Create an extremely shortened representation of a log line.
 	global describe: function(rec: Info): string;
 	global log_smtp: event(rec: Info);
 }
-redef record connection += { 
+redef record connection += {
 	smtp:       Info  &optional;
 	smtp_state: State &optional;
 };
@ -95,7 +95,7 @@ event bro_init() &priority=5
 	Log::create_stream(SMTP::LOG, [$columns=SMTP::Info, $ev=log_smtp]);
 	Analyzer::register_for_ports(Analyzer::ANALYZER_SMTP, ports);
 	}
-	
+
 function find_address_in_smtp_header(header: string): string
 {
 	local ips = find_ip_addresses(header);
@ -116,17 +116,17 @@ function new_smtp_log(c: connection): Info
 	l$ts=network_time();
 	l$uid=c$uid;
 	l$id=c$id;
-	# The messages_transferred count isn't incremented until the message is 
+	# The messages_transferred count isn't incremented until the message is
 	# finished so we need to increment the count by 1 here.
 	l$trans_depth = c$smtp_state$messages_transferred+1;
-	
+
 	if ( c$smtp_state?$helo )
 		l$helo = c$smtp_state$helo;
-	
+
 	# The path will always end with the hosts involved in this connection.
 	# The lower values in the vector are the end of the path.
 	l$path = vector(c$id$resp_h, c$id$orig_h);
-	
+
 	return l;
 	}
@ -134,7 +134,7 @@ function set_smtp_session(c: connection)
 	{
 	if ( ! c?$smtp_state )
 		c$smtp_state = [];
-	
+
 	if ( ! c?$smtp )
 		c$smtp = new_smtp_log(c);
 	}
@ -142,17 +142,17 @@ function set_smtp_session(c: connection)
 function smtp_message(c: connection)
 	{
 	if ( c$smtp$has_client_activity )
 		{
 		Log::write(SMTP::LOG, c$smtp);
 		c$smtp = new_smtp_log(c);
 		}
 	}
-	
+
 event smtp_request(c: connection, is_orig: bool, command: string, arg: string) &priority=5
 	{
 	set_smtp_session(c);
 	local upper_command = to_upper(command);
 	if ( upper_command != "QUIT" )
 		c$smtp$has_client_activity = T;
 	if ( upper_command == "HELO" || upper_command == "EHLO" )
 		{
 		c$smtp_state$helo = arg;
@ -161,23 +161,28 @@ event smtp_request(c: connection, is_orig: bool, command: string, arg: string) &
 	else if ( upper_command == "RCPT" && /^[tT][oO]:/ in arg )
 		{
-		if ( ! c$smtp?$rcptto ) 
+		if ( ! c$smtp?$rcptto )
 			c$smtp$rcptto = set();
 		add c$smtp$rcptto[split1(arg, /:[[:blank:]]*/)[2]];
 		c$smtp$has_client_activity = T;
 		}
 	else if ( upper_command == "MAIL" && /^[fF][rR][oO][mM]:/ in arg )
 		{
 		# Flush last message in case we didn't see the server's acknowledgement.
 		smtp_message(c);
 		local partially_done = split1(arg, /:[[:blank:]]*/)[2];
 		c$smtp$mailfrom = split1(partially_done, /[[:blank:]]?/)[1];
 		c$smtp$has_client_activity = T;
 		}
 	}
-	
+
 event smtp_reply(c: connection, is_orig: bool, code: count, cmd: string,
                 msg: string, cont_resp: bool) &priority=5
 	{
 	set_smtp_session(c);
-	
+
 	# This continually overwrites, but we want the last reply,
 	# so this actually works fine.
 	c$smtp$last_reply = fmt("%d %s", code, msg);
@ -198,7 +203,6 @@ event smtp_reply(c: connection, is_orig: bool, code: count, cmd: string,
 event mime_one_header(c: connection, h: mime_header_rec) &priority=5
 	{
 	if ( ! c?$smtp ) return;
 	c$smtp$has_client_activity = T;
 	if ( h$name == "MESSAGE-ID" )
 		c$smtp$msg_id = h$value;
@ -241,19 +245,19 @@ event mime_one_header(c: connection, h: mime_header_rec) &priority=5
 		if ( 1 in addresses )
 			c$smtp$x_originating_ip = to_addr(addresses[1]);
 		}
-	
+
 	else if ( h$name == "X-MAILER" ||
 	          h$name == "USER-AGENT" ||
 	          h$name == "X-USER-AGENT" )
 		c$smtp$user_agent = h$value;
 	}
-	
+
-# This event handler builds the "Received From" path by reading the 
+# This event handler builds the "Received From" path by reading the
 # headers in the mail
 event mime_one_header(c: connection, h: mime_header_rec) &priority=3
 	{
 	# If we've decided that we're done watching the received headers for
-	# whatever reason, we're done.  Could be due to only watching until 
+	# whatever reason, we're done.  Could be due to only watching until
 	# local addresses are seen in the received from headers.
 	if ( ! c?$smtp || h$name != "RECEIVED" || ! c$smtp$process_received_from )
 		return;
@ -263,7 +267,7 @@ event mime_one_header(c: connection, h: mime_header_rec) &priority=3
 		return;
 	local ip = to_addr(text_ip);
-	if ( ! addr_matches_host(ip, mail_path_capture) && 
+	if ( ! addr_matches_host(ip, mail_path_capture) &&
 	     ! Site::is_private_addr(ip) )
 		{
 		c$smtp$process_received_from = F;
@ -281,7 +285,10 @@ event connection_state_remove(c: connection) &priority=-5
 event smtp_starttls(c: connection) &priority=5
 	{
 	if ( c?$smtp )
 		{
 		c$smtp$tls = T;
 		c$smtp$has_client_activity = T;
 		}
 	}
 function describe(rec: Info): string
--- a/scripts/base/protocols/ssl/consts.bro
+++ b/scripts/base/protocols/ssl/consts.bro
@ -26,6 +26,21 @@ export {
 	const V2_CLIENT_MASTER_KEY = 302;
 	const V2_SERVER_HELLO = 304;
 	## TLS Handshake types:
 	const HELLO_REQUEST       = 0;
 	const CLIENT_HELLO        = 1;
 	const SERVER_HELLO        = 2;
 	const SESSION_TICKET      = 4; # RFC 5077
 	const CERTIFICATE         = 11;
 	const SERVER_KEY_EXCHANGE = 12;
 	const CERTIFICATE_REQUEST = 13;
 	const SERVER_HELLO_DONE   = 14;
 	const CERTIFICATE_VERIFY  = 15;
 	const CLIENT_KEY_EXCHANGE = 16;
 	const FINISHED            = 20;
 	const CERTIFICATE_URL     = 21; # RFC 3546
 	const CERTIFICATE_STATUS  = 22; # RFC 3546
 	## Mapping between numeric codes and human readable strings for alert
 	## levels.
 	const alert_levels: table[count] of string = {
@ -94,6 +109,10 @@ export {
 		[16] = "application_layer_protocol_negotiation",
 		[17] = "status_request_v2",
 		[18] = "signed_certificate_timestamp",
 		[19] = "client_certificate_type",
 		[20] = "server_certificate_type",
 		[21] = "padding", # temporary till 2015-03-12
 		[22] = "encrypt_then_mac", # temporary till 2015-06-05
 		[35] = "SessionTicket TLS",
 		[40] = "extended_random",
 		[13172] = "next_protocol_negotiation",
--- a/scripts/policy/misc/load-balancing.bro
+++ b/scripts/policy/misc/load-balancing.bro
@ -82,7 +82,7 @@ event bro_init() &priority=5
 			++lb_proc_track[that_node$ip, that_node$interface];
 			if ( total_lb_procs > 1 )
 				{
-				that_node$lb_filter = PacketFilter::sample_filter(total_lb_procs, this_lb_proc);
+				that_node$lb_filter = PacketFilter::sampling_filter(total_lb_procs, this_lb_proc);
 				Communication::nodes[no]$capture_filter = that_node$lb_filter;
 				}
 			}
--- a/scripts/policy/protocols/ssl/expiring-certs.bro
+++ b/scripts/policy/protocols/ssl/expiring-certs.bro
@ -39,27 +39,31 @@ event ssl_established(c: connection) &priority=3
 	# If there are no certificates or we are not interested in the server, just return.
 	if ( ! c$ssl?$cert_chain || |c$ssl$cert_chain| == 0 ||
 	     ! addr_matches_host(c$id$resp_h, notify_certs_expiration) ||
-	     ! c$ssl$cert_chain[0]?$x509 )
+	     ! c$ssl$cert_chain[0]?$x509 || ! c$ssl$cert_chain[0]?$sha1 )
 		return;
 	local fuid = c$ssl$cert_chain_fuids[0];
 	local cert = c$ssl$cert_chain[0]$x509$certificate;
 	local hash = c$ssl$cert_chain[0]$sha1;
 	if ( cert$not_valid_before > network_time() )
 		NOTICE([$note=Certificate_Not_Valid_Yet,
 		        $conn=c, $suppress_for=1day,
 		        $msg=fmt("Certificate %s isn't valid until %T", cert$subject, cert$not_valid_before),
 		        $identifier=cat(c$id$resp_h, c$id$resp_p, hash),
 		        $fuid=fuid]);
 	else if ( cert$not_valid_after < network_time() )
 		NOTICE([$note=Certificate_Expired,
 		        $conn=c, $suppress_for=1day,
 		        $msg=fmt("Certificate %s expired at %T", cert$subject, cert$not_valid_after),
 		        $identifier=cat(c$id$resp_h, c$id$resp_p, hash),
 		        $fuid=fuid]);
 	else if ( cert$not_valid_after - notify_when_cert_expiring_in < network_time() )
 		NOTICE([$note=Certificate_Expires_Soon,
 		        $msg=fmt("Certificate %s is going to expire at %T", cert$subject, cert$not_valid_after),
 		        $conn=c, $suppress_for=1day,
 		        $identifier=cat(c$id$resp_h, c$id$resp_p, hash),
 		        $fuid=fuid]);
 	}
--- a/scripts/policy/tuning/logs-to-elasticsearch.bro
+++ b/scripts/policy/tuning/logs-to-elasticsearch.bro
@ -1,36 +0,0 @@
 ##! Load this script to enable global log output to an ElasticSearch database.
 module LogElasticSearch;
 export {
 	## An elasticsearch specific rotation interval.
 	const rotation_interval = 3hr &redef;
 	## Optionally ignore any :bro:type:`Log::ID` from being sent to
 	## ElasticSearch with this script.
 	const excluded_log_ids: set[Log::ID] &redef;
 	## If you want to explicitly only send certain :bro:type:`Log::ID` 
 	## streams, add them to this set.  If the set remains empty, all will 
 	## be sent.  The :bro:id:`LogElasticSearch::excluded_log_ids` option
 	## will remain in effect as well.
 	const send_logs: set[Log::ID] &redef;
 }
 event bro_init() &priority=-5
 	{
 	if ( server_host == "" )
 		return;
 	for ( stream_id in Log::active_streams )
 		{
 		if ( stream_id in excluded_log_ids ||
 		     (|send_logs| > 0 && stream_id !in send_logs) )
 			next;
 		local filter: Log::Filter = [$name = "default-es",
 		                             $writer = Log::WRITER_ELASTICSEARCH,
 		                             $interv = LogElasticSearch::rotation_interval];
 		Log::add_filter(stream_id, filter);
 		}
 	}
--- a/scripts/test-all-policy.bro
+++ b/scripts/test-all-policy.bro
@ -98,7 +98,4 @@
@load tuning/defaults/packet-fragments.bro
@load tuning/defaults/warnings.bro
@load tuning/json-logs.bro
@load tuning/logs-to-elasticsearch.bro
@load tuning/track-all-assets.bro
 redef LogElasticSearch::server_host = "";
--- a/src/CMakeLists.txt
+++ b/src/CMakeLists.txt
@ -7,7 +7,8 @@ include_directories(BEFORE
 set(bro_ALL_GENERATED_OUTPUTS  CACHE INTERNAL "automatically generated files" FORCE)
 # This collects bif inputs that we'll load automatically.
-set(bro_AUTO_BIFS CACHE INTERNAL "BIFs for automatic inclusion" FORCE)
+set(bro_AUTO_BIFS     CACHE INTERNAL "BIFs for automatic inclusion" FORCE)
 set(bro_REGISTER_BIFS CACHE INTERNAL "BIFs for automatic registering" FORCE)
 set(bro_BASE_BIF_SCRIPTS CACHE INTERNAL "Bro script stubs for BIFs in base distribution of Bro" FORCE)
 set(bro_PLUGIN_BIF_SCRIPTS CACHE INTERNAL "Bro script stubs for BIFs in Bro plugins" FORCE)
@ -117,8 +118,6 @@ include(BifCl)
 set(BIF_SRCS
    bro.bif
    logging.bif
    input.bif
    event.bif
    const.bif
    types.bif
@ -155,21 +154,25 @@ set(bro_SUBDIR_LIBS CACHE INTERNAL "subdir libraries" FORCE)
 set(bro_PLUGIN_LIBS CACHE INTERNAL "plugin libraries" FORCE)
 add_subdirectory(analyzer)
 add_subdirectory(file_analysis)
 add_subdirectory(probabilistic)
 add_subdirectory(broxygen)
 add_subdirectory(file_analysis)
 add_subdirectory(input)
 add_subdirectory(iosource)
 add_subdirectory(logging)
 add_subdirectory(probabilistic)
 set(bro_SUBDIRS
-    ${bro_SUBDIR_LIBS}
+    # Order is important here.
    ${bro_PLUGIN_LIBS}
    ${bro_SUBDIR_LIBS}
 )
 if ( NOT bro_HAVE_OBJECT_LIBRARIES )
    foreach (_plugin ${bro_PLUGIN_LIBS})
        string(REGEX REPLACE "plugin-" "" _plugin "${_plugin}")
        string(REGEX REPLACE "-" "_" _plugin "${_plugin}")
-        set(_decl "namespace plugin { namespace ${_plugin} { class Plugin; extern Plugin __plugin; } };")
+        set(_decl "namespace plugin { namespace ${_plugin} { class Plugin; extern Plugin plugin; } };")
-        set(_use "i += (size_t)(&(plugin::${_plugin}::__plugin));")
+        set(_use "i += (size_t)(&(plugin::${_plugin}::plugin));")
        set(__BRO_DECL_PLUGINS "${__BRO_DECL_PLUGINS}${_decl}\n")
        set(__BRO_USE_PLUGINS "${__BRO_USE_PLUGINS}${_use}\n")
    endforeach()
@ -252,7 +255,6 @@ set(bro_SRCS
    Anon.cc
    Attr.cc
    Base64.cc
    BPF_Program.cc
    Brofiler.cc
    BroString.cc
    CCL.cc
@ -277,14 +279,12 @@ set(bro_SRCS
    EventRegistry.cc
    Expr.cc
    File.cc
    FlowSrc.cc
    Frag.cc
    Frame.cc
    Func.cc
    Hash.cc
    ID.cc
    IntSet.cc
    IOSource.cc
    IP.cc
    IPAddr.cc
    List.cc
@ -297,7 +297,6 @@ set(bro_SRCS
    OSFinger.cc
    PacketFilter.cc
    PersistenceSerializer.cc
    PktSrc.cc
    PolicyFile.cc
    PrefixTable.cc
    PriorityQueue.cc
@ -346,24 +345,6 @@ set(bro_SRCS
    threading/formatters/Ascii.cc
    threading/formatters/JSON.cc
    logging/Manager.cc
    logging/WriterBackend.cc
    logging/WriterFrontend.cc
    logging/writers/Ascii.cc
    logging/writers/DataSeries.cc
    logging/writers/SQLite.cc
    logging/writers/ElasticSearch.cc
    logging/writers/None.cc
    input/Manager.cc
    input/ReaderBackend.cc
    input/ReaderFrontend.cc
    input/readers/Ascii.cc
    input/readers/Raw.cc
    input/readers/Benchmark.cc
    input/readers/Binary.cc
    input/readers/SQLite.cc
    3rdparty/sqlite3.c
    plugin/Component.cc
@ -371,7 +352,6 @@ set(bro_SRCS
    plugin/TaggedComponent.h
    plugin/Manager.cc
    plugin/Plugin.cc
    plugin/Macros.h
    nb_dns.c
    digest.h
@ -387,22 +367,31 @@ else ()
    target_link_libraries(bro ${bro_SUBDIRS} ${brodeps} ${CMAKE_THREAD_LIBS_INIT} ${CMAKE_DL_LIBS})
 endif ()
 if ( NOT "${bro_LINKER_FLAGS}" STREQUAL "" )
    set_target_properties(bro PROPERTIES LINK_FLAGS "${bro_LINKER_FLAGS}")
 endif ()
 install(TARGETS bro DESTINATION bin)
 set(BRO_EXE bro
    CACHE STRING "Bro executable binary" FORCE)
 set(BRO_EXE_PATH ${CMAKE_CURRENT_BINARY_DIR}/bro
    CACHE STRING "Path to Bro executable binary" FORCE)
 # Target to create all the autogenerated files.
 add_custom_target(generate_outputs_stage1)
 add_dependencies(generate_outputs_stage1 ${bro_ALL_GENERATED_OUTPUTS})
 # Target to create the joint includes files that pull in the bif code.
-bro_bif_create_includes(generate_outputs_stage2 ${CMAKE_CURRENT_BINARY_DIR} "${bro_AUTO_BIFS}")
+bro_bif_create_includes(generate_outputs_stage2a ${CMAKE_CURRENT_BINARY_DIR} "${bro_AUTO_BIFS}")
-add_dependencies(generate_outputs_stage2 generate_outputs_stage1)
+bro_bif_create_register(generate_outputs_stage2b ${CMAKE_CURRENT_BINARY_DIR} "${bro_REGISTER_BIFS}")
 add_dependencies(generate_outputs_stage2a generate_outputs_stage1)
 add_dependencies(generate_outputs_stage2b generate_outputs_stage1)
 # Global target to trigger creation of autogenerated code.
 add_custom_target(generate_outputs)
-add_dependencies(generate_outputs generate_outputs_stage2)
+add_dependencies(generate_outputs generate_outputs_stage2a generate_outputs_stage2b)
 # Build __load__.bro files for standard *.bif.bro.
 bro_bif_create_loader(bif_loader "${bro_BASE_BIF_SCRIPTS}")
--- a/src/DNS_Mgr.cc
+++ b/src/DNS_Mgr.cc
@ -35,6 +35,7 @@
 #include "Net.h"
 #include "Var.h"
 #include "Reporter.h"
 #include "iosource/Manager.h"
 extern "C" {
 extern int select(int, fd_set *, fd_set *, fd_set *, struct timeval *);
@ -404,17 +405,17 @@ DNS_Mgr::~DNS_Mgr()
 	delete [] dir;
 	}
-bool DNS_Mgr::Init()
+void DNS_Mgr::InitPostScript()
 	{
 	if ( did_init )
-		return true;
+		return;
 	const char* cache_dir = dir ? dir : ".";
 	if ( mode == DNS_PRIME && ! ensure_dir(cache_dir) )
 		{
 		did_init = 0;
-		return false;
+		return;
 		}
 	cache_name = new char[strlen(cache_dir) + 64];
@ -433,14 +434,12 @@ bool DNS_Mgr::Init()
 	did_init = 1;
-	io_sources.Register(this, true);
+	iosource_mgr->Register(this, true);
 	// We never set idle to false, having the main loop only calling us from
 	// time to time. If we're issuing more DNS requests than we can handle
 	// in this way, we are having problems anyway ...
-	idle = true;
+	SetIdle(true);
 	return true;
 	}
 static TableVal* fake_name_lookup_result(const char* name)
--- a/src/DNS_Mgr.h
+++ b/src/DNS_Mgr.h
@ -12,7 +12,7 @@
 #include "BroList.h"
 #include "Dict.h"
 #include "EventHandler.h"
-#include "IOSource.h"
+#include "iosource/IOSource.h"
 #include "IPAddr.h"
 class Val;
@ -40,12 +40,12 @@ enum DNS_MgrMode {
 // Number of seconds we'll wait for a reply.
 #define DNS_TIMEOUT 5
-class DNS_Mgr : public IOSource {
+class DNS_Mgr : public iosource::IOSource {
 public:
 	DNS_Mgr(DNS_MgrMode mode);
 	virtual ~DNS_Mgr();
-	bool Init();
+	void InitPostScript();
 	void Flush();
 	// Looks up the address or addresses of the given host, and returns
--- a/src/DebugLogger.cc
+++ b/src/DebugLogger.cc
@ -5,6 +5,7 @@
 #include "DebugLogger.h"
 #include "Net.h"
 #include "plugin/Plugin.h"
 DebugLogger debug_logger("debug");
@ -15,9 +16,10 @@ DebugLogger::Stream DebugLogger::streams[NUM_DBGS] = {
 	{ "compressor", 0, false }, {"string", 0, false },
 	{ "notifiers", 0, false },  { "main-loop", 0, false },
 	{ "dpd", 0, false }, { "tm", 0, false },
-	{ "logging", 0, false }, {"input", 0, false }, 
+	{ "logging", 0, false }, {"input", 0, false },
 	{ "threading", 0, false }, { "file_analysis", 0, false },
-	{ "plugins", 0, false }, { "broxygen", 0, false }
+	{ "plugins", 0, false }, { "broxygen", 0, false },
 	{ "pktio", 0, false}
 };
 DebugLogger::DebugLogger(const char* filename)
@ -73,10 +75,12 @@ void DebugLogger::EnableStreams(const char* s)
 			{
 			if ( strcasecmp("verbose", tok) == 0 )
 				verbose = true;
-			else
+			else if ( strncmp(tok, "plugin-", 7) != 0 )
 				reporter->FatalError("unknown debug stream %s\n", tok);
 			}
 		enabled_streams.insert(tok);
 		tok = strtok(0, ",");
 		}
@ -105,4 +109,24 @@ void DebugLogger::Log(DebugStream stream, const char* fmt, ...)
 	fflush(file);
 	}
 void DebugLogger::Log(const plugin::Plugin& plugin, const char* fmt, ...)
 	{
 	string tok = string("plugin-") + plugin.Name();
 	tok = strreplace(tok, "::", "-");
 	if ( enabled_streams.find(tok) == enabled_streams.end() )
 		return;
 	fprintf(file, "%17.06f/%17.06f [plugin %s] ",
 			network_time, current_time(true), plugin.Name().c_str());
 	va_list ap;
 	va_start(ap, fmt);
 	vfprintf(file, fmt, ap);
 	va_end(ap);
 	fputc('\n', file);
 	fflush(file);
 	}
 #endif
--- a/src/DebugLogger.h
+++ b/src/DebugLogger.h
@ -7,6 +7,8 @@
 #ifdef DEBUG
 #include <stdio.h>
 #include <string>
 #include <set>
 // To add a new debugging stream, add a constant here as well as
 // an entry to DebugLogger::streams in DebugLogger.cc.
@ -27,8 +29,9 @@ enum DebugStream {
 	DBG_INPUT,	// Input streams
 	DBG_THREADING,	// Threading system
 	DBG_FILE_ANALYSIS,	// File analysis
-	DBG_PLUGINS,
+	DBG_PLUGINS,	// Plugin system
-	DBG_BROXYGEN,
+	DBG_BROXYGEN,	// Broxygen
 	DBG_PKTIO,	// Packet sources and dumpers.
 	NUM_DBGS // Has to be last
 };
@ -42,6 +45,10 @@ enum DebugStream {
 #define DBG_PUSH(stream) debug_logger.PushIndent(stream)
 #define DBG_POP(stream) debug_logger.PopIndent(stream)
 #define PLUGIN_DBG_LOG(plugin, args...) debug_logger.Log(plugin, args)
 namespace plugin { class Plugin; }
 class DebugLogger {
 public:
 	// Output goes to stderr per default.
@ -49,6 +56,7 @@ public:
 	~DebugLogger();
 	void Log(DebugStream stream, const char* fmt, ...);
 	void Log(const plugin::Plugin& plugin, const char* fmt, ...);
 	void PushIndent(DebugStream stream)
 		{ ++streams[int(stream)].indent; }
@ -79,6 +87,8 @@ private:
 		bool enabled;
 	};
 	std::set<std::string> enabled_streams;
 	static Stream streams[NUM_DBGS];
 };
@ -89,6 +99,7 @@ extern DebugLogger debug_logger;
 #define DBG_LOG_VERBOSE(args...)
 #define DBG_PUSH(stream)
 #define DBG_POP(stream)
 #define PLUGIN_DBG_LOG(plugin, args...)
 #endif
 #endif
--- a/src/Event.cc
+++ b/src/Event.cc
@ -6,6 +6,7 @@
 #include "Func.h"
 #include "NetVar.h"
 #include "Trigger.h"
 #include "plugin/Manager.h"
 EventMgr mgr;
@ -77,6 +78,11 @@ EventMgr::~EventMgr()
 void EventMgr::QueueEvent(Event* event)
 	{
 	bool done = PLUGIN_HOOK_WITH_RESULT(HOOK_QUEUE_EVENT, HookQueueEvent(event), false);
 	if ( done )
 		return;
 	if ( ! head )
 		head = tail = event;
 	else
@ -115,6 +121,8 @@ void EventMgr::Drain()
 	SegmentProfiler(segment_logger, "draining-events");
 	PLUGIN_HOOK_VOID(HOOK_DRAIN_EVENTS, HookDrainEvents());
 	draining = true;
 	while ( head )
 		Dispatch();
--- a/src/Event.h
+++ b/src/Event.h
@ -24,6 +24,8 @@ public:
 	SourceID Source() const		{ return src; }
 	analyzer::ID Analyzer() const	{ return aid; }
 	TimerMgr* Mgr() const		{ return mgr; }
 	EventHandlerPtr Handler() const	{ return handler; }
 	val_list* Args() const	{ return args; }
 	void Describe(ODesc* d) const;
--- a/src/EventHandler.cc
+++ b/src/EventHandler.cc
@ -13,6 +13,7 @@ EventHandler::EventHandler(const char* arg_name)
 	type = 0;
 	error_handler = false;
 	enabled = true;
 	generate_always = false;
 	}
 EventHandler::~EventHandler()
@ -23,7 +24,9 @@ EventHandler::~EventHandler()
 EventHandler::operator bool() const
 	{
-	return enabled && ((local && local->HasBodies()) || receivers.length());
+	return enabled && ((local && local->HasBodies())
 			   || receivers.length()
 			   || generate_always);
 	}
 FuncType* EventHandler::FType()
--- a/src/EventHandler.h
+++ b/src/EventHandler.h
@ -43,6 +43,11 @@ public:
 	void SetEnable(bool arg_enable)	{ enabled = arg_enable; }
 	// Flags the event as interesting even if there is no body defined. In
 	// particular, this will then still pass the event on to plugins.
 	void SetGenerateAlways()	{ generate_always = true; }
 	bool GenerateAlways()	{ return generate_always; }
 	// We don't serialize the handler(s) itself here, but
 	// just the reference to it.
 	bool Serialize(SerialInfo* info) const;
@ -57,6 +62,7 @@ private:
 	bool used;		// this handler is indeed used somewhere
 	bool enabled;
 	bool error_handler;	// this handler reports error messages.
 	bool generate_always;
 	declare(List, SourceID);
 	typedef List(SourceID) receiver_list;
--- a/src/EventRegistry.cc
+++ b/src/EventRegistry.cc
@ -71,6 +71,23 @@ EventRegistry::string_list* EventRegistry::UsedHandlers()
 	return names;
 	}
 EventRegistry::string_list* EventRegistry::AllHandlers()
 	{
 	string_list* names = new string_list;
 	IterCookie* c = handlers.InitForIteration();
 	HashKey* k;
 	EventHandler* v;
 	while ( (v = handlers.NextEntry(k, c)) )
 		{
 		names->append(v->Name());
 		delete k;
 		}
 	return names;
 	}
 void EventRegistry::PrintDebug()
 	{
 	IterCookie* c = handlers.InitForIteration();
--- a/src/EventRegistry.h
+++ b/src/EventRegistry.h
@ -33,6 +33,8 @@ public:
 	string_list* UnusedHandlers();
 	string_list* UsedHandlers();
 	string_list* AllHandlers();
 	void PrintDebug();
 private:
--- a/src/Expr.cc
+++ b/src/Expr.cc
@ -4330,7 +4330,7 @@ Val* TableCoerceExpr::Fold(Val* v) const
 	if ( tv->Size() > 0 )
 		Internal("coercion of non-empty table/set");
-	return new TableVal(Type()->Ref()->AsTableType(), tv->Attrs());
+	return new TableVal(Type()->AsTableType(), tv->Attrs());
 	}
 IMPLEMENT_SERIAL(TableCoerceExpr, SER_TABLE_COERCE_EXPR);
--- a/src/Expr.h
+++ b/src/Expr.h
@ -608,6 +608,10 @@ public:
 	CondExpr(Expr* op1, Expr* op2, Expr* op3);
 	~CondExpr();
 	const Expr* Op1() const	{ return op1; }
 	const Expr* Op2() const	{ return op2; }
 	const Expr* Op3() const	{ return op3; }
 	Expr* Simplify(SimplifyType simp_type);
 	Val* Eval(Frame* f) const;
 	int IsPure() const;
@ -706,6 +710,7 @@ public:
 	~FieldExpr();
 	int Field() const	{ return field; }
 	const char* FieldName() const	{ return field_name; }
 	int CanDel() const;
@ -737,6 +742,8 @@ public:
 	HasFieldExpr(Expr* op, const char* field_name);
 	~HasFieldExpr();
 	const char* FieldName() const	{ return field_name; }
 protected:
 	friend class Expr;
 	HasFieldExpr()	{ field_name = 0; }
--- a/src/FlowSrc.cc
+++ b/src/FlowSrc.cc
@ -1,228 +0,0 @@
 // See the file "COPYING" in the main distribution directory for copyright.
 //
 // Written by Bernhard Ager, TU Berlin (2006/2007).
 #include <stdio.h>
 #include <unistd.h>
 #include <fcntl.h>
 #include <netdb.h>
 #include "FlowSrc.h"
 #include "Net.h"
 #include "analyzer/protocol/netflow/netflow_pac.h"
 #include <errno.h>
 FlowSrc::FlowSrc()
 	{ // TODO: v9.
 	selectable_fd = -1;
 	idle = false;
 	data = 0;
 	pdu_len = -1;
 	exporter_ip = 0;
 	current_timestamp = next_timestamp = 0.0;
 	netflow_analyzer = new binpac::NetFlow::NetFlow_Analyzer();
 	}
 FlowSrc::~FlowSrc()
 	{
 	delete netflow_analyzer;
 	}
 void FlowSrc::GetFds(int* read, int* write, int* except)
 	{
 	if ( selectable_fd >= 0 )
 		*read = selectable_fd;
 	}
 double FlowSrc::NextTimestamp(double* network_time)
 	{
 	if ( ! data && ! ExtractNextPDU() )
 		return -1.0;
 	else
 		return next_timestamp;
 	}
 void FlowSrc::Process()
 	{
 	if ( ! data && ! ExtractNextPDU() )
 		return;
 	// This is normally done by calling net_packet_dispatch(),
 	// but as we don't have a packet to dispatch ...
 	network_time = next_timestamp;
 	expire_timers();
 	netflow_analyzer->downflow()->set_exporter_ip(exporter_ip);
 	// We handle exceptions in NewData (might have changed w/ new binpac).
 	netflow_analyzer->NewData(0, data, data + pdu_len);
 	data = 0;
 	}
 void FlowSrc::Close()
 	{
 	safe_close(selectable_fd);
 	}
 FlowSocketSrc::~FlowSocketSrc()
 	{
 	}
 int FlowSocketSrc::ExtractNextPDU()
 	{
 	sockaddr_in from;
 	socklen_t fromlen = sizeof(from);
 	pdu_len = recvfrom(selectable_fd, buffer, NF_MAX_PKT_SIZE, 0,
 				(struct sockaddr*) &from, &fromlen);
 	if ( pdu_len < 0 )
 		{
 		reporter->Error("problem reading NetFlow data from socket");
 		data = 0;
 		next_timestamp = -1.0;
 		closed = 1;
 		return 0;
 		}
 	if ( fromlen != sizeof(from) )
 		{
 		reporter->Error("malformed NetFlow PDU");
 		return 0;
 		}
 	data = buffer;
 	exporter_ip = from.sin_addr.s_addr;
 	next_timestamp = current_time();
 	if ( next_timestamp < current_timestamp )
 		next_timestamp = current_timestamp;
 	else
 		current_timestamp = next_timestamp;
 	return 1;
 	}
 FlowSocketSrc::FlowSocketSrc(const char* listen_parms)
 	{
 	int n = strlen(listen_parms) + 1;
 	char laddr[n], port[n], ident[n];
 	laddr[0] = port[0] = ident[0] = '\0';
 	int ret = sscanf(listen_parms, "%[^:]:%[^=]=%s", laddr, port, ident);
 	if ( ret < 2 )
 		{
 		snprintf(errbuf, BRO_FLOW_ERRBUF_SIZE,
 			"parsing your listen-spec went nuts: laddr='%s', port='%s'\n",
 			laddr[0] ? laddr : "", port[0] ? port : "");
 		closed = 1;
 		return;
 		}
 	const char* id = (ret == 3) ? ident : listen_parms;
 	netflow_analyzer->downflow()->set_identifier(id);
 	struct addrinfo aiprefs = {
 		0, PF_INET, SOCK_DGRAM, IPPROTO_UDP, 0, NULL, NULL, NULL
 	};
 	struct addrinfo* ainfo = 0;
 	if ( (ret = getaddrinfo(laddr, port, &aiprefs, &ainfo)) != 0 )
 		{
 		snprintf(errbuf, BRO_FLOW_ERRBUF_SIZE,
 				"getaddrinfo(%s, %s, ...): %s",
 				laddr, port, gai_strerror(ret));
 		closed = 1;
 		return;
 		}
 	if ( (selectable_fd = socket (PF_INET, SOCK_DGRAM, 0)) < 0 )
 		{
 		snprintf(errbuf, BRO_FLOW_ERRBUF_SIZE,
 				"socket: %s", strerror(errno));
 		closed = 1;
 		goto cleanup;
 		}
 	if ( bind (selectable_fd, ainfo->ai_addr, ainfo->ai_addrlen) < 0 )
 		{
 		snprintf(errbuf, BRO_FLOW_ERRBUF_SIZE,
 				"bind: %s", strerror(errno));
 		closed = 1;
 		goto cleanup;
 		}
 cleanup:
 	freeaddrinfo(ainfo);
 	}
 FlowFileSrc::~FlowFileSrc()
 	{
 	delete [] readfile;
 	}
 int FlowFileSrc::ExtractNextPDU()
 	{
 	FlowFileSrcPDUHeader pdu_header;
 	if ( read(selectable_fd, &pdu_header, sizeof(pdu_header)) <
 	     int(sizeof(pdu_header)) )
 		return Error(errno, "read header");
 	if ( pdu_header.pdu_length > NF_MAX_PKT_SIZE )
 		{
 		reporter->Error("NetFlow packet too long");
 		// Safely skip over the too-long PDU.
 		if ( lseek(selectable_fd, pdu_header.pdu_length, SEEK_CUR) < 0 )
 			return Error(errno, "lseek");
 		return 0;
 		}
 	if ( read(selectable_fd, buffer, pdu_header.pdu_length) <
 	     pdu_header.pdu_length )
 		return Error(errno, "read data");
 	if ( next_timestamp < pdu_header.network_time )
 		{
 		next_timestamp = pdu_header.network_time;
 		current_timestamp = pdu_header.network_time;
 		}
 	else
 		current_timestamp = next_timestamp;
 	data = buffer;
 	pdu_len = pdu_header.pdu_length;
 	exporter_ip = pdu_header.ipaddr;
 	return 1;
 	}
 FlowFileSrc::FlowFileSrc(const char* readfile)
 	{
 	int n = strlen(readfile) + 1;
 	char ident[n];
 	this->readfile = new char[n];
 	int ret = sscanf(readfile, "%[^=]=%s", this->readfile, ident);
 	const char* id = (ret == 2) ? ident : this->readfile;
 	netflow_analyzer->downflow()->set_identifier(id);
 	selectable_fd = open(this->readfile, O_RDONLY);
 	if ( selectable_fd < 0 )
 		{
 		closed = 1;
 		snprintf(errbuf, BRO_FLOW_ERRBUF_SIZE,
 				"open: %s", strerror(errno));
 		}
 	}
 int FlowFileSrc::Error(int errlvl, const char* errmsg)
 	{
 	snprintf(errbuf, BRO_FLOW_ERRBUF_SIZE,
 			"%s: %s", errmsg, strerror(errlvl));
 	data = 0;
 	next_timestamp = -1.0;
 	closed = 1;
 	return 0;
 	}
--- a/src/FlowSrc.h
+++ b/src/FlowSrc.h
@ -1,84 +0,0 @@
 // See the file "COPYING" in the main distribution directory for copyright.
 //
 // Written by Bernhard Ager, TU Berlin (2006/2007).
 #ifndef flowsrc_h
 #define flowsrc_h
 #include "IOSource.h"
 #include "NetVar.h"
 #include "binpac.h"
 #define BRO_FLOW_ERRBUF_SIZE 512
 // TODO: 1500 is enough for v5 - how about the others?
 // 65536 would be enough for any UDP packet.
 #define NF_MAX_PKT_SIZE 8192
 struct FlowFileSrcPDUHeader {
 	double network_time;
 	int pdu_length;
 	uint32 ipaddr;
 };
 // Avoid including netflow_pac.h by explicitly declaring the NetFlow_Analyzer.
 namespace binpac {
 	namespace NetFlow {
 		class NetFlow_Analyzer;
 	}
 }
 class FlowSrc : public IOSource {
 public:
 	virtual ~FlowSrc();
 	// IOSource interface:
 	bool IsReady();
 	void GetFds(int* read, int* write, int* except);
 	double NextTimestamp(double* network_time);
 	void Process();
 	const char* Tag()		{ return "FlowSrc"; }
 	const char* ErrorMsg() const	{ return errbuf; }
 protected:
 	FlowSrc();
 	virtual int ExtractNextPDU() = 0;
 	virtual void Close();
 	int selectable_fd;
 	double current_timestamp;
 	double next_timestamp;
 	binpac::NetFlow::NetFlow_Analyzer* netflow_analyzer;
 	u_char buffer[NF_MAX_PKT_SIZE];
 	u_char* data;
 	int pdu_len;
 	uint32 exporter_ip;	// in network byte order
 	char errbuf[BRO_FLOW_ERRBUF_SIZE];
 };
 class FlowSocketSrc : public FlowSrc {
 public:
 	FlowSocketSrc(const char* listen_parms);
 	virtual ~FlowSocketSrc();
 	int ExtractNextPDU();
 };
 class FlowFileSrc : public FlowSrc {
 public:
 	FlowFileSrc(const char* readfile);
 	~FlowFileSrc();
 	int ExtractNextPDU();
 protected:
 	int Error(int errlvl, const char* errmsg);
 	char* readfile;
 };
 #endif
--- a/src/Func.cc
+++ b/src/Func.cc
@ -46,6 +46,7 @@
 #include "Event.h"
 #include "Traverse.h"
 #include "Reporter.h"
 #include "plugin/Manager.h"
 extern	RETSIGTYPE sig_handler(int signo);
@ -226,7 +227,7 @@ TraversalCode Func::Traverse(TraversalCallback* cb) const
 	HANDLE_TC_STMT_PRE(tc);
 	// FIXME: Traverse arguments to builtin functions, too.
-	if ( kind == BRO_FUNC )
+	if ( kind == BRO_FUNC && scope )
 		{
 		tc = scope->Traverse(cb);
 		HANDLE_TC_STMT_PRE(tc);
@ -244,6 +245,49 @@ TraversalCode Func::Traverse(TraversalCallback* cb) const
 	HANDLE_TC_STMT_POST(tc);
 	}
 Val* Func::HandlePluginResult(Val* plugin_result, val_list* args, function_flavor flavor) const
 	{
 	// Helper function factoring out this code from BroFunc:Call() for better
 	// readability.
 	switch ( flavor ) {
 	case FUNC_FLAVOR_EVENT:
 		Unref(plugin_result);
 		plugin_result = 0;
 		break;
 	case FUNC_FLAVOR_HOOK:
 		if ( plugin_result->Type()->Tag() != TYPE_BOOL )
 			reporter->InternalError("plugin returned non-bool for hook");
 		break;
 	case FUNC_FLAVOR_FUNCTION:
 		{
 		BroType* yt = FType()->YieldType();
 		if ( (! yt) || yt->Tag() == TYPE_VOID )
 			{
 			Unref(plugin_result);
 			plugin_result = 0;
 			}
 		else
 			{
 			if ( plugin_result->Type()->Tag() != yt->Tag() )
 				reporter->InternalError("plugin returned wrong type for function call");
 			}
 		break;
 		}
 	}
 	loop_over_list(*args, i)
 		Unref((*args)[i]);
 	return plugin_result;
 	}
 BroFunc::BroFunc(ID* arg_id, Stmt* arg_body, id_list* aggr_inits,
 		int arg_frame_size, int priority)
 : Func(BRO_FUNC)
@ -281,6 +325,17 @@ Val* BroFunc::Call(val_list* args, Frame* parent) const
 #ifdef PROFILE_BRO_FUNCTIONS
 	DEBUG_MSG("Function: %s\n", id->Name());
 #endif
 	SegmentProfiler(segment_logger, location);
 	if ( sample_logger )
 		sample_logger->FunctionSeen(this);
 	Val* plugin_result = PLUGIN_HOOK_WITH_RESULT(HOOK_CALL_FUNCTION, HookCallFunction(this, args), 0);
 	if ( plugin_result )
 		return HandlePluginResult(plugin_result, args, Flavor());
 	if ( bodies.empty() )
 		{
 		// Can only happen for events and hooks.
@ -291,7 +346,6 @@ Val* BroFunc::Call(val_list* args, Frame* parent) const
 		return Flavor() == FUNC_FLAVOR_HOOK ? new Val(true, TYPE_BOOL) : 0;
 		}
 	SegmentProfiler(segment_logger, location);
 	Frame* f = new Frame(frame_size, this, args);
 	// Hand down any trigger.
@ -319,9 +373,6 @@ Val* BroFunc::Call(val_list* args, Frame* parent) const
 	Val* result = 0;
 	if ( sample_logger )
 		sample_logger->FunctionSeen(this);
 	for ( size_t i = 0; i < bodies.size(); ++i )
 		{
 		if ( sample_logger )
@ -497,6 +548,11 @@ Val* BuiltinFunc::Call(val_list* args, Frame* parent) const
 	if ( sample_logger )
 		sample_logger->FunctionSeen(this);
 	Val* plugin_result = PLUGIN_HOOK_WITH_RESULT(HOOK_CALL_FUNCTION, HookCallFunction(this, args), 0);
 	if ( plugin_result )
 		return HandlePluginResult(plugin_result, args, FUNC_FLAVOR_FUNCTION);
 	if ( g_trace_state.DoTrace() )
 		{
 		ODesc d;
@ -550,18 +606,15 @@ void builtin_error(const char* msg, BroObj* arg)
 	}
 #include "bro.bif.func_h"
 #include "logging.bif.func_h"
 #include "input.bif.func_h"
 #include "reporter.bif.func_h"
 #include "strings.bif.func_h"
 #include "bro.bif.func_def"
 #include "logging.bif.func_def"
 #include "input.bif.func_def"
 #include "reporter.bif.func_def"
 #include "strings.bif.func_def"
 #include "__all__.bif.cc" // Autogenerated for compiling in the bif_target() code.
 #include "__all__.bif.register.cc" // Autogenerated for compiling in the bif_target() code.
 void init_builtin_funcs()
 	{
@ -572,16 +625,17 @@ void init_builtin_funcs()
 	gap_info = internal_type("gap_info")->AsRecordType();
 #include "bro.bif.func_init"
 #include "logging.bif.func_init"
 #include "input.bif.func_init"
 #include "reporter.bif.func_init"
 #include "strings.bif.func_init"
 #include "__all__.bif.init.cc" // Autogenerated for compiling in the bif_target() code.
 	did_builtin_init = true;
 	}
 void init_builtin_funcs_subdirs()
 {
 	#include "__all__.bif.init.cc" // Autogenerated for compiling in the bif_target() code.
 }
 bool check_built_in_call(BuiltinFunc* f, CallExpr* call)
 	{
 	if ( f->TheFunc() != BifFunc::bro_fmt )
--- a/src/Func.h
+++ b/src/Func.h
@ -52,6 +52,7 @@ public:
 	Kind GetKind() const	{ return kind; }
 	const char* Name() const { return name.c_str(); }
 	void SetName(const char* arg_name)	{ name = arg_name; }
 	virtual void Describe(ODesc* d) const = 0;
 	virtual void DescribeDebug(ODesc* d, const val_list* args) const;
@ -69,6 +70,9 @@ public:
 protected:
 	Func();
 	// Helper function for handling result of plugin hook.
 	Val* HandlePluginResult(Val* plugin_result, val_list* args, function_flavor flavor) const;
 	DECLARE_ABSTRACT_SERIAL(Func);
 	vector<Body> bodies;
@ -130,6 +134,7 @@ protected:
 extern void builtin_error(const char* msg, BroObj* arg = 0);
 extern void init_builtin_funcs();
 extern void init_builtin_funcs_subdirs();
 extern bool check_built_in_call(BuiltinFunc* f, CallExpr* call);
--- a/src/ID.h
+++ b/src/ID.h
@ -32,9 +32,11 @@ public:
 	void SetType(BroType* t)	{ Unref(type); type = t; }
 	BroType* Type()			{ return type; }
 	const BroType* Type() const	{ return type; }
 	void MakeType()			{ is_type = 1; }
 	BroType* AsType()		{ return is_type ? Type() : 0; }
 	const BroType* AsType() const	{ return is_type ? Type() : 0; }
 	// If weak_ref is false, the Val is assumed to be already ref'ed
 	// and will be deref'ed when the ID is deleted.
--- a/src/IOSource.h
+++ b/src/IOSource.h
@ -1,103 +0,0 @@
 // Interface for classes providing/consuming data during Bro's main loop.
 #ifndef iosource_h
 #define iosource_h
 #include <list>
 #include "Timer.h"
 using namespace std;
 class IOSource {
 public:
 	IOSource()	{ idle = closed = false; }
 	virtual ~IOSource()	{}
 	// Returns true if source has nothing ready to process.
 	bool IsIdle() const	{ return idle; }
 	// Returns true if more data is to be expected in the future.
 	// Otherwise, source may be removed.
 	bool IsOpen() const	{ return ! closed; }
 	// Returns select'able fds (leaves args untouched if we don't have
 	// selectable fds).
 	virtual void GetFds(int* read, int* write, int* except) = 0;
 	// The following two methods are only called when either IsIdle()
 	// returns false or select() on one of the fds indicates that there's
 	// data to process.
 	// Returns timestamp (in global network time) associated with next
 	// data item.  If the source wants the data item to be processed
 	// with a local network time, it sets the argument accordingly.
 	virtual double NextTimestamp(double* network_time) = 0;
 	// Processes and consumes next data item.
 	virtual void Process() = 0;
 	// Returns tag of timer manager associated with last processed
 	// data item, nil for global timer manager.
 	virtual TimerMgr::Tag* GetCurrentTag()	{ return 0; }
 	// Returns a descriptual tag for debugging.
 	virtual const char* Tag() = 0;
 protected:
 	// Derived classed are to set this to true if they have gone dry
 	// temporarily.
 	bool idle;
 	// Derived classed are to set this to true if they have gone dry
 	// permanently.
 	bool closed;
 };
 class IOSourceRegistry {
 public:
 	IOSourceRegistry()	{ call_count = 0; dont_counts = 0; }
 	~IOSourceRegistry();
 	// If dont_count is true, this source does not contribute to the
 	// number of IOSources returned by Size().  The effect is that
 	// if all sources but the non-counting ones have gone dry,
 	// processing will shut down.
 	void Register(IOSource* src, bool dont_count = false);
 	// This may block for some time.
 	IOSource* FindSoonest(double* ts);
 	int Size() const	{ return sources.size() - dont_counts; }
 	// Terminate IOSource processing immediately by removing all
 	// sources (and therefore returning a Size() of zero).
 	void Terminate()	{ RemoveAll(); }
 protected:
 	// When looking for a source with something to process,
 	// every SELECT_FREQUENCY calls we will go ahead and
 	// block on a select().
 	static const int SELECT_FREQUENCY = 25;
 	// Microseconds to wait in an empty select if no source is ready.
 	static const int SELECT_TIMEOUT = 50;
 	void RemoveAll();
 	unsigned int call_count;
 	int dont_counts;
 	struct Source {
 		IOSource* src;
 		int fd_read;
 		int fd_write;
 		int fd_except;
 	};
 	typedef list<Source*> SourceList;
 	SourceList sources;
 };
 extern IOSourceRegistry io_sources;
 #endif
--- a/src/Net.cc
+++ b/src/Net.cc
@ -29,6 +29,10 @@
 #include "Anon.h"
 #include "Serializer.h"
 #include "PacketDumper.h"
 #include "iosource/Manager.h"
 #include "iosource/PktSrc.h"
 #include "iosource/PktDumper.h"
 #include "plugin/Manager.h"
 extern "C" {
 #include "setsignal.h"
@ -38,10 +42,7 @@ extern "C" {
 extern int select(int, fd_set *, fd_set *, fd_set *, struct timeval *);
 }
-PList(PktSrc) pkt_srcs;
+iosource::PktDumper* pkt_dumper = 0;
 // FIXME: We should really merge PktDumper and PacketDumper.
 PktDumper* pkt_dumper = 0;
 int reading_live = 0;
 int reading_traces = 0;
@ -62,8 +63,8 @@ const u_char* current_pkt = 0;
 int current_dispatched = 0;
 int current_hdr_size = 0;
 double current_timestamp = 0.0;
-PktSrc* current_pktsrc = 0;
+iosource::PktSrc* current_pktsrc = 0;
-IOSource* current_iosrc;
+iosource::IOSource* current_iosrc = 0;
 std::list<ScannedFile> files_scanned;
 std::vector<string> sig_files;
@ -112,17 +113,21 @@ RETSIGTYPE watchdog(int /* signo */)
 					// saving the packet which caused the
 					// watchdog to trigger may be helpful,
 					// so we'll save that one nevertheless.
-					pkt_dumper = new PktDumper("watchdog-pkt.pcap");
+					pkt_dumper = iosource_mgr->OpenPktDumper("watchdog-pkt.pcap", false);
-					if ( pkt_dumper->IsError() )
+					if ( ! pkt_dumper || pkt_dumper->IsError() )
 						{
-						reporter->Error("watchdog: can't open watchdog-pkt.pcap for writing\n");
+						reporter->Error("watchdog: can't open watchdog-pkt.pcap for writing");
 						delete pkt_dumper;
 						pkt_dumper = 0;
 						}
 					}
 				if ( pkt_dumper )
-					pkt_dumper->Dump(current_hdr, current_pkt);
+					{
 					iosource::PktDumper::Packet p;
 					p.hdr = current_hdr;
 					p.data = current_pkt;
 					pkt_dumper->Dump(&p);
 					}
 				}
 			net_get_final_stats();
@ -141,121 +146,47 @@ RETSIGTYPE watchdog(int /* signo */)
 	return RETSIGVAL;
 	}
-void net_init(name_list& interfaces, name_list& readfiles,
+void net_update_time(double new_network_time)
 	      name_list& netflows, name_list& flowfiles,
 	        const char* writefile, const char* filter,
 			const char* secondary_filter, int do_watchdog)
 	{
-	init_net_var();
+	network_time = new_network_time;
 	PLUGIN_HOOK_VOID(HOOK_UPDATE_NETWORK_TIME, HookUpdateNetworkTime(new_network_time));
 	}
-	if ( readfiles.length() > 0 || flowfiles.length() > 0 )
+void net_init(name_list& interfaces, name_list& readfiles,
 	      const char* writefile, int do_watchdog)
 	{
 	if ( readfiles.length() > 0 )
 		{
 		reading_live = pseudo_realtime > 0.0;
 		reading_traces = 1;
 		for ( int i = 0; i < readfiles.length(); ++i )
 			{
-			PktFileSrc* ps = new PktFileSrc(readfiles[i], filter);
+			iosource::PktSrc* ps = iosource_mgr->OpenPktSrc(readfiles[i], false);
 			assert(ps);
 			if ( ! ps->IsOpen() )
-				reporter->FatalError("%s: problem with trace file %s - %s\n",
+				reporter->FatalError("problem with trace file %s (%s)",
-					prog, readfiles[i], ps->ErrorMsg());
+						     readfiles[i],
-			else
+						     ps->ErrorMsg());
 				{
 				pkt_srcs.append(ps);
 				io_sources.Register(ps);
 				}
 			if ( secondary_filter )
 				{
 				// We use a second PktFileSrc for the
 				// secondary path.
 				PktFileSrc* ps = new PktFileSrc(readfiles[i],
 							secondary_filter,
 							TYPE_FILTER_SECONDARY);
 				if ( ! ps->IsOpen() )
 					reporter->FatalError("%s: problem with trace file %s - %s\n",
 						prog, readfiles[i],
 						ps->ErrorMsg());
 				else
 					{
 					pkt_srcs.append(ps);
 					io_sources.Register(ps);
 					}
 				ps->AddSecondaryTablePrograms();
 				}
 			}
 		for ( int i = 0; i < flowfiles.length(); ++i )
 			{
 			FlowFileSrc* fs = new FlowFileSrc(flowfiles[i]);
 			if ( ! fs->IsOpen() )
 				reporter->FatalError("%s: problem with netflow file %s - %s\n",
 					prog, flowfiles[i], fs->ErrorMsg());
 			else
 				{
 				io_sources.Register(fs);
 				}
 			}
 		}
-	else if ((interfaces.length() > 0 || netflows.length() > 0))
+	else if ( interfaces.length() > 0 )
 		{
 		reading_live = 1;
 		reading_traces = 0;
 		for ( int i = 0; i < interfaces.length(); ++i )
 			{
-			PktSrc* ps;
+			iosource::PktSrc* ps = iosource_mgr->OpenPktSrc(interfaces[i], true);
-			ps = new PktInterfaceSrc(interfaces[i], filter);
+			assert(ps);
 			if ( ! ps->IsOpen() )
-				reporter->FatalError("%s: problem with interface %s - %s\n",
+				reporter->FatalError("problem with interface %s (%s)",
-					prog, interfaces[i], ps->ErrorMsg());
+						     interfaces[i],
-			else
+						     ps->ErrorMsg());
 				{
 				pkt_srcs.append(ps);
 				io_sources.Register(ps);
 				}
 			if ( secondary_filter )
 				{
 				PktSrc* ps;
 				ps = new PktInterfaceSrc(interfaces[i],
 					filter, TYPE_FILTER_SECONDARY);
 				if ( ! ps->IsOpen() )
 					reporter->Error("%s: problem with interface %s - %s\n",
 						prog, interfaces[i],
 						ps->ErrorMsg());
 				else
 					{
 					pkt_srcs.append(ps);
 					io_sources.Register(ps);
 					}
 				ps->AddSecondaryTablePrograms();
 				}
 			}
 		for ( int i = 0; i < netflows.length(); ++i )
 			{
 			FlowSocketSrc* fs = new FlowSocketSrc(netflows[i]);
 			if ( ! fs->IsOpen() )
 				{
 				reporter->Error("%s: problem with netflow socket %s - %s\n",
 					prog, netflows[i], fs->ErrorMsg());
 				delete fs;
 				}
 			else
 				io_sources.Register(fs);
 			}
 		}
 	else
@ -267,12 +198,12 @@ void net_init(name_list& interfaces, name_list& readfiles,
 	if ( writefile )
 		{
-		// ### This will fail horribly if there are multiple
+		pkt_dumper = iosource_mgr->OpenPktDumper(writefile, false);
-		// interfaces with different-lengthed media.
+		assert(pkt_dumper);
-		pkt_dumper = new PktDumper(writefile);
+
-		if ( pkt_dumper->IsError() )
+		if ( ! pkt_dumper->IsOpen() )
-			reporter->FatalError("%s: can't open write file \"%s\" - %s\n",
+			reporter->FatalError("problem opening dump file %s (%s)",
-				prog, writefile, pkt_dumper->ErrorMsg());
+					     writefile, pkt_dumper->ErrorMsg());
 		ID* id = global_scope()->Lookup("trace_output_file");
 		if ( ! id )
@ -293,7 +224,7 @@ void net_init(name_list& interfaces, name_list& readfiles,
 		}
 	}
-void expire_timers(PktSrc* src_ps)
+void expire_timers(iosource::PktSrc* src_ps)
 	{
 	SegmentProfiler(segment_logger, "expiring-timers");
 	TimerMgr* tmgr =
@ -306,8 +237,8 @@ void expire_timers(PktSrc* src_ps)
 	}
 void net_packet_dispatch(double t, const struct pcap_pkthdr* hdr,
-			const u_char* pkt, int hdr_size,
+			 const u_char* pkt, int hdr_size,
-			PktSrc* src_ps)
+			 iosource::PktSrc* src_ps)
 	{
 	if ( ! bro_start_network_time )
 		bro_start_network_time = t;
@ -315,7 +246,7 @@ void net_packet_dispatch(double t, const struct pcap_pkthdr* hdr,
 	TimerMgr* tmgr = sessions->LookupTimerMgr(src_ps->GetCurrentTag());
 	// network_time never goes back.
-	network_time = tmgr->Time() < t ? t : tmgr->Time();
+	net_update_time(tmgr->Time() < t ? t : tmgr->Time());
 	current_pktsrc = src_ps;
 	current_iosrc = src_ps;
@ -363,11 +294,11 @@ void net_run()
 	{
 	set_processing_status("RUNNING", "net_run");
-	while ( io_sources.Size() ||
+	while ( iosource_mgr->Size() ||
 		(BifConst::exit_only_after_terminate && ! terminating) )
 		{
 		double ts;
-		IOSource* src = io_sources.FindSoonest(&ts);
+		iosource::IOSource* src = iosource_mgr->FindSoonest(&ts);
 #ifdef DEBUG
 		static int loop_counter = 0;
@ -395,7 +326,7 @@ void net_run()
 				{
 				// Take advantage of the lull to get up to
 				// date on timers and events.
-				network_time = ct;
+				net_update_time(ct);
 				expire_timers();
 				usleep(1); // Just yield.
 				}
@ -408,7 +339,7 @@ void net_run()
 			// date on timers and events.  Because we only
 			// have timers as sources, going to sleep here
 			// doesn't risk blocking on other inputs.
-			network_time = current_time();
+			net_update_time(current_time());
 			expire_timers();
 			// Avoid busy-waiting - pause for 100 ms.
@ -465,16 +396,19 @@ void net_run()
 void net_get_final_stats()
 	{
-	loop_over_list(pkt_srcs, i)
+	const iosource::Manager::PktSrcList& pkt_srcs(iosource_mgr->GetPktSrcs());
 	for ( iosource::Manager::PktSrcList::const_iterator i = pkt_srcs.begin();
 	      i != pkt_srcs.end(); i++ )
 		{
-		PktSrc* ps = pkt_srcs[i];
+		iosource::PktSrc* ps = *i;
 		if ( ps->IsLive() )
 			{
-			struct PktSrc::Stats s;
+			iosource::PktSrc::Stats s;
 			ps->Statistics(&s);
-			reporter->Info("%d packets received on interface %s, %d dropped\n",
+			reporter->Info("%d packets received on interface %s, %d dropped",
-					s.received, ps->Interface(), s.dropped);
+					s.received, ps->Path().c_str(), s.dropped);
 			}
 		}
 	}
@ -494,8 +428,6 @@ void net_finish(int drain_events)
 			sessions->Done();
 		}
 	delete pkt_dumper;
 #ifdef DEBUG
 	extern int reassem_seen_bytes, reassem_copied_bytes;
 	// DEBUG_MSG("Reassembly (TCP and IP/Frag): %d bytes seen, %d bytes copied\n",
@ -516,29 +448,6 @@ void net_delete()
 		delete ip_anonymizer[i];
 	}
 // net_packet_match
 //
 // Description:
 //  - Checks if a packet matches a filter. It just wraps up a call to
 //    [pcap.h's] bpf_filter().
 //
 // Inputs:
 //  - fp: a BPF-compiled filter
 //  - pkt: a pointer to the packet
 //  - len: the original packet length
 //  - caplen: the captured packet length. This is pkt length
 //
 // Output:
 //  - return: 1 if the packet matches the filter, 0 otherwise
 int net_packet_match(BPF_Program* fp, const u_char* pkt,
 		     u_int len, u_int caplen)
 	{
 	// NOTE: I don't like too much un-const'ing the pkt variable.
 	return bpf_filter(fp->GetProgram()->bf_insns, (u_char*) pkt, len, caplen);
 	}
 int _processing_suspended = 0;
 static double suspend_start = 0;
@ -556,8 +465,12 @@ void net_continue_processing()
 	if ( _processing_suspended == 1 )
 		{
 		reporter->Info("processing continued");
-		loop_over_list(pkt_srcs, i)
+
-			pkt_srcs[i]->ContinueAfterSuspend();
+		const iosource::Manager::PktSrcList& pkt_srcs(iosource_mgr->GetPktSrcs());
 		for ( iosource::Manager::PktSrcList::const_iterator i = pkt_srcs.begin();
 		      i != pkt_srcs.end(); i++ )
 			(*i)->ContinueAfterSuspend();
 		}
 	--_processing_suspended;
--- a/src/Net.h
+++ b/src/Net.h
@ -5,27 +5,24 @@
 #include "net_util.h"
 #include "util.h"
 #include "BPF_Program.h"
 #include "List.h"
 #include "PktSrc.h"
 #include "FlowSrc.h"
 #include "Func.h"
 #include "RemoteSerializer.h"
 #include "iosource/IOSource.h"
 #include "iosource/PktSrc.h"
 #include "iosource/PktDumper.h"
 extern void net_init(name_list& interfaces, name_list& readfiles,
-		name_list& netflows, name_list& flowfiles,
+		const char* writefile, int do_watchdog);
 		const char* writefile, const char* filter,
 		const char* secondary_filter, int do_watchdog);
 extern void net_run();
 extern void net_get_final_stats();
 extern void net_finish(int drain_events);
 extern void net_delete();	// Reclaim all memory, etc.
 extern void net_update_time(double new_network_time);
 extern void net_packet_dispatch(double t, const struct pcap_pkthdr* hdr,
 			const u_char* pkt, int hdr_size,
-			PktSrc* src_ps);
+			iosource::PktSrc* src_ps);
-extern int net_packet_match(BPF_Program* fp, const u_char* pkt,
+extern void expire_timers(iosource::PktSrc* src_ps = 0);
 			    u_int len, u_int caplen);
 extern void expire_timers(PktSrc* src_ps = 0);
 extern void termination_signal();
 // Functions to temporarily suspend processing of live input (network packets
@ -82,13 +79,10 @@ extern const u_char* current_pkt;
 extern int current_dispatched;
 extern int current_hdr_size;
 extern double current_timestamp;
-extern PktSrc* current_pktsrc;
+extern iosource::PktSrc* current_pktsrc;
-extern IOSource* current_iosrc;
+extern iosource::IOSource* current_iosrc;
-declare(PList,PktSrc);
+extern iosource::PktDumper* pkt_dumper;	// where to save packets
 extern PList(PktSrc) pkt_srcs;
 extern PktDumper* pkt_dumper;	// where to save packets
 extern char* writefile;
--- a/src/NetVar.cc
+++ b/src/NetVar.cc
@ -245,8 +245,6 @@ bro_uint_t bits_per_uid;
 #include "const.bif.netvar_def"
 #include "types.bif.netvar_def"
 #include "event.bif.netvar_def"
 #include "logging.bif.netvar_def"
 #include "input.bif.netvar_def"
 #include "reporter.bif.netvar_def"
 void init_event_handlers()
@ -311,8 +309,6 @@ void init_net_var()
 	{
 #include "const.bif.netvar_init"
 #include "types.bif.netvar_init"
 #include "logging.bif.netvar_init"
 #include "input.bif.netvar_init"
 #include "reporter.bif.netvar_init"
 	conn_id = internal_type("conn_id")->AsRecordType();
--- a/src/NetVar.h
+++ b/src/NetVar.h
@ -255,8 +255,6 @@ extern void init_net_var();
 #include "const.bif.netvar_h"
 #include "types.bif.netvar_h"
 #include "event.bif.netvar_h"
 #include "logging.bif.netvar_h"
 #include "input.bif.netvar_h"
 #include "reporter.bif.netvar_h"
 #endif
--- a/src/Obj.cc
+++ b/src/Obj.cc
@ -7,6 +7,7 @@
 #include "Obj.h"
 #include "Serializer.h"
 #include "File.h"
 #include "plugin/Manager.h"
 Location no_location("<no location>", 0, 0, 0, 0);
 Location start_location("<start uninitialized>", 0, 0, 0, 0);
@ -92,6 +93,9 @@ int BroObj::suppress_errors = 0;
 BroObj::~BroObj()
 	{
 	if ( notify_plugins )
 		PLUGIN_HOOK_VOID(HOOK_BRO_OBJ_DTOR, HookBroObjDtor(this));
 	delete location;
 	}
--- a/src/Obj.h
+++ b/src/Obj.h
@ -92,6 +92,7 @@ public:
 		{
 		ref_cnt = 1;
 		in_ser_cache = false;
 		notify_plugins = false;
 		// A bit of a hack.  We'd like to associate location
 		// information with every object created when parsing,
@ -151,6 +152,9 @@ public:
 	// extend compound objects such as statement lists.
 	virtual void UpdateLocationEndInfo(const Location& end);
 	// Enable notification of plugins when this objects gets destroyed.
 	void NotifyPluginsOnDtor()	{ notify_plugins = true; }
 	int RefCnt() const	{ return ref_cnt; }
 	// Helper class to temporarily suppress errors
@ -181,6 +185,7 @@ private:
 	friend inline void Ref(BroObj* o);
 	friend inline void Unref(BroObj* o);
 	bool notify_plugins;
 	int ref_cnt;
 	// If non-zero, do not print runtime errors.  Useful for
--- a/src/PktSrc.cc
+++ b/src/PktSrc.cc
@ -1,804 +0,0 @@
 // See the file "COPYING" in the main distribution directory for copyright.
 #include <errno.h>
 #include <sys/stat.h>
 #include "config.h"
 #include "util.h"
 #include "PktSrc.h"
 #include "Hash.h"
 #include "Net.h"
 #include "Sessions.h"
 // ### This needs auto-confing.
 #ifdef HAVE_PCAP_INT_H
 #include <pcap-int.h>
 #endif
 PktSrc::PktSrc()
 	{
 	interface = readfile = 0;
 	data = last_data = 0;
 	memset(&hdr, 0, sizeof(hdr));
 	hdr_size = 0;
 	datalink = 0;
 	netmask = 0xffffff00;
 	pd = 0;
 	idle = false;
 	next_sync_point = 0;
 	first_timestamp = current_timestamp = next_timestamp = 0.0;
 	first_wallclock = current_wallclock = 0;
 	stats.received = stats.dropped = stats.link = 0;
 	}
 PktSrc::~PktSrc()
 	{
 	Close();
 	loop_over_list(program_list, i)
 		delete program_list[i];
 	BPF_Program* code;
 	IterCookie* cookie = filters.InitForIteration();
 	while ( (code = filters.NextEntry(cookie)) )
 		delete code;
 	delete [] interface;
 	delete [] readfile;
 	}
 void PktSrc::GetFds(int* read, int* write, int* except)
 	{
 	if ( pseudo_realtime )
 		{
 		// Select would give erroneous results. But we simulate it
 		// by setting idle accordingly.
 		idle = CheckPseudoTime() == 0;
 		return;
 		}
 	if ( selectable_fd >= 0 )
 		*read = selectable_fd;
 	}
 int PktSrc::ExtractNextPacket()
 	{
 	// Don't return any packets if processing is suspended (except for the
 	// very first packet which we need to set up times).
 	if ( net_is_processing_suspended() && first_timestamp )
 		{
 		idle = true;
 		return 0;
 		}
 	data = last_data = pcap_next(pd, &hdr);
 	if ( data && (hdr.len == 0 || hdr.caplen == 0) )
 		{
 		sessions->Weird("empty_pcap_header", &hdr, data);
 		return 0;
 		}
 	if ( data )
 		next_timestamp = hdr.ts.tv_sec + double(hdr.ts.tv_usec) / 1e6;
 	if ( pseudo_realtime )
 		current_wallclock = current_time(true);
 	if ( ! first_timestamp )
 		first_timestamp = next_timestamp;
 	idle = (data == 0);
 	if ( data )
 		++stats.received;
 	// Source has gone dry.  If it's a network interface, this just means
 	// it's timed out. If it's a file, though, then the file has been
 	// exhausted.
 	if ( ! data && ! IsLive() )
 		{
 		closed = true;
 		if ( pseudo_realtime && using_communication )
 			{
 			if ( remote_trace_sync_interval )
 				remote_serializer->SendFinalSyncPoint();
 			else
 				remote_serializer->Terminate();
 			}
 		}
 	return data != 0;
 	}
 double PktSrc::NextTimestamp(double* local_network_time)
 	{
 	if ( ! data && ! ExtractNextPacket() )
 		return -1.0;
 	if ( pseudo_realtime )
 		{
 		// Delay packet if necessary.
 		double packet_time = CheckPseudoTime();
 		if ( packet_time )
 			return packet_time;
 		idle = true;
 		return -1.0;
 		}
 	return next_timestamp;
 	}
 void PktSrc::ContinueAfterSuspend()
 	{
 	current_wallclock = current_time(true);
 	}
 double PktSrc::CurrentPacketWallClock()
 	{
 	// We stop time when we are suspended.
 	if ( net_is_processing_suspended() )
 		current_wallclock = current_time(true);
 	return current_wallclock;
 	}
 double PktSrc::CheckPseudoTime()
 	{
 	if ( ! data && ! ExtractNextPacket() )
 		return 0;
 	if ( ! current_timestamp )
 		return bro_start_time;
 	if ( remote_trace_sync_interval )
 		{
 		if ( next_sync_point == 0 || next_timestamp >= next_sync_point )
 			{
 			int n = remote_serializer->SendSyncPoint();
 			next_sync_point = first_timestamp +
 						n * remote_trace_sync_interval;
 			remote_serializer->Log(RemoteSerializer::LogInfo,
 				fmt("stopping at packet %.6f, next sync-point at %.6f",
 					current_timestamp, next_sync_point));
 			return 0;
 			}
 		}
 	double pseudo_time = next_timestamp - first_timestamp;
 	double ct = (current_time(true) - first_wallclock) * pseudo_realtime;
 	return pseudo_time <= ct ? bro_start_time + pseudo_time : 0;
 	}
 void PktSrc::Process()
 	{
 	if ( ! data && ! ExtractNextPacket() )
 		return;
 	current_timestamp = next_timestamp;
 	int pkt_hdr_size = hdr_size;
 	// Unfortunately some packets on the link might have MPLS labels
 	// while others don't. That means we need to ask the link-layer if
 	// labels are in place.
 	bool have_mpls = false;
 	int protocol = 0;
 	switch ( datalink ) {
 	case DLT_NULL:
 		{
 		protocol = (data[3] << 24) + (data[2] << 16) + (data[1] << 8) + data[0];
 		// From the Wireshark Wiki: "AF_INET6, unfortunately, has
 		// different values in {NetBSD,OpenBSD,BSD/OS},
 		// {FreeBSD,DragonFlyBSD}, and {Darwin/Mac OS X}, so an IPv6
 		// packet might have a link-layer header with 24, 28, or 30
 		// as the AF_ value." As we may be reading traces captured on
 		// platforms other than what we're running on, we accept them
 		// all here.
 		if ( protocol != AF_INET
 		     && protocol != AF_INET6
 		     && protocol != 24
 		     && protocol != 28
 		     && protocol != 30 )
 			{
 			sessions->Weird("non_ip_packet_in_null_transport", &hdr, data);
 			data = 0;
 			return;
 			}
 		break;
 		}
 	case DLT_EN10MB:
 		{
 		// Get protocol being carried from the ethernet frame.
 		protocol = (data[12] << 8) + data[13];
 		switch ( protocol )
 			{
 			// MPLS carried over the ethernet frame.
 			case 0x8847:
 				// Remove the data link layer and denote a
 				// header size of zero before the IP header.
 				have_mpls = true;
 				data += get_link_header_size(datalink);
 				pkt_hdr_size = 0;
 				break;
 			// VLAN carried over the ethernet frame.
 			case 0x8100:
 				data += get_link_header_size(datalink);
 				// Check for MPLS in VLAN.
 				if ( ((data[2] << 8) + data[3]) == 0x8847 )
 					have_mpls = true;
 				data += 4; // Skip the vlan header
 				pkt_hdr_size = 0;
 				// Check for 802.1ah (Q-in-Q) containing IP.
 				// Only do a second layer of vlan tag
 				// stripping because there is no
 				// specification that allows for deeper
 				// nesting.
 				if ( ((data[2] << 8) + data[3]) == 0x0800 )
 					data += 4;
 				break;
 			// PPPoE carried over the ethernet frame.
 			case 0x8864:
 				data += get_link_header_size(datalink);
 				protocol = (data[6] << 8) + data[7];
 				data += 8; // Skip the PPPoE session and PPP header
 				pkt_hdr_size = 0;
 				if ( protocol != 0x0021 && protocol != 0x0057 )
 					{
 					// Neither IPv4 nor IPv6.
 					sessions->Weird("non_ip_packet_in_pppoe_encapsulation", &hdr, data);
 					data = 0;
 					return;
 					}
 				break;
 			}
 		break;
 		}
 	case DLT_PPP_SERIAL:
 		{
 		// Get PPP protocol.
 		protocol = (data[2] << 8) + data[3];
 		if ( protocol == 0x0281 )
 			{
 			// MPLS Unicast. Remove the data link layer and
 			// denote a header size of zero before the IP header.
 			have_mpls = true;
 			data += get_link_header_size(datalink);
 			pkt_hdr_size = 0;
 			}
 		else if ( protocol != 0x0021 && protocol != 0x0057 )
 			{
 			// Neither IPv4 nor IPv6.
 			sessions->Weird("non_ip_packet_in_ppp_encapsulation", &hdr, data);
 			data = 0;
 			return;
 			}
 		break;
 		}
 	}
 	if ( have_mpls )
 		{
 		// Skip the MPLS label stack.
 		bool end_of_stack = false;
 		while ( ! end_of_stack )
 			{
 			end_of_stack = *(data + 2) & 0x01;
 			data += 4;
 			}
 		}
 	if ( pseudo_realtime )
 		{
 		current_pseudo = CheckPseudoTime();
 		net_packet_dispatch(current_pseudo, &hdr, data, pkt_hdr_size, this);
 		if ( ! first_wallclock )
 			first_wallclock = current_time(true);
 		}
 	else
 		net_packet_dispatch(current_timestamp, &hdr, data, pkt_hdr_size, this);
 	data = 0;
 	}
 bool PktSrc::GetCurrentPacket(const struct pcap_pkthdr** arg_hdr,
 				const u_char** arg_pkt)
 	{
 	if ( ! last_data )
 		return false;
 	*arg_hdr = &hdr;
 	*arg_pkt = last_data;
 	return true;
 	}
 int PktSrc::PrecompileFilter(int index, const char* filter)
 	{
 	// Compile filter.
 	BPF_Program* code = new BPF_Program();
 	if ( ! code->Compile(pd, filter, netmask, errbuf, sizeof(errbuf)) )
 		{
 		delete code;
 		return 0;
 		}
 	// Store it in hash.
 	HashKey* hash = new HashKey(HashKey(bro_int_t(index)));
 	BPF_Program* oldcode = filters.Lookup(hash);
 	if ( oldcode )
 		delete oldcode;
 	filters.Insert(hash, code);
 	delete hash;
 	return 1;
 	}
 int PktSrc::SetFilter(int index)
 	{
 	// We don't want load-level filters for the secondary path.
 	if ( filter_type == TYPE_FILTER_SECONDARY && index > 0 )
 		return 1;
 	HashKey* hash = new HashKey(HashKey(bro_int_t(index)));
 	BPF_Program* code = filters.Lookup(hash);
 	delete hash;
 	if ( ! code )
 		{
 		safe_snprintf(errbuf, sizeof(errbuf),
 			      "No precompiled pcap filter for index %d",
 			      index);
 		return 0;
 		}
 	if ( pcap_setfilter(pd, code->GetProgram()) < 0 )
 		{
 		safe_snprintf(errbuf, sizeof(errbuf),
 			      "pcap_setfilter(%d): %s",
 			      index, pcap_geterr(pd));
 		return 0;
 		}
 #ifndef HAVE_LINUX
 	// Linux doesn't clear counters when resetting filter.
 	stats.received = stats.dropped = stats.link = 0;
 #endif
 	return 1;
 	}
 void PktSrc::SetHdrSize()
 	{
 	int dl = pcap_datalink(pd);
 	hdr_size = get_link_header_size(dl);
 	if ( hdr_size < 0 )
 		{
 		safe_snprintf(errbuf, sizeof(errbuf),
 			 "unknown data link type 0x%x", dl);
 		Close();
 		}
 	datalink = dl;
 	}
 void PktSrc::Close()
 	{
 	if ( pd )
 		{
 		pcap_close(pd);
 		pd = 0;
 		closed = true;
 		}
 	}
 void PktSrc::AddSecondaryTablePrograms()
 	{
 	BPF_Program* program;
 	loop_over_list(secondary_path->EventTable(), i)
 		{
 		SecondaryEvent* se = secondary_path->EventTable()[i];
 		program = new BPF_Program();
 		if ( ! program->Compile(snaplen, datalink, se->Filter(),
 					netmask, errbuf, sizeof(errbuf)) )
 			{
 			delete program;
 			Close();
 			return;
 			}
 		SecondaryProgram* sp = new SecondaryProgram(program, se);
 		program_list.append(sp);
 		}
 	}
 void PktSrc::Statistics(Stats* s)
 	{
 	if ( reading_traces )
 		s->received = s->dropped = s->link = 0;
 	else
 		{
 		struct pcap_stat pstat;
 		if ( pcap_stats(pd, &pstat) < 0 )
 			{
 			reporter->Error("problem getting packet filter statistics: %s",
 					ErrorMsg());
 			s->received = s->dropped = s->link = 0;
 			}
 		else
 			{
 			s->dropped = pstat.ps_drop;
 			s->link = pstat.ps_recv;
 			}
 		}
 	s->received = stats.received;
 	if ( pseudo_realtime )
 		s->dropped = 0;
 	stats.dropped = s->dropped;
 	}
 PktInterfaceSrc::PktInterfaceSrc(const char* arg_interface, const char* filter,
 					PktSrc_Filter_Type ft)
 : PktSrc()
 	{
 	char tmp_errbuf[PCAP_ERRBUF_SIZE];
 	filter_type = ft;
 	// Determine interface if not specified.
 	if ( ! arg_interface && ! (arg_interface = pcap_lookupdev(tmp_errbuf)) )
 		{
 		safe_snprintf(errbuf, sizeof(errbuf),
 			 "pcap_lookupdev: %s", tmp_errbuf);
 		return;
 		}
 	interface = copy_string(arg_interface);
 	// Determine network and netmask.
 	uint32 net;
 	if ( pcap_lookupnet(interface, &net, &netmask, tmp_errbuf) < 0 )
 		{
 		// ### The lookup can fail if no address is assigned to
 		// the interface; and libpcap doesn't have any useful notion
 		// of error codes, just error strings - how bogus - so we
 		// just kludge around the error :-(.
 		// sprintf(errbuf, "pcap_lookupnet %s", tmp_errbuf);
 		// return;
 		net = 0;
 		netmask = 0xffffff00;
 		}
 	// We use the smallest time-out possible to return almost immediately if
 	// no packets are available. (We can't use set_nonblocking() as it's
 	// broken on FreeBSD: even when select() indicates that we can read
 	// something, we may get nothing if the store buffer hasn't filled up
 	// yet.)
 	pd = pcap_open_live(interface, snaplen, 1, 1, tmp_errbuf);
 	if ( ! pd )
 		{
 		safe_snprintf(errbuf, sizeof(errbuf),
 			 "pcap_open_live: %s", tmp_errbuf);
 		closed = true;
 		return;
 		}
 	// ### This needs autoconf'ing.
 #ifdef HAVE_PCAP_INT_H
 	reporter->Info("pcap bufsize = %d\n", ((struct pcap *) pd)->bufsize);
 #endif
 #ifdef HAVE_LINUX
 	if ( pcap_setnonblock(pd, 1, tmp_errbuf) < 0 )
 		{
 		safe_snprintf(errbuf, sizeof(errbuf),
 			 "pcap_setnonblock: %s", tmp_errbuf);
 		pcap_close(pd);
 		closed = true;
 		return;
 		}
 #endif
 	selectable_fd = pcap_fileno(pd);
 	if ( PrecompileFilter(0, filter) && SetFilter(0) )
 		{
 		SetHdrSize();
 		if ( closed )
 			// Couldn't get header size.
 			return;
 		reporter->Info("listening on %s, capture length %d bytes\n", interface, snaplen);
 		}
 	else
 		closed = true;
 	}
 PktFileSrc::PktFileSrc(const char* arg_readfile, const char* filter,
 			PktSrc_Filter_Type ft)
 : PktSrc()
 	{
 	readfile = copy_string(arg_readfile);
 	filter_type = ft;
 	pd = pcap_open_offline((char*) readfile, errbuf);
 	if ( pd && PrecompileFilter(0, filter) && SetFilter(0) )
 		{
 		SetHdrSize();
 		if ( closed )
 			// Unknown link layer type.
 			return;
 		// We don't put file sources into non-blocking mode as
 		// otherwise we would not be able to identify the EOF.
 		selectable_fd = fileno(pcap_file(pd));
 		if ( selectable_fd < 0 )
 			reporter->InternalError("OS does not support selectable pcap fd");
 		}
 	else
 		closed = true;
 	}
 SecondaryPath::SecondaryPath()
 	{
 	filter = 0;
 	// Glue together the secondary filter, if exists.
 	Val* secondary_fv = internal_val("secondary_filters");
 	if ( secondary_fv->AsTableVal()->Size() == 0 )
 		return;
 	int did_first = 0;
 	const TableEntryValPDict* v = secondary_fv->AsTable();
 	IterCookie* c = v->InitForIteration();
 	TableEntryVal* tv;
 	HashKey* h;
 	while ( (tv = v->NextEntry(h, c)) )
 		{
 		// Get the index values.
 		ListVal* index =
 			secondary_fv->AsTableVal()->RecoverIndex(h);
 		const char* str =
 			index->Index(0)->Ref()->AsString()->CheckString();
 		if ( ++did_first == 1 )
 			{
 			filter = copy_string(str);
 			}
 		else
 			{
 			if ( strlen(filter) > 0 )
 				{
 				char* tmp_f = new char[strlen(str) + strlen(filter) + 32];
 				if ( strlen(str) == 0 )
 					sprintf(tmp_f, "%s", filter);
 				else
 					sprintf(tmp_f, "(%s) or (%s)", filter, str);
 				delete [] filter;
 				filter = tmp_f;
 				}
 			}
 		// Build secondary_path event table item and link it.
 		SecondaryEvent* se =
 			new SecondaryEvent(index->Index(0)->Ref()->AsString()->CheckString(),
 				tv->Value()->AsFunc() );
 		event_list.append(se);
 		delete h;
 		Unref(index);
 		}
 	}
 SecondaryPath::~SecondaryPath()
 	{
 	loop_over_list(event_list, i)
 		delete event_list[i];
 	delete [] filter;
 	}
 SecondaryProgram::~SecondaryProgram()
 	{
 	delete program;
 	}
 PktDumper::PktDumper(const char* arg_filename, bool arg_append)
 	{
 	filename[0] = '\0';
 	is_error = false;
 	append = arg_append;
 	dumper = 0;
 	open_time = 0.0;
 	// We need a pcap_t with a reasonable link-layer type. We try to get it
 	// from the packet sources. If not available, we fall back to Ethernet.
 	// FIXME: Perhaps we should make this configurable?
 	int linktype = -1;
 	if ( pkt_srcs.length() )
 		linktype = pkt_srcs[0]->LinkType();
 	if ( linktype < 0 )
 		linktype = DLT_EN10MB;
 	pd = pcap_open_dead(linktype, snaplen);
 	if ( ! pd )
 		{
 		Error("error for pcap_open_dead");
 		return;
 		}
 	if ( arg_filename )
 		Open(arg_filename);
 	}
 bool PktDumper::Open(const char* arg_filename)
 	{
 	if ( ! arg_filename && ! *filename )
 		{
 		Error("no filename given");
 		return false;
 		}
 	if ( arg_filename )
 		{
 		if ( dumper && streq(arg_filename, filename) )
 			// Already open.
 			return true;
 		safe_strncpy(filename, arg_filename, FNBUF_LEN);
 		}
 	if ( dumper )
 		Close();
 	struct stat s;
 	int exists = 0;
 	if ( append )
 		{
 		// See if output file already exists (and is non-empty).
 		exists = stat(filename, &s); ;
 		if ( exists < 0 && errno != ENOENT )
 			{
 			Error(fmt("can't stat file %s: %s", filename, strerror(errno)));
 			return false;
 			}
 		}
 	if ( ! append || exists < 0 || s.st_size == 0 )
 		{
 		// Open new file.
 		dumper =  pcap_dump_open(pd, filename);
 		if ( ! dumper )
 			{
 			Error(pcap_geterr(pd));
 			return false;
 			}
 		}
 	else
 		{
 		// Old file and we need to append, which, unfortunately,
 		// is not supported by libpcap. So, we have to hack a
 		// little bit, knowing that pcap_dumpter_t is, in fact,
 		// a FILE ... :-(
 		dumper = (pcap_dumper_t*) fopen(filename, "a");
 		if ( ! dumper )
 			{
 			Error(fmt("can't open dump %s: %s", filename, strerror(errno)));
 			return false;
 			}
 		}
 	open_time = network_time;
 	is_error = false;
 	return true;
 	}
 bool PktDumper::Close()
 	{
 	if ( dumper )
 		{
 		pcap_dump_close(dumper);
 		dumper = 0;
 		is_error = false;
 		}
 	return true;
 	}
 bool PktDumper::Dump(const struct pcap_pkthdr* hdr, const u_char* pkt)
 	{
 	if ( ! dumper )
 		return false;
 	if ( ! open_time )
 		open_time = network_time;
 	pcap_dump((u_char*) dumper, hdr, pkt);
 	return true;
 	}
 void PktDumper::Error(const char* errstr)
 	{
 	safe_strncpy(errbuf, errstr, sizeof(errbuf));
 	is_error = true;
 	}
 int get_link_header_size(int dl)
 	{
 	switch ( dl ) {
 	case DLT_NULL:
 		return 4;
 	case DLT_EN10MB:
 		return 14;
 	case DLT_FDDI:
 		return 13 + 8;	// fddi_header + LLC
 #ifdef DLT_LINUX_SLL
 	case DLT_LINUX_SLL:
 		return 16;
 #endif
 	case DLT_PPP_SERIAL:	// PPP_SERIAL
 		return 4;
 	case DLT_RAW:
 		return 0;
 	}
 	return -1;
 	}
--- a/src/PktSrc.h
+++ b/src/PktSrc.h
@ -1,258 +0,0 @@
 // See the file "COPYING" in the main distribution directory for copyright.
 #ifndef pktsrc_h
 #define pktsrc_h
 #include "Dict.h"
 #include "Expr.h"
 #include "BPF_Program.h"
 #include "IOSource.h"
 #include "RemoteSerializer.h"
 #define BRO_PCAP_ERRBUF_SIZE   PCAP_ERRBUF_SIZE + 256
 extern "C" {
 #include <pcap.h>
 }
 declare(PDict,BPF_Program);
 // Whether a PktSrc object is used by the normal filter structure or the
 // secondary-path structure.
 typedef enum {
 	TYPE_FILTER_NORMAL,  // the normal filter
 	TYPE_FILTER_SECONDARY,  // the secondary-path filter
 } PktSrc_Filter_Type;
 // {filter,event} tuples conforming the secondary path.
 class SecondaryEvent {
 public:
 	SecondaryEvent(const char* arg_filter, Func* arg_event)
 		{
 		filter = arg_filter;
 		event = arg_event;
 		}
 	const char* Filter()	{ return filter; }
 	Func* Event()		{ return event; }
 private:
 	const char* filter;
 	Func* event;
 };
 declare(PList,SecondaryEvent);
 typedef PList(SecondaryEvent) secondary_event_list;
 class SecondaryPath {
 public:
 	SecondaryPath();
 	~SecondaryPath();
 	secondary_event_list& EventTable()	{ return event_list; }
 	const char* Filter()			{ return filter; }
 private:
 	secondary_event_list event_list;
 	// OR'ed union of all SecondaryEvent filters
 	char* filter;
 };
 // Main secondary-path object.
 extern SecondaryPath* secondary_path;
 // {program, {filter,event}} tuple table.
 class SecondaryProgram {
 public:
 	SecondaryProgram(BPF_Program* arg_program, SecondaryEvent* arg_event)
 		{
 		program = arg_program;
 		event = arg_event;
 		}
 	~SecondaryProgram();
 	BPF_Program* Program()  { return program; }
 	SecondaryEvent* Event()	{ return event; }
 private:
 	// Associated program.
 	BPF_Program *program;
 	// Event that is run in case the program is matched.
 	SecondaryEvent* event;
 };
 declare(PList,SecondaryProgram);
 typedef PList(SecondaryProgram) secondary_program_list;
 class PktSrc : public IOSource {
 public:
 	~PktSrc();
 	// IOSource interface
 	bool IsReady();
 	void GetFds(int* read, int* write, int* except);
 	double NextTimestamp(double* local_network_time);
 	void Process();
 	const char* Tag()	{ return "PktSrc"; }
 	const char* ErrorMsg() const	{ return errbuf; }
 	void ClearErrorMsg()		{ *errbuf ='\0'; }
 	// Returns the packet last processed; false if there is no
 	// current packet available.
 	bool GetCurrentPacket(const pcap_pkthdr** hdr, const u_char** pkt);
 	int HdrSize() const		{ return hdr_size; }
 	int DataLink() const		{ return datalink; }
 	void ConsumePacket()	{ data = 0; }
 	int IsLive() const		{ return interface != 0; }
 	pcap_t* PcapHandle() const	{ return pd; }
 	int LinkType() const		{ return pcap_datalink(pd); }
 	const char* ReadFile() const	{ return readfile; }
 	const char* Interface() const	{ return interface; }
 	PktSrc_Filter_Type FilterType() const	{ return filter_type; }
 	void AddSecondaryTablePrograms();
 	const secondary_program_list& ProgramTable() const
 		{ return program_list; }
 	// Signal packet source that processing was suspended and is now going
 	// to be continued.
 	void ContinueAfterSuspend();
 	// Only valid in pseudo-realtime mode.
 	double CurrentPacketTimestamp()	{ return current_pseudo; }
 	double CurrentPacketWallClock();
 	struct Stats {
 		unsigned int received;	// pkts received (w/o drops)
 		unsigned int dropped;	// pkts dropped
 		unsigned int link;	// total packets on link
 					// (not always not available)
 	};
 	virtual void Statistics(Stats* stats);
 	// Precompiles a filter and associates the given index with it.
 	// Returns true on success, 0 if a problem occurred.
 	virtual int PrecompileFilter(int index, const char* filter);
 	// Activates the filter with the given index.
 	// Returns true on success, 0 if a problem occurred.
 	virtual int SetFilter(int index);
 protected:
 	PktSrc();
 	static const int PCAP_TIMEOUT = 20;
 	void SetHdrSize();
 	virtual void Close();
 	// Returns 1 on success, 0 on time-out/gone dry.
 	virtual int ExtractNextPacket();
 	// Checks if the current packet has a pseudo-time <= current_time.
 	// If yes, returns pseudo-time, otherwise 0.
 	double CheckPseudoTime();
 	double current_timestamp;
 	double next_timestamp;
 	// Only set in pseudo-realtime mode.
 	double first_timestamp;
 	double first_wallclock;
 	double current_wallclock;
 	double current_pseudo;
 	struct pcap_pkthdr hdr;
 	const u_char* data;	// contents of current packet
 	const u_char* last_data;	// same, but unaffected by consuming
 	int hdr_size;
 	int datalink;
 	double next_sync_point; // For trace synchronziation in pseudo-realtime
 	char* interface;	// nil if not reading from an interface
 	char* readfile;		// nil if not reading from a file
 	pcap_t* pd;
 	int selectable_fd;
 	uint32 netmask;
 	char errbuf[BRO_PCAP_ERRBUF_SIZE];
 	Stats stats;
 	PDict(BPF_Program) filters; // precompiled filters
 	PktSrc_Filter_Type filter_type; // normal path or secondary path
 	secondary_program_list program_list;
 };
 class PktInterfaceSrc : public PktSrc {
 public:
 	PktInterfaceSrc(const char* interface, const char* filter,
 			PktSrc_Filter_Type ft=TYPE_FILTER_NORMAL);
 };
 class PktFileSrc : public PktSrc {
 public:
 	PktFileSrc(const char* readfile, const char* filter,
 			PktSrc_Filter_Type ft=TYPE_FILTER_NORMAL);
 };
 extern int get_link_header_size(int dl);
 class PktDumper {
 public:
 	PktDumper(const char* file = 0, bool append = false);
 	~PktDumper()	{ Close(); }
 	bool Open(const char* file = 0);
 	bool Close();
 	bool Dump(const struct pcap_pkthdr* hdr, const u_char* pkt);
 	pcap_dumper_t* PcapDumper() 	{ return dumper; }
 	const char* FileName() const	{ return filename; }
 	bool IsError() const		{ return is_error; }
 	const char* ErrorMsg() const	{ return errbuf; }
 	// This heuristic will horribly fail if we're using packets
 	// with different link layers.  (If we can't derive a reasonable value
 	// from the packet sources, our fall-back is Ethernet.)
 	int HdrSize() const
 		{ return get_link_header_size(pcap_datalink(pd)); }
 	// Network time when dump file was opened.
 	double OpenTime() const		{ return open_time; }
 private:
 	void InitPd();
 	void Error(const char* str);
 	static const int FNBUF_LEN = 1024;
 	char filename[FNBUF_LEN];
 	bool append;
 	pcap_dumper_t* dumper;
 	pcap_t* pd;
 	double open_time;
 	bool is_error;
 	char errbuf[BRO_PCAP_ERRBUF_SIZE];
 };
 #endif
--- a/src/RemoteSerializer.cc
+++ b/src/RemoteSerializer.cc
@ -188,10 +188,11 @@
 #include "File.h"
 #include "Conn.h"
 #include "Reporter.h"
 #include "threading/SerialTypes.h"
 #include "logging/Manager.h"
 #include "IPAddr.h"
 #include "bro_inet_ntop.h"
 #include "iosource/Manager.h"
 #include "logging/Manager.h"
 #include "logging/logging.bif.h"
 extern "C" {
 #include "setsignal.h"
@ -284,10 +285,10 @@ struct ping_args {
 	\
 	if ( ! c ) \
 		{ \
-		idle = io->IsIdle();\
+		SetIdle(io->IsIdle());\
 		return true; \
 		} \
-	idle = false; \
+	SetIdle(false); \
 	}
 static const char* msgToStr(int msg)
@ -533,7 +534,6 @@ RemoteSerializer::RemoteSerializer()
 	current_sync_point = 0;
 	syncing_times = false;
 	io = 0;
 	closed = false;
 	terminating = false;
 	in_sync = 0;
 	last_flush = 0;
@ -558,7 +558,7 @@ RemoteSerializer::~RemoteSerializer()
 	delete io;
 	}
-void RemoteSerializer::Init()
+void RemoteSerializer::Enable()
 	{
 	if ( initialized )
 		return;
@ -571,7 +571,7 @@ void RemoteSerializer::Init()
 	Fork();
-	io_sources.Register(this);
+	iosource_mgr->Register(this);
 	Log(LogInfo, fmt("communication started, parent pid is %d, child pid is %d", getpid(), child_pid));
 	initialized = 1;
@ -1275,7 +1275,7 @@ bool RemoteSerializer::Listen(const IPAddr& ip, uint16 port, bool expect_ssl,
 		return false;
 	listening = true;
-	closed = false;
+	SetClosed(false);
 	return true;
 	}
@ -1344,7 +1344,7 @@ bool RemoteSerializer::StopListening()
 		return false;
 	listening = false;
-	closed = ! IsActive();
+	SetClosed(! IsActive());
 	return true;
 	}
@ -1382,7 +1382,7 @@ double RemoteSerializer::NextTimestamp(double* local_network_time)
 	if ( received_logs > 0 )
 		{
 		// If we processed logs last time, assume there's more.
-		idle = false;
+		SetIdle(false);
 		received_logs = 0;
 		return timer_mgr->Time();
 		}
@ -1397,7 +1397,7 @@ double RemoteSerializer::NextTimestamp(double* local_network_time)
 		pt = timer_mgr->Time();
 	if ( packets.length() )
-		idle = false;
+		SetIdle(false);
 	if ( et >= 0 && (et < pt || pt < 0) )
 		return et;
@ -1452,7 +1452,7 @@ void RemoteSerializer::Process()
 		// FIXME: The following chunk of code is copied from
 		// net_packet_dispatch().  We should change that function
 		// to accept an IOSource instead of the PktSrc.
-		network_time = p->time;
+		net_update_time(p->time);
 		SegmentProfiler(segment_logger, "expiring-timers");
 		TimerMgr* tmgr = sessions->LookupTimerMgr(GetCurrentTag());
@ -1476,7 +1476,7 @@ void RemoteSerializer::Process()
 		}
 	if ( packets.length() )
-		idle = false;
+		SetIdle(false);
 	}
 void RemoteSerializer::Finish()
@ -1508,7 +1508,7 @@ bool RemoteSerializer::Poll(bool may_block)
 		}
 	io->Flush();
-	idle = false;
+	SetIdle(false);
 	switch ( msgstate ) {
 	case TYPE:
@ -1690,7 +1690,7 @@ bool RemoteSerializer::DoMessage()
 	case MSG_TERMINATE:
 		assert(terminating);
-		io_sources.Terminate();
+		iosource_mgr->Terminate();
 		return true;
 	case MSG_REMOTE_PRINT:
@ -1878,7 +1878,7 @@ void RemoteSerializer::RemovePeer(Peer* peer)
 	delete peer->cache_out;
 	delete peer;
-	closed = ! IsActive();
+	SetClosed(! IsActive());
 	if ( in_sync == peer )
 		in_sync = 0;
@ -2723,8 +2723,8 @@ bool RemoteSerializer::ProcessLogCreateWriter()
 	fmt.EndRead();
-	id_val = new EnumVal(id, BifType::Enum::Log::ID);
+	id_val = new EnumVal(id, internal_type("Log::ID")->AsEnumType());
-	writer_val = new EnumVal(writer, BifType::Enum::Log::Writer);
+	writer_val = new EnumVal(writer, internal_type("Log::Writer")->AsEnumType());
 	if ( ! log_mgr->CreateWriter(id_val, writer_val, info, num_fields, fields,
 	                             true, false, true) )
@ -2796,8 +2796,8 @@ bool RemoteSerializer::ProcessLogWrite()
 				}
 			}
-		id_val = new EnumVal(id, BifType::Enum::Log::ID);
+		id_val = new EnumVal(id, internal_type("Log::ID")->AsEnumType());
-		writer_val = new EnumVal(writer, BifType::Enum::Log::Writer);
+		writer_val = new EnumVal(writer, internal_type("Log::Writer")->AsEnumType());
 		success = log_mgr->Write(id_val, writer_val, path, num_fields, vals);
@ -2833,13 +2833,14 @@ void RemoteSerializer::GotEvent(const char* name, double time,
 	if ( ! current_peer )
 		{
 		Error("unserialized event from unknown peer");
 		delete_vals(args);
 		return;
 		}
 	BufferedEvent* e = new BufferedEvent;
 	// Our time, not the time when the event was generated.
-	e->time = pkt_srcs.length() ?
+	e->time = iosource_mgr->GetPktSrcs().size() ?
 			time_t(network_time) : time_t(timer_mgr->Time());
 	e->src = current_peer->id;
@ -2882,6 +2883,7 @@ void RemoteSerializer::GotFunctionCall(const char* name, double time,
 	if ( ! current_peer )
 		{
 		Error("unserialized function from unknown peer");
 		delete_vals(args);
 		return;
 		}
@ -3083,7 +3085,7 @@ RecordVal* RemoteSerializer::GetPeerVal(PeerID id)
 void RemoteSerializer::ChildDied()
 	{
 	Log(LogError, "child died");
-	closed = true;
+	SetClosed(true);
 	child_pid = 0;
 	// Shut down the main process as well.
@ -3182,7 +3184,7 @@ void RemoteSerializer::FatalError(const char* msg)
 	Log(LogError, msg);
 	reporter->Error("%s", msg);
-	closed = true;
+	SetClosed(true);
 	if ( kill(child_pid, SIGQUIT) < 0 )
 		reporter->Warning("warning: cannot kill child pid %d, %s", child_pid, strerror(errno));
@ -4209,6 +4211,7 @@ bool SocketComm::Listen()
 				safe_close(fd);
 				CloseListenFDs();
 				listen_next_try = time(0) + bind_retry_interval;
 				freeaddrinfo(res0);
 				return false;
 				}
--- a/src/RemoteSerializer.h
+++ b/src/RemoteSerializer.h
@ -6,7 +6,7 @@
 #include "Dict.h"
 #include "List.h"
 #include "Serializer.h"
-#include "IOSource.h"
+#include "iosource/IOSource.h"
 #include "Stats.h"
 #include "File.h"
 #include "logging/WriterBackend.h"
@ -22,13 +22,13 @@ namespace threading {
 }
 // This class handles the communication done in Bro's main loop.
-class RemoteSerializer : public Serializer, public IOSource {
+class RemoteSerializer : public Serializer, public iosource::IOSource {
 public:
 	RemoteSerializer();
 	virtual ~RemoteSerializer();
 	// Initialize the remote serializer (calling this will fork).
-	void Init();
+	void Enable();
 	// FIXME: Use SourceID directly (or rename everything to Peer*).
 	typedef SourceID PeerID;
--- a/src/RuleAction.cc
+++ b/src/RuleAction.cc
@ -65,11 +65,11 @@ RuleActionAnalyzer::RuleActionAnalyzer(const char* arg_analyzer)
 void RuleActionAnalyzer::PrintDebug()
 	{
 	if ( ! child_analyzer )
-		fprintf(stderr, "|%s|\n", analyzer_mgr->GetComponentName(analyzer));
+		fprintf(stderr, "|%s|\n", analyzer_mgr->GetComponentName(analyzer).c_str());
 	else
 		fprintf(stderr, "|%s:%s|\n",
-			analyzer_mgr->GetComponentName(analyzer),
+			analyzer_mgr->GetComponentName(analyzer).c_str(),
-			analyzer_mgr->GetComponentName(child_analyzer));
+			analyzer_mgr->GetComponentName(child_analyzer).c_str());
 	}
--- a/src/Serializer.cc
+++ b/src/Serializer.cc
@ -19,6 +19,7 @@
 #include "Conn.h"
 #include "Timer.h"
 #include "RemoteSerializer.h"
 #include "iosource/Manager.h"
 Serializer::Serializer(SerializationFormat* arg_format)
 	{
@ -1045,7 +1046,7 @@ EventPlayer::EventPlayer(const char* file)
 		Error(fmt("event replayer: cannot open %s", file));
 	if ( ReadHeader() )
-		io_sources.Register(this);
+		iosource_mgr->Register(this);
 	}
 EventPlayer::~EventPlayer()
@ -1085,7 +1086,7 @@ double EventPlayer::NextTimestamp(double* local_network_time)
 		{
 		UnserialInfo info(this);
 		Unserialize(&info);
-		closed = io->Eof();
+		SetClosed(io->Eof());
 		}
 	if ( ! ne_time )
@ -1142,7 +1143,7 @@ bool Packet::Serialize(SerialInfo* info) const
 static BroFile* profiling_output = 0;
 #ifdef DEBUG
-static PktDumper* dump = 0;
+static iosource::PktDumper* dump = 0;
 #endif
 Packet* Packet::Unserialize(UnserialInfo* info)
@ -1188,7 +1189,7 @@ Packet* Packet::Unserialize(UnserialInfo* info)
 	p->hdr = hdr;
 	p->pkt = (u_char*) pkt;
 	p->tag = tag;
-	p->hdr_size = get_link_header_size(p->link_type);
+	p->hdr_size = iosource::PktSrc::GetLinkHeaderSize(p->link_type);
 	delete [] tag;
@ -1213,9 +1214,15 @@ Packet* Packet::Unserialize(UnserialInfo* info)
 	if ( debug_logger.IsEnabled(DBG_TM) )
 		{
 		if ( ! dump )
-			dump = new PktDumper("tm.pcap");
+			dump = iosource_mgr->OpenPktDumper("tm.pcap", true);
-		dump->Dump(p->hdr, p->pkt);
+		if ( dump )
 			{
 			iosource::PktDumper::Packet dp;
 			dp.hdr = p->hdr;
 			dp.data = p->pkt;
 			dump->Dump(&dp);
 			}
 		}
 #endif
--- a/src/Serializer.h
+++ b/src/Serializer.h
@ -15,7 +15,7 @@
 #include "SerialInfo.h"
 #include "IP.h"
 #include "Timer.h"
-#include "IOSource.h"
+#include "iosource/IOSource.h"
 #include "Reporter.h"
 class SerializationCache;
@ -350,7 +350,7 @@ public:
 };
 // Plays a file of events back.
-class EventPlayer : public FileSerializer, public IOSource {
+class EventPlayer : public FileSerializer, public iosource::IOSource {
 public:
 	EventPlayer(const char* file);
 	virtual ~EventPlayer();
--- a/src/Sessions.cc
+++ b/src/Sessions.cc
@ -167,7 +167,7 @@ void NetSessions::Done()
 void NetSessions::DispatchPacket(double t, const struct pcap_pkthdr* hdr,
 			const u_char* pkt, int hdr_size,
-			PktSrc* src_ps)
+			iosource::PktSrc* src_ps)
 	{
 	const struct ip* ip_hdr = 0;
 	const u_char* ip_data = 0;
@ -184,10 +184,7 @@ void NetSessions::DispatchPacket(double t, const struct pcap_pkthdr* hdr,
 		// Blanket encapsulation
 		hdr_size += encap_hdr_size;
-	if ( src_ps->FilterType() == TYPE_FILTER_NORMAL )
+	NextPacket(t, hdr, pkt, hdr_size);
 		NextPacket(t, hdr, pkt, hdr_size);
 	else
 		NextPacketSecondary(t, hdr, pkt, hdr_size, src_ps);
 	}
 void NetSessions::NextPacket(double t, const struct pcap_pkthdr* hdr,
@ -262,53 +259,6 @@ void NetSessions::NextPacket(double t, const struct pcap_pkthdr* hdr,
 		DumpPacket(hdr, pkt);
 	}
 void NetSessions::NextPacketSecondary(double /* t */, const struct pcap_pkthdr* hdr,
 				const u_char* const pkt, int hdr_size,
 				const PktSrc* src_ps)
 	{
 	SegmentProfiler(segment_logger, "processing-secondary-packet");
 	++num_packets_processed;
 	uint32 caplen = hdr->caplen - hdr_size;
 	if ( caplen < sizeof(struct ip) )
 		{
 		Weird("truncated_IP", hdr, pkt);
 		return;
 		}
 	const struct ip* ip = (const struct ip*) (pkt + hdr_size);
 	if ( ip->ip_v == 4 )
 		{
 		const secondary_program_list& spt = src_ps->ProgramTable();
 		loop_over_list(spt, i)
 			{
 			SecondaryProgram* sp = spt[i];
 			if ( ! net_packet_match(sp->Program(), pkt,
 						hdr->len, hdr->caplen) )
 				continue;
 			val_list* args = new val_list;
 			StringVal* cmd_val =
 				new StringVal(sp->Event()->Filter());
 			args->append(cmd_val);
 			IP_Hdr ip_hdr(ip, false);
 			args->append(ip_hdr.BuildPktHdrVal());
 			// ### Need to queue event here.
 			try
 				{
 				sp->Event()->Event()->Call(args);
 				}
 			catch ( InterpreterException& e )
 				{ /* Already reported. */ }
 			delete args;
 			}
 		}
 	}
 int NetSessions::CheckConnectionTag(Connection* conn)
 	{
 	if ( current_iosrc->GetCurrentTag() )
@ -1440,14 +1390,24 @@ void NetSessions::DumpPacket(const struct pcap_pkthdr* hdr,
 		return;
 	if ( len == 0 )
-		pkt_dumper->Dump(hdr, pkt);
+		{
 		iosource::PktDumper::Packet p;
 		p.hdr = hdr;
 		p.data = pkt;
 		pkt_dumper->Dump(&p);
 		}
 	else
 		{
 		struct pcap_pkthdr h = *hdr;
 		h.caplen = len;
 		if ( h.caplen > hdr->caplen )
 			reporter->InternalError("bad modified caplen");
-		pkt_dumper->Dump(&h, pkt);
+
 		iosource::PktDumper::Packet p;
 		p.hdr = &h;
 		p.data = pkt;
 		pkt_dumper->Dump(&p);
 		}
 	}
--- a/src/Sessions.h
+++ b/src/Sessions.h
@ -69,11 +69,11 @@ public:
 	~NetSessions();
 	// Main entry point for packet processing. Dispatches the packet
-	// either through NextPacket() or NextPacketSecondary(), optionally
+	// either through NextPacket(), optionally employing the packet
-	// employing the packet sorter first.
+	// sorter first.
 	void DispatchPacket(double t, const struct pcap_pkthdr* hdr,
 			const u_char* const pkt, int hdr_size,
-			PktSrc* src_ps);
+			iosource::PktSrc* src_ps);
 	void Done();	// call to drain events before destructing
@ -221,10 +221,6 @@ protected:
 	void NextPacket(double t, const struct pcap_pkthdr* hdr,
 			const u_char* const pkt, int hdr_size);
 	void NextPacketSecondary(double t, const struct pcap_pkthdr* hdr,
 			const u_char* const pkt, int hdr_size,
 			const PktSrc* src_ps);
 	// Record the given packet (if a dumper is active).  If len=0
 	// then the whole packet is recorded, otherwise just the first
 	// len bytes.
--- a/src/Stmt.cc
+++ b/src/Stmt.cc
@ -660,8 +660,13 @@ void Case::Describe(ODesc* d) const
 TraversalCode Case::Traverse(TraversalCallback* cb) const
 	{
-	TraversalCode tc = cases->Traverse(cb);
+	TraversalCode tc;
-	HANDLE_TC_STMT_PRE(tc);
+
 	if ( cases )
 		{
 		tc = cases->Traverse(cb);
 		HANDLE_TC_STMT_PRE(tc);
 		}
 	tc = s->Traverse(cb);
 	HANDLE_TC_STMT_PRE(tc);
--- a/src/Tag.cc
+++ b/src/Tag.cc
@ -55,6 +55,7 @@ Tag& Tag::operator=(const Tag& other)
 		{
 		type = other.type;
 		subtype = other.subtype;
 		Unref(val);
 		val = other.val;
 		if ( val )
--- a/src/Type.cc
+++ b/src/Type.cc
@ -449,6 +449,11 @@ BroType* IndexType::YieldType()
 	return yield_type;
 	}
 const BroType* IndexType::YieldType() const
 	{
 	return yield_type;
 	}
 void IndexType::Describe(ODesc* d) const
 	{
 	BroType::Describe(d);
@ -742,6 +747,11 @@ BroType* FuncType::YieldType()
 	return yield;
 	}
 const BroType* FuncType::YieldType() const
 	{
 	return yield;
 	}
 int FuncType::MatchesIndex(ListExpr*& index) const
 	{
 	return check_and_promote_args(index, args) ?
@ -1371,6 +1381,11 @@ void OpaqueType::Describe(ODesc* d) const
 	d->Add(name.c_str());
 	}
 void OpaqueType::DescribeReST(ODesc* d, bool roles_only) const
 	{
 	d->Add(fmt(":bro:type:`%s` of %s", type_name(Tag()), name.c_str()));
 	}
 IMPLEMENT_SERIAL(OpaqueType, SER_OPAQUE_TYPE);
 bool OpaqueType::DoSerialize(SerialInfo* info) const
@ -1393,6 +1408,23 @@ bool OpaqueType::DoUnserialize(UnserialInfo* info)
 	return true;
 	}
 EnumType::EnumType(const string& name)
 	: BroType(TYPE_ENUM)
 	{
 	counter = 0;
 	SetName(name);
 	}
 EnumType::EnumType(EnumType* e)
 	: BroType(TYPE_ENUM)
 	{
 	counter = e->counter;
 	SetName(e->GetName());
 	for ( NameMap::iterator it = e->names.begin(); it != e->names.end(); ++it )
 		names[copy_string(it->first)] = it->second;
 	}
 EnumType::~EnumType()
 	{
 	for ( NameMap::iterator iter = names.begin(); iter != names.end(); ++iter )
@ -1449,10 +1481,19 @@ void EnumType::CheckAndAddName(const string& module_name, const char* name,
 		}
 	else
 		{
 		// We allow double-definitions if matching exactly. This is so that
 		// we can define an enum both in a *.bif and *.bro for avoiding
 		// cyclic dependencies.
 		if ( id->Name() != make_full_var_name(module_name.c_str(), name)
 		     || (id->HasVal() && val != id->ID_Val()->AsEnum()) )
 			{
 			Unref(id);
 			reporter->Error("identifier or enumerator value in enumerated type definition already exists");
 			SetError();
 			return;
 			}
 		Unref(id);
 		reporter->Error("identifier or enumerator value in enumerated type definition already exists");
 		SetError();
 		return;
 		}
 	AddNameInternal(module_name, name, val, is_export);
@ -1473,9 +1514,9 @@ void EnumType::AddNameInternal(const string& module_name, const char* name,
 	names[copy_string(fullname.c_str())] = val;
 	}
-bro_int_t EnumType::Lookup(const string& module_name, const char* name)
+bro_int_t EnumType::Lookup(const string& module_name, const char* name) const
 	{
-	NameMap::iterator pos =
+	NameMap::const_iterator pos =
 		names.find(make_full_var_name(module_name.c_str(), name).c_str());
 	if ( pos == names.end() )
@ -1484,9 +1525,9 @@ bro_int_t EnumType::Lookup(const string& module_name, const char* name)
 		return pos->second;
 	}
-const char* EnumType::Lookup(bro_int_t value)
+const char* EnumType::Lookup(bro_int_t value) const
 	{
-	for ( NameMap::iterator iter = names.begin();
+	for ( NameMap::const_iterator iter = names.begin();
 	      iter != names.end(); ++iter )
 		if ( iter->second == value )
 			return iter->first;
@ -1494,6 +1535,16 @@ const char* EnumType::Lookup(bro_int_t value)
 	return 0;
 	}
 EnumType::enum_name_list EnumType::Names() const
 	{
 	enum_name_list n;
 	for ( NameMap::const_iterator iter = names.begin();
 	      iter != names.end(); ++iter )
 		n.push_back(std::make_pair(iter->first, iter->second));
 	return n;
 	}
 void EnumType::DescribeReST(ODesc* d, bool roles_only) const
 	{
 	d->Add(":bro:type:`enum`");
@ -1644,6 +1695,23 @@ BroType* VectorType::YieldType()
 	return yield_type;
 	}
 const BroType* VectorType::YieldType() const
 	{
 	// Work around the fact that we use void internally to mark a vector
 	// as being unspecified. When looking at its yield type, we need to
 	// return any as that's what other code historically expects for type
 	// comparisions.
 	if ( IsUnspecifiedVector() )
 		{
 		BroType* ret = ::base_type(TYPE_ANY);
 		Unref(ret); // unref, because this won't be held by anyone.
 		assert(ret);
 		return ret;
 		}
 	return yield_type;
 	}
 int VectorType::MatchesIndex(ListExpr*& index) const
 	{
 	expr_list& el = index->Exprs();
@ -1691,7 +1759,17 @@ void VectorType::Describe(ODesc* d) const
 	yield_type->Describe(d);
 	}
-BroType* base_type(TypeTag tag)
+void VectorType::DescribeReST(ODesc* d, bool roles_only) const
 	{
 	d->Add(fmt(":bro:type:`%s` of ", type_name(Tag())));
 	if ( yield_type->GetName().empty() )
 		yield_type->DescribeReST(d, roles_only);
 	else
 		d->Add(fmt(":bro:type:`%s`", yield_type->GetName().c_str()));
 	}
 BroType* base_type_no_ref(TypeTag tag)
 	{
 	static BroType* base_types[NUM_TYPES];
@ -1707,7 +1785,7 @@ BroType* base_type(TypeTag tag)
 		base_types[t]->SetLocationInfo(&l);
 		}
-	return base_types[t]->Ref();
+	return base_types[t];
 	}
@ -1732,7 +1810,7 @@ static int is_init_compat(const BroType* t1, const BroType* t2)
 	return 0;
 	}
-int same_type(const BroType* t1, const BroType* t2, int is_init)
+int same_type(const BroType* t1, const BroType* t2, int is_init, bool match_record_field_names)
 	{
 	if ( t1 == t2 ||
 	     t1->Tag() == TYPE_ANY ||
@ -1788,7 +1866,7 @@ int same_type(const BroType* t1, const BroType* t2, int is_init)
 		if ( tl1 || tl2 )
 			{
-			if ( ! tl1 || ! tl2 || ! same_type(tl1, tl2, is_init) )
+			if ( ! tl1 || ! tl2 || ! same_type(tl1, tl2, is_init, match_record_field_names) )
 				return 0;
 			}
@ -1797,7 +1875,7 @@ int same_type(const BroType* t1, const BroType* t2, int is_init)
 		if ( y1 || y2 )
 			{
-			if ( ! y1 || ! y2 || ! same_type(y1, y2, is_init) )
+			if ( ! y1 || ! y2 || ! same_type(y1, y2, is_init, match_record_field_names) )
 				return 0;
 			}
@ -1815,7 +1893,7 @@ int same_type(const BroType* t1, const BroType* t2, int is_init)
 		if ( t1->YieldType() || t2->YieldType() )
 			{
 			if ( ! t1->YieldType() || ! t2->YieldType() ||
-			     ! same_type(t1->YieldType(), t2->YieldType(), is_init) )
+			     ! same_type(t1->YieldType(), t2->YieldType(), is_init, match_record_field_names) )
 				return 0;
 			}
@ -1835,8 +1913,8 @@ int same_type(const BroType* t1, const BroType* t2, int is_init)
 			const TypeDecl* td1 = rt1->FieldDecl(i);
 			const TypeDecl* td2 = rt2->FieldDecl(i);
-			if ( ! streq(td1->id, td2->id) ||
+			if ( (match_record_field_names && ! streq(td1->id, td2->id)) ||
-			     ! same_type(td1->type, td2->type, is_init) )
+			     ! same_type(td1->type, td2->type, is_init, match_record_field_names) )
 				return 0;
 			}
@ -1852,7 +1930,7 @@ int same_type(const BroType* t1, const BroType* t2, int is_init)
 			return 0;
 		loop_over_list(*tl1, i)
-			if ( ! same_type((*tl1)[i], (*tl2)[i], is_init) )
+			if ( ! same_type((*tl1)[i], (*tl2)[i], is_init, match_record_field_names) )
 				return 0;
 		return 1;
@ -1860,7 +1938,7 @@ int same_type(const BroType* t1, const BroType* t2, int is_init)
 	case TYPE_VECTOR:
 	case TYPE_FILE:
-		return same_type(t1->YieldType(), t2->YieldType(), is_init);
+		return same_type(t1->YieldType(), t2->YieldType(), is_init, match_record_field_names);
 	case TYPE_OPAQUE:
 		{
@ -1870,7 +1948,7 @@ int same_type(const BroType* t1, const BroType* t2, int is_init)
 		}
 	case TYPE_TYPE:
-		return same_type(t1, t2, is_init);
+		return same_type(t1, t2, is_init, match_record_field_names);
 	case TYPE_UNION:
 		reporter->Error("union type in same_type()");
--- a/src/Type.h
+++ b/src/Type.h
@ -6,6 +6,7 @@
 #include <string>
 #include <set>
 #include <map>
 #include <list>
 #include "Obj.h"
 #include "Attr.h"
@ -334,6 +335,7 @@ public:
 	TypeList* Indices() const		{ return indices; }
 	const type_list* IndexTypes() const	{ return indices->Types(); }
 	BroType* YieldType();
 	const BroType* YieldType() const;
 	void Describe(ODesc* d) const;
 	void DescribeReST(ODesc* d, bool roles_only = false) const;
@ -396,6 +398,7 @@ public:
 	RecordType* Args() const	{ return args; }
 	BroType* YieldType();
 	const BroType* YieldType() const;
 	void SetYieldType(BroType* arg_yield)	{ yield = arg_yield; }
 	function_flavor Flavor() const { return flavor; }
 	string FlavorString() const;
@ -531,6 +534,7 @@ public:
 	const string& Name() const { return name; }
 	void Describe(ODesc* d) const;
 	void DescribeReST(ODesc* d, bool roles_only = false) const;
 protected:
 	OpaqueType() { }
@ -542,7 +546,10 @@ protected:
 class EnumType : public BroType {
 public:
-	EnumType() : BroType(TYPE_ENUM) { counter = 0; }
+	typedef std::list<std::pair<string, bro_int_t> > enum_name_list;
 	EnumType(EnumType* e);
 	EnumType(const string& arg_name);
 	~EnumType();
 	// The value of this name is next internal counter value, starting
@ -555,12 +562,18 @@ public:
 	void AddName(const string& module_name, const char* name, bro_int_t val, bool is_export);
 	// -1 indicates not found.
-	bro_int_t Lookup(const string& module_name, const char* name);
+	bro_int_t Lookup(const string& module_name, const char* name) const;
-	const char* Lookup(bro_int_t value); // Returns 0 if not found
+	const char* Lookup(bro_int_t value) const; // Returns 0 if not found
 	// Returns the list of defined names with their values. The names
 	// will be fully qualified with their module name.
 	enum_name_list Names() const;
 	void DescribeReST(ODesc* d, bool roles_only = false) const;
 protected:
 	EnumType() { counter = 0; }
 	DECLARE_SERIAL(EnumType)
 	void AddNameInternal(const string& module_name,
@ -586,6 +599,7 @@ public:
 	VectorType(BroType* t);
 	virtual ~VectorType();
 	BroType* YieldType();
 	const BroType* YieldType() const;
 	int MatchesIndex(ListExpr*& index) const;
@ -594,6 +608,7 @@ public:
 	bool IsUnspecifiedVector() const;
 	void Describe(ODesc* d) const;
 	void DescribeReST(ODesc* d, bool roles_only = false) const;
 protected:
 	VectorType()	{ yield_type = 0; }
@ -612,15 +627,22 @@ extern OpaqueType* topk_type;
 extern OpaqueType* bloomfilter_type;
 extern OpaqueType* x509_opaque_type;
 // Returns the Bro basic (non-parameterized) type with the given type.
 // The reference count of the type is not increased.
 BroType* base_type_no_ref(TypeTag tag);
 // Returns the BRO basic (non-parameterized) type with the given type.
-extern BroType* base_type(TypeTag tag);
+// The caller assumes responsibility for a reference to the type.
 inline BroType* base_type(TypeTag tag)
 	{ return base_type_no_ref(tag)->Ref(); }
 // Returns the BRO basic error type.
 inline BroType* error_type()	{ return base_type(TYPE_ERROR); }
-// True if the two types are equivalent.  If is_init is true then the
+// True if the two types are equivalent.  If is_init is true then the test is
-// test is done in the context of an initialization.
+// done in the context of an initialization. If match_record_field_names is
-extern int same_type(const BroType* t1, const BroType* t2, int is_init=0);
+// true then for record types the field names have to match, too.
 extern int same_type(const BroType* t1, const BroType* t2, int is_init=0, bool match_record_field_names=true);
 // True if the two attribute lists are equivalent.
 extern int same_attrs(const Attributes* a1, const Attributes* a2);
--- a/src/Val.cc
+++ b/src/Val.cc
@ -465,10 +465,7 @@ void Val::Describe(ODesc* d) const
 		d->SP();
 		}
-	if ( d->IsReadable() )
+	ValDescribe(d);
 		ValDescribe(d);
 	else
 		Val::ValDescribe(d);
 	}
 void Val::DescribeReST(ODesc* d) const
@ -1152,7 +1149,7 @@ bool PatternVal::DoUnserialize(UnserialInfo* info)
 	}
 ListVal::ListVal(TypeTag t)
-: Val(new TypeList(t == TYPE_ANY ? 0 : base_type(t)))
+: Val(new TypeList(t == TYPE_ANY ? 0 : base_type_no_ref(t)))
 	{
 	tag = t;
 	}
@ -1471,13 +1468,20 @@ int TableVal::Assign(Val* index, HashKey* k, Val* new_val, Opcode op)
 		}
 	TableEntryVal* new_entry_val = new TableEntryVal(new_val);
 	HashKey k_copy(k->Key(), k->Size(), k->Hash());
 	TableEntryVal* old_entry_val = AsNonConstTable()->Insert(k, new_entry_val);
 	// If the dictionary index already existed, the insert may free up the
 	// memory allocated to the key bytes, so have to assume k is invalid
 	// from here on out.
 	delete k;
 	k = 0;
 	if ( subnets )
 		{
 		if ( ! index )
 			{
-			Val* v = RecoverIndex(k);
+			Val* v = RecoverIndex(&k_copy);
 			subnets->Insert(v, new_entry_val);
 			Unref(v);
 			}
@ -1489,7 +1493,7 @@ int TableVal::Assign(Val* index, HashKey* k, Val* new_val, Opcode op)
 		{
 		Val* rec_index = 0;
 		if ( ! index )
-			index = rec_index = RecoverIndex(k);
+			index = rec_index = RecoverIndex(&k_copy);
 		if ( new_val )
 			{
@ -1547,7 +1551,6 @@ int TableVal::Assign(Val* index, HashKey* k, Val* new_val, Opcode op)
 	if ( old_entry_val && attrs && attrs->FindAttr(ATTR_EXPIRE_CREATE) )
 		new_entry_val->SetExpireAccess(old_entry_val->ExpireAccessTime());
 	delete k;
 	if ( old_entry_val )
 		{
 		old_entry_val->Unref();
--- a/src/Var.cc
+++ b/src/Var.cc
@ -9,6 +9,7 @@
 #include "Serializer.h"
 #include "RemoteSerializer.h"
 #include "EventRegistry.h"
 #include "Traverse.h"
 static Val* init_val(Expr* init, const BroType* t, Val* aggr)
 	{
@ -392,6 +393,34 @@ void begin_func(ID* id, const char* module_name, function_flavor flavor,
 		}
 	}
 class OuterIDBindingFinder : public TraversalCallback {
 public:
 	OuterIDBindingFinder(Scope* s)
 		: scope(s) { }
 	virtual TraversalCode PreExpr(const Expr*);
 	Scope* scope;
 	vector<const NameExpr*> outer_id_references;
 };
 TraversalCode OuterIDBindingFinder::PreExpr(const Expr* expr)
 	{
 	if ( expr->Tag() != EXPR_NAME )
 		return TC_CONTINUE;
 	const NameExpr* e = static_cast<const NameExpr*>(expr);
 	if ( e->Id()->IsGlobal() )
 		return TC_CONTINUE;
 	if ( scope->GetIDs()->Lookup(e->Id()->Name()) )
 		return TC_CONTINUE;
 	outer_id_references.push_back(e);
 	return TC_CONTINUE;
 	}
 void end_func(Stmt* body, attr_list* attrs)
 	{
 	int frame_size = current_scope()->Length();
@ -429,6 +458,16 @@ void end_func(Stmt* body, attr_list* attrs)
 			}
 		}
 	if ( streq(id->Name(), "anonymous-function") )
 		{
 		OuterIDBindingFinder cb(scope);
 		body->Traverse(&cb);
 		for ( size_t i = 0; i < cb.outer_id_references.size(); ++i )
 			cb.outer_id_references[i]->Error(
 						"referencing outer function IDs not supported");
 		}
 	if ( id->HasVal() )
 		id->ID_Val()->AsFunc()->AddBody(body, inits, frame_size, priority);
 	else
--- a/src/analyzer/Analyzer.cc
+++ b/src/analyzer/Analyzer.cc
@ -4,6 +4,7 @@
 #include "Analyzer.h"
 #include "Manager.h"
 #include "binpac.h"
 #include "analyzer/protocol/pia/PIA.h"
 #include "../Event.h"
@ -75,7 +76,7 @@ analyzer::ID Analyzer::id_counter = 0;
 const char* Analyzer::GetAnalyzerName() const
 	{
 	assert(tag);
-	return analyzer_mgr->GetComponentName(tag);
+	return analyzer_mgr->GetComponentName(tag).c_str();
 	}
 void Analyzer::SetAnalyzerTag(const Tag& arg_tag)
@ -87,7 +88,7 @@ void Analyzer::SetAnalyzerTag(const Tag& arg_tag)
 bool Analyzer::IsAnalyzer(const char* name)
 	{
 	assert(tag);
-	return strcmp(analyzer_mgr->GetComponentName(tag), name) == 0;
+	return strcmp(analyzer_mgr->GetComponentName(tag).c_str(), name) == 0;
 	}
 // Used in debugging output.
@ -642,12 +643,12 @@ void Analyzer::FlipRoles()
 	resp_supporters = tmp;
 	}
-void Analyzer::ProtocolConfirmation()
+void Analyzer::ProtocolConfirmation(Tag arg_tag)
 	{
 	if ( protocol_confirmed )
 		return;
-	EnumVal* tval = tag.AsEnumVal();
+	EnumVal* tval = arg_tag ? arg_tag.AsEnumVal() : tag.AsEnumVal();
 	Ref(tval);
 	val_list* vl = new val_list;
--- a/src/analyzer/Analyzer.h
+++ b/src/analyzer/Analyzer.h
@ -97,8 +97,8 @@ public:
 	/**
 	 * Constructor. As this version of the constructor does not receive a
-	 * name or tag, setTag() must be called before the instance can be
+	 * name or tag, SetAnalyzerTag() must be called before the instance
-	 * used.
+	 * can be used.
 	 *
 	 * @param conn The connection the analyzer is associated with.
 	 */
@ -471,8 +471,11 @@ public:
 	 * may turn into \c protocol_confirmed event at the script-layer (but
 	 * only once per analyzer for each connection, even if the method is
 	 * called multiple times).
 	 *
 	 * If tag is given, it overrides the analyzer tag passed to the
 	 * scripting layer; the default is the one of the analyzer itself.
 	 */
-	 virtual void ProtocolConfirmation();
+	virtual void ProtocolConfirmation(Tag tag = Tag());
 	/**
 	 * Signals Bro's protocol detection that the analyzer has found a
--- a/src/analyzer/Component.cc
+++ b/src/analyzer/Component.cc
@ -8,62 +8,29 @@
 using namespace analyzer;
-Component::Component(const char* arg_name, factory_callback arg_factory, Tag::subtype_t arg_subtype, bool arg_enabled, bool arg_partial)
+Component::Component(const std::string& name, factory_callback arg_factory, Tag::subtype_t arg_subtype, bool arg_enabled, bool arg_partial)
-	: plugin::Component(plugin::component::ANALYZER),
+	: plugin::Component(plugin::component::ANALYZER, name),
 	  plugin::TaggedComponent<analyzer::Tag>(arg_subtype)
 	{
 	name = copy_string(arg_name);
 	canon_name = canonify_name(arg_name);
 	factory = arg_factory;
 	enabled = arg_enabled;
 	partial = arg_partial;
 	}
-Component::Component(const Component& other)
+	analyzer_mgr->RegisterComponent(this, "ANALYZER_");
 	: plugin::Component(Type()),
 	  plugin::TaggedComponent<analyzer::Tag>(other)
 	{
 	name = copy_string(other.name);
 	canon_name = copy_string(other.canon_name);
 	factory = other.factory;
 	enabled = other.enabled;
 	partial = other.partial;
 	}
 Component::~Component()
 	{
 	delete [] name;
 	delete [] canon_name;
 	}
-void Component::Describe(ODesc* d) const
+void Component::DoDescribe(ODesc* d) const
 	{
 	plugin::Component::Describe(d);
 	d->Add(name);
 	d->Add(" (");
 	if ( factory )
 		{
 		d->Add("ANALYZER_");
-		d->Add(canon_name);
+		d->Add(CanonicalName());
 		d->Add(", ");
 		}
 	d->Add(enabled ? "enabled" : "disabled");
 	d->Add(")");
 	}
 Component& Component::operator=(const Component& other)
 	{
 	plugin::TaggedComponent<analyzer::Tag>::operator=(other);
 	if ( &other != this )
 		{
 		name = copy_string(other.name);
 		factory = other.factory;
 		enabled = other.enabled;
 		partial = other.partial;
 		}
 	return *this;
 	}
--- a/src/analyzer/Component.h
+++ b/src/analyzer/Component.h
@ -1,7 +1,7 @@
 // See the file "COPYING" in the main distribution directory for copyright.
-#ifndef ANALYZER_PLUGIN_COMPONENT_H
+#ifndef ANALYZER_COMPONENT_H
-#define ANALYZER_PLUGIN_COMPONENT_H
+#define ANALYZER_COMPONENT_H
 #include "Tag.h"
 #include "plugin/Component.h"
@ -56,34 +56,13 @@ public:
 	 * connections has generally not seen much testing yet as virtually
 	 * no existing analyzer supports it.
 	 */
-	Component(const char* name, factory_callback factory, Tag::subtype_t subtype = 0, bool enabled = true, bool partial = false);
+	Component(const std::string& name, factory_callback factory, Tag::subtype_t subtype = 0, bool enabled = true, bool partial = false);
 	/**
 	 * Copy constructor.
 	 */
 	Component(const Component& other);
 	/**
 	 * Destructor.
 	 */
 	~Component();
 	/**
 	 * Returns the name of the analyzer. This name is unique across all
 	 * analyzers and used to identify it. The returned name is derived
 	 * from what's passed to the constructor but upper-cased and
 	 * canonified to allow being part of a script-level ID.
 	 */
 	virtual const char* Name() const	{ return name; }
 	/**
 	 * Returns a canonocalized version of the analyzer's name.  The
 	 * returned name is derived from what's passed to the constructor but
 	 * upper-cased and transformed to allow being part of a script-level
 	 * ID.
 	 */
 	const char* CanonicalName() const	{ return canon_name; }
 	/**
 	 * Returns the analyzer's factory function.
 	 */
@ -110,17 +89,13 @@ public:
 	 */
 	void SetEnabled(bool arg_enabled)	{ enabled = arg_enabled; }
 protected:
 	/**
-	 * Generates a human-readable description of the component's main
+	  * Overriden from plugin::Component.
-	 * parameters. This goes into the output of \c "bro -NN".
+	  */
-	 */
+	virtual void DoDescribe(ODesc* d) const;
 	virtual void Describe(ODesc* d) const;
 	Component& operator=(const Component& other);
 private:
 	const char* name;	// The analyzer's name.
 	const char* canon_name;	// The analyzer's canonical name.
 	factory_callback factory;	// The analyzer's factory callback.
 	bool partial;	// True if the analyzer supports partial connections.
 	bool enabled;	// True if the analyzer is enabled.
--- a/src/analyzer/Manager.cc
+++ b/src/analyzer/Manager.cc
@ -60,7 +60,7 @@ bool Manager::ConnIndex::operator<(const ConnIndex& other) const
 	}
 Manager::Manager()
-	: plugin::ComponentManager<analyzer::Tag, analyzer::Component>("Analyzer")
+	: plugin::ComponentManager<analyzer::Tag, analyzer::Component>("Analyzer", "Tag")
 	{
 	}
@ -86,11 +86,6 @@ Manager::~Manager()
 void Manager::InitPreScript()
 	{
 	std::list<Component*> analyzers = plugin_mgr->Components<Component>();
 	for ( std::list<Component*>::const_iterator i = analyzers.begin(); i != analyzers.end(); i++ )
 		RegisterComponent(*i, "ANALYZER_");
 	// Cache these tags.
 	analyzer_backdoor = GetComponentTag("BACKDOOR");
 	analyzer_connsize = GetComponentTag("CONNSIZE");
@ -109,7 +104,8 @@ void Manager::DumpDebug()
 	DBG_LOG(DBG_ANALYZER, "Available analyzers after bro_init():");
 	list<Component*> all_analyzers = GetComponents();
 	for ( list<Component*>::const_iterator i = all_analyzers.begin(); i != all_analyzers.end(); ++i )
-		DBG_LOG(DBG_ANALYZER, "    %s (%s)", (*i)->Name(), IsEnabled((*i)->Tag()) ? "enabled" : "disabled");
+		DBG_LOG(DBG_ANALYZER, "    %s (%s)", (*i)->Name().c_str(),
 			IsEnabled((*i)->Tag()) ? "enabled" : "disabled");
 	DBG_LOG(DBG_ANALYZER, "");
 	DBG_LOG(DBG_ANALYZER, "Analyzers by port:");
@ -148,7 +144,7 @@ bool Manager::EnableAnalyzer(Tag tag)
 	if ( ! p  )
 		return false;
-	DBG_LOG(DBG_ANALYZER, "Enabling analyzer %s", p->Name());
+	DBG_LOG(DBG_ANALYZER, "Enabling analyzer %s", p->Name().c_str());
 	p->SetEnabled(true);
 	return true;
@ -161,7 +157,7 @@ bool Manager::EnableAnalyzer(EnumVal* val)
 	if ( ! p  )
 		return false;
-	DBG_LOG(DBG_ANALYZER, "Enabling analyzer %s", p->Name());
+	DBG_LOG(DBG_ANALYZER, "Enabling analyzer %s", p->Name().c_str());
 	p->SetEnabled(true);
 	return true;
@ -174,7 +170,7 @@ bool Manager::DisableAnalyzer(Tag tag)
 	if ( ! p  )
 		return false;
-	DBG_LOG(DBG_ANALYZER, "Disabling analyzer %s", p->Name());
+	DBG_LOG(DBG_ANALYZER, "Disabling analyzer %s", p->Name().c_str());
 	p->SetEnabled(false);
 	return true;
@ -187,7 +183,7 @@ bool Manager::DisableAnalyzer(EnumVal* val)
 	if ( ! p  )
 		return false;
-	DBG_LOG(DBG_ANALYZER, "Disabling analyzer %s", p->Name());
+	DBG_LOG(DBG_ANALYZER, "Disabling analyzer %s", p->Name().c_str());
 	p->SetEnabled(false);
 	return true;
@ -202,6 +198,11 @@ void Manager::DisableAllAnalyzers()
 		(*i)->SetEnabled(false);
 	}
 analyzer::Tag Manager::GetAnalyzerTag(const char* name)
 	{
 	return GetComponentTag(name);
 	}
 bool Manager::IsEnabled(Tag tag)
 	{
 	if ( ! tag )
@ -254,7 +255,7 @@ bool Manager::RegisterAnalyzerForPort(Tag tag, TransportProto proto, uint32 port
 		return false;
 #ifdef DEBUG
-	const char* name = GetComponentName(tag);
+	const char* name = GetComponentName(tag).c_str();
 	DBG_LOG(DBG_ANALYZER, "Registering analyzer %s for port %" PRIu32 "/%d", name, port, proto);
 #endif
@ -270,7 +271,7 @@ bool Manager::UnregisterAnalyzerForPort(Tag tag, TransportProto proto, uint32 po
 		return true;  // still a "successful" unregistration
 #ifdef DEBUG
-	const char* name = GetComponentName(tag);
+	const char* name = GetComponentName(tag).c_str();
 	DBG_LOG(DBG_ANALYZER, "Unregistering analyzer %s for port %" PRIu32 "/%d", name, port, proto);
 #endif
@ -293,7 +294,8 @@ Analyzer* Manager::InstantiateAnalyzer(Tag tag, Connection* conn)
 	if ( ! c->Factory() )
 		{
-		reporter->InternalWarning("analyzer %s cannot be instantiated dynamically", GetComponentName(tag));
+		reporter->InternalWarning("analyzer %s cannot be instantiated dynamically",
 					  GetComponentName(tag).c_str());
 		return 0;
 		}
@ -413,7 +415,7 @@ bool Manager::BuildInitialAnalyzerTree(Connection* conn)
 					root->AddChildAnalyzer(analyzer, false);
 					DBG_ANALYZER_ARGS(conn, "activated %s analyzer due to port %d",
-							  analyzer_mgr->GetComponentName(*j), resp_port);
+							  analyzer_mgr->GetComponentName(*j).c_str(), resp_port);
 					}
 				}
 			}
@ -532,7 +534,7 @@ void Manager::ExpireScheduledAnalyzers()
 			conns.erase(i);
 			DBG_LOG(DBG_ANALYZER, "Expiring expected analyzer %s for connection %s",
-				analyzer_mgr->GetComponentName(a->analyzer),
+				analyzer_mgr->GetComponentName(a->analyzer).c_str(),
 				fmt_conn_id(a->conn.orig, 0, a->conn.resp, a->conn.resp_p));
 			delete a;
@ -638,7 +640,7 @@ bool Manager::ApplyScheduledAnalyzers(Connection* conn, bool init, TransportLaye
 		conn->Event(scheduled_analyzer_applied, 0, tag);
 		DBG_ANALYZER_ARGS(conn, "activated %s analyzer as scheduled",
-		                  analyzer_mgr->GetComponentName(*it));
+		                  analyzer_mgr->GetComponentName(*it).c_str());
 		}
 	return expected.size();
--- a/src/analyzer/Manager.h
+++ b/src/analyzer/Manager.h
@ -45,10 +45,6 @@ namespace analyzer {
 * sets up their initial analyzer tree, including adding the right \c PIA,
 * respecting well-known ports, and tracking any analyzers specifically
 * scheduled for individidual connections.
 *
 * Note that we keep the public interface of this class free of std::*
 * classes. This allows to external analyzer code to potentially use a
 * different C++ standard library.
 */
 class Manager : public plugin::ComponentManager<Tag, Component> {
 public:
@ -133,6 +129,14 @@ public:
 	 */
 	void DisableAllAnalyzers();
 	/**
 	 * Returns the tag associated with an analyer name, or the tag
 	 * associated with an error if no such analyzer exists.
 	 *
 	 * @param name The canonical analyzer name to check.
 	 */
 	Tag GetAnalyzerTag(const char* name);
 	/**
 	 * Returns true if an analyzer is enabled.
 	 *
--- a/src/analyzer/protocol/arp/Plugin.cc
+++ b/src/analyzer/protocol/arp/Plugin.cc
@ -1,7 +1,21 @@
 // See the file  in the main distribution directory for copyright.
 #include "plugin/Plugin.h"
-BRO_PLUGIN_BEGIN(Bro, ARP)
+namespace plugin {
-	BRO_PLUGIN_DESCRIPTION("ARP Parsing Code");
+namespace Bro_ARP {
-	BRO_PLUGIN_BIF_FILE(events);
+
-BRO_PLUGIN_END
+class Plugin : public plugin::Plugin {
 public:
 	plugin::Configuration Configure()
 		{
 		plugin::Configuration config;
 		config.name = "Bro::ARP";
 		config.description = "ARP Parsing";
 		return config;
 		}
 } plugin;
 }
 }
--- a/src/analyzer/protocol/ayiya/AYIYA.h
+++ b/src/analyzer/protocol/ayiya/AYIYA.h
@ -14,7 +14,7 @@ public:
 	virtual void DeliverPacket(int len, const u_char* data, bool orig,
 					uint64 seq, const IP_Hdr* ip, int caplen);
-	static analyzer::Analyzer* InstantiateAnalyzer(Connection* conn)
+	static analyzer::Analyzer* Instantiate(Connection* conn)
 		{ return new AYIYA_Analyzer(conn); }
 protected:
--- a/src/analyzer/protocol/ayiya/Plugin.cc
+++ b/src/analyzer/protocol/ayiya/Plugin.cc
@ -1,10 +1,25 @@
 // See the file  in the main distribution directory for copyright.
 #include "plugin/Plugin.h"
 #include "AYIYA.h"
-BRO_PLUGIN_BEGIN(Bro, AYIYA)
+namespace plugin {
-	BRO_PLUGIN_DESCRIPTION("AYIYA Analyzer");
+namespace Bro_AYIYA {
-	BRO_PLUGIN_ANALYZER("AYIYA", ayiya::AYIYA_Analyzer);
+
-	BRO_PLUGIN_BIF_FILE(events);
+class Plugin : public plugin::Plugin {
-BRO_PLUGIN_END
+public:
 	plugin::Configuration Configure()
 		{
 		AddComponent(new ::analyzer::Component("AYIYA", ::analyzer::ayiya::AYIYA_Analyzer::Instantiate));
 		plugin::Configuration config;
 		config.name = "Bro::AYIYA";
 		config.description = "AYIYA Analyzer";
 		return config;
 		}
 } plugin;
 }
 }
--- a/src/analyzer/protocol/backdoor/BackDoor.h
+++ b/src/analyzer/protocol/backdoor/BackDoor.h
@ -73,7 +73,7 @@ public:
 	virtual void Done();
 	void StatTimer(double t, int is_expire);
-	static analyzer::Analyzer* InstantiateAnalyzer(Connection* conn)
+	static analyzer::Analyzer* Instantiate(Connection* conn)
 		{ return new BackDoor_Analyzer(conn); }
 protected:
--- a/src/analyzer/protocol/backdoor/Plugin.cc
+++ b/src/analyzer/protocol/backdoor/Plugin.cc
@ -1,10 +1,25 @@
 // See the file  in the main distribution directory for copyright.
 #include "plugin/Plugin.h"
 #include "BackDoor.h"
-BRO_PLUGIN_BEGIN(Bro, BackDoor)
+namespace plugin {
-	BRO_PLUGIN_DESCRIPTION("Backdoor Analyzer (deprecated)");
+namespace Bro_BackDoor {
-	BRO_PLUGIN_ANALYZER("BackDoor", backdoor::BackDoor_Analyzer);
+
-	BRO_PLUGIN_BIF_FILE(events);
+class Plugin : public plugin::Plugin {
-BRO_PLUGIN_END
+public:
 	plugin::Configuration Configure()
 		{
 		AddComponent(new ::analyzer::Component("BackDoor", ::analyzer::backdoor::BackDoor_Analyzer::Instantiate));
 		plugin::Configuration config;
 		config.name = "Bro::BackDoor";
 		config.description = "Backdoor Analyzer deprecated";
 		return config;
 		}
 } plugin;
 }
 }
--- a/src/analyzer/protocol/bittorrent/BitTorrent.h
+++ b/src/analyzer/protocol/bittorrent/BitTorrent.h
@ -19,7 +19,7 @@ public:
 	virtual void Undelivered(uint64 seq, int len, bool orig);
 	virtual void EndpointEOF(bool is_orig);
-	static analyzer::Analyzer* InstantiateAnalyzer(Connection* conn)
+	static analyzer::Analyzer* Instantiate(Connection* conn)
 		{ return new BitTorrent_Analyzer(conn); }
 protected:
--- a/src/analyzer/protocol/bittorrent/BitTorrentTracker.h
+++ b/src/analyzer/protocol/bittorrent/BitTorrentTracker.h
@ -52,7 +52,7 @@ public:
 	virtual void Undelivered(uint64 seq, int len, bool orig);
 	virtual void EndpointEOF(bool is_orig);
-	static analyzer::Analyzer* InstantiateAnalyzer(Connection* conn)
+	static analyzer::Analyzer* Instantiate(Connection* conn)
 		{ return new BitTorrentTracker_Analyzer(conn); }
 protected:
--- a/Show more
+++ b/Show more
		`@ -1 +1 @@`
			`Subproject commit ec1e052afd5a8cd3d1d2cbb28fcd688018e379a5`				`Subproject commit 3a4684801aafa0558383199e9abd711650b53af9`
		`@ -1 +1 @@`
			`Subproject commit 5721df4f5f6fa84de6257cca6582a28e45831786`				`Subproject commit 9ea20c3905bd3fd5109849c474a2f2b4ed008357`
		`@ -1 +1 @@`
			`Subproject commit c2f5dd2cb7876158fdf9721aebd22567db840db1`				`Subproject commit 33d0ed4a54a6ecf08a0b5fe18831aa413b437066`
		`@ -1 +1 @@`
			`Subproject commit ca3b12421269fce655028c05939e0ff2ee82ca2d`				`Subproject commit 2f808bc8541378b1a4953cca02c58c43945d154f`
		`@ -1 +1 @@`
			`Subproject commit 4da1bd24038d4977e655f2b210f34e37f0b73b78`				`Subproject commit 1efa4d10f943351efea96def68e598b053fd217a`
		`@ -0,0 +1 @@`
							`Subproject commit 23055b473c689a79da12b2825d8388f71f28c709`
		`@ -1 +1 @@`
			`Subproject commit 0f301aa08a970150195a2ea5b3ed43d2d98b35b3`				`Subproject commit 03de0cc467d2334dcb851eddd843d59fef217909`