Update NEWS

Corrected some typos, fixed some reST formatting, and added some more
useful info.

NEWS

@@ -31,26 +31,29 @@ New Functionality
      transferred over SMB can be analyzed.
    - Includes GSSAPI and NTLM analyzer and reimplements the DCE-RPC
      analyzer.
-   - New logs: smb_files.log, smb_mapping.log, ntlm.log, and dce_rpc.log
+   - New logs: smb_cmd.log, smb_files.log, smb_mapping.log, ntlm.log, and dce_rpc.log
    - Not every possible SMB command or functionality is implemented, but
      generally, file handling should work whenever files are transferred.
      Please speak up on the mailing list if there is an obvious oversight.
 
 - Bro now includes the NetControl framework. The framework allows for easy
   interaction of Bro with hard- and software switches, firewalls, etc.
+  New log files: net_control.log, netcontrol_catch_release.log,
+  netcontrol_drop.log, and netcontrol_shunt.log.
 
 - Bro's Intelligence Framework was refactored and new functionality
   has been added:
 
   - The framework now supports the new indicator type Intel::SUBNET.
-    As subnets are matched against seen addresses, the field 'matched'
+    As subnets are matched against seen addresses, the new field 'matched'
-    was introduced to indicate which indicator type(s) caused the hit.
+    in intel.log was introduced to indicate which indicator type(s) caused
+    the hit.
 
   - The new function remove() allows to delete intelligence items.
 
   - The intel framework now supports expiration of intelligence items.
-    Expiration can be configured by using Intel::item_expiration and
+    Expiration can be configured using the new Intel::item_expiration constant
-    can be handled by using the item_expired() hook. The new script
+    and can be handled by using the item_expired() hook. The new script
     do_expire.bro removes expired items.
 
   - The new hook extend_match() allows extending the framework. The new
@@ -62,26 +65,23 @@ New Functionality
 - There is a new file entropy analyzer for files.
 
 - Bro now supports the remote framebuffer protocol (RFB) that is used by
-  VNC servers for remote graphical displays.
+  VNC servers for remote graphical displays.  New log file: rfb.log.
 
 - Bro now supports the Radiotap header for 802.11 frames.
 
-- Bro now has rudimentary IMAP and XMPP analyzers examinig the initial
+- Bro now has rudimentary IMAP and XMPP analyzers examining the initial
-  phases of the protocol. Right now these analyzer only identify
+  phases of the protocol. Right now these analyzers only identify
-  STARTTLS sessions, handing them over to TLS analysis. The analyzer
+  STARTTLS sessions, handing them over to TLS analysis. These analyzers
-  does not yet analyze any further IMAP/XMPP content.
+  do not yet analyze any further IMAP/XMPP content.
 
-- The new event ssl_extension_signature_algorithm allows access to the
+- The new event ssl_extension_signature_algorithm() allows access to the
   TLS signature_algorithms extension that lists client supported signature
   and hash algorithm pairs.
 
 - Bro now tracks VLAN IDs. To record them inside the connection log,
   load protocols/conn/vlan-logging.bro.
 
-- The new misc/stats.bro records Bro executions statistics in a
+- A new dns_CAA_reply() event gives access to DNS Certification Authority
-  standard Bro log file.
-
-- A new dns_CAA_reply event gives access to DNS Certification Authority
   Authorization replies.
 
 - A new per-packet event raw_packet() provides access to layer 2
@@ -93,10 +93,10 @@ New Functionality
   argument that will be used for decoding errors into weird.log
   (instead of reporter.log).
 
-- A new get_current_packet_header bif returns the headers of the current
+- A new get_current_packet_header() bif returns the headers of the current
   packet.
 
-- Two new built-in functions for handling set[subnet] and table[subnet]:
+- Three new built-in functions for handling set[subnet] and table[subnet]:
 
   - check_subnet(subnet, table) checks if a specific subnet is a member
     of a set/table. This is different from the "in" operator, which always
@@ -120,22 +120,25 @@ New Functionality
 
   - subnet_width(subnet) returns the width of a subnet.
 
-- The IRC analyzer now recognizes StartTLS sessions and enable the SSL
+- The IRC analyzer now recognizes StartTLS sessions and enables the SSL
   analyzer for them.
 
-- A set of new built-in function for gathering execution statistics:
+- The misc/stats.bro script is now loaded by default and logs more Bro
+  execution statistics to the stats.log file than it did previously.
+
+- A set of new built-in functions for gathering execution statistics:
 
       get_net_stats(), get_conn_stats(), get_proc_stats(),
       get_event_stats(), get_reassembler_stats(), get_dns_stats(),
       get_timer_stats(), get_file_analysis_stats(), get_thread_stats(),
-      get_gap_stats(), get_matcher_stats(),
+      get_gap_stats(), get_matcher_stats()
 
 - Two new functions haversine_distance() and haversine_distance_ip()
-  for calculating geographic distances. They requires that Bro be
+  for calculating geographic distances. The latter function requires that Bro
-  built with libgeoip.
+  be built with libgeoip.
 
 - Table expiration timeout expressions are evaluated dynamically as
-  timestmaps are updated.
+  timestamps are updated.
 
 - The pcap buffer size can be set through the new option Pcap::bufsize.
 
@@ -144,7 +147,7 @@ New Functionality
 
 - The logging framework now supports user-defined record separators,
   renaming of column names, as well as extension data columns that can
-  be added to specific or all logfiles (e.g., to add noew names).
+  be added to specific or all logfiles (e.g., to add new names).
 
 - The new "bro-config" script can be used to determine the Bro installation
   paths.
@@ -185,7 +188,7 @@ New Functionality
     - pf_ring: Native PF_RING support.
     - postgresql: A PostgreSQL reader/writer.
     - redis: An experimental log writer for Redis.
-    - tcprs: An TCP-level analyzer detecting retransmissions, reordering, and more.
+    - tcprs: A TCP-level analyzer detecting retransmissions, reordering, and more.
 
 Changed Functionality
 ---------------------
@@ -195,10 +198,14 @@ Changed Functionality
     - Connections
 
         The 'history' field gains two new flags: '^' indicates that
-        Bro heuristically flipped to direction of the connection.
+        Bro heuristically flipped the direction of the connection.
         't/T' indicates the first TCP payload retransmission from
         originator or responder, respectively.
 
+    - Intelligence
+
+        New field 'matched' to indicate which indicator type(s) caused the hit.
+
     - DNS
 
         New 'rtt' field to indicate the round trip time between when a
@@ -211,42 +218,58 @@ Changed Functionality
 
         Changes in 'mailfrom' and 'rcptto' fields to remove some
         non-address cruft that will tend to be found.  The main
-        example is the change from "<user@domain>" to
+        example is the change from ``"<user@domain>"`` to
-        "user@domain.com".
+        ``"user@domain.com"``.
 
     - HTTP
 
-        Removed 'filename' field.
+        Removed 'filename' field (which was never filled out in the first
+        place).
 
         New 'orig_filenames' and 'resp_filenames' fields which each
         contain a vector of filenames seen in entities transferred.
 
+    - stats.log
+
+        The following fields have been added: active_tcp_conns,
+        active_udp_conns, active_icmp_conns, tcp_conns, udp_conns,
+        icmp_conns, timers, active_timers, files, active_files, dns_requests,
+        active_dns_requests, reassem_tcp_size, reassem_file_size,
+        reassem_frag_size, reassem_unknown_size.
+
+        The following fields have been renamed: lag -> pkt_lag.
+
+        The following fields have been removed: pkts_recv.
+
 - The BrokerComm and BrokerStore namespaces were renamed to Broker.
-  The Broker "print" function was renamed to Broker::send_print, and
+  The Broker "print()" function was renamed to Broker::send_print(), and
-  "event" to "Broker::send_event".
+  the "event()" function was renamed to Broker::send_event().
 
-- ``SSH::skip_processing_after_detection`` was removed. The functionality was
+- The constant ``SSH::skip_processing_after_detection`` was removed. The
-  replaced by ``SSH::disable_analyzer_after_detection``.
+  functionality was replaced by the new constant
+  ``SSH::disable_analyzer_after_detection``.
 
-- ``net_stats()`` and ``resource_usage()`` have been superseded by the
+- The ``net_stats()`` and ``resource_usage()`` functions have been
-  new execution statistics functions (see above).
+  removed, and their functionality is now provided by the new execution
+  statistics functions (see above).
 
-- Some script-level identifier have changed their names:
+- Some script-level identifiers have changed their names:
 
-      snaplen                  -> Pcap::snaplen
+      - snaplen                  -> Pcap::snaplen
-      precompile_pcap_filter() -> Pcap::precompile_pcap_filter()
+      - precompile_pcap_filter() -> Pcap::precompile_pcap_filter()
-      install_pcap_filter()    -> Pcap::install_pcap_filter()
+      - install_pcap_filter()    -> Pcap::install_pcap_filter()
-      pcap_error()             -> Pcap::pcap_error()
+      - pcap_error()             -> Pcap::error()
-
-- In http.log, the "filename" field (which it turns out was never
-  filled out in the first place) has been split into to
-  "orig_filenames" and "resp_filenames".
 
 - TCP analysis was changed to process connections without the initial
   SYN packet. In the past, connections without a full handshake were
   treated as partial, meaning that most application-layer analyzers
   would refuse to inspect the payload. Now, Bro will consider these
-  connections as complete and all analyzers will process them notmally.
+  connections as complete and all analyzers will process them normally.
+
+- The ``policy/misc/capture-loss.bro`` script is now loaded by default.
+
+- The traceroute detection script package ``policy/misc/detect-traceroute``
+  is no longer loaded by default.
 
 - Changed BroControl functionality in aux/broctl:
 
@@ -284,33 +307,34 @@ Changed Functionality
 Removed Functionality
 ---------------------
 
-    - The app-stats scripts have been removed because they weren't
+- The app-stats scripts have been removed because they weren't
-      being maintained and they were becoming inaccurate. They
+  being maintained and they were becoming inaccurate (as a result, the
-      were also prone to needing more regular updates as the internet
+  app_stats.log is also gone). They were also prone to needing more regular
-      changed and will likely be more relevant if maintained externally.
+  updates as the internet changed and will likely be more relevant if
+  maintained externally.
 
-    - The event ack_above_hole() has been removed, as it was a subset
+- The event ack_above_hole() has been removed, as it was a subset
-      of content_gap() and led to plenty noise.
+  of content_gap() and led to plenty of noise.
 
-    - The command line options --set-seed and --md5-hashkey have been
+- The command line options ``--analyze``, ``--set-seed``, and
-      removed.
+  ``--md5-hashkey`` have been removed.
 
-    - The packaging scripts pkg/make-\*-packages are gone. They aren't
+- The packaging scripts pkg/make-\*-packages are gone. They aren't
-      used anymore for the binary Bro packages that the projects
+  used anymore for the binary Bro packages that the project
-      distributes; haven't been supported in a while; and have
+  distributes; haven't been supported in a while; and have
-      problems.
+  problems.
 
 Deprecated Functionality
 ------------------------
 
-    - The built-in functions decode_base64_custom() and
+- The built-in functions decode_base64_custom() and
-      encode_base64_custom() are no longer needed and will be removed
+  encode_base64_custom() are no longer needed and will be removed
-      in the future. Their functionality is now provided directly by
+  in the future. Their functionality is now provided directly by
-      decode_base64() and encode_base64(), which take an optional
+  decode_base64() and encode_base64(), which take an optional
-      parameter to change the Base64 alphabet.
+  parameter to change the Base64 alphabet.
 
-    - The ElasticSearch log writer hasn't been maintained for a while
+- The ElasticSearch log writer hasn't been maintained for a while
-      and is now deprecated. It will be removed with the next release.
+  and is now deprecated. It will be removed with the next release.
 
 
 Bro 2.4