Merge remote-tracking branch 'origin/topic/timw/5.2-NEWS-updates'

* origin/topic/timw/5.2-NEWS-updates:
  Whitespace cleanup in 5.2 NEWS block
  Updates to the NEWS for upcoming release/5.2 branching

CHANGES

@@ -1,3 +1,9 @@
+5.2.0-dev.569 | 2023-01-30 09:19:23 -0700
+
+  * Whitespace cleanup in 5.2 NEWS block (Tim Wojtulewicz, Corelight)
+
+  * Updates to the NEWS for upcoming release/5.2 branching (Tim Wojtulewicz, Corelight)
+
 5.2.0-dev.566 | 2023-01-30 11:05:40 +0100
 
   * GH-2698: broker/store: Extend SQLiteOptions around data safety and performance (Arne Welzel, Corelight)

NEWS

@@ -25,6 +25,22 @@ Breaking Changes
   adapted accordingly. Users of ``mysql_ok()`` likely need to switch to
   ``mysql_eof()``.
 
+- Zeek will now exit at startup if an external plugin (e.g. from a package) is
+  discovered to have the same name as a built-in plugin. See below for the
+  change regarding the AF_PACKET plugin now being built-in for an example of
+  this potentially being triggered.
+
+- DNS query type strings were updated to match the current standardized list of
+  strings. This changes the string reported for a small subset of query types:
+
+	30: Changed from "EID" to "NXT"
+	31: Changed from "NIMLOC" to "EID"
+	32: Changed from "NB" to "NIMLOC"
+
+- The ``--with-caf`` option for the ``configure`` script was removed. Broker now
+  requires specific versions of CAF per Zeek release, and passing an
+  externally-built version of CAF often lead to build failures.
+
 New Functionality
 -----------------
 
@@ -36,27 +52,31 @@ New Functionality
   available in the Zeek documentation. Note also that Spicy is currently
   unsupported and will be fixed in the future.
 
+  The feature as checked into the repository is not considered production-ready.
+  There are many bugs to squash and features to improve, and we will be steadily
+  fixing things over the next few months.
+
   The Zeek team wants to give a huge thank you to the team at Microsoft for all
   of their effort in completing this port.
 
-- New ``analyzer_confirmation_info`` and ``analyzer_violation_info``
+- New ``analyzer_confirmation_info`` and ``analyzer_violation_info`` events with
-  events with accompanying record types ``AnalyzerConfirmationInfo``
+  accompanying record types ``AnalyzerConfirmationInfo`` and
-  and ``AnalyzerViolationInfo`` have been added. These supersede
+  ``AnalyzerViolationInfo`` have been added. These supersede
-  ``analyzer_confirmation`` and ``analyzer_violation``, which have
+  ``analyzer_confirmation`` and ``analyzer_violation``, which have been
-  been deprecated.
+  deprecated.
 
-- Added helpers to determine protocol, packet or file analyzer based
+- Added helpers to determine protocol, packet or file analyzer based on
-  on ``AllAnalyzers::Tag`` values named ``is_protocol_analyzer()``,
+  ``AllAnalyzers::Tag`` values named ``is_protocol_analyzer()``,
   ``is_packet_analyzer()`` and ``is_file_analyzer()``.
 
-- File analyzers can now raise analyzer violations to the script-layer via
+- File analyzers can now raise analyzer violations to the script-layer via the
-  the new ``AnalyzerViolation()`` method.
+  new ``AnalyzerViolation()`` method.
 
-- Packet and file analyzers can now be disabled and enabled at runtime using
+- Packet and file analyzers can now be disabled and enabled at runtime using the
-  the ``Analyzer::enable_analyzer()`` and ``Analyzer::disable_analyzer()``
+  ``Analyzer::enable_analyzer()`` and ``Analyzer::disable_analyzer()``
-  wrappers. While initially for protocol analyzers only, these have
+  wrappers. While initially for protocol analyzers only, these have been
-  been extended to work for packet and file analyzers. This now allows to
+  extended to work for packet and file analyzers. This now allows to leverage
-  leverage ``Analyzer::disabled_analyzers`` for these kinds of analyzers.
+  ``Analyzer::disabled_analyzers`` for these kinds of analyzers.
 
 - The blank identifier ``_`` can now be used to ignore loop variables of
   different types without type clash errors. This allows to do the following
@@ -77,8 +97,8 @@ New Functionality
 	for ( _, v in tab )
 		print v;
 
-  It's also possible ignore individual indices of different types with the
+  It's also possible ignore individual indices of different types with the blank
-  blank identifier ``_`` as follows:
+  identifier ``_`` as follows:
 
 	for ( [_, i, _], v in tab )
 		print i, v;
@@ -87,8 +107,8 @@ New Functionality
   referenced in expression anymore.
 
 - It is now possible to put trailing commas within table, vector, set and record
-  construction. For example, the following code is now valid, which can make
+  construction. For example, the following code is now valid, which can make for
-  for more uniform style and smaller diffs.
+  more uniform style and smaller diffs.
 
 	local vec = vector(
 		"1",
@@ -112,25 +132,25 @@ New Functionality
 	);
 
 - Re-introduce event groups. Allow the ``&group`` attribute on event and hook
-  handlers for annotating them with one or more event groups. These groups
+  handlers for annotating them with one or more event groups. These groups can
-  can be disabled and enable during runtime. Disabling an event group implies
+  be disabled and enable during runtime. Disabling an event group implies
   disabling all event and hook handlers that are part of it.
 
   The main difference to a previous implementation in (very) old Zeek versions
   is its granularity: It is now possible to toggle individual event handlers
   (event handler bodies). The original implementation worked at the level of
   events, disabling or enabling all event handlers for a given event at once.
-  Additionally, support for hooks was added as these are structurally similar
+  Additionally, support for hooks was added as these are structurally similar to
-  to events.
+  events.
 
   The BIFs ``disable_event_group()`` and ``enable_event_group()`` are
   re-instantiated and allow controlling event groups based on the group
   attribute.
 
-  Additionally, event and hook handlers are implicitly placed into event
+  Additionally, event and hook handlers are implicitly placed into event module
-  module groups based on the module they are implemented in. All events
+  groups based on the module they are implemented in. All events implemented in
-  implemented in a given module can be toggled with ``disable_module_events()``
+  a given module can be toggled with ``disable_module_events()`` and
-  and ``enable_module_events()``.
+  ``enable_module_events()``.
 
 - Extend the ``Logging::Stream`` record with an ``event_groups`` field and
   toggle these during ``Log::disable_stream`` and ``Log::enable_stream``
@@ -138,31 +158,34 @@ New Functionality
   by turning off event handlers at runtime that are only needed for log
   generation.
 
-- On Linux, the AF_PACKET packet source plugin (https://github.com/zeek/zeek-af_packet-plugin)
+- On Linux, the AF_PACKET packet source plugin
-  is included as builtin plugin by default. To select this packet source, prefix
+  (https://github.com/zeek/zeek-af_packet-plugin) is included as a builtin
-  the interface name with ``af_packet``.
+  plugin by default. To select this packet source, prefix the interface name
+  with ``af_packet``.
 
 	zeek -i af_packet::eth0
 
 - Usage of ``break`` and ``next`` statements is now validated. It was previously
-  possible to place these outside of ``for``, ``while`` or ``switch``
+  possible to place these outside of ``for``, ``while`` or ``switch`` statements
-  statements without any error indication.
+  without any error indication.
 
-- Add two BIFs ``get_identifier_declaring_script()`` and ``get_record_field_declaring_script()``
+- Add two BIFs ``get_identifier_declaring_script()`` and
-  to query the declaring scripts for identifiers and record fields from Zeek scripts.
+  ``get_record_field_declaring_script()`` to query the declaring scripts for
+  identifiers and record fields from Zeek scripts.
 
 - Extend the SSH analyzer to produce new events (``ssh2_ecc_init``,
-  ``ssh2_gh_gex_init``, ``ssh2_gss_init``, ssh2_rsa_secret``) to detect
+  ``ssh2_gh_gex_init``, ``ssh2_gss_init``, ssh2_rsa_secret``) to detect when SSH
-  when SSH client and server roles are reversed.
+  client and server roles are reversed.
 
 - Analyzers found in the new ``Analyzer::requested_analyzers`` set will be
-  enabled at ``zeek_init()`` time. The set can be populated via :zeek:see:`redef`.
+  enabled at ``zeek_init()`` time. The set can be populated via
-  This change only has an effect in settings where ``Analyzer::disable_all``
+  :zeek:see:`redef`.  This change only has an effect in settings where
-  is changed to ``T``. By default, all analyzers continue to be enabled.
+  ``Analyzer::disable_all`` is changed to ``T``. By default, all analyzers
+  continue to be enabled.
 
-- A new ``analyzer.log`` was added to log all analyzer violations and
+- A new ``analyzer.log`` was added to log all analyzer violations and optionally
-  optionally analyzer confirmations. This log can be useful during development
+  analyzer confirmations. This log can be useful during development of new
-  of new analyzers as well as for collecting operational data in production
+  analyzers as well as for collecting operational data in production
   environments.
 
 - Expose configurability of for SQLite's synchronous and journal_mode PRAGMAs
@@ -174,31 +197,49 @@ New Functionality
   Additionally, add integrity_check and failure_mode options to support
   detecting and deleting corrupted SQLite database at store initialization.
 
+- A new ``join_string_set`` BIF was added, replacing the existing script-level
+  version from utils/strings.zeek.
+
+- A new ``&ordered`` attribute for tables and sets was added. This attribute
+  causes iteration over a table/set to return elements in the order of their
+  insertion.
+
+- A new ``-D`` argument was added to the ``configure`` script to allow passing
+  parameters directly to the underlying CMake call.
+
+- Added parsing for the challenge and response fields to the NTLM analyzer.
+
+- A new ``FTP::max_command_length`` value was added to script-land, defaulting
+  to 100. This value is used by the FTP analyzer to limit the size of commands
+  accepted by the analyzer. A ``FTP_max_command_length_exceeded`` weird is
+  raised for any violations of that length.
+
 Changed Functionality
 ---------------------
 
-- Violations for packet analyzers that have sessions attached with them
+- Violations for packet analyzers that have sessions attached with them will be
-  will be raised once only. Further, analyzer confirmations are not raised
+  raised once only. Further, analyzer confirmations are not raised after a
-  after a violation.
+  violation.
 
-- The parameter given to ``enum_names()`` can now be a string naming the
+- The parameter given to ``enum_names()`` can now be a string naming the enum
-  enum type, rather than the type itself.
+  type, rather than the type itself.
 
-- The ``type_name`` of enum types produced by ``record_fields()`` now
+- The ``type_name`` of enum types produced by ``record_fields()`` now includes
-  includes the actual type name rather than just ``"enum"``.
+  the actual type name rather than just ``"enum"``.
 
 - Passing non-string ``sep`` and ``def`` arguments to ``cat_sep()`` isn't a
   fatal error anymore. More descriptive error messages are produced, too.
 
 - The number of analyzer violation events that can be raised by protocol
-  analyzer instances is now capped by the const ``max_analyzer_violation_events``.
+  analyzer instances is now capped by the const
+  ``max_analyzer_violation_events``.
 
 - The number of analyzer violation events that can be raised by protocol and
-  file analyzer instances is now capped by the const ``max_analyzer_violation_events``.
+  file analyzer instances is now capped by the const
-  Its default is 1000 and the main purpose is to prevent analyzers from
+  ``max_analyzer_violation_events``.  Its default is 1000 and the main purpose
-  scheduling too many ``analyzer_violation_info`` events before the
+  is to prevent analyzers from scheduling too many ``analyzer_violation_info``
-  DPD ``max_violations`` script-level logic has a chance to run and disable
+  events before the DPD ``max_violations`` script-level logic has a chance to
-  the problematic analyzer.
+  run and disable the problematic analyzer.
 
 - The TCP analyzer now continues processing payload for some
   connections missing initial packets where it would previously have
@@ -214,8 +255,8 @@ Changed Functionality
 	expression warning in <command line>, line 1: count underflow (1 - 2)
 	18446744073709551615
 
-- The MQTT scripts registering the analyzer and DPD signatures have
+- The MQTT scripts registering the analyzer and DPD signatures have been moved
-  been moved from the policy folder to base and are loaded by default.
+  from the policy folder to base and are loaded by default.
 
 - Notices created for files transferred over multiple connections will now be
   associated with one of the connections rather than none.
@@ -223,6 +264,13 @@ Changed Functionality
 - The MySQL analyzer has been switched to parse in little endian. This avoids
   analyzer violations due to out of bound errors for length encoded strings.
 
+- Non-fatal errors when setting up BPF filtering will no longer cause Zeek to
+  exit, but instead will log the error in reporter.log and continue processing.
+
+- The languages reported for the ``keyboard_layout`` field in rdp.log were
+  updated to match the current standardized set of languages. Unknown layout
+  values now attempt to fallback to a "parent" layout if one is available.
+
 Deprecated Functionality
 ------------------------
 
@@ -230,9 +278,9 @@ Deprecated Functionality
   deprecated in favor of the more generic ``analyzer_confirmation_info`` and
   ``analyzer_violation_info`` events.
 
-- The const values for toggling individual tunnel packet analyzers have
+- The const values for toggling individual tunnel packet analyzers have been
-  been deprecated in favor of using ``Analyzer::disable_analyzer()``
+  deprecated in favor of using ``Analyzer::disable_analyzer()`` directly. This
-  directly. This affects:
+  affects:
 
 	Tunnel::enable_ip
 	Tunnel::enable_ayiya
@@ -252,6 +300,11 @@ Deprecated Functionality
 - The pre-authentication data field (pa_data) available in certain Kerberos
   events now exposes the (encrypted) PA-ENC-TIMESTAMP field (padata-type=2).
 
+- The ``SupressWeirds()`` method in the ContentLine analyzer was deprecated in
+  favor of the correctly-spelled ``SuppressWeirds()`` method.
+
+- The `bro` symlink has finally been removed.
+
 Zeek 5.1.0
 ==========
 

VERSION

@@ -1 +1 @@
-5.2.0-dev.566
+5.2.0-dev.569