Merge remote-tracking branch 'origin/topic/dnthayer/ticket1905'

Includes slight editing of the new text.

Closes BIT-1905.

* origin/topic/dnthayer/ticket1905:
  Add documentation of the configuration framework
  Add documentation of "option" declarations
  Improve config framework documentation comments
  Fix minor typos and doc build warnings in NEWS

NEWS

@@ -20,14 +20,14 @@ New Functionality
     using normal assignments. Instead, they can be changed using the
     new function Option::set.
 
-    It is possible to "subscribe" to option through
+    It is possible to "subscribe" to an option through
     Option::set_change_handler, which will trigger a handler callback
     when an option changes. Change handlers can optionally modify
     values before they are applied by returning the desired value, or
     reject updates by returning the old value. Priorities can be
     specified if there are several handlers for one option.
 
-    Example script:
+    Example script::
 
         option testbool: bool = T;
 
@@ -55,7 +55,7 @@ New Functionality
     Configuration files to read can be specified by adding them to
     Config::config_files.
 
-    Usage example:
+    Usage example::
 
         redef Config::config_files += { "/path/to/config.dat" };
 
@@ -65,18 +65,18 @@ New Functionality
             option testbool: bool = F;
         }
 
-    The specified file will now monitored contionoulsy for changes, so
+    The specified file will now be monitored continuously for changes, so
     that writing "testbool T" into /path/to/config.dat will
     automatically update the option's value accordingly.
 
-    The configutation framework creates a config.log that shows all
+    The configuration framework creates a config.log that shows all
     value changes that took place.
 
-  - Config reader: Internally, the configuration framework use a new
+  - Config reader: Internally, the configuration framework uses a new
     type of input reader to read such configuration files into Bro.
     The reader uses the option name to look up the type that variable
     has, converts the read value to the correct type, and then updates
-    the option's value. Example script use:
+    the option's value. Example script use::
 
         type Idx: record {
             option_name: string;
@@ -173,12 +173,15 @@ Changed Functionality
   the default configuration of logs, this field will show "-" instead of
   "(empty)" for connections that lack any tunelling.
 
-- SMB event argument changes
+- SMB event argument changes:
+
   - smb1_transaction_request now has two additional arguments, "parameters"
     and "data" strings
+
   - smb1_transaction2_request now has an additional "args" record argument
 
 - SSL event argument changes:
+
   - event ssl_server_signature now has an additional argument
     "signature_and_hashalgorithm".
 

doc/frameworks/configuration.rst

@@ -0,0 +1,150 @@
+
+.. _framework-configuration:
+
+=======================
+Configuration Framework
+=======================
+
+.. rst-class:: opening
+
+Bro includes a "configuration framework" that allows
+updating script options dynamically at runtime. This functionality
+consists of several components: an "option" declaration, the
+ability to specify input files to enable changing the value of options at
+runtime, a couple of built-in functions, and a log file "config.log"
+which contains information about every change to option values.
+
+
+.. contents::
+
+
+Introduction
+------------
+
+The configuration framework provides an alternative to using Bro
+script constants to store various Bro settings.
+
+In general, traditional constants can be used when a value is not
+expected to change at runtime, but they cannot be used for values that
+need to be modified occasionally. While a "redef" allows a
+re-definition of an already defined constant in Bro, these
+redefinitions can only be performed when Bro first starts. Afterwards,
+constants no longer be modified.
+
+However, it is clearly desirable to be able to change at runtime many
+of the configuration options that Bro offers. Having to restart Bro
+can be time-consuming and causes Bro to lose all connection state and
+knowledge that it accumulated. Bro's configuration framework solves
+this problem by allowing changing configuration options at runtime.
+
+Declaring options
+-----------------
+
+The "option" keyword allows variables to be declared as configuration options.
+
+.. code:: bro
+
+    module TestModule;
+
+    export {
+        option my_networks: set[subnet] = {};
+        option enable_feature = F;
+        option hostname = "testsystem";
+    }
+
+The rules regarding options can be thought of as being in between global
+variables and constants.  Like global variables, options cannot be declared
+inside a function, hook, or event handler.  Like constants, options must be
+initialized when declared.  The value of an option can change at runtime,
+but options cannot be assigned a new value using normal assignments.
+
+
+Changing options
+----------------
+
+The configuration framework facilitates reading in new option values
+from external files at runtime.
+
+Configuration files contain a mapping between option names and their values.
+The format for these files looks like this:
+
+    [option name][tab/spaces][new value]
+
+Configuration files can be specified by adding them to Config::config_files.
+For example, simply add something like this to local.bro:
+
+.. code:: bro
+
+    redef Config::config_files += { "/path/to/config.dat" };
+
+The specified configuration file will then be monitored continuously for changes,
+so that writing ``TestModule::enable_feature T`` into that file will
+automatically update the option's value accordingly.  Here is an example
+configuration file::
+
+    TestModule::my_networks 10.0.12.0/24,192.168.17.0/24
+    TestModule::enable_feature  T
+    TestModule::hostname  host-1
+
+Internally, the configuration framework uses the Bro input framework
+with a type of input reader specifically for reading config files. Users
+familiar with the Bro input framework might be aware that the input framework
+is usually very strict about the syntax of input files. This is not true
+for configuration files: the files need no header lines and either
+tabs or spaces are accepted as separators.
+
+If you inspect the configuration framework scripts, you will notice that the
+scripts simply catch events from the input framework and then a built-in
+function :bro:see:`Option::set` is called to set an option to the new value.
+If you want to change an option yourself during runtime, you can
+call Option::set directly from a script.
+
+The log file "config.log" contains information about each configuration
+change that occurs during runtime.
+
+
+Change handlers
+---------------
+
+A change handler is a user-defined function that is called automatically
+each time an option value changes.  This example shows how to register a
+change handler for an option that has a data type of "addr" (for other
+data types, the return type and 2nd parameter data type must be adjusted
+accordingly):
+
+.. code:: bro
+
+    option testaddr = 127.0.0.1;
+
+    # Note: the data type of 2nd parameter and return type must match
+    function change_addr(ID: string, new_value: addr): addr
+        {
+        print fmt("Value of %s changed from %s to %s", ID, testaddr, new_value);
+        return new_value;
+        }
+
+    event bro_init()
+        {
+        Option::set_change_handler("testaddr", change_addr);
+        }
+
+Each time the specified option value is changed, the change handler
+function will be called before the change is performed.  The value returned
+by the change handler is the value finally assigned to the option. This
+allows, for example, checking of values to reject invalid input (the original
+value can be returned to reject the change).
+
+A change handler can optionally have a third argument, which is the location
+string (this is normally the pathname of the configuration file that triggered
+the change).
+
+It is also possible to chain together multiple change handlers.  In this
+case, the value returned by the first change handler is the "new value" seen
+by the next change handler, and so on.  The built-in function
+:bro:see:`Option::set_change_handler` takes an optional third argument
+that can specify a priority for the handlers.
+
+Note that change handlers are also used internally by the
+configuration framework. If you look at the script level source code of
+the config framework, you can see that change handlers are used for
+logging the option changes to config.log.

doc/frameworks/index.rst

@@ -6,6 +6,7 @@ Frameworks
 .. toctree::
    :maxdepth: 1
 
+   configuration
    file-analysis
    geoip
    input

doc/script-reference/statements.rst

@@ -20,6 +20,9 @@ Declarations
 +----------------------------+-----------------------------+
 | :bro:keyword:`const`       | Declare a constant          |
 +----------------------------+-----------------------------+
+| :bro:keyword:`option`      | Declare a configuration     |
+|                            | option                      |
++----------------------------+-----------------------------+
 | :bro:keyword:`type`        | Declare a user-defined type |
 +----------------------------+-----------------------------+
 | :bro:keyword:`redef`       | Redefine a global value or  |
@@ -176,6 +179,25 @@ all loaded Bro scripts.
     or "global" keywords (i.e., "const" replaces "local" and "global").
 
 
+.. bro:keyword:: option
+
+    A variable declared with the "option" keyword is a configuration option.
+
+    Options are required to be initialized at the
+    time of declaration.  Normally, the type is inferred from the initializer,
+    but the type can be explicitly specified.  Example::
+
+        option hostname = "host-1";
+        option peers: set[addr] = {};
+
+    The value of an option cannot be changed by an assignment statement.
+
+    The scope of an option is global.
+
+    Note that an "option" declaration cannot also use the "local", "global",
+    or "const" keywords.
+
+
 .. bro:keyword:: type
 
    The "type" keyword is used to declare a user-defined type.  The name

scripts/base/frameworks/config/README

@@ -1,2 +1,2 @@
-The configuration famework provides a way to change the Bro configuration
+The configuration framework provides a way to change the Bro configuration
 in "option" values at run-time.

scripts/base/frameworks/config/main.bro

@@ -1,6 +1,6 @@
 ##! The configuration framework provides a way to change Bro options
-##! (as specified by the option keyword) at runtime. It also logs runtime changes
+##! (as specified by the "option" keyword) at runtime. It also logs runtime
-##! to options to config.log.
+##! changes to options to config.log.
 
 module Config;
 

scripts/base/frameworks/input/readers/config.bro

@@ -33,12 +33,12 @@ export {
 	## and also does not track removals. If you need this, combine the event
 	## with a table reader.
 	##
-	## name: name of the input stream.
+	## name: Name of the input stream.
 	##
-	## source: source of the input stream.
+	## source: Source of the input stream.
 	##
 	## id: ID of the configuration option being set.
 	##
-	## value: new value of the configuration option being set.
+	## value: New value of the configuration option being set.
 	global new_value: event(name: string, source: string, id: string, value: any);
 }

src/option.bif

@@ -1,6 +1,5 @@
-##! The option built-in functions allow the scripting layer to
+##! Definitions of built-in functions that allow the scripting layer to
-##! change the value of option-values and to be notified when
+##! change the value of options and to be notified when option values change.
-##! option values change.
 
 module Option;
 
@@ -8,16 +7,16 @@ module Option;
 #include "NetVar.h"
 %%}
 
-## Sets an option to a new value. This change will also cause the option change handlers
+## Set an option to a new value. This change will also cause the option change
-## to be called.
+## handlers to be called.
 ##
 ## ID: The ID of the option to update.
 ##
 ## val: The new value of the option.
 ##
-## location: optional parameter detailing where this change originated from.
+## location: Optional parameter detailing where this change originated from.
 ##
-## Returns: true on success, false when an error occured.
+## Returns: true on success, false when an error occurred.
 ##
 ## .. bro:see:: Option::set_change_handler
 function Option::set%(ID: string, val: any, location: string &default=""%): bool
@@ -74,24 +73,26 @@ function Option::set%(ID: string, val: any, location: string &default=""%): bool
 	return new Val(1, TYPE_BOOL);
 	%}
 
-## Set the change handler for the option *ID*. The change handler will be called anytime
+## Set a change handler for an option. The change handler will be
-## :bro:id:`Option::set` is called fot *ID*.
+## called anytime :bro:id:`Option::set` is called for the option.
 ##
 ## ID: The ID of the option for which change notifications are desired.
 ##
-## on_change: The function that will be called when a change occurs. The function can choose to
+## on_change: The function that will be called when a change occurs. The
-##            receive two or three parameters: the first parameter is a string containing *ID*,
+##            function can choose to receive two or three parameters: the first
-##            the second parameter is the new option value. The third, optional, parameter is the
+##            parameter is a string containing *ID*, the second parameter is
-##            location string as passed to Option::set. Note that the global value is not yet changed
+##            the new option value. The third, optional, parameter is the
-##            when the function is called. The passed function has to return the new value that
+##            location string as passed to Option::set. Note that the global
-##            it wants the option to be set to. This enables it to reject changes, or change values
+##            value is not yet changed when the function is called. The passed
-##            that are being set. When several change handlers are set for an option they are chained;
+##            function has to return the new value that it wants the option to
-##            the second change handler will see the return value of the first change handler as the
+##            be set to. This enables it to reject changes, or change values
-##            "new value".
+##            that are being set. When several change handlers are set for an
+##            option they are chained; the second change handler will see the
+##            return value of the first change handler as the "new value".
 ##
-## priority: The priority of the function that was added; functions with higher priority are
+## priority: The priority of the function that was added; functions with higher
-##           called first, functions with the same priority are called in the order in which
+##           priority are called first, functions with the same priority are
-##           they were added.
+##           called in the order in which they were added.
 ##
 ## Returns: true when the change handler was set, false when an error occurred.
 ##