From c759583d116b279d8b66269b48101a47d4fdc32d Mon Sep 17 00:00:00 2001 From: Daniel Thayer Date: Thu, 15 Mar 2018 14:13:54 -0500 Subject: [PATCH 1/4] Fix minor typos and doc build warnings in NEWS --- NEWS | 49 ++++++++++++++++++++++++++----------------------- 1 file changed, 26 insertions(+), 23 deletions(-) diff --git a/NEWS b/NEWS index 4e518a4261..aa43682047 100644 --- a/NEWS +++ b/NEWS @@ -20,14 +20,14 @@ New Functionality using normal assignments. Instead, they can be changed using the new function Option::set. - It is possible to "subscribe" to option through + It is possible to "subscribe" to an option through Option::set_change_handler, which will trigger a handler callback when an option changes. Change handlers can optionally modify values before they are applied by returning the desired value, or reject updates by returning the old value. Priorities can be specified if there are several handlers for one option. - Example script: + Example script:: option testbool: bool = T; @@ -55,7 +55,7 @@ New Functionality Configuration files to read can be specified by adding them to Config::config_files. - Usage example: + Usage example:: redef Config::config_files += { "/path/to/config.dat" }; @@ -65,38 +65,38 @@ New Functionality option testbool: bool = F; } - The specified file will now monitored contionoulsy for changes, so + The specified file will now be monitored continuously for changes, so that writing "testbool T" into /path/to/config.dat will automatically update the option's value accordingly. - The configutation framework creates a config.log that shows all + The configuration framework creates a config.log that shows all value changes that took place. - - Config reader: Internally, the configuration framework use a new + - Config reader: Internally, the configuration framework uses a new type of input reader to read such configuration files into Bro. The reader uses the option name to look up the type that variable has, converts the read value to the correct type, and then updates - the option's value. Example script use: + the option's value. Example script use:: - type Idx: record { - option_name: string; - }; + type Idx: record { + option_name: string; + }; - type Val: record { - option_val: string; - }; + type Val: record { + option_val: string; + }; - global currconfig: table[string] of string = table(); + global currconfig: table[string] of string = table(); - event InputConfig::new_value(name: string, source: string, id: string, value: any) - { - print id, value; - } + event InputConfig::new_value(name: string, source: string, id: string, value: any) + { + print id, value; + } - event bro_init() - { - Input::add_table([$reader=Input::READER_CONFIG, $source="../configfile", $name="configuration", $idx=Idx, $val=Val, $destination=currconfig, $want_record=F]); - } + event bro_init() + { + Input::add_table([$reader=Input::READER_CONFIG, $source="../configfile", $name="configuration", $idx=Idx, $val=Val, $destination=currconfig, $want_record=F]); + } - Support for OCSP and Signed Certificate Timestamp. This adds the following events and BIFs: @@ -173,12 +173,15 @@ Changed Functionality the default configuration of logs, this field will show "-" instead of "(empty)" for connections that lack any tunelling. -- SMB event argument changes +- SMB event argument changes: + - smb1_transaction_request now has two additional arguments, "parameters" and "data" strings + - smb1_transaction2_request now has an additional "args" record argument - SSL event argument changes: + - event ssl_server_signature now has an additional argument "signature_and_hashalgorithm". From f3e42874b80ebfe86e7ca7db228ec1ffa34d4445 Mon Sep 17 00:00:00 2001 From: Daniel Thayer Date: Thu, 15 Mar 2018 14:16:00 -0500 Subject: [PATCH 2/4] Improve config framework documentation comments Fixed typos and formatting. --- scripts/base/frameworks/config/README | 2 +- scripts/base/frameworks/config/main.bro | 4 +- .../base/frameworks/input/readers/config.bro | 6 +-- src/option.bif | 43 ++++++++++--------- 4 files changed, 28 insertions(+), 27 deletions(-) diff --git a/scripts/base/frameworks/config/README b/scripts/base/frameworks/config/README index 95193b49a3..3640d1e8c4 100644 --- a/scripts/base/frameworks/config/README +++ b/scripts/base/frameworks/config/README @@ -1,2 +1,2 @@ -The configuration famework provides a way to change the Bro configuration +The configuration framework provides a way to change the Bro configuration in "option" values at run-time. diff --git a/scripts/base/frameworks/config/main.bro b/scripts/base/frameworks/config/main.bro index 372bb10eb4..ffd9e764ef 100644 --- a/scripts/base/frameworks/config/main.bro +++ b/scripts/base/frameworks/config/main.bro @@ -1,6 +1,6 @@ ##! The configuration framework provides a way to change Bro options -##! (as specified by the option keyword) at runtime. It also logs runtime changes -##! to options to config.log. +##! (as specified by the "option" keyword) at runtime. It also logs runtime +##! changes to options to config.log. module Config; diff --git a/scripts/base/frameworks/input/readers/config.bro b/scripts/base/frameworks/input/readers/config.bro index 166228e81f..0d334b9b65 100644 --- a/scripts/base/frameworks/input/readers/config.bro +++ b/scripts/base/frameworks/input/readers/config.bro @@ -33,12 +33,12 @@ export { ## and also does not track removals. If you need this, combine the event ## with a table reader. ## - ## name: name of the input stream. + ## name: Name of the input stream. ## - ## source: source of the input stream. + ## source: Source of the input stream. ## ## id: ID of the configuration option being set. ## - ## value: new value of the configuration option being set. + ## value: New value of the configuration option being set. global new_value: event(name: string, source: string, id: string, value: any); } diff --git a/src/option.bif b/src/option.bif index 59f4a0fec4..011db2f069 100644 --- a/src/option.bif +++ b/src/option.bif @@ -1,6 +1,5 @@ -##! The option built-in functions allow the scripting layer to -##! change the value of option-values and to be notified when -##! option values change. +##! Definitions of built-in functions that allow the scripting layer to +##! change the value of options and to be notified when option values change. module Option; @@ -8,16 +7,16 @@ module Option; #include "NetVar.h" %%} -## Sets an option to a new value. This change will also cause the option change handlers -## to be called. +## Set an option to a new value. This change will also cause the option change +## handlers to be called. ## ## ID: The ID of the option to update. ## ## val: The new value of the option. ## -## location: optional parameter detailing where this change originated from. +## location: Optional parameter detailing where this change originated from. ## -## Returns: true on success, false when an error occured. +## Returns: true on success, false when an error occurred. ## ## .. bro:see:: Option::set_change_handler function Option::set%(ID: string, val: any, location: string &default=""%): bool @@ -74,24 +73,26 @@ function Option::set%(ID: string, val: any, location: string &default=""%): bool return new Val(1, TYPE_BOOL); %} -## Set the change handler for the option *ID*. The change handler will be called anytime -## :bro:id:`Option::set` is called fot *ID*. +## Set a change handler for an option. The change handler will be +## called anytime :bro:id:`Option::set` is called for the option. ## ## ID: The ID of the option for which change notifications are desired. ## -## on_change: The function that will be called when a change occurs. The function can choose to -## receive two or three parameters: the first parameter is a string containing *ID*, -## the second parameter is the new option value. The third, optional, parameter is the -## location string as passed to Option::set. Note that the global value is not yet changed -## when the function is called. The passed function has to return the new value that -## it wants the option to be set to. This enables it to reject changes, or change values -## that are being set. When several change handlers are set for an option they are chained; -## the second change handler will see the return value of the first change handler as the -## "new value". +## on_change: The function that will be called when a change occurs. The +## function can choose to receive two or three parameters: the first +## parameter is a string containing *ID*, the second parameter is +## the new option value. The third, optional, parameter is the +## location string as passed to Option::set. Note that the global +## value is not yet changed when the function is called. The passed +## function has to return the new value that it wants the option to +## be set to. This enables it to reject changes, or change values +## that are being set. When several change handlers are set for an +## option they are chained; the second change handler will see the +## return value of the first change handler as the "new value". ## -## priority: The priority of the function that was added; functions with higher priority are -## called first, functions with the same priority are called in the order in which -## they were added. +## priority: The priority of the function that was added; functions with higher +## priority are called first, functions with the same priority are +## called in the order in which they were added. ## ## Returns: true when the change handler was set, false when an error occurred. ## From 79afd99229fc5abc19fbe973271778e424ae4613 Mon Sep 17 00:00:00 2001 From: Daniel Thayer Date: Thu, 15 Mar 2018 14:29:26 -0500 Subject: [PATCH 3/4] Add documentation of "option" declarations --- doc/script-reference/statements.rst | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/doc/script-reference/statements.rst b/doc/script-reference/statements.rst index 14e0cc3c32..963d4dc7a2 100644 --- a/doc/script-reference/statements.rst +++ b/doc/script-reference/statements.rst @@ -20,6 +20,9 @@ Declarations +----------------------------+-----------------------------+ | :bro:keyword:`const` | Declare a constant | +----------------------------+-----------------------------+ +| :bro:keyword:`option` | Declare a configuration | +| | option | ++----------------------------+-----------------------------+ | :bro:keyword:`type` | Declare a user-defined type | +----------------------------+-----------------------------+ | :bro:keyword:`redef` | Redefine a global value or | @@ -176,6 +179,25 @@ all loaded Bro scripts. or "global" keywords (i.e., "const" replaces "local" and "global"). +.. bro:keyword:: option + + A variable declared with the "option" keyword is a configuration option. + + Options are required to be initialized at the + time of declaration. Normally, the type is inferred from the initializer, + but the type can be explicitly specified. Example:: + + option hostname = "host-1"; + option peers: set[addr] = {}; + + The value of an option cannot be changed by an assignment statement. + + The scope of an option is global. + + Note that an "option" declaration cannot also use the "local", "global", + or "const" keywords. + + .. bro:keyword:: type The "type" keyword is used to declare a user-defined type. The name From 8aeedba066877633e9bf18576ab6c6e5130df1ca Mon Sep 17 00:00:00 2001 From: Daniel Thayer Date: Thu, 15 Mar 2018 14:50:42 -0500 Subject: [PATCH 4/4] Add documentation of the configuration framework --- doc/frameworks/configuration.rst | 156 +++++++++++++++++++++++++++++++ doc/frameworks/index.rst | 1 + 2 files changed, 157 insertions(+) create mode 100644 doc/frameworks/configuration.rst diff --git a/doc/frameworks/configuration.rst b/doc/frameworks/configuration.rst new file mode 100644 index 0000000000..fba8b46f5f --- /dev/null +++ b/doc/frameworks/configuration.rst @@ -0,0 +1,156 @@ + +.. _framework-configuration: + +======================= +Configuration Framework +======================= + +.. rst-class:: opening + +Bro now includes a "configuration framework" that allows +updating script options dynamically at runtime. This functionality +consists of several components: an "option" declaration, the +ability to specify input files to enable changing the value of options at +runtime, a couple of built-in functions, and a log file "config.log" +which contains information about every change to option values. + + +.. contents:: + + +Introduction +------------ + +The configuration framework provides an alternative to using Bro script +constants to store various Bro settings. In general, constants can be used +when a value is not expected to change at runtime, but they cannot be used +for values that need to be modified occasionally. + +A "redef" allows the re-definition of an already defined constant in Bro. +For example, to modify the constant "Site::local_nets", you can use code +similar to this: + + redef Site::local_nets += { 10.1.0.0/16 }; + +A disadvantage of redefs is that these redefinitions can only be +performed when Bro first starts. Afterwards, Site::local_nets is just a +normal constant and can no longer be modified. + +However, it is clearly desirable to be able to change at runtime many of the +configuration options that Bro offers. Having to restart Bro can be +time-consuming and causes Bro to lose all connection state and knowledge +that it accumulated. To solve this problem, the Bro configuration framework +was created, which allows changing configuration options at runtime. + + +Declaring options +----------------- + +The "option" keyword allows variables to be declared as configuration options. + +.. code:: bro + + module TestModule; + + export { + option my_networks: set[subnet] = {}; + option enable_feature = F; + option hostname = "testsystem"; + } + +The rules regarding options can be thought of as being in between global +variables and constants. Like global variables, options cannot be declared +inside a function, hook, or event handler. Like constants, options must be +initialized when declared. The value of an option can change at runtime, +but options cannot be assigned a new value using normal assignments. + + +Changing options +---------------- + +The configuration framework facilitates reading in new option values +from external files at runtime. + +Configuration files contain a mapping between option names and their values. +The format for these files looks like this: + + [option name][tab/spaces][new value] + +Configuration files can be specified by adding them to Config::config_files. +For example, simply add something like this to local.bro: + +.. code:: bro + + redef Config::config_files += { "/path/to/config.dat" }; + +The specified configuration file will be monitored continuously for changes, +so that writing "TestModule::enable_feature T" into that file will +automatically update the option's value accordingly. Here is an example +configuration file:: + + TestModule::my_networks 10.0.12.0/24,192.168.17.0/24 + TestModule::enable_feature T + TestModule::hostname host-1 + +Internally, the configuration framework uses the Bro input framework +with a type of input reader specifically for reading config files. Users +familiar with the Bro input framework might be aware that the input framework +is usually very strict about the syntax of input files. This is not true +for configuration files: the files need no header lines and either +tabs or spaces are accepted as separators. + +If you inspect the configuration framework scripts, you will notice that the +scripts simply catch events from the input framework and then a built-in +function :bro:see:`Option::set` is called to set an option to the new value. +If you want to change an option yourself during runtime, you can +call Option::set directly from a script. + +The log file "config.log" contains information about each configuration +change that occurs during runtime. + + +Change handlers +--------------- + +A change handler is a user-defined function that is called automatically +each time an option value changes. This example shows how to register a +change handler for an option that has a data type of "addr" (for other +data types, the return type and 2nd parameter data type must be adjusted +accordingly): + +.. code:: bro + + option testaddr = 127.0.0.1; + + # Note: the data type of 2nd parameter and return type must match + function change_addr(ID: string, new_value: addr): addr + { + print fmt("Value of %s changed from %s to %s", ID, testaddr, new_value); + return new_value; + } + + event bro_init() + { + Option::set_change_handler("testaddr", change_addr); + } + +Each time the specified option value is changed, the change handler +function will be called before the change is performed. The value returned +by the change handler is the value finally assigned to the option. This +allows, for example, checking of values to reject invalid input (the original +value can be returned to reject the change). + +A change handler can optionally have a third argument, which is the location +string (this is normally the pathname of the configuration file that triggered +the change). + +It is also possible to chain together multiple change handlers. In this +case, the value returned by the first change handler is the "new value" seen +by the next change handler, and so on. The built-in function +:bro:see:`Option::set_change_handler` takes an optional third argument +that can specify a priority for the handlers. + +Note that change handlers are also used internally by the +configuration framework. If you look at the script level source code of +the config framework, you can see that change handlers are used for +logging the option changes to config.log. diff --git a/doc/frameworks/index.rst b/doc/frameworks/index.rst index 4f87df3b53..ab46939c38 100644 --- a/doc/frameworks/index.rst +++ b/doc/frameworks/index.rst @@ -6,6 +6,7 @@ Frameworks .. toctree:: :maxdepth: 1 + configuration file-analysis geoip input