Add config framework.

The configuration framework consists of three mostly distinct parts:

* option variables
* the config reader
* the script level framework

I will describe the three elements in the following.

Internally, this commit also performs a range of changes to the Input
manager; it marks a lot of functions as const and introduces a new
ValueToVal method (which could in theory replace the already existing
one - it is a bit more powerful).

This also changes SerialTypes to have a subtype for Values, just as
Fields already have it; I think it was mostly an oversight that this was
not introduced from the beginning. This should not necessitate any code
changes for people already using SerialTypes.

option variable
===============

The option keyword allows variables to be specified as run-tine options.
Such variables cannot be changed using normal assignments. Instead, they
can be changed using Option::set. It is possible to "subscribe" to
options and be notified when an option value changes.

Change handlers can also change values before they are applied; this
gives them the opportunity to reject changes. Priorities can be
specified if there are several handlers for one option.

Example script:

option testbool: bool = T;

function option_changed(ID: string, new_value: bool): bool
  {
  print fmt("Value of %s changed from %s to %s", ID, testbool, new_value);
  return new_value;
  }

event bro_init()
  {
  print "Old value", testbool;
  Option::set_change_handler("testbool", option_changed);
  Option::set("testbool", F);
  print "New value", testbool;
  }

config reader
=============

The config reader provides a way to read configuration files back into
Bro. Most importantly it automatically converts values to the correct
types. This is important because it is at least inconvenient (and
sometimes near impossible) to perform the necessary type conversions in
Bro scripts themselves. This is especially true for sets/vectors.

Configuration generally look like this:

[option name][tab/spaces][new variable value]

so, for example:

testaddr 2607:f8b0:4005:801::200e
testinterval 60
testtime 1507321987
test_set a	b	c	d	erdbeerschnitzel

The reader uses the option name to look up the type that variable has in
the Bro core and automatically converts the value to the correct type.

Example script use:

type Idx: record {
  option_name: string;
};

type Val: record {
  option_val: string;
};

global currconfig: table[string] of string = table();

event InputConfig::new_value(name: string, source: string, id: string, value: any)
  {
  print id, value;
  }

event bro_init()
  {
  Input::add_table([$reader=Input::READER_CONFIG, $source="../configfile", $name="configuration", $idx=Idx, $val=Val, $destination=currconfig, $want_record=F]);
  }

Script-level config framework
=============================

The script-level framework ties these two features together and makes
them a bit more convenient to use. Configuration files can simply be
specified by placing them into Config::config_files. The framework also
creates a config.log that shows all value changes that took place.

Usage example:

redef Config::config_files += {configfile};

export {
  option testbool : bool = F;
}

The file is now monitored for changes; when a change occurs the
respective option values are automatically updated and the value change
is written to config.log.

testing/btest/scripts/base/frameworks/input/basic.bro

@@ -7,9 +7,9 @@ redef exit_only_after_terminate = T;
 @TEST-START-FILE input.log
 #separator \x09
 #path	ssh
-#fields	b	i	e	c	p	sn	a	d	t	iv	s	sc	ss	se	vc	ve	ns
-#types	bool	int	enum	count	port	subnet	addr	double	time	interval	string	table	table	table	vector	vector	string
-T	-42	SSH::LOG	21	123	10.0.0.0/24	1.2.3.4	3.14	1315801931.273616	100.000000	hurz	2,4,1,3	CC,AA,BB	EMPTY	10,20,30	EMPTY	4242
+#fields	b	i	e	c	p	pp	sn	a	d	t	iv	s	sc	ss	se	vc	ve	ns
+#types	bool	int	enum	count	port	port	subnet	addr	double	time	interval	string	table	table	table	vector	vector	string
+T	-42	SSH::LOG	21	123	5/icmp	10.0.0.0/24	1.2.3.4	3.14	1315801931.273616	100.000000	hurz	2,4,1,3	CC,AA,BB	EMPTY	10,20,30	EMPTY	4242
 @TEST-END-FILE
 
 @load base/protocols/ssh
@@ -29,6 +29,7 @@ type Val: record {
 	e: Log::ID;
 	c: count;
 	p: port;
+	pp: port;
 	sn: subnet;
 	a: addr;
 	d: double;

testing/btest/scripts/base/frameworks/input/config/basic.bro

@@ -0,0 +1,75 @@
+# @TEST-EXEC: btest-bg-run bro bro -b %INPUT
+# @TEST-EXEC: btest-bg-wait 10
+# @TEST-EXEC: btest-diff out
+
+redef exit_only_after_terminate = T;
+redef InputConfig::empty_field = "EMPTY";
+redef InputConfig::set_separator = "\t";
+
+@TEST-START-FILE configfile
+testbool F
+testcount    1
+testcount 2
+testcount 2
+testint		-1
+testenum Conn::LOG
+testport 45
+testportandproto 45/udp
+testaddr 127.0.0.1
+testaddr 2607:f8b0:4005:801::200e
+testinterval 60
+testtime 1507321987
+test_set a	b	c	d	erdbeerschnitzel
+test_vector 1	2	3	4	5	6
+test_set (empty)
+test_set EMPTY
+test_set -
+@TEST-END-FILE
+
+@load base/protocols/ssh
+@load base/protocols/conn
+
+global outfile: file;
+
+export {
+	option testbool: bool = T;
+	option testcount: count = 0;
+	option testint: int = 0;
+	option testenum = SSH::LOG;
+	option testport = 42/tcp;
+	option testportandproto = 42/tcp;
+	option testaddr = 127.0.0.1;
+	option testtime = network_time();
+	option testinterval = 1sec;
+	option teststring = "a";
+	option test_set: set[string] = {};
+	option test_vector: vector of count = {};
+}
+
+type Idx: record {
+	option_name: string;
+};
+
+type Val: record {
+	option_val: string;
+};
+
+global currconfig: table[string] of string = table();
+
+event InputConfig::new_value(name: string, source: string, id: string, value: any)
+	{
+	print outfile, id, value;
+	}
+
+event Input::end_of_data(name: string, source:string)
+	{
+	close(outfile);
+	terminate();
+	}
+
+event bro_init()
+	{
+	outfile = open("../out");
+	Input::add_table([$reader=Input::READER_CONFIG, $source="../configfile", $name="configuration", $idx=Idx, $val=Val, $destination=currconfig, $want_record=F]);
+	}
+

testing/btest/scripts/base/frameworks/input/config/errors.bro

@@ -0,0 +1,66 @@
+# @TEST-EXEC: btest-bg-run bro bro -b %INPUT
+# @TEST-EXEC: btest-bg-wait 10
+# @TEST-EXEC: tail -n +2 .stderr > errout
+# @TEST-EXEC: btest-diff errout
+
+redef exit_only_after_terminate = T;
+
+@TEST-START-FILE configfile
+testbool A
+testtesttesttesttesttest
+testbool A B
+testcount A
+testenum unknown
+testbooool T
+test_any F
+test_table whatever
+@TEST-END-FILE
+
+@load base/protocols/ssh
+@load base/protocols/conn
+
+global outfile: file;
+
+export {
+	option testbool: bool = T;
+	option testcount: count = 0;
+	option testint: int = 0;
+	option testenum = SSH::LOG;
+	option testport = 42/tcp;
+	option testaddr = 127.0.0.1;
+	option testtime = network_time();
+	option testinterval = 1sec;
+	option teststring = "a";
+	option test_set: set[string] = {};
+	option test_vector: vector of count = {};
+	option test_any: any = 5;
+	option test_table: table[string] of string = {};
+}
+
+type Idx: record {
+	option_name: string;
+};
+
+type Val: record {
+	option_val: string;
+};
+
+global currconfig: table[string] of string = table();
+
+event InputConfig::new_value(name: string, source: string, id: string, value: any)
+	{
+	print outfile, id, value;
+	}
+
+event Input::end_of_data(name: string, source:string)
+	{
+	close(outfile);
+	terminate();
+	}
+
+event bro_init()
+	{
+	outfile = open("../out");
+	Input::add_table([$reader=Input::READER_CONFIG, $source="../configfile", $name="configuration", $idx=Idx, $val=Val, $destination=currconfig, $want_record=F]);
+	}
+