The configuration framework consists of three mostly distinct parts:
* option variables
* the config reader
* the script level framework
I will describe the three elements in the following.
Internally, this commit also performs a range of changes to the Input
manager; it marks a lot of functions as const and introduces a new
ValueToVal method (which could in theory replace the already existing
one - it is a bit more powerful).
This also changes SerialTypes to have a subtype for Values, just as
Fields already have it; I think it was mostly an oversight that this was
not introduced from the beginning. This should not necessitate any code
changes for people already using SerialTypes.
option variable
===============
The option keyword allows variables to be specified as run-tine options.
Such variables cannot be changed using normal assignments. Instead, they
can be changed using Option::set. It is possible to "subscribe" to
options and be notified when an option value changes.
Change handlers can also change values before they are applied; this
gives them the opportunity to reject changes. Priorities can be
specified if there are several handlers for one option.
Example script:
option testbool: bool = T;
function option_changed(ID: string, new_value: bool): bool
{
print fmt("Value of %s changed from %s to %s", ID, testbool, new_value);
return new_value;
}
event bro_init()
{
print "Old value", testbool;
Option::set_change_handler("testbool", option_changed);
Option::set("testbool", F);
print "New value", testbool;
}
config reader
=============
The config reader provides a way to read configuration files back into
Bro. Most importantly it automatically converts values to the correct
types. This is important because it is at least inconvenient (and
sometimes near impossible) to perform the necessary type conversions in
Bro scripts themselves. This is especially true for sets/vectors.
Configuration generally look like this:
[option name][tab/spaces][new variable value]
so, for example:
testaddr 2607:f8b0:4005:801::200e
testinterval 60
testtime 1507321987
test_set a b c d erdbeerschnitzel
The reader uses the option name to look up the type that variable has in
the Bro core and automatically converts the value to the correct type.
Example script use:
type Idx: record {
option_name: string;
};
type Val: record {
option_val: string;
};
global currconfig: table[string] of string = table();
event InputConfig::new_value(name: string, source: string, id: string, value: any)
{
print id, value;
}
event bro_init()
{
Input::add_table([$reader=Input::READER_CONFIG, $source="../configfile", $name="configuration", $idx=Idx, $val=Val, $destination=currconfig, $want_record=F]);
}
Script-level config framework
=============================
The script-level framework ties these two features together and makes
them a bit more convenient to use. Configuration files can simply be
specified by placing them into Config::config_files. The framework also
creates a config.log that shows all value changes that took place.
Usage example:
redef Config::config_files += {configfile};
export {
option testbool : bool = F;
}
The file is now monitored for changes; when a change occurs the
respective option values are automatically updated and the value change
is written to config.log.
These were caused by earlier code updates.
This commit also removes the ocsp stapling logging script; it was
defunctional and the information provided by it wnever really has been
especially interesting.
The linker was complaining about linking files that didn't
have any symbols. These were actually empty files so I just
got rid of them and removed references to them.
* origin/topic/seth/smb-auth-fixes:
Disable SMB2 error data parsing.
Removed some files that aren't being used.
Fix SMB tree connect handling.
Fix a small issue where DCE_RPC commands were improperly being logged.
SMB fixes and cleanup.
Including a test for raw NTLM in SMB
Updates for SMB auth handling from Martin van Hensbergen.
BIT-1721 #merged
One more name change: version_num -> version_number
BIT-1707 #merged
* origin/topic/johanna/version:
Slight naming changes for Bro version information.
Add convenient way to access version information to Bro.
ninja said:
ninja: warning: multiple rules generate
scripts/base/bif/const.bif.bro. builds involving this target will
not be correct; continuing anyway [-w dupbuild=warn]
Looks like there's a larger problem here involving *.bif of the same
name at different locations of the source tree. For now, I'ved fixed
this one by merging src/iosource/pcap/{const,functions}.bif into
pcap.bif.
File Analysis Framework related code has been moved into a separate
script. Using redefinitions of the corresponding records causes the
file-related columns to appear last.
We now extract email addresses in the fields that one would expect
to contain addresses. This makes further downstream processing of
these fields easier like log analysis or using these fields in the
Intel framework. The primary downside is that any other content
in these fields is no longer available such as full name and any
group information. I believe the simplification of the content in
these fields is worth the change.
Added "cc" to the script that feeds information from SMTP into the
Intel framework.
A new script for email handling utility functions has been created
as a side effect of these changes.
(Cleaned up some code a little bit.)
* origin/topic/seth/stats-improvement:
Fixing tests for stats improvements
Rename the reporting interval variable for stats.
Removing more broken functionality due to changed stats apis.
Removing some references to resource_usage()
Removing Broker stats, it was broken and incomplete.
Fixing default stats collection interval to every 5 minutes.
Add DNS stats to the stats.log
Small stats script tweaks and beginning broker stats.
Continued stats cleanup and extension.
More stats collection extensions.
More stats improvements
Slight change to Mach API for collecting memory usage.
Fixing some small mistakes.
Updating the cmake submodule for the stats updates.
Fix memory usage collection on Mac OS X.
Cleaned up stats collection.
BIT-1581 #merged
BIT-1449 #merged
* origin/topic/dnthayer/ticket1449:
Rename broker BIF wrapper functions in a few more places
Sync the core/leaks/broker/data.bro test with broker/data.bro
Add missing tests for broker data BIFs
Code cleanup for some broker tests
Add script wrapper functions for broker data BIFs
Add script wrapper functions for broker BIFs
Also renamed the "print" function to "send_print" and the "event"
function to "send_event" because Bro shows a syntax error when a
Bro script function is named "event" or "print".
* origin/topic/dnthayer/broker-namespace:
Split the broker main.bro into two scripts
Rename the BrokerStore namespace to Broker
Rename the BrokerComm namespace to Broker
BIT-1563 #merged
* topic/seth/file-entropy:
Add a file entropy test.
Fixing a test.
Updated tests for file entropy analyzer.
Update and clean up to file entropy measurement.
First commit of file entropy analyzer.
* martin/topic/fox/rfb:
Fixed issue in state machine
Some styling tweaks
Implement protocol confirmation
Analyzer and bro script for RFB protocol (VNC)
* <seth> I also applied a bit of clean up to the base
script to make it match other scripts better and
updated tests.
Separate the former BrokerComm and BrokerStore portions of the script
because these files will be much larger when script wrappers for BIFs are
written.
I reworked this a bit:
- Moved the globals into a new Pcap::* namespace, and renamed them
slightly.
- Moved the definitions of the globals into pcap/const.bif.
- Also moved the existing 'snaplen' into Pcap::* and removed
SnapLen() from the PktSrc API (it's really a pcap thing).
- Likewise moved the existing functions precompile_pcap_filter,
install_pcap_filter, and pcap_error, into Pcap::*.
- Did some more refactoring for the pcap code.
* 'master' of https://github.com/knielander/bro:
Refactored patch (removed options, less ambiguous name)
Allow Bro to run in fanout mode.
Allow libpcap buffer size to be set manually.
Allow Bro to run in fanout mode.
Allowed libpcap buffer size to be set via configuration.
* origin/topic/vladg/sip:
Update NEWS.
Update baselines.
Spruce up SIP events.bif documentation a bit.
Register SIP analyzer to well known port.
Fix indenting issue in main.bro
Add SIP btests.
Small update for the SIP logs and DPD sig.
SIP: Fix up DPD and the TCP analyzer a bit.
SIP: Move to the new string BIFs
SIP: Move to new analyzer format.
Move the SIP analyzer to uint64 sequences, and a number of other small SIP fixes.
Rely on content inspection and not just is_orig to determine client/server.
Enable SIP in CMakeLists.txt
Merge topic/seth/faf-updates.
BIT-1370 #merged
* origin/topic/vladg/kerberos: (27 commits)
Add Kerberos to NEWS.
Add Kerberos memleak btest.
Add Kerberos analyzer btest.
Update baselines for Kerberos analyzer.
Add known ports to krb/main.bro
KRB: Clean up krb.log a bit.
Kerberos: Remove debugging output.
Kerberos: Fix a memleak.
Kerberos: A couple small tweaks.
Kerberos: Fix parsing of the cipher in tickets, and add it to the log.
Kerberos: A couple more formatting fixes.
Change krb Info string to success bool
Clean up formatting.
Documentation update, and rework events a bit.
Add support for the SAFE message type.
Add support for AP_REQ, AP_REP, PRIV, and CRED message types.
Fix parsing error for KRB_Ticket_Sequence
Continue clean-up. Some reformatting, removing hard-coded values, documentation, etc.
Kerberos analyzer updates: - Split up the (quite length) krb-protocol.pac into krb-protocol, krb-defs, krb-types and krb-padata - Add some supporting types to get rid of awkward and difficult to read case true/false statements - Clean up the conversion code in krb-analyzer.pac
Improve Kerberos DPD and fix a few parse errors.
...
BIT-1369 #merged
Since the command-line option for reading NetFlow went away, the has
been neither used nor tested anymore. We might bring this back later,
but for now I'd rather remove it than having dead code that seems to
suggest that we support it.
* origin/topic/seth/rdp: (31 commits)
Improved transition into SSL/TLS from RDP.
Fixes tests in RDP branch.
add a special case to the X509 code that deals with RDP certificates.
A few more changes to handling encryption in RDP.
Adds some comments and fixes a broxygen warning.
Fixes another optional part of an RDP unit.
Support RDP negotiation requests optionally and support zero length cookies.
Changed UTF-16 to UTF-8 conversion to be more lenient.
Fixed an issue with parse failure on an optional field.
Removing a stray printf from RDP analyzer.
Another big RDP update.
New script to add a field to rdp.log when the connection is upgraded to SSL.
Huge updates to the RDP analyzer from Josh Liburdi.
FreeRDP test trace showing SSL encryption -- RDP analyzer does not currently handle this and SSL analyzer does not identify it either
Wireshark test trace for native encryption -- generates a binpac error
Delete RDP-004.pcap
Delete nla_win7_win2k8r2.pcap
Update dpd.sig
Fixed typo
Added check for connection existence
...
BIT-1340 #merged
