The load-duplicates.bro test would never fail because loading
the provided script code twice wouldn't trigger an error.
Fixed this by changing the sample script content. Also added a
test case to verify that an error occurs as expected
when two scripts with the same content are loaded.
* 'master' of https://github.com/rdenniston/zeek:
Add linux netfilter NFLOG capture functionality initial commit
I made modifications:
- Formatting / code style
- More error handling and validity checks
- The Type and Length value of TLVs is technically host order
- Changed / fixed the Length value padding check: it's generally
32-bit alignment, not just aligning any TLV less than 8 bytes.
* Better parsing/error-checking of VXLAN and encapsulated packet headers
* Add/implement the "vxlan_packet" event
* Add "Tunnel::vxlan_ports" option to tune the set of VXLAN ports to
analyze/decapsulate
* Add "Tunnel::validate_vxlan_checksums" option to allow for tuning of how
checksums associated with the outer UDP header of a possible VXLAN
tunnel are handled
Fixes GH-250
* origin/topic/jsiwek/gh-211:
GH-208: change invalid subnet expressions to a runtime error
GH-211: improve consistency of how scripting errors are handled
Removed the 'allow_init_errors' option.
Scripting errors/mistakes now consistently generate a runtime error
which have the behavior of unwinding the call stack all the way out of
the current event handler.
Before, such errors were not treated consistently and either aborted
the process entirely or emitted a message while continuing to execute
subsequent statements without well-defined behavior (possibly causing
a cascade of errors).
The previous behavior also would only unwind out of the current
function (if within a function body), not out the current event
handler, which is especially problematic for functions that return
a value: the caller is essentially left a mess with no way to deal
with it.
This also changes the behavior of the startup/initialization process
to abort if there's errors during bro_init() rather than continue one
to the main run loop. The `allow_init_errors` option may change this
new, default behavior.
Added check for the optional sub-header in ERSPAN Type III as well
as additional truncation checks to the GRE parsing logic in general.
Also added a unit test for ERSPAN Type II.
This fixes some use-after-free issues in the shutdown order of
various systems: I/O, plugin, logging, and reporter systems may
interact during shutdown if there's errors emitted (or maybe just
still pending) during the shutdown.
Fixes GH-188
* 'fatemabw/bro' of https://github.com/fatemabw/bro:
DNSSEC support in Bro
I made several changes:
- renamed event/record types
- reformatted the info added to dns.log
- removed the "addl" scripts that added extended dnssec info to dns.log
- simplifications/improvements to the internal parsing logic
This change allows a weird sampling rate of 0, which completely suppresses
all notifications (previously this crashed Bro). If also fixes the sampling
threshold to work with sampling rates of 0.
* origin/topic/jsiwek/empty-lines:
Add 'smtp_excessive_pending_cmds' weird
Fix SMTP command string comparisons
Improve handling of empty lines in several text protocol analyzers
Add rate-limiting sampling mechanism for weird events
Teach timestamp canonifier about timestamps before ~2001
* origin/topic/vern/vec-append:
d'oh, still have a (deprecated) string_array rather than string_vector
forgot to update test suite results for v += e
reap the fruits of v += e
test case for v += e
documentation of v += e
v += e implemented
Fixed a mistake in find_ip_addresses()
* origin/topic/vern/set-ops2:
documentation, test suite update
implemented set relationals
bug fix for set intersection
set intersection implemented
mirroring previous topic/vern/set-ops to get branch up to date, since I'm a n00b
Fixed a couple memory leaks and added a leak test
The generation of weird events, by default, are now rate-limited
according to these tunable options:
- Weird::sampling_whitelist
- Weird::sampling_threshold
- Weird::sampling_rate
- Weird::sampling_duration
The new get_reporter_stats() BIF also allows one to query the
total number of weirds generated (pre-sampling) which the new
policy/misc/weird-stats.bro script uses periodically to populate
a weird_stats.log.
There's also new reporter BIFs to allow generating weirds from the
script-layer such that they go through the same, internal
rate-limiting/sampling mechanisms:
- Reporter::conn_weird
- Reporter::flow_weird
- Reporter::net_weird
Some of the code was adapted from previous work by Johanna Amann.
In some cases one can get the Type() of unaryexpr to be ANY. Vectors so
far did not deal gracefully with this and crashed because trying to
convert any to a vectortype.
This patch fixes this by just using the original vector-type in this
case.
* origin/topic/vern/case-insensitive-patterns:
use PCRE syntax instead of the beautiful new (?i ...) syntax
nitlet in NEWS entry
test suite update for case-insensitive patterns
document use of double quotes to escape case-insensitivity
bug fix for recent memory leak patch
documentation updates for case-insensitive patterns
d'oh there's isalpha. I looked earlier for isletter :-P
fix for handling [:(lower|upper):] in case-insensitive patterns
implemented /re/i for case-insensitive patterns
Mostly trying to standardize the way tests sleep for arbitrary amounts
of time to make it easier to tell at which particular point the
unit test actually may need the timeout interval increased (or else
debugged further).
* 'Reporter/MessageFix' of https://github.com/catenacyber/bro:
Better reporter for Brostring with embedded NUL
I slightly changed the code for beautification purposes and added a
testcase. No functional changes.
The configuration framework consists of three mostly distinct parts:
* option variables
* the config reader
* the script level framework
I will describe the three elements in the following.
Internally, this commit also performs a range of changes to the Input
manager; it marks a lot of functions as const and introduces a new
ValueToVal method (which could in theory replace the already existing
one - it is a bit more powerful).
This also changes SerialTypes to have a subtype for Values, just as
Fields already have it; I think it was mostly an oversight that this was
not introduced from the beginning. This should not necessitate any code
changes for people already using SerialTypes.
option variable
===============
The option keyword allows variables to be specified as run-tine options.
Such variables cannot be changed using normal assignments. Instead, they
can be changed using Option::set. It is possible to "subscribe" to
options and be notified when an option value changes.
Change handlers can also change values before they are applied; this
gives them the opportunity to reject changes. Priorities can be
specified if there are several handlers for one option.
Example script:
option testbool: bool = T;
function option_changed(ID: string, new_value: bool): bool
{
print fmt("Value of %s changed from %s to %s", ID, testbool, new_value);
return new_value;
}
event bro_init()
{
print "Old value", testbool;
Option::set_change_handler("testbool", option_changed);
Option::set("testbool", F);
print "New value", testbool;
}
config reader
=============
The config reader provides a way to read configuration files back into
Bro. Most importantly it automatically converts values to the correct
types. This is important because it is at least inconvenient (and
sometimes near impossible) to perform the necessary type conversions in
Bro scripts themselves. This is especially true for sets/vectors.
Configuration generally look like this:
[option name][tab/spaces][new variable value]
so, for example:
testaddr 2607:f8b0:4005:801::200e
testinterval 60
testtime 1507321987
test_set a b c d erdbeerschnitzel
The reader uses the option name to look up the type that variable has in
the Bro core and automatically converts the value to the correct type.
Example script use:
type Idx: record {
option_name: string;
};
type Val: record {
option_val: string;
};
global currconfig: table[string] of string = table();
event InputConfig::new_value(name: string, source: string, id: string, value: any)
{
print id, value;
}
event bro_init()
{
Input::add_table([$reader=Input::READER_CONFIG, $source="../configfile", $name="configuration", $idx=Idx, $val=Val, $destination=currconfig, $want_record=F]);
}
Script-level config framework
=============================
The script-level framework ties these two features together and makes
them a bit more convenient to use. Configuration files can simply be
specified by placing them into Config::config_files. The framework also
creates a config.log that shows all value changes that took place.
Usage example:
redef Config::config_files += {configfile};
export {
option testbool : bool = F;
}
The file is now monitored for changes; when a change occurs the
respective option values are automatically updated and the value change
is written to config.log.
handlers.
It's well known that changes to mutable event arguments, like tables,
become visible to all places where those values are used, including
subsequent handlers of the same event. However, there's a related case
that's more suprising: simply assigning *a new value* to an event
argument passes through, too. This commit fixes that behaviour. (We
even had a btest with a baseline reflecting the problen).
