These have been lingering for a while and they generally annoy
everyone because of the sheer volume. They also don't really add
any useful information for debugging and they were generated differently
than most other weirds anyway (which was a little weird...).
This introduces a new option, SOCKS::default_capture_password which can
be used to specify if Socks passwords are logged by default
Like fot FTP/HTTP, this option is set to false by default.
Addresses BIT-1791
Now we only parse the SignatureAndHashalgorithm field in cases where it
is present. This change also takes care to respect SCTs, which do
include the SignatureAndHashalgorithm in their digitally-signed struct,
even when used in protocol versions that do not have the
SignatureAndHashalgorithm in the protocols digitally-signed struct.
I also added tests to make sure this does indeed work with TLS 1.1 - it
turns out that so far we did not have a single TLS 1.1 pcap.
The configuration framework consists of three mostly distinct parts:
* option variables
* the config reader
* the script level framework
I will describe the three elements in the following.
Internally, this commit also performs a range of changes to the Input
manager; it marks a lot of functions as const and introduces a new
ValueToVal method (which could in theory replace the already existing
one - it is a bit more powerful).
This also changes SerialTypes to have a subtype for Values, just as
Fields already have it; I think it was mostly an oversight that this was
not introduced from the beginning. This should not necessitate any code
changes for people already using SerialTypes.
option variable
===============
The option keyword allows variables to be specified as run-tine options.
Such variables cannot be changed using normal assignments. Instead, they
can be changed using Option::set. It is possible to "subscribe" to
options and be notified when an option value changes.
Change handlers can also change values before they are applied; this
gives them the opportunity to reject changes. Priorities can be
specified if there are several handlers for one option.
Example script:
option testbool: bool = T;
function option_changed(ID: string, new_value: bool): bool
{
print fmt("Value of %s changed from %s to %s", ID, testbool, new_value);
return new_value;
}
event bro_init()
{
print "Old value", testbool;
Option::set_change_handler("testbool", option_changed);
Option::set("testbool", F);
print "New value", testbool;
}
config reader
=============
The config reader provides a way to read configuration files back into
Bro. Most importantly it automatically converts values to the correct
types. This is important because it is at least inconvenient (and
sometimes near impossible) to perform the necessary type conversions in
Bro scripts themselves. This is especially true for sets/vectors.
Configuration generally look like this:
[option name][tab/spaces][new variable value]
so, for example:
testaddr 2607:f8b0:4005:801::200e
testinterval 60
testtime 1507321987
test_set a b c d erdbeerschnitzel
The reader uses the option name to look up the type that variable has in
the Bro core and automatically converts the value to the correct type.
Example script use:
type Idx: record {
option_name: string;
};
type Val: record {
option_val: string;
};
global currconfig: table[string] of string = table();
event InputConfig::new_value(name: string, source: string, id: string, value: any)
{
print id, value;
}
event bro_init()
{
Input::add_table([$reader=Input::READER_CONFIG, $source="../configfile", $name="configuration", $idx=Idx, $val=Val, $destination=currconfig, $want_record=F]);
}
Script-level config framework
=============================
The script-level framework ties these two features together and makes
them a bit more convenient to use. Configuration files can simply be
specified by placing them into Config::config_files. The framework also
creates a config.log that shows all value changes that took place.
Usage example:
redef Config::config_files += {configfile};
export {
option testbool : bool = F;
}
The file is now monitored for changes; when a change occurs the
respective option values are automatically updated and the value change
is written to config.log.
This commit fixes a few small issues.
* server key exchange parameters are only parsed when a named curve is
given.
* I removed the ssl-verbose.bro and moved the functionality into the
testcase.
The information that we get with these events is likely irrelevant to
the majority of Bro users; I do not think that we have to ship a
script that uses them by default. A script like this would be
something to publish via the Bro package manager instead; this is the
approach that we have taken with a number of the recent SSL addition.
* I marked the ssl_server_curve event as deprecated. More information is
contained in the new ssl_ecdh_server_params event.
This is an events that is probably seldomly (or never) directly used
by anyone; I plan to completely remove it right after the 2.6 release.
* 'topic/corelight/load-hook' of https://github.com/corelight/bro:
Fix and extend behavior of HookLoadFile
I refactored some parts of scan.l to avoid the ambiguity of some
branches returning 0 and some branches not returning anything.
The hook being added is:
bool HookReporter(const std::string& prefix, const EventHandlerPtr event,
const Connection* conn, const val_list* addl, bool location,
const Location* location1, const Location* location2,
bool time, const std::string& buffer) override;
This hook gives access to basically all information that is available in
the function in Reporter.cc that performs the logging. The hook is
called each time when anything passes through the reporter in the cases
in which an event usually would be called. This includes weirds. The
hook can return false to prevent the normal reporter events from being
raised.
This commit fixes and extends the behavior of HookLoadFile. Before this
change, HookLoadFile appended ".bro" to each path that was @loaded, even
if the path specified directory names. Furthermore it only gave the path
of the file as it was specified in the Bro script without revealing the
final path of the file that it was going to load.
This patch changes this behavior - in addition to giving the unmodified
path given in the @load command, the hook now returns the resolved path
of the file or directory it is going to load (if found). The hook is
furthermore raises for @load-sigs and @load-plugin; a enum specifies the
kind of load that is happening.
Increased the size of a buffer to be large enough to contain all the
characters of the largest possible "double" value when scientific
notation is not being used (previously, the nonsensical "NAN.0" would be
written to ASCII logs for any value >= 1e248).
In ContentLine_Analyzer, prevent excessively long lines being assembled.
The line length will default to just under 16MB, but can be overriden on
a per-analyzer basis. This is done for the finger,ident, and irc
analyzers.
handlers.
It's well known that changes to mutable event arguments, like tables,
become visible to all places where those values are used, including
subsequent handlers of the same event. However, there's a related case
that's more suprising: simply assigning *a new value* to an event
argument passes through, too. This commit fixes that behaviour. (We
even had a btest with a baseline reflecting the problen).
IP packets that have a header length that is greater than the total
length of the packet cause a integer overflow, which cause range-checks
to fail, which causes OOB reads.
Furthermore Bro does not currently check the version field of IP packets
that are read from tunnels. I added this check - otherwhise Bro reports
bogus IP information in its error messages, just converting the data
from the place where the IP information is supposed to be to IPs.
This behavior brings us closer to what other software (e.g. Wireshark)
displays in these cases.
Signatures using an eval-condition that had no return value caused a
segmentation fault. This fix just returns false in this case, as it is
done for an interpreter error.
This switches in from using strstr to use strnstr (implementation from
FreeBSD on systems which do not bring their own implementation).
It is especially likely that users come accross this when using the
DATA_EVENT analyzer with files that contain binary data - the test uses
exactly this case.
It turns out that the serial number field in all events was never
populated correctly. Instead, the previous field (issuer key hash) was
re-read and repeated in all events.
It turns out that Chrome supports an experimental mode to support TLS
1.3, which uses a non-standard way to negotiate TLS 1.3 with a server.
This non-standard way to negotiate TLS 1.3 breaks the current draft RFC
and re-uses an extension on the server-side with a different binary
formatting, causing us to throw a binpac exception.
This patch ignores the extension when sent by the server, continuing to
correctly parse the server_hello reply (as far as possible).
From what I can tell this seems to be google working around the fact
that MITM equipment cannot deal with TLS 1.3 server hellos; this change
makes the fact that TLS 1.3 is used completely opaque unless one looks
into a few extensions.
We currently log this as TLS 1.2.
The expire-redef.bro test was sometimes failing due to the second "Run"
message being printed after (should happen before) the "Expired"
message. Fixed by increasing the time interval between events.
Also reduced the number of events raised to make the test finish more
quickly.
This adds a slight patch to the HTTP analyzer, which recognizez when a connection is
upgraded to a different protocol (using a 101 reply with a few specific headers being
set).
In this case, the analyzer stops further processing of the connection (which will
result in DPD errors) and raises a new event:
event http_connection_upgrade(c: connection, protocol: string);
Protocol contains the name of the protocol that is being upgraded to, as specified in
one of the header values.
Closes#1830.
* origin/topic/johanna/ocsp-sct-validate: (82 commits)
Tiny script changes for SSL.
Update CT Log list
SSL: Update OCSP/SCT scripts and documentation.
Revert "add parameter 'status_type' to event ssl_stapled_ocsp"
Revert "parse multiple OCSP stapling responses"
SCT: Fix script error when mime type of file unknown.
SCT: another memory leak in SCT parsing.
SCT validation: fix small memory leak (public keys were not freed)
Change end-of-connection handling for validation
OCSP/TLS/SCT: Fix a number of test failures.
SCT Validate: make caching a bit less aggressive.
SSL: Fix type of ssl validation result
TLS-SCT: compile on old versions of OpenSSL (1.0.1...)
SCT: Add caching support for validation
SCT: Add signed certificate timestamp validation script.
SCT: Allow verification of SCTs in Certs.
SCT: only compare correct OID/NID for Cert/OCSP.
SCT: add validation of proofs for extensions and OCSP.
SCT: pass timestamp as uint64 instead of time
Add CT log information to Bro
...
I added another small change - since we are inlining Configure(), we can
just set bro_plugin directly to BRO_PLUGIN_BRO_VERSION in
src/plugin/Plugin.h, instead of depending on the plugin to do it. This
also means we do not need to change init-plugin in bro-aux at this
moment.
BIT-1828 #closed
* origin/topic/robin/plugin-version-check:
Adding plugin API number into versioned function name, and removing old runtime API version check.
Extend plugin infrastructure to catch Bro version mismatches at link time.
- Addresses Philip Romero's question from the Bro mailing list.
- Adds Microsoft Edge as a detected browser.
- We are now unescaping encoded characters in software names.
* origin/topic/dnthayer/ticket1821:
Remove loading of listen.bro in tests that do not need it
Serialize tests that load listen.bro
Fix race condition causing some tests to fail
Fix a race condition in some failing tests
The broccoli-v6addrs "-r" option was renamed to "-R"
Fix a race condition in some failing tests
Removed loading of the "frameworks/communication/listen" script for
a couple of tests that don't need this functionality. This was causing
failures of some broccoli-related tests in the "istate" test directory
due to two instances of Bro trying to listen on the same port.
* 'nfs_changes' of https://github.com/jwallior/bro:
Add nfs unittest. Includes an example for the new nfs_proc_rename.
Added rename event to rpc/nfs protocol analyzer. This event identifies and reports information about nfs/rpc calls and replies of the type rename.
Expand parsing of RPC Call packets to add Uid, Gid, Stamp, MachineName and AuxGIDs
Fix NFS protocol parser.
These were caused by earlier code updates.
This commit also removes the ocsp stapling logging script; it was
defunctional and the information provided by it wnever really has been
especially interesting.
