There are two new script level functions to query and lookup files
from the core by their IDs. These are adding feature parity for
similarly named functions for files. The function prototypes are
as follows:
Files::file_exists(fuid: string): bool
Files::lookup_File(fuid: string): fa_file
Add the folowing option types:
- 55 Parameters Request List;
- 58 Renewal time;
- 59 Rebinding time;
- 61 Client Identifier;
- 82 Relay Agent Information.
Extend the following events with new parameters, specifically:
- dhcp_discover exports client identifier and parameters request list;
- dhcp_request exports client_identifier and parameters request list;
- dhcp_ack exports rebinding time, renewal time and list of suboptions value of
dhcp relay agent information option;
- dhcp_inform exports parameters request list.
Add option type specific variables within the scope of DHCP module
(see src/analyzer/protocol/dhcp/types.bif).
Move protocol specific variables "dhcp_msg" and "dhcp_router_list" from scope Global to DHCP::
and adapt inet_net_var in src/NetVar.cc consequently.
Extend src/analyzer/protocols/dhcp/main.bro to handle the new events and to log
dhcp_ack, dhcp_request and dhcp_discover.
Modify scripts/policy/protocols/dhcp/known-devices-and-hostnames.bro to
include new events' variables.
These have been lingering for a while and they generally annoy
everyone because of the sheer volume. They also don't really add
any useful information for debugging and they were generated differently
than most other weirds anyway (which was a little weird...).
This introduces a new option, SOCKS::default_capture_password which can
be used to specify if Socks passwords are logged by default
Like fot FTP/HTTP, this option is set to false by default.
Addresses BIT-1791
The configuration framework consists of three mostly distinct parts:
* option variables
* the config reader
* the script level framework
I will describe the three elements in the following.
Internally, this commit also performs a range of changes to the Input
manager; it marks a lot of functions as const and introduces a new
ValueToVal method (which could in theory replace the already existing
one - it is a bit more powerful).
This also changes SerialTypes to have a subtype for Values, just as
Fields already have it; I think it was mostly an oversight that this was
not introduced from the beginning. This should not necessitate any code
changes for people already using SerialTypes.
option variable
===============
The option keyword allows variables to be specified as run-tine options.
Such variables cannot be changed using normal assignments. Instead, they
can be changed using Option::set. It is possible to "subscribe" to
options and be notified when an option value changes.
Change handlers can also change values before they are applied; this
gives them the opportunity to reject changes. Priorities can be
specified if there are several handlers for one option.
Example script:
option testbool: bool = T;
function option_changed(ID: string, new_value: bool): bool
{
print fmt("Value of %s changed from %s to %s", ID, testbool, new_value);
return new_value;
}
event bro_init()
{
print "Old value", testbool;
Option::set_change_handler("testbool", option_changed);
Option::set("testbool", F);
print "New value", testbool;
}
config reader
=============
The config reader provides a way to read configuration files back into
Bro. Most importantly it automatically converts values to the correct
types. This is important because it is at least inconvenient (and
sometimes near impossible) to perform the necessary type conversions in
Bro scripts themselves. This is especially true for sets/vectors.
Configuration generally look like this:
[option name][tab/spaces][new variable value]
so, for example:
testaddr 2607:f8b0:4005:801::200e
testinterval 60
testtime 1507321987
test_set a b c d erdbeerschnitzel
The reader uses the option name to look up the type that variable has in
the Bro core and automatically converts the value to the correct type.
Example script use:
type Idx: record {
option_name: string;
};
type Val: record {
option_val: string;
};
global currconfig: table[string] of string = table();
event InputConfig::new_value(name: string, source: string, id: string, value: any)
{
print id, value;
}
event bro_init()
{
Input::add_table([$reader=Input::READER_CONFIG, $source="../configfile", $name="configuration", $idx=Idx, $val=Val, $destination=currconfig, $want_record=F]);
}
Script-level config framework
=============================
The script-level framework ties these two features together and makes
them a bit more convenient to use. Configuration files can simply be
specified by placing them into Config::config_files. The framework also
creates a config.log that shows all value changes that took place.
Usage example:
redef Config::config_files += {configfile};
export {
option testbool : bool = F;
}
The file is now monitored for changes; when a change occurs the
respective option values are automatically updated and the value change
is written to config.log.
This commit fixes a few small issues.
* server key exchange parameters are only parsed when a named curve is
given.
* I removed the ssl-verbose.bro and moved the functionality into the
testcase.
The information that we get with these events is likely irrelevant to
the majority of Bro users; I do not think that we have to ship a
script that uses them by default. A script like this would be
something to publish via the Bro package manager instead; this is the
approach that we have taken with a number of the recent SSL addition.
* I marked the ssl_server_curve event as deprecated. More information is
contained in the new ssl_ecdh_server_params event.
This is an events that is probably seldomly (or never) directly used
by anyone; I plan to completely remove it right after the 2.6 release.
In ContentLine_Analyzer, prevent excessively long lines being assembled.
The line length will default to just under 16MB, but can be overriden on
a per-analyzer basis. This is done for the finger,ident, and irc
analyzers.
a broctl print triggers this error
Reporter::ERROR no such index (Cluster::nodes[Intel::p$descr])
/usr/local/bro/share/bro/base/frameworks/intel/./cluster.bro, line 39
when broctl connects p$descr is empty. It should probably be set to
'control' somewhere inside broctl, but that would only fix broctl, not
other clients.
diff --git a/aux/bro-aux b/aux/bro-aux
index 02f710a43..43f4b90bb 160000
--- a/aux/bro-aux
+++ b/aux/bro-aux
@@ -1 +1 @@
-Subproject commit 02f710a436dfe285bae0d48d7f7bc498783e11a8
+Subproject commit 43f4b90bbaf87dae1a1073e7bf13301e58866011
diff --git a/aux/broctl b/aux/broctl
index e960be2c1..d3e6cdfba 160000
--- a/aux/broctl
+++ b/aux/broctl
@@ -1 +1 @@
-Subproject commit e960be2c192a02f1244ebca3ec31ca57d64e23dc
+Subproject commit d3e6cdfba496879bd55542c668ea959f524bd723
diff --git a/aux/btest b/aux/btest
index 2810ccee2..e638fc65a 160000
--- a/aux/btest
+++ b/aux/btest
@@ -1 +1 @@
-Subproject commit 2810ccee25f6f20be5cd241155f12d02a79d592a
+Subproject commit e638fc65aa12bd136594451b8c185a7a01ef3e9a
diff --git a/scripts/base/frameworks/intel/cluster.bro b/scripts/base/frameworks/intel/cluster.bro
index 820a5497a..e75bdd057 100644
--- a/scripts/base/frameworks/intel/cluster.bro
+++ b/scripts/base/frameworks/intel/cluster.bro
@@ -32,7 +32,7 @@ event remote_connection_handshake_done(p: event_peer)
{
# When a worker connects, send it the complete minimal data store.
# It will be kept up to date after this by the cluster_new_item event.
- if ( Cluster::nodes[p$descr]$node_type == Cluster::WORKER )
+ if ( p$descr in Cluster::nodes && Cluster::nodes[p$descr]$node_type == Cluster::WORKER )
{
send_id(p, "Intel::min_data_store");
}
The bytes_threshold_crossed event in the gridftp analyzer is not first
checking to see if the connection passed the initial criteria. This
causes the script to add the gridftp-data service to any connection that
crosses a threshold that is the same as or greater than the gridftp
size_threshold.
The HTTP "Origin" header is a useful header for CSRF, Chrome plugins making requests, and other scenarios where referrer may not be present.
Reference:
https://tools.ietf.org/html/rfc6454#section-7 ---- "In some sense, the origin granularity is a historical artifact of how the security model evolved."
Especially useful if origin/referrer is a "file://" ---- https://tools.ietf.org/html/rfc6454#section-4
When Bro writes a compressed log, it uses a file extension of ".gz".
However, upon log rotation the ascii writer script function
"default_rotation_postprocessor_func" was discarding the ".gz"
file extension. Fixed so that the correct file extension is
preserved after rotation.
Closes#1830.
* origin/topic/johanna/ocsp-sct-validate: (82 commits)
Tiny script changes for SSL.
Update CT Log list
SSL: Update OCSP/SCT scripts and documentation.
Revert "add parameter 'status_type' to event ssl_stapled_ocsp"
Revert "parse multiple OCSP stapling responses"
SCT: Fix script error when mime type of file unknown.
SCT: another memory leak in SCT parsing.
SCT validation: fix small memory leak (public keys were not freed)
Change end-of-connection handling for validation
OCSP/TLS/SCT: Fix a number of test failures.
SCT Validate: make caching a bit less aggressive.
SSL: Fix type of ssl validation result
TLS-SCT: compile on old versions of OpenSSL (1.0.1...)
SCT: Add caching support for validation
SCT: Add signed certificate timestamp validation script.
SCT: Allow verification of SCTs in Certs.
SCT: only compare correct OID/NID for Cert/OCSP.
SCT: add validation of proofs for extensions and OCSP.
SCT: pass timestamp as uint64 instead of time
Add CT log information to Bro
...
- Addresses Philip Romero's question from the Bro mailing list.
- Adds Microsoft Edge as a detected browser.
- We are now unescaping encoded characters in software names.
* 'nfs_changes' of https://github.com/jwallior/bro:
Add nfs unittest. Includes an example for the new nfs_proc_rename.
Added rename event to rpc/nfs protocol analyzer. This event identifies and reports information about nfs/rpc calls and replies of the type rename.
Expand parsing of RPC Call packets to add Uid, Gid, Stamp, MachineName and AuxGIDs
Fix NFS protocol parser.
Move from using CCS (before: established) to just doing certificate
validation at the end of the connection.
This is (again) more robust in the case of aborted connection. I am
moving this into a hook because of the complexity of the
end-of-connection handling for SSL.
This should probably be extended to not just handle SSL validation, but
all other logging constructs that are currently called in _established.
With this commit, the data structure that is transfered for notice
suppression is much smaller than before, not including potentially
complex data structures like the fa_file record.
This feature can be enabled globally for all logs by setting
LogAscii::gzip_level to a value greater than 0.
This feature can be enabled on a per-log basis by setting gzip-level in
$confic to a value greater than 0.
The dpd signature missed a few cases that are used for TLS 1.3,
especially when draft versions (which are all that we are seeing at the
moment) are being negotiated.
This fix mostly allows draft versions in the server hello (identified by
7F[version]; since we do not know how many drafts there will be, we are
currently allowing a rather safe upper limit.
This commit add the table SSL::ct_logs to Bro. This table is populated
with information about the currently active certificate transparency
logs (data from Google). The data can, e.g., be used to identify which
Logs are being used in SCTs.
The changes are now a bit more succinct with less code changes required.
Behavior is tested a little bit more thoroughly and a memory problem
when reading incomplete lines was fixed. ReadHeader also always directly
returns if header reading failed.
Error messages now are back to what they were before the change, if the
new behavior is not used.
I also tweaked the documentation text a bit.
By default, the ASCII reader does not fail on errors anymore.
If there is a problem parsing a line, a reporter warning is
written and parsing continues. If the file is missing or can't
be read, the input thread just tries again on the next heartbeat.
Options have been added to recreate the previous behavior...
const InputAscii::fail_on_invalid_lines: bool;
and
const InputAscii::fail_on_file_problem: bool;
They are both set to `F` by default which makes the input readers
resilient to failure.
- This fixes BIT-1769 by logging all requests even in the absence of a
reply. The way that request and replying matching were being handled
was restructured to mostly ignore the transaction ids because they
aren't that helpful for network monitoring and it makes the script
structure more complicated.
- Add `framed_addr` field to the radius log to indicate if the radius
server is hinting at an address for the client.
- Add `ttl` field to indicate how quickly the radius server is replying
to the network access server.
- Fix a bunch of indentation inconsistencies.
