- The feature was primarily added to allow the value to be
modified for cluster based intermediate threshold checks
without requiring the user to write the metrics filter
differently for cluster consideration. It's also a nice
way to calculate some related information to the metric
without accidently applying thresholds to that value.
- Fixed a few small bugs in ftp detect-bruteforcing script
and adapted it to the new threshold value selection feature.
- Fixed several state maintenance issues for intermediate updates.
- Added a new tuning variable Metrics::max_outstanding_global_views
which limits the number of in-flight intermediate updates per
metric filter.
- Changed the default global view threshold percent to 20% (up from 10%)
- Removed default logging. Now a function is available for the new
$period_finished filter field to get the same behavior for logging
named Metrics::write_log.
- Added index rollups for getting multiple metrics result values
as the same time.
- New script measures a couple of aspects of SMTP traffic.
- Existing metrics scripts had a small amount of work done
to make them work with changes to metrics framework.
- Metrics:ID enum has been replaced with strings.
- Uniqueness can now be measured with the Metrics::add_unique function.
- Filters can change the index value with the $normalize_func field.
- Fixed a bug where notices were being passed to proxies.
This was a mistake and should greatly reduce load on
many clusters.
- Cluster event regex variables renamed to:
- Notice::manager2worker_events
- Notice::manager2proxy_events
- Notice::worker2manager_events
- Notice::worker2proxy_events
- Notice::proxy2manager_events
- Notice::proxy2worker_events
- The default Notice::policy set is cleared for all cluster
nodes except for managers to cause all default notice
processing to occur on managers. This should reduce load
on workers slightly.
- Log path's are generated in the scripting land
now. The default Log stream ID to path string
mapping works like this:
- Notice::LOG -> "notice"
- Notice::POLICY_LOG -> "notice_policy"
- TestModule::LOG -> "test_module"
- Logging streams updated across all of the shipped
scripts to be more user friendly. Instead of
the logging stream ID HTTP::HTTP, we now have
HTTP::LOG, etc.
- The priorities on some bro_init handlers have
been adjusted to make the process of applying
filters or disabling streams easier for users.
- Since each host in a cluster has it's own view of the metrics
the only time the manager would get a chance for a global view
is the break_interval. This update improves that time. If a
worker crosses 10% of the full threshold, it will send it's
value to the manager which can then ask the rest of the cluster
for a global view. The manager then adds all of the values for
each workers metric indexes together and will do the notice
if it crosses the threshold so that it isn't dependent on
waiting for the break interval to hit. This functionality
works completely independently of the break_interval too. Logging
will happen as normal.
- Small update for SSH bruteforcer detection to match additions in
the metrics framework API.
- The hope is that this update is mostly invisible from anyone's
perspective. The only affect it should have on users is to better
the detection of metric values crossing thresholds on cluster
deployments.
- fixing some Metrics::add_data() call signatures
- slight refactors to cluster framework @if and adding a NONE NodeType for
so local_node_type() will return that instead of just emitting an error
when cluster mode isn't enabled
- `make restdoc` target now uses bro's bare-mode
- bro scripts generated from bifs now really only live in the build/src/base/
directory and changed the DocSourcesList.cmake to dynamically figure out
what bifs exist by looking in src/ instead of build/src/
- add some missing @load dependencies
- Metrics now work on cluster deployments with no caveats. It should be
completely transparent. Intermediate updates to speed some detection
will come later.
- Metrics API is much more similar to the Logging framework's API now.
- Filters define all output and metrics collection now.
- Initial attempt at thresholding and generating notices.
- policy/ renamed to scripts/
- By default BROPATH now contains:
- scripts/
- scripts/policy
- scripts/site
- *Nearly* all tests pass.
- All of scripts/base/ is loaded by main.cc
- Can be disabled by setting $BRO_NO_BASE_SCRIPTS
- Scripts in scripts/base/ don't use relative path loading to ease use of BRO_NO_BASE_SCRIPTS (to copy and paste that script).
- The scripts in scripts/base/protocols/ only (or soon will only) do logging and state building.
- The scripts in scripts/base/frameworks/ add functionality without causing any additional overhead.
- All "detection" activity happens through scripts in scripts/policy/.
- Communications framework modified temporarily to need an environment variable to actually enable (ENABLE_COMMUNICATION=1)
- This is so the communications framework can be loaded as part
of the base without causing trouble when it's not needed.
- This will be removed once a resolution to ticket #540 is reached.
