- $result is renamed to $action to reflect changes to the notice framework
since there is already another result-like field ($suppress_for) and
there may be more in the future.
- Slipped in a change to add connection information to notice emails too.
- Simplified the communication API and made it easier to change
to encrypted connections by not having separate variables to
define encrypted and unencrypted ports.
- Now, to enable listening without configuring nodes just
load the frameworks/communication/listen script.
- If encrypted listening is desired set the following:
redef Communication::listen_encrypted=T;
- Accompanying test updates.
- New script extracted from weird.bro to implement the
connection related "weird" data into an optionally
loaded script.
- Adjusted the default notice tuning to stop ignoring
the connection related weirds since they aren't loaded
by default anymore.
- Fixed a bug where notices were being passed to proxies.
This was a mistake and should greatly reduce load on
many clusters.
- Cluster event regex variables renamed to:
- Notice::manager2worker_events
- Notice::manager2proxy_events
- Notice::worker2manager_events
- Notice::worker2proxy_events
- Notice::proxy2manager_events
- Notice::proxy2worker_events
- The default Notice::policy set is cleared for all cluster
nodes except for managers to cause all default notice
processing to occur on managers. This should reduce load
on workers slightly.
- scan.bro and hot.conn.bro will be returning soon.
- The rest are going to return as updated protocol analysis
scripts and new/updated frameworks later.
Updated README and collected coverage-related tests in a common dir.
There are still coverage failures resulting from either the following
scripts not being @load'd in the default bro mode:
base/frameworks/time-machine/notice.bro
base/protocols/http/partial-content.bro
base/protocols/rpc/main.bro
Or the following result in errors when @load'd:
policy/protocols/conn/scan.bro
policy/hot.conn.bro
If these are all scripts-in-progress, can we move them all to live
outside the main scripts/ directory until they're ready?
- Fixed the notice_policy log.
- Predicate functions are now logged.
- Predicate functions are now optional. If not given, it's assumed that
the result should always apply.
* origin/topic/seth/notice-suppression:
Updated a notice related baseline and added a necessary @load line.
Notice suppression clean up and notice/cluster integrtion fixes.
Updates for notice suppression to use the &create_expire attribute
Small, mostly cosmetic updates and fixing a test.
Fix crash on exit (addresses #607).
Duplicate notice suppression.
Closes#623.
* origin/fastpath:
Fix PktSrc setting next_timestamp even when no packet available
Fix lack of NUL-termination in to_upper/to_lower BIF's return val.
Fixing unit tests and some minor bugs.
Fix broctl cluster log rotation (addresses #619)
* master:
- Worker raised notices are printed a single time by the manager now.
- Cluster/notices integration cleaned up.
- New tests for cluster/notice integration.
- Notice suppression fixes and tests.
* origin/topic/jsiwek/filter-rotation:
Remove Log::rotation_control (addresses #572).
Add an optional Log::RotationControl to Log::Filter records.
Closes#572.
- The test needed to append the second notice the event queue so that
the notice would be suppressed correctly.
- Setting $suppress_for manually in apply_policy since it doesn't pick
up the &default when directly creating the record inside the NOTICE call.
- Duplicate notices are discovered with the new Notice::Info
field $identifier. It's a string that is left up to the
notice implementor to define which would indicate a
fundamentally duplicate notice. The field is optional and
if it's not included it's not possible for notice
suppression to take place.
- Duplicate notices are suppressed by default for the interval
defined by the Notice::default_suppression_interval variable
(1 hour by default).
- A new notice action was defined ACTION_NO_SUPPRESS to prevent
suppression for a specific notice instance. A convenience set
named not_suppressed_types was also created to not suppress
entire notice types.
- A new field was added to the PolicyItem type to modify the length
of time a notice should be suppressed if the predicate matches.
The field is named $suppress_for. This name makes the code more
readable like this: $suppress_for = 1day
- New events were created to give visibility into the notice
framework's suppression activity.
- event Notice::begin_suppression(n: Notice::Info)
- event Notice::suppressed(n: Notice::Info)
- event Notice::end_suppression(n: Notice::Info)
- The suppression.bro script doesn't have a baseline because
it is causing a segfault in Bro. This one test is the
reason that this is being integrated into a branch instead
of master.
Log rotation is now controlled directly through Filter records.
Also addressed a TODO in the default_path_func regarding the
LogMgr::AddFilter function generating internal filter path
suggestions/fallbacks. Now, if the user doesn't explicitly set a filter
path, the filter's path will be the result of the first call to
default_path_func (happens during the first write to the log). And in
that case the path suggestion argument to the path_func is an empty
string.
- Reorganize top-level 'doc' Makefile target so submodules can easily
add their own doc-generating routines to it. e.g. the Bro project
makes a placeholder 'doc' target, then adds 'restdoc', 'sphinxdoc';
later Broccoli can add it's own target as a dependency for generating
API docs.
- Fixed generated docs for BIFs not being organized under a base/
subdirectory like the original source files.
- Fixed documentation style for function parameters not applying to
functions declared as record fields.
- Misc. script documentation tweaks to address warnings given by Sphinx.
* origin/topic/gilbert/ascii-header:
Updated tests; removed net type from type conversion code.
Updated header format (see #558)
Header modification to LogWriterAscii to make it easier for scripts to understand bro log files.
Notes:
- I've refactored the code a bit, also adapting the style a bit.
Also edited the header format slightly.
- I'm skipping the testing/btest/profiles directory, which seems
unrelated.
- I'm also skipping the baseline updates as they weren't
up-to-date anymore. Will update them in a subsequent commit.
- Log path's are generated in the scripting land
now. The default Log stream ID to path string
mapping works like this:
- Notice::LOG -> "notice"
- Notice::POLICY_LOG -> "notice_policy"
- TestModule::LOG -> "test_module"
- Logging streams updated across all of the shipped
scripts to be more user friendly. Instead of
the logging stream ID HTTP::HTTP, we now have
HTTP::LOG, etc.
- The priorities on some bro_init handlers have
been adjusted to make the process of applying
filters or disabling streams easier for users.
This allows for the RotationControl to be automatically added to the
Log::rotation_control table for the filter's (writer, path) when it is
added to a stream via Log::add_filter. Log::remove_filter now also
removes any RotationControl's associated with the filter from the
Log::rotation_control table.
This can be used from a Bro script to cause Bro to exit with non-zero
status and a message to stderr. Uses of the exit() BIF in scripts
for terminating Bro because of an error were replaced with this one,
which addresses #564.
