Functions are now assigned a unique integer on construction which
CompositeHash can base hashes on. Recovery then just involves
looking up the function pointer associated with that unique number.
* origin/topic/jsiwek/http-1xx-replies:
Change logging of HTTP 1xx responses to occur in their own columns.
Fix handling of HTTP 1xx response codes (addresses #411).
Updated README and collected coverage-related tests in a common dir.
There are still coverage failures resulting from either the following
scripts not being @load'd in the default bro mode:
base/frameworks/time-machine/notice.bro
base/protocols/http/partial-content.bro
base/protocols/rpc/main.bro
Or the following result in errors when @load'd:
policy/protocols/conn/scan.bro
policy/hot.conn.bro
If these are all scripts-in-progress, can we move them all to live
outside the main scripts/ directory until they're ready?
Changed the parser to not treat 1xx response codes as a final answer
to an unanswered request -- a later response is still expected.
The scripting layer will also not finish a request-reply pair when
seeing 1xx's, instead it logs both the 1xx and final response messages
with associated information of the current request as they're seen.
* origin/topic/seth/notice-suppression:
Updated a notice related baseline and added a necessary @load line.
Notice suppression clean up and notice/cluster integrtion fixes.
Updates for notice suppression to use the &create_expire attribute
Small, mostly cosmetic updates and fixing a test.
Fix crash on exit (addresses #607).
Duplicate notice suppression.
Closes#623.
- Worker raised notices are printed a single time by the manager now.
- Cluster/notices integration cleaned up.
- New tests for cluster/notice integration.
- Notice suppression fixes and tests.
- The table tracking notice suppressions is now done with a table
attribute instead of "manually" with scheduled events.
- The $suppress_for field can be set within PolicyItem's now too to
affect the suppression interval through the notice policy.
* origin/topic/jsiwek/path-func-record-demote:
Fix filter path_func to allow record argument as a subset of stream's columns.
Conflicts:
src/LogMgr.cc
Closes#600.
- The test needed to append the second notice the event queue so that
the notice would be suppressed correctly.
- Setting $suppress_for manually in apply_policy since it doesn't pick
up the &default when directly creating the record inside the NOTICE call.
- Duplicate notices are discovered with the new Notice::Info
field $identifier. It's a string that is left up to the
notice implementor to define which would indicate a
fundamentally duplicate notice. The field is optional and
if it's not included it's not possible for notice
suppression to take place.
- Duplicate notices are suppressed by default for the interval
defined by the Notice::default_suppression_interval variable
(1 hour by default).
- A new notice action was defined ACTION_NO_SUPPRESS to prevent
suppression for a specific notice instance. A convenience set
named not_suppressed_types was also created to not suppress
entire notice types.
- A new field was added to the PolicyItem type to modify the length
of time a notice should be suppressed if the predicate matches.
The field is named $suppress_for. This name makes the code more
readable like this: $suppress_for = 1day
- New events were created to give visibility into the notice
framework's suppression activity.
- event Notice::begin_suppression(n: Notice::Info)
- event Notice::suppressed(n: Notice::Info)
- event Notice::end_suppression(n: Notice::Info)
- The suppression.bro script doesn't have a baseline because
it is causing a segfault in Bro. This one test is the
reason that this is being integrated into a branch instead
of master.
This required adding the ability for RecordVal::CoerceTo functions to
optionally allow orphaning fields. The default is to not allow it, but
now before writing to a log, the value of the stream's columns is coerced
down, if necessary, before passing it on to the filter's path_func.
Addresses #600.
* origin/topic/gilbert/rand-pool:
Updating tests.
Updated uid pools to use integer values instead of strings.
Updating tests.
Test no longer relevant. Need a way to generate and test collisions.
A few minor tweaks to make code less braindead. Fixed-length piece of pool name now only used to hash when determinism is not required; otherwise, whole pool name is used. Note that collisions between pool name hashes will lead to sensitivity to initialization order within the UID generator.
Testing long (>32 character) pool names.
Simple test to verify various pools are not affecting each other.
Some working code. Adds UID pools classified by string. Just compiles and runs; need to go back through and make sure this code is actually doing what I want it to do.
Note, I've removed the collision detection. Seems unlikely to occur
and even if, it's not really that bad.
- The "private interface" section is still tracked, but no longer rendered.
We can judge the usefulness of it later and add back if there's demand.
- Documentation of event handlers was being treated the same as event
declarations. Now handlers are tracked separately, but not currently
rendered in the generated doc output since usefulness is questionable.
- Reorganize top-level 'doc' Makefile target so submodules can easily
add their own doc-generating routines to it. e.g. the Bro project
makes a placeholder 'doc' target, then adds 'restdoc', 'sphinxdoc';
later Broccoli can add it's own target as a dependency for generating
API docs.
- Fixed generated docs for BIFs not being organized under a base/
subdirectory like the original source files.
- Fixed documentation style for function parameters not applying to
functions declared as record fields.
- Misc. script documentation tweaks to address warnings given by Sphinx.
- Log path's are generated in the scripting land
now. The default Log stream ID to path string
mapping works like this:
- Notice::LOG -> "notice"
- Notice::POLICY_LOG -> "notice_policy"
- TestModule::LOG -> "test_module"
- Logging streams updated across all of the shipped
scripts to be more user friendly. Instead of
the logging stream ID HTTP::HTTP, we now have
HTTP::LOG, etc.
- The priorities on some bro_init handlers have
been adjusted to make the process of applying
filters or disabling streams easier for users.
* origin/topic/gregor/script-polishing:
Tune when c$conn is set.
Set c$conn (for logging) in new_connection() event.
(Semiautomatically) convert the comments in bare-init.bro into autodoc ones.
Add ConnSize_Analyzer's fields to conn.log
SSH base scripts: make sure ConnSizeAnalyzer variables are available before using them.
Make reference to the other script a link
- Since each host in a cluster has it's own view of the metrics
the only time the manager would get a chance for a global view
is the break_interval. This update improves that time. If a
worker crosses 10% of the full threshold, it will send it's
value to the manager which can then ask the rest of the cluster
for a global view. The manager then adds all of the values for
each workers metric indexes together and will do the notice
if it crosses the threshold so that it isn't dependent on
waiting for the break interval to hit. This functionality
works completely independently of the break_interval too. Logging
will happen as normal.
- Small update for SSH bruteforcer detection to match additions in
the metrics framework API.
- The hope is that this update is mostly invisible from anyone's
perspective. The only affect it should have on users is to better
the detection of metric values crossing thresholds on cluster
deployments.
- fixing some Metrics::add_data() call signatures
- slight refactors to cluster framework @if and adding a NONE NodeType for
so local_node_type() will return that instead of just emitting an error
when cluster mode isn't enabled
- `make restdoc` target now uses bro's bare-mode
- bro scripts generated from bifs now really only live in the build/src/base/
directory and changed the DocSourcesList.cmake to dynamically figure out
what bifs exist by looking in src/ instead of build/src/
- add some missing @load dependencies
