observed elements.
Add methods to merge with and without pruning (before only merge
method was with pruning, which invalidates the number of total
observed elements)
* origin/topic/seth/metrics-merge: (70 commits)
Added protocol to the traceroute detection script.
Added an automatic state limiter for threshold based SumStats.
Removed some dead code in scan.bro
Renamed a plugin hook in sumstats framework.
Move loading variance back to where it should be alphabetically.
Fix a bug with path building in FTP. Came up when changing the path utils.
Fix a few tests.
SumStats test checkpoint.
SumStats tests pass.
Checkpoint for SumStats rename.
Fix another occasional reporter error.
Small updates to hopefully correct reporter errors leading to lost memory.
Trying to fix a state maintenance issue.
Updating DocSourcesList
Updated FTP bruteforce detection and a few other small changes.
Test updates and cleanup.
Fixed the measurement "sample" plugin.
Fix path compression to include removing "/./".
Removed the example metrics scripts. Better real world examples exist now.
Measurement framework is ready for testing.
...
I am not (entirely) sure that this is mathematically correct, but
I am (more and more) getting the feeling that it... might be.
In any case - this was the last step and now it should work
in cluster settings.
Include explanation of various Notice::policy hook actions.
Add two btest scripts. framework_notice_hook_01.bro shows adding an
action to the n$action set while framework_notice_suppression.bro shows
how to add a custom n$suppress_for value for a notice through a policy
hook. While both scripts include an @load directive, it is left out in
RST document so as to avoid confusion.
Note: merging top-k data structures is not yet possible (and is
actually quite awkward/expensive). I will have to think about
how to do that for a bit...
and fix up the hll scripts for it.
Conflicts:
scripts/base/frameworks/sumstats/plugins/__load__.bro
testing/btest/scripts/base/frameworks/measurement/basic.bro
- It's derived from the magic database of libmagic 5.14, but with most
everything not related to mime types removed.
- The custom database is always used by default for mime detection, but
the more verbose file type detection will fall back on the default
libmagic installation's database. The result is: mime type strings
are now guaranteed to be consistent across platforms, but the verbose
file type descriptions are not.
- The custom database gets installed in $prefix/share/bro/magic, and
should even be extensible if files with new patterns are added inside
the directory.
- The search path for the mime magic database can be controlled via
BROMAGIC environment variable.
- Remove mime_desc field from ftp.log.
- Stop using the mime/file type canonifier with unit tests.
- libmagic >= 5.04 is now a requirement.
- FileAnalysis::Info is now just a record used for logging, the fa_file
record type is defined in init-bare.bro as the analogue to a
connection record.
- Starting to transfer policy hook triggers and analyzer results to
events.
