- On-demand access to sumstats results through "return from"
functions named SumStats::request and Sumstats::request_key.
Both functions are tested in standalone and clustered modes.
- $name field has returned to SumStats which simplifies cluster
code and makes the on-demand access stuff possible.
- Clustered results can only be collected for 1 minute from their
time of creation now instead of time of last read.
- Thresholds use doubles instead of counts everywhere now.
- Calculation dependency resolution occurs at start up time now
instead of doing it at observation time which provide a minor
cpu performance improvement. A new plugin registration mechanism
was created to support this change.
- AppStats now has a minimal doc string and is broken into hook-based
plugins.
- AppStats and traceroute detection added to local.bro
Thanks to git this merge was less troublesome that I was afraid it
would be. Not all tests pass yet though (and file hashes have changed
unfortunately).
Conflicts:
cmake
doc/scripts/DocSourcesList.cmake
scripts/base/init-bare.bro
scripts/base/protocols/ftp/main.bro
scripts/base/protocols/irc/dcc-send.bro
scripts/test-all-policy.bro
src/AnalyzerTags.h
src/CMakeLists.txt
src/analyzer/Analyzer.cc
src/analyzer/protocol/file/File.cc
src/analyzer/protocol/file/File.h
src/analyzer/protocol/http/HTTP.cc
src/analyzer/protocol/http/HTTP.h
src/analyzer/protocol/mime/MIME.cc
src/event.bif
src/main.cc
src/util-config.h.in
testing/btest/Baseline/coverage.bare-load-baseline/canonified_loaded_scripts.log
testing/btest/Baseline/coverage.default-load-baseline/canonified_loaded_scripts.log
testing/btest/Baseline/istate.events-ssl/receiver.http.log
testing/btest/Baseline/istate.events-ssl/sender.http.log
testing/btest/Baseline/istate.events/receiver.http.log
testing/btest/Baseline/istate.events/sender.http.log
Closes#1003.
* origin/topic/bernhard/metrics-samples:
finishing touches, make test more robust, rename function in last again
change names of data structures after talking with seth
make last plugin nicer and samplify sqli detector
add tests for sampler
reservoir sampler. untested.
Closes#997.
* topic/robin/sqlite-merge: (25 commits)
Fix to make sqlite test consistent, and updating coverage baselines
Avoid a CMake warning about 3rdparty looking like a number.
Fixing linker error.
and there is no has-reader.
make sqlite3 executable required and add test-cases for errors
Renaming src/external -> src/3rdparty
fix a few small rough edges (mostly comments that do no longer apply)
fix bug in input-manager regarding enums that a writer reads without 0-terminating the string
actually make sqlite work again (tests passed because the writer was not actually defined because of the define.)
add sqlite distribution.
fix warnings, update baselines, handle rotation
add sqlite tests and fix small vector/set escaping bugs
fix small bug with vectors and sets.
make work with newer AsciiFormatter.
start adding a different text for empty records for the sqlite writer.
no, you will never guess from where I copied this file...
make sqlite support more or less work for logging and input
make sqlite-writer more stable.
make it compile with new version of AsciiInputOutput
and adapt to AsciiInputOutput - seems to work...
...
Conflicts:
scripts/base/frameworks/input/__load__.bro
src/CMakeLists.txt
src/input.bif
src/input/Manager.cc
src/main.cc
src/types.bif
testing/btest/Baseline/coverage.bare-load-baseline/canonified_loaded_scripts.log
testing/btest/Baseline/coverage.default-load-baseline/canonified_loaded_scripts.log
logging: table does not contain all required columns (when extending
data structures)
input: table does not contain all required columns (when extending
data structure), wrong sql statement
and fix up the hll scripts for it.
Conflicts:
scripts/base/frameworks/sumstats/plugins/__load__.bro
testing/btest/scripts/base/frameworks/measurement/basic.bro
- FileAnalysis::Info is now just a record used for logging, the fa_file
record type is defined in init-bare.bro as the analogue to a
connection record.
- Starting to transfer policy hook triggers and analyzer results to
events.
I've used the opportunity to also cleanup DPD's expect_connection()
infrastructure, and renamed that bif to schedule_analyzer(), which
seems more appropiate. One can now also schedule more than one
analyzer per connection.
TODOs:
- "make install" is probably broken.
- Broxygen is probably broken for plugin-defined events.
- event groups are broken (do we want to keep them?)
- parallel btest is broken, but I'm not sure why ...
(tests all pass individually, but lots of error when running
in parallel; must be related to *.bif restructuring).
- Document API for src/plugin/*
- Document API for src/analyzer/Analyzer.h
- Document API for scripts/base/frameworks/analyzer
Other misc:
- Remove HTTP::MD5 notice.
- Add "last_active" field to FileAnalysis::Info record.
- Replace "conn_uids", "conn_ids" fields in FileAnalysis::Info record
with just a "conns" fields containing full connection records.
- The http-methods unit test is failing now, but I think it will be
fixed once I change the file handle callback mechanism to use events
instead.
