Seth Hall
22b1e78ec1
Changing how threshold reseting happens (actually do it on managers!)
2013-08-21 14:08:14 -04:00
Vlad Grigorescu
73a5106bd9
Fix some reporter.log errors by adding a few extra key membership checks.
2013-08-21 07:29:22 -04:00
Bernhard Amann
d83edf8068
Merge remote-tracking branch 'origin/master' into topic/bernhard/hyperloglog
...
Conflicts:
src/NetVar.cc
src/NetVar.h
src/SerialTypes.h
src/probabilistic/CMakeLists.txt
testing/btest/scripts/base/frameworks/sumstats/basic-cluster.bro
testing/btest/scripts/base/frameworks/sumstats/basic.bro
2013-08-12 09:47:53 -07:00
Seth Hall
1eadeaec3c
Fix a major memory issue in the SumStats framework.
...
- There are still problems, but this should prevent a deadlock issue
and help with memory use.
2013-08-03 01:57:51 -04:00
Seth Hall
6b58ef12d7
Still fixing bugs in sumstats updated api cluster support.
2013-08-02 16:53:56 -04:00
Seth Hall
135094428e
Hopefully fix the SumStats cluster support.
2013-08-02 16:30:34 -04:00
Seth Hall
7b8073556e
Fix the SumStats top-k plugin and test.
2013-08-02 16:30:20 -04:00
Seth Hall
d6edbd27b1
Merge remote-tracking branch 'origin/master' into topic/seth/sumstats-updates
2013-08-02 13:17:48 -04:00
Seth Hall
4f8100774c
Updates for SumStats API to deal with high memory stats.
...
- The code is a mess and will need to be cleaned up, but the
tests do pass.
2013-08-02 12:44:33 -04:00
Robin Sommer
81dcda3eb4
Merge remote-tracking branch 'origin/topic/bernhard/topk'
...
* origin/topic/bernhard/topk:
adapt to new folder structure
fix opaqueval-related memleak
synchronize pruned attribute
potentially found wrong Ref.
add sum function that can be used to get the number of total observed elements.
in cluster settings, the resultvals can apparently been uninitialized in some special cases
fix memory leaks
fix warnings
add topk cluster test
make size of topk-list configureable when using sumstats
implement merging for top-k.
add serialization for topk
make the get function const
topk for sumstats
well, a test that works..
implement topk.
2013-08-01 10:27:18 -07:00
Seth Hall
7db531e162
Merge remote-tracking branch 'origin/master' into topic/seth/sumstats-updates
...
Conflicts:
doc/scripts/DocSourcesList.cmake
scripts/test-all-policy.bro
2013-07-30 11:48:03 -04:00
Seth Hall
0e23a8bc9e
Beginning rework of SumStats API.
2013-07-30 11:46:51 -04:00
Bernhard Amann
9e0fd963e0
Merge remote-tracking branch 'origin/topic/robin/bloom-filter-merge' into topic/bernhard/hyperloglog
...
Conflicts:
scripts/base/frameworks/sumstats/plugins/__load__.bro
src/CMakeLists.txt
src/NetVar.cc
src/NetVar.h
src/OpaqueVal.h
src/SerialTypes.h
src/bro.bif
2013-07-23 21:31:05 -07:00
Seth Hall
26f8bd7ad7
Fix a reporter message in sumstats.
2013-07-20 01:21:01 -04:00
Bernhard Amann
03b584c34a
Merge remote-tracking branch 'origin/master' into topic/bernhard/topk
2013-07-09 14:56:05 -07:00
Jon Siwek
e45933562e
Fix broken/missing documentation.
2013-05-23 16:53:42 -05:00
Seth Hall
4f4ef99a6b
SumStats changes to how thresholding works to simplify and reduce memory use.
2013-05-23 10:12:17 -04:00
Seth Hall
6bd9ab3bd6
More adjustments to try and correct SumStats memory use.
2013-05-22 16:41:46 -04:00
Seth Hall
c4a1f30a87
Hopefully fixing a strange error.
2013-05-22 14:59:31 -04:00
Seth Hall
0a18b62d12
Merge remote-tracking branch 'origin/master' into topic/seth/sumstats-updates
...
Conflicts:
scripts/base/frameworks/sumstats/cluster.bro
scripts/base/frameworks/sumstats/plugins/average.bro
scripts/base/frameworks/sumstats/plugins/max.bro
scripts/base/frameworks/sumstats/plugins/min.bro
scripts/base/frameworks/sumstats/plugins/sample.bro
scripts/base/frameworks/sumstats/plugins/std-dev.bro
scripts/base/frameworks/sumstats/plugins/sum.bro
scripts/base/frameworks/sumstats/plugins/unique.bro
scripts/base/frameworks/sumstats/plugins/variance.bro
scripts/policy/protocols/http/detect-sqli.bro
testing/btest/scripts/base/frameworks/sumstats/cluster-intermediate-update.bro
2013-05-21 22:33:16 -04:00
Seth Hall
bec965b66f
Large update for the SumStats framework.
...
- On-demand access to sumstats results through "return from"
functions named SumStats::request and Sumstats::request_key.
Both functions are tested in standalone and clustered modes.
- $name field has returned to SumStats which simplifies cluster
code and makes the on-demand access stuff possible.
- Clustered results can only be collected for 1 minute from their
time of creation now instead of time of last read.
- Thresholds use doubles instead of counts everywhere now.
- Calculation dependency resolution occurs at start up time now
instead of doing it at observation time which provide a minor
cpu performance improvement. A new plugin registration mechanism
was created to support this change.
- AppStats now has a minimal doc string and is broken into hook-based
plugins.
- AppStats and traceroute detection added to local.bro
2013-05-21 15:52:59 -04:00
Robin Sommer
f76446fb4e
Merge remote-tracking branch 'origin/topic/bernhard/metrics-samples'
...
Closes #1003 .
* origin/topic/bernhard/metrics-samples:
finishing touches, make test more robust, rename function in last again
change names of data structures after talking with seth
make last plugin nicer and samplify sqli detector
add tests for sampler
reservoir sampler. untested.
2013-05-15 16:11:27 -07:00
Bernhard Amann
ab6d5b08a8
finishing touches, make test more robust, rename function in last again
2013-05-15 11:33:25 -07:00
Bernhard Amann
80962ad74b
change names of data structures after talking with seth
2013-05-15 09:44:43 -07:00
Bernhard Amann
b0c4dcdfed
make last plugin nicer and samplify sqli detector
2013-05-15 01:09:52 -07:00
Bernhard Amann
d939c2bdfc
add tests for sampler
2013-05-13 22:11:17 -07:00
Bernhard Amann
fa58e26aa0
Merge remote-tracking branch 'origin/master' into topic/bernhard/metrics-samples
2013-05-13 21:20:25 -07:00
Bernhard Amann
56ab9285a4
Merge remote-tracking branch 'origin/master' into topic/bernhard/topk
2013-05-13 21:03:23 -07:00
Bernhard Amann
70f3f4343a
prevent merge-hook of sumstats unique plugin from damaging source data.
2013-05-07 11:16:59 -07:00
Bernhard Amann
663082e2d5
reservoir sampler. untested.
2013-05-05 11:19:53 -07:00
Bernhard Amann
6acbbe0231
Merge remote-tracking branch 'origin/master' into topic/bernhard/topk
2013-05-03 23:04:22 -07:00
Bernhard Amann
3e74cdc6e0
Merge remote-tracking branch 'origin/master' into topic/bernhard/hyperloglog
2013-05-03 22:58:02 -07:00
Robin Sommer
8992dc6cff
Merge remote-tracking branch 'origin/topic/bernhard/metrics-bug'
...
* origin/topic/bernhard/metrics-bug:
add comment for seth to make us not forget about the copy statements
fix the fix (thanks seth)
duct-tape fix of values not propagating after intermediate check in cluster environments.
Fixing coverage.bare-mode-errors test.
2013-05-02 12:47:36 -07:00
Bernhard Amann
2cfef36116
add comment for seth to make us not forget about the copy statements
2013-05-02 11:42:34 -07:00
Bernhard Amann
fe779575d5
fix the fix (thanks seth)
2013-05-02 11:38:40 -07:00
Bernhard Amann
d984243a77
duct-tape fix of values not propagating after intermediate check in cluster environments.
2013-05-02 11:34:33 -07:00
Robin Sommer
9d483b7e74
Fixing coverage.bare-mode-errors test.
2013-05-01 17:52:16 -07:00
Robin Sommer
9ea5a470e6
Fixing coverage.bare-mode-errors test.
2013-05-01 15:28:45 -07:00
Bernhard Amann
321dfadaab
Merge remote-tracking branch 'origin/topic/robin/metrics-merge' into topic/bernhard/topk
2013-04-29 14:08:17 -07:00
Bernhard Amann
07ecd31bbd
in cluster settings, the resultvals can apparently
...
been uninitialized in some special cases
2013-04-28 21:21:22 -07:00
Robin Sommer
b9249ecf9d
Layout tweaks for the sumstats code, and preliminary updates for NEWS.
...
The layout changes are mostly whitespace and some comment rewrapping.
No functional changes.
2013-04-28 15:35:21 -07:00
Bernhard Amann
5608caf79a
make error rate configureable
2013-04-25 14:20:13 -07:00
Bernhard Amann
9802e2332d
Merge branch 'topic/bernhard/hyperloglog-with-measurement' into topic/bernhard/hyperloglog
2013-04-25 13:46:36 -07:00
Bernhard Amann
166fc4765a
Merge remote-tracking branch 'origin/topic/seth/metrics-merge' into topic/bernhard/topk
2013-04-25 13:21:18 -07:00
Seth Hall
48cbb31747
Added an automatic state limiter for threshold based SumStats.
2013-04-25 12:51:55 -04:00
Bernhard Amann
c0890f2a0f
make size of topk-list configureable when using sumstats
2013-04-24 15:01:06 -07:00
Bernhard Amann
2f48008c42
implement merging for top-k.
...
I am not (entirely) sure that this is mathematically correct, but
I am (more and more) getting the feeling that it... might be.
In any case - this was the last step and now it should work
in cluster settings.
2013-04-24 06:17:51 -07:00
Bernhard Amann
567fee6439
Merge remote-tracking branch 'origin/topic/seth/metrics-merge' into topic/bernhard/hyperloglog-with-measurement
...
Conflicts:
scripts/base/frameworks/sumstats/plugins/__load__.bro
2013-04-23 15:27:17 -07:00
Bernhard Amann
de5769a88f
topk for sumstats
2013-04-23 15:19:01 -07:00
Seth Hall
91362717da
Renamed a plugin hook in sumstats framework.
2013-04-22 15:27:03 -04:00
