Commit graph

2220 commits

Author SHA1 Message Date
Bernhard Amann
80962ad74b change names of data structures after talking with seth 2013-05-15 09:44:43 -07:00
Bernhard Amann
b0c4dcdfed make last plugin nicer and samplify sqli detector 2013-05-15 01:09:52 -07:00
Robin Sommer
de88645d05 Merge remote-tracking branch 'origin/topic/bernhard/sqlite'
* origin/topic/bernhard/sqlite:
  fix a few small rough edges (mostly comments that do no longer apply)
  fix bug in input-manager regarding enums that a writer reads without 0-terminating the string
  actually make sqlite work again (tests passed because the writer was not actually defined because of the define.)
  add sqlite distribution.
  fix warnings, update baselines, handle rotation
  add sqlite tests and fix small vector/set escaping bugs
  fix small bug with vectors and sets.
  make work with newer AsciiFormatter.
  start adding a different text for empty records for the sqlite writer.
  no, you will never guess from where I copied this file...
  make sqlite support more or less work for logging and input
  make sqlite-writer more stable.
  make it compile with new version of AsciiInputOutput
  and adapt to AsciiInputOutput - seems to work...
  make it compile
  add SQLite reader.
  ...adapt to new api...
  now the writer supports tables and vectors.
  basic sqlite writer seems to work.
2013-05-14 17:11:09 -07:00
Bernhard Amann
d939c2bdfc add tests for sampler 2013-05-13 22:11:17 -07:00
Bernhard Amann
fa58e26aa0 Merge remote-tracking branch 'origin/master' into topic/bernhard/metrics-samples 2013-05-13 21:20:25 -07:00
Bernhard Amann
bb1e2f57b9 Merge remote-tracking branch 'origin/master' into topic/bernhard/thread-cleanup 2013-05-13 21:19:09 -07:00
Bernhard Amann
56ab9285a4 Merge remote-tracking branch 'origin/master' into topic/bernhard/topk 2013-05-13 21:03:23 -07:00
Bernhard Amann
6392acecd2 fix warnings, update baselines, handle rotation 2013-05-12 20:48:17 -07:00
Bernhard Amann
747ba68030 Merge remote branch 'origin/master' into topic/bernhard/sqlite 2013-05-12 20:47:55 -07:00
Bernhard Amann
70f3f4343a prevent merge-hook of sumstats unique plugin from damaging source data. 2013-05-07 11:16:59 -07:00
Jon Siwek
ec50cad9db Merge branch 'master' into topic/jsiwek/file-analysis
Conflicts:
	scripts/base/protocols/ftp/main.bro
	src/OpaqueVal.h
	testing/btest/Baseline/coverage.bare-load-baseline/canonified_loaded_scripts.log
	testing/btest/Baseline/coverage.default-load-baseline/canonified_loaded_scripts.log
2013-05-06 10:21:16 -05:00
Bernhard Amann
663082e2d5 reservoir sampler. untested. 2013-05-05 11:19:53 -07:00
Bernhard Amann
6acbbe0231 Merge remote-tracking branch 'origin/master' into topic/bernhard/topk 2013-05-03 23:04:22 -07:00
Bernhard Amann
3e74cdc6e0 Merge remote-tracking branch 'origin/master' into topic/bernhard/hyperloglog 2013-05-03 22:58:02 -07:00
Robin Sommer
8992dc6cff Merge remote-tracking branch 'origin/topic/bernhard/metrics-bug'
* origin/topic/bernhard/metrics-bug:
  add comment for seth to make us not forget about the copy statements
  fix the fix (thanks seth)
  duct-tape fix of values not propagating after intermediate check in cluster environments.
  Fixing coverage.bare-mode-errors test.
2013-05-02 12:47:36 -07:00
Bernhard Amann
2cfef36116 add comment for seth to make us not forget about the copy statements 2013-05-02 11:42:34 -07:00
Bernhard Amann
fe779575d5 fix the fix (thanks seth) 2013-05-02 11:38:40 -07:00
Bernhard Amann
d984243a77 duct-tape fix of values not propagating after intermediate check in cluster environments. 2013-05-02 11:34:33 -07:00
Robin Sommer
9d483b7e74 Fixing coverage.bare-mode-errors test. 2013-05-01 17:52:16 -07:00
Robin Sommer
9ea5a470e6 Fixing coverage.bare-mode-errors test. 2013-05-01 15:28:45 -07:00
Bernhard Amann
321dfadaab Merge remote-tracking branch 'origin/topic/robin/metrics-merge' into topic/bernhard/topk 2013-04-29 14:08:17 -07:00
Bernhard Amann
b968103c92 Merge remote-tracking branch 'origin/master' into topic/bernhard/sqlite 2013-04-28 22:06:34 -07:00
Bernhard Amann
07ecd31bbd in cluster settings, the resultvals can apparently
been uninitialized in some special cases
2013-04-28 21:21:22 -07:00
Robin Sommer
b9249ecf9d Layout tweaks for the sumstats code, and preliminary updates for NEWS.
The layout changes are mostly whitespace and some comment rewrapping.
No functional changes.
2013-04-28 15:35:21 -07:00
Bernhard Amann
5608caf79a make error rate configureable 2013-04-25 14:20:13 -07:00
Bernhard Amann
9802e2332d Merge branch 'topic/bernhard/hyperloglog-with-measurement' into topic/bernhard/hyperloglog 2013-04-25 13:46:36 -07:00
Bernhard Amann
166fc4765a Merge remote-tracking branch 'origin/topic/seth/metrics-merge' into topic/bernhard/topk 2013-04-25 13:21:18 -07:00
Seth Hall
317252b5ae Another checkpoint 2013-04-25 13:44:12 -04:00
Seth Hall
48cbb31747 Added an automatic state limiter for threshold based SumStats. 2013-04-25 12:51:55 -04:00
Bernhard Amann
c0890f2a0f make size of topk-list configureable when using sumstats 2013-04-24 15:01:06 -07:00
Seth Hall
d72980828f Merge remote-tracking branch 'origin/topic/jsiwek/file-analysis' into topic/seth/file-analysis-exe-analyzer
Conflicts:
	src/file_analysis/ActionSet.cc
	src/types.bif
2013-04-24 13:01:39 -04:00
Seth Hall
4cc9ca4243 Checkpoint 2013-04-24 12:56:20 -04:00
Bernhard Amann
2f48008c42 implement merging for top-k.
I am not (entirely) sure that this is mathematically correct, but
I am (more and more) getting the feeling that it... might be.

In any case - this was the last step and now it should work
in cluster settings.
2013-04-24 06:17:51 -07:00
Bernhard Amann
567fee6439 Merge remote-tracking branch 'origin/topic/seth/metrics-merge' into topic/bernhard/hyperloglog-with-measurement
Conflicts:
	scripts/base/frameworks/sumstats/plugins/__load__.bro
2013-04-23 15:27:17 -07:00
Bernhard Amann
de5769a88f topk for sumstats 2013-04-23 15:19:01 -07:00
Jon Siwek
f07760ba00 FileAnalysis: add is_orig field to fa_file & Info. 2013-04-23 10:50:43 -05:00
Seth Hall
08348b2bc2 Update to make Dir::monitor watch inodes instead of file names. 2013-04-22 21:53:00 -04:00
Seth Hall
035b668f73 Updates to use new input framework mechanism to execute command line programs. 2013-04-22 21:52:21 -04:00
Seth Hall
91362717da Renamed a plugin hook in sumstats framework. 2013-04-22 15:27:03 -04:00
Seth Hall
9574499382 Move loading variance back to where it should be alphabetically. 2013-04-22 14:15:37 -04:00
Seth Hall
8f987e5066 Fix a bug with path building in FTP. Came up when changing the path utils. 2013-04-22 14:15:20 -04:00
Jon Siwek
98f7907dbb FileAnalysis: optimize file handle construction.
cat is slightly faster than fmt.
2013-04-19 11:38:11 -05:00
Robin Sommer
aeddca6523 More API documentation. 2013-04-16 14:28:23 -07:00
Bernhard Amann
dc18a6d6e3 Merge remote-tracking branch 'origin/topic/seth/metrics-merge' into topic/bernhard/hyperloglog-with-measurement
and fix up the hll scripts for it.

Conflicts:
	scripts/base/frameworks/sumstats/plugins/__load__.bro
	testing/btest/scripts/base/frameworks/measurement/basic.bro
2013-04-16 05:25:10 -07:00
Seth Hall
1cac89e4f8 SumStats test checkpoint. 2013-04-16 00:54:41 -04:00
Seth Hall
437815454d SumStats tests pass. 2013-04-15 15:28:11 -04:00
Seth Hall
fbe967e16a Checkpoint for SumStats rename. 2013-04-15 15:12:28 -04:00
Jon Siwek
037d582b0e FileAnalysis: add custom libmagic database.
- It's derived from the magic database of libmagic 5.14, but with most
  everything not related to mime types removed.

- The custom database is always used by default for mime detection, but
  the more verbose file type detection will fall back on the default
  libmagic installation's database.  The result is: mime type strings
  are now guaranteed to be consistent across platforms, but the verbose
  file type descriptions are not.

- The custom database gets installed in $prefix/share/bro/magic, and
  should even be extensible if files with new patterns are added inside
  the directory.

- The search path for the mime magic database can be controlled via
  BROMAGIC environment variable.

- Remove mime_desc field from ftp.log.

- Stop using the mime/file type canonifier with unit tests.

- libmagic >= 5.04 is now a requirement.
2013-04-12 11:58:19 -05:00
Seth Hall
8165d6077d Fix another occasional reporter error. 2013-04-12 11:20:45 -04:00
Seth Hall
e93fd69cf2 Small updates to hopefully correct reporter errors leading to lost memory. 2013-04-12 09:28:38 -04:00