Commit graph

845 commits

Author SHA1 Message Date
Bernhard Amann
6acbbe0231 Merge remote-tracking branch 'origin/master' into topic/bernhard/topk 2013-05-03 23:04:22 -07:00
Bernhard Amann
3e74cdc6e0 Merge remote-tracking branch 'origin/master' into topic/bernhard/hyperloglog 2013-05-03 22:58:02 -07:00
Robin Sommer
8992dc6cff Merge remote-tracking branch 'origin/topic/bernhard/metrics-bug'
* origin/topic/bernhard/metrics-bug:
  add comment for seth to make us not forget about the copy statements
  fix the fix (thanks seth)
  duct-tape fix of values not propagating after intermediate check in cluster environments.
  Fixing coverage.bare-mode-errors test.
2013-05-02 12:47:36 -07:00
Bernhard Amann
2cfef36116 add comment for seth to make us not forget about the copy statements 2013-05-02 11:42:34 -07:00
Bernhard Amann
fe779575d5 fix the fix (thanks seth) 2013-05-02 11:38:40 -07:00
Bernhard Amann
d984243a77 duct-tape fix of values not propagating after intermediate check in cluster environments. 2013-05-02 11:34:33 -07:00
Robin Sommer
9d483b7e74 Fixing coverage.bare-mode-errors test. 2013-05-01 17:52:16 -07:00
Robin Sommer
9ea5a470e6 Fixing coverage.bare-mode-errors test. 2013-05-01 15:28:45 -07:00
Bernhard Amann
321dfadaab Merge remote-tracking branch 'origin/topic/robin/metrics-merge' into topic/bernhard/topk 2013-04-29 14:08:17 -07:00
Bernhard Amann
b968103c92 Merge remote-tracking branch 'origin/master' into topic/bernhard/sqlite 2013-04-28 22:06:34 -07:00
Bernhard Amann
07ecd31bbd in cluster settings, the resultvals can apparently
been uninitialized in some special cases
2013-04-28 21:21:22 -07:00
Robin Sommer
b9249ecf9d Layout tweaks for the sumstats code, and preliminary updates for NEWS.
The layout changes are mostly whitespace and some comment rewrapping.
No functional changes.
2013-04-28 15:35:21 -07:00
Bernhard Amann
5608caf79a make error rate configureable 2013-04-25 14:20:13 -07:00
Bernhard Amann
9802e2332d Merge branch 'topic/bernhard/hyperloglog-with-measurement' into topic/bernhard/hyperloglog 2013-04-25 13:46:36 -07:00
Bernhard Amann
166fc4765a Merge remote-tracking branch 'origin/topic/seth/metrics-merge' into topic/bernhard/topk 2013-04-25 13:21:18 -07:00
Seth Hall
48cbb31747 Added an automatic state limiter for threshold based SumStats. 2013-04-25 12:51:55 -04:00
Bernhard Amann
c0890f2a0f make size of topk-list configureable when using sumstats 2013-04-24 15:01:06 -07:00
Bernhard Amann
2f48008c42 implement merging for top-k.
I am not (entirely) sure that this is mathematically correct, but
I am (more and more) getting the feeling that it... might be.

In any case - this was the last step and now it should work
in cluster settings.
2013-04-24 06:17:51 -07:00
Bernhard Amann
567fee6439 Merge remote-tracking branch 'origin/topic/seth/metrics-merge' into topic/bernhard/hyperloglog-with-measurement
Conflicts:
	scripts/base/frameworks/sumstats/plugins/__load__.bro
2013-04-23 15:27:17 -07:00
Bernhard Amann
de5769a88f topk for sumstats 2013-04-23 15:19:01 -07:00
Jon Siwek
f07760ba00 FileAnalysis: add is_orig field to fa_file & Info. 2013-04-23 10:50:43 -05:00
Seth Hall
91362717da Renamed a plugin hook in sumstats framework. 2013-04-22 15:27:03 -04:00
Seth Hall
9574499382 Move loading variance back to where it should be alphabetically. 2013-04-22 14:15:37 -04:00
Robin Sommer
aeddca6523 More API documentation. 2013-04-16 14:28:23 -07:00
Bernhard Amann
dc18a6d6e3 Merge remote-tracking branch 'origin/topic/seth/metrics-merge' into topic/bernhard/hyperloglog-with-measurement
and fix up the hll scripts for it.

Conflicts:
	scripts/base/frameworks/sumstats/plugins/__load__.bro
	testing/btest/scripts/base/frameworks/measurement/basic.bro
2013-04-16 05:25:10 -07:00
Seth Hall
1cac89e4f8 SumStats test checkpoint. 2013-04-16 00:54:41 -04:00
Seth Hall
437815454d SumStats tests pass. 2013-04-15 15:28:11 -04:00
Seth Hall
fbe967e16a Checkpoint for SumStats rename. 2013-04-15 15:12:28 -04:00
Seth Hall
8165d6077d Fix another occasional reporter error. 2013-04-12 11:20:45 -04:00
Seth Hall
e93fd69cf2 Small updates to hopefully correct reporter errors leading to lost memory. 2013-04-12 09:28:38 -04:00
Jon Siwek
b8c98b8bf7 FileAnalysis: change terminology s/action/analyzer 2013-04-11 14:53:54 -05:00
Jon Siwek
e81f2ae7b0 FileAnalysis: libmagic tweaks.
Remove verbose file type detection and automatically strip out charset
from mime type.
2013-04-11 13:11:46 -05:00
Jon Siwek
2fba37e277 FileAnalysis: add bif for setting timeout interval 2013-04-11 12:08:46 -05:00
Seth Hall
a615601269 Trying to fix a state maintenance issue. 2013-04-11 09:42:46 -04:00
Jon Siwek
a2d9b47bcd FileAnalysis: finish switching hooks to events. 2013-04-10 11:13:43 -05:00
Bernhard Amann
f10ed9e29a change plugin after feedback of seth 2013-04-10 10:45:45 -04:00
Jon Siwek
641154f8e8 FileAnalysis: checkpoint in middle of big reorganization.
- FileAnalysis::Info is now just a record used for logging, the fa_file
  record type is defined in init-bare.bro as the analogue to a
  connection record.

- Starting to transfer policy hook triggers and analyzer results to
  events.
2013-04-09 15:49:58 -05:00
Bernhard Amann
07d44f3aa0 Merge remote-tracking branch 'origin/topic/seth/metrics-merge' into topic/bernhard/hyperloglog-with-measurement 2013-04-08 10:56:18 +02:00
Bernhard Amann
bcd610fd50 Forgot a file. Again. Like always. Basically. 2013-04-08 10:55:00 +02:00
Bernhard Amann
7eee2f0d17 measurement framework with hll unique 2013-04-08 10:00:34 +02:00
Bernhard Amann
2cc1f82425 Merge remote-tracking branch 'origin/master' into topic/bernhard/thread-cleanup 2013-04-07 20:43:47 +02:00
Robin Sommer
bccaea6883 Adding options Analyzer::disable_all to disable all analyzers at
startup.

One can then selectively enable the ones one wants inside a bro_init()
handler.
2013-04-04 15:24:15 -07:00
Seth Hall
e8b60d1ba8 Updated FTP bruteforce detection and a few other small changes. 2013-04-02 00:55:25 -04:00
Seth Hall
423bf3b3bf Test updates and cleanup. 2013-04-02 00:30:14 -04:00
Seth Hall
0e3c84e863 Fixed the measurement "sample" plugin. 2013-04-02 00:19:06 -04:00
Seth Hall
b477d2b02d Measurement framework is ready for testing.
- New, expanded API.
 - Calculations moved into plugins.
 - Scripts using measurement framework ported.
 - Updated the script-land queue implementation to make it more generic.
 -
2013-04-01 17:04:15 -04:00
Robin Sommer
e0c4bd1a82 Lots of cleanup and API documentation for the analyzer/* classes.
I've used the opportunity to also cleanup DPD's expect_connection()
infrastructure, and renamed that bif to schedule_analyzer(), which
seems more appropiate. One can now also schedule more than one
analyzer per connection.

TODOs:
        - "make install" is probably broken.
        - Broxygen is probably broken for plugin-defined events.
        - event groups are broken (do we want to keep them?)
        - parallel btest is broken, but I'm not sure why ...
          (tests all pass individually, but lots of error when running
          in parallel; must be related to *.bif restructuring).
        - Document API for src/plugin/*
        - Document API for src/analyzer/Analyzer.h
        - Document API for scripts/base/frameworks/analyzer
2013-04-01 13:12:21 -07:00
Seth Hall
93eca70e6b Merge remote-tracking branch 'origin/master' into topic/seth/metrics-merge 2013-04-01 14:16:46 -04:00
Seth Hall
53f9948b02 Measurement framework tests all pass now. 2013-04-01 14:16:37 -04:00
Robin Sommer
19c1816ebb Infrastructure for modularizing protocol analyzers.
There's now a new directory "src/protocols/", and the plan is for each
protocol analyzer to eventually have its own subdirectory in there
that contains everything it defines (C++/pac/bif). The infrastructure
to make that happen is in place, and two analyzers have been
converted to the new model, HTTP and SSL; there's no further
HTTP/SSL-specific code anywhere else in the core anymore (I believe :-)

Further changes:

    - -N lists available plugins, -NN lists more details on what these
      plugins provide (analyzers, bif elements). (The latter does not
      work for analyzers that haven't been converted yet).

    - *.bif.bro files now go into scripts/base/bif/; and
      scripts/base/bif/plugins/ for bif files provided by plugins.

    - I've factored out the bifcl/binpac CMake magic from
      src/CMakeLists.txt to cmake/{BifCl,Binpac}

    - There's a new cmake/BroPlugin that contains magic to allow
      plugins to have a simple CMakeLists.txt. The hope is that
      eventually the same CMakeLists.txt can be used for compiling a
      plugin either statically or dynamically.

    - bifcl has a new option -c that changes the code it generates so
      that it can be used with a plugin.

TODOs:
    - "make install" is probably broken.
    - Broxygen is probably broken for plugin-defined events.
    - event groups are broken (do we want to keep them?)
2013-03-29 19:59:31 -07:00