Commit graph

1220 commits

Author SHA1 Message Date
Robin Sommer
4dcf8fc0db Merge remote-tracking branch 'origin/topic/bernhard/hyperloglog'
* origin/topic/bernhard/hyperloglog: (32 commits)
  add clustered leak test for hll. No issues.
  make gcc happy
  (hopefully) fix refcounting problem in hll/bloom-filter opaque vals. Thanks Robin.
  re-use same hash class for all add operations
  get hll ready for merging
  and forgot a file...
  adapt to new structure
  fix opaqueval-related memleak.
  make it compile on case-sensitive file systems and fix warnings
  make error rate configureable
  add persistence test not using predetermined random seeds.
  update cluster test to also use hll
  persistence really works.
  well, with this commit synchronizing the data structure should work.. ...if we had consistent hashing.
  and also serialize the other things we need
  ok, this bug was hard to find.
  serialization compiles.
  change plugin after feedback of seth
  Forgot a file. Again. Like always. Basically.
  do away with old file.
  ...
2013-08-30 11:30:05 -07:00
Bernhard Amann
2dd0d057e6 Merge remote-tracking branch 'origin/master' into topic/bernhard/hyperloglog
Conflicts:
	src/NetVar.cc
	src/NetVar.h
2013-08-30 08:43:47 -07:00
Jon Siwek
dc2e3d6e04 Fix global opaque val segfault, addresses BIT-1071
The opaque types need to be created before scripts are parsed.
2013-08-29 17:17:40 -05:00
Bernhard Amann
4cd20c6375 add clustered leak test for hll. No issues. 2013-08-29 14:01:22 -07:00
Jon Siwek
fb8b78840b Fix bloom filter memory leaks. 2013-08-29 11:24:24 -05:00
Robin Sommer
6373d817a5 Merge remote-tracking branch 'origin/topic/jsiwek/uid'
Thanks!

* origin/topic/jsiwek/uid:
  Add bits_per_uid unit test, addressing BIT0-1016.
  UID optimizations addressing BIT-1016.

BIT-1016 #merged
2013-08-28 18:57:00 -07:00
Bernhard Amann
dc9fd36497 Merge remote branch 'origin/master' into topic/bernhard/hyperloglog 2013-08-28 17:48:59 -07:00
Jon Siwek
7a2972d0e4 Add bits_per_uid unit test, addressing BIT0-1016. 2013-08-28 16:12:11 -05:00
Seth Hall
0592b528ef Merge branch 'master' of ssh://git.bro-ids.org/bro 2013-08-28 00:58:03 -04:00
Seth Hall
2b78922af8 Added a $unique_max field to Reducers for the SumStats::UNIQUE calculation
- Using the new option in scan.bro and the FTP bruteforce detection.
2013-08-28 00:57:44 -04:00
Robin Sommer
f46f692185 Merge remote-tracking branch 'origin/topic/bernhard/hexstr'
BIT-1069 #merged

* origin/topic/bernhard/hexstr:
  add hexstr_to_bytestring bif that does exactly the opposite of bytestring_to_hexstr.
2013-08-27 13:30:25 -07:00
Bernhard Amann
8a5a2b5b39 add hexstr_to_bytestring bif that does exactly the opposite of
bytestring_to_hexstr.
2013-08-27 12:20:03 -07:00
Jon Siwek
50dbbab52c Fix UID compiler warning/error & missed baselines. 2013-08-26 16:11:20 -05:00
Jon Siwek
22bf3e1196 Increase UIDs to 96 bits w/ C/F prefix - BIT-1016
- The bit-length is adjustable via redef'ing bits_per_uid.

- Prefix 'C' is used for connection UIDS (including IP tunnels) and
  'F' for files.
2013-08-26 15:36:31 -05:00
Bernhard Amann
74f96d22ef Merge remote branch 'origin/master' into topic/bernhard/hyperloglog
Conflicts:
	src/3rdparty
2013-08-26 12:53:13 -07:00
Hui Lin
3e3ca1bb74 fixed number of object bug in dnp3-protocol pac; update two base test trases 2013-08-23 18:10:30 -05:00
Jon Siwek
89ae4ffd05 Add options to limit extracted file sizes w/ 100MB default. 2013-08-22 16:37:58 -05:00
Jon Siwek
71704ab002 Merge branch 'master' into topic/jsiwek/misc 2013-08-19 14:40:09 -05:00
Robin Sommer
ab8d13889e Merge remote-tracking branch 'origin/topic/matthias/bloom-filter'
* origin/topic/matthias/bloom-filter:
  Use Bro-style platform-independent integer types.
  Change bloom filter's dependence on size_t.
  Remove debugging code.
  Update baseline with now correct FP tests.
  Add debugging code to find FP inconsistency.

Conflicts:
	src/3rdparty
2013-08-19 11:26:29 -07:00
Robin Sommer
95f74313d0 Merge branch 'master' of https://github.com/anthonykasza/bro
* 'master' of https://github.com/anthonykasza/bro:
  levenshtein distance function unit test
  levenshtein distance

Conflicts:
	src/3rdparty
2013-08-19 11:20:50 -07:00
Matthias Vallentin
8660c1ab55 Merge ssh://git.bro.org/bro into topic/matthias/bloom-filter 2013-08-16 21:10:37 +02:00
Jon Siwek
e202500f5f Fix a unit test. 2013-08-15 15:53:00 -05:00
Jon Siwek
57b193593d Make mem leak tests able to time out.
This is a workaround for what seems to be a deadlock in gperftools
(seen in their 2.0 and 2.1 releases) that happens occasionally.
2013-08-15 14:27:02 -05:00
Jon Siwek
4198414118 Add a diff canonifier to a test in need of it. 2013-08-14 15:10:47 -05:00
Jon Siwek
c18d1100ab Merge branch 'master' into topic/jsiwek/misc 2013-08-14 14:19:54 -05:00
Jon Siwek
85084f2493 Input framework unit test adjustments.
- Generally increased the time allowed before they timeout.

- For tests w/ a clear termination condition (most of them), made
  timeouts result in a test failure.

- Seemed to be a race in some cases between tests generating output and
  the input reader stream getting removed/closed, so moved stream removal
  closer to termination time, when all output should be available.
2013-08-14 14:10:04 -05:00
Robin Sommer
a646fde884 Merge remote-tracking branch 'origin/topic/seth/unified2-analyzer' into topic/robin/unified2-analyzer-merge
* origin/topic/seth/unified2-analyzer:
  Fixed a problem where the Unified2 analyzer was attached to every file.
  Fixing intel framework tests.
  Updating submodule(s).
  Add file name support to intel framework.
  Add file support to intel framework and slightly restructure intel http handling.

Conflicts:
	CHANGES
	VERSION
	scripts/base/files/unified2/main.bro
	testing/btest/Baseline/coverage.bare-load-baseline/canonified_loaded_scripts.log
	testing/btest/Baseline/coverage.default-load-baseline/canonified_loaded_scripts.log
2013-08-14 10:27:26 -07:00
Jon Siwek
d3dad31bdc Raw input reader command execution "fixes".
- Primarily working around an issue that occurs when threads
  concurrently create pipes and fork a child process.  See comment in
  code...

- Other minor cleanup of the code:  making sure the child process calls
  _exit() versus exit(), limits itself to few select system calls before
  the exec(), and closes more unused file descriptors.
2013-08-14 11:37:30 -05:00
Seth Hall
f8f465e259 Fixed a problem where the Unified2 analyzer was attached to every file. 2013-08-14 01:01:03 -04:00
Seth Hall
efca3c0840 Merge remote-tracking branch 'origin/master' into topic/seth/unified2-analyzer
Conflicts:
	testing/btest/Baseline/coverage.bare-load-baseline/canonified_loaded_scripts.log
	testing/btest/Baseline/coverage.default-load-baseline/canonified_loaded_scripts.log
2013-08-14 00:54:41 -04:00
Seth Hall
a98c78c0d1 Fixing intel framework tests. 2013-08-13 23:49:39 -04:00
Robin Sommer
83eae53f54 Merge remote-tracking branch 'origin/topic/seth/unified2-analyzer'
BIT-1054 #merged

* origin/topic/seth/unified2-analyzer:
  Fixes in case a packet isn't seen that matches an event.
  Finished work on unified2 analyzer.
  Fixed some tests.
  Working unified2 analyzer.
  Unified2 file analyzer updated to new plugin style.
  Adding the unified2 analyzer.

Conflicts:
	testing/btest/Baseline/coverage.bare-load-baseline/canonified_loaded_scripts.log
	testing/btest/Baseline/coverage.default-load-baseline/canonified_loaded_scripts.log
2013-08-13 18:37:52 -07:00
Seth Hall
f7c6dd7f7e Finished work on unified2 analyzer. 2013-08-13 03:21:43 -04:00
anthonykasza
c9313df382 levenshtein distance function unit test 2013-08-12 21:29:57 -05:00
Seth Hall
95161a920c Fixed some tests. 2013-08-12 15:31:31 -04:00
Robin Sommer
45f1b89f60 Merge branch 'topic/robin/dnp3-merge-v3'
Includes a bit more docs/comments cleanup. We should eventually
document the events further but it should suffice for now.

* topic/robin/dnp3-merge-v3:
  Tiny bit of cleanup and adapting the new test.
  added a test case for dnp3 packets with only link layer
  added condition to check DNP3 packet without app layer data
  Fixing well-known port.
  Pluginizing the DNP3 analyzer, plus a basic script logging requests and replies.
2013-08-12 11:39:03 -07:00
Bernhard Amann
2a684cd486 Merge remote-tracking branch 'origin/master' into topic/bernhard/hyperloglog 2013-08-12 09:48:03 -07:00
Bernhard Amann
d83edf8068 Merge remote-tracking branch 'origin/master' into topic/bernhard/hyperloglog
Conflicts:
	src/NetVar.cc
	src/NetVar.h
	src/SerialTypes.h
	src/probabilistic/CMakeLists.txt
	testing/btest/scripts/base/frameworks/sumstats/basic-cluster.bro
	testing/btest/scripts/base/frameworks/sumstats/basic.bro
2013-08-12 09:47:53 -07:00
Robin Sommer
a927189bdb Tiny bit of cleanup and adapting the new test. 2013-08-11 16:20:08 -07:00
Hui Lin
bced60f7a8 added a test case for dnp3 packets with only link layer 2013-08-11 16:02:44 -07:00
Robin Sommer
36c2433075 Fixing well-known port.
This fixes the remaining test.
2013-08-11 15:59:32 -07:00
Robin Sommer
0e7f51f78c Merge branch 'master' into topic/robin/dnp3-merge-v3
Conflicts:
	scripts/base/init-default.bro
2013-08-09 17:11:51 -07:00
Robin Sommer
214272b8e3 Merge remote-tracking branch 'origin/topic/jsiwek/load-order-fix'
* origin/topic/jsiwek/load-order-fix:
  Update coverage baselines for canonical load order of scripts.
2013-08-09 15:32:30 -07:00
Jon Siwek
4bdbd1762d Update coverage baselines for canonical load order of scripts. 2013-08-09 11:26:49 -05:00
Jon Siwek
db7b6661fe Fix a unit test outdated by recent sumstats changes. 2013-08-09 09:50:05 -05:00
Robin Sommer
2f0671aeeb Updating tests for DHCP. 2013-08-03 20:50:33 -07:00
Robin Sommer
e7aefcdf36 Merge branch 'master' into topic/robin/dhcp-merge 2013-08-03 19:14:01 -07:00
Robin Sommer
308db797c3 Merge remote-tracking branch 'origin/topic/vladg/dhcp'
* origin/topic/vladg/dhcp:
  DHCP: Adding unit tests.
  DHCP:
  Rework the DHCP analyzer to make it compatible again.
2013-08-03 19:14:00 -07:00
Matthias Vallentin
c526ebcfeb Update baseline with now correct FP tests. 2013-08-03 16:54:47 +02:00
Seth Hall
56de65461e Merge remote-tracking branch 'origin/topic/seth/sumstats-updates'
* origin/topic/seth/sumstats-updates:
  Still fixing bugs in sumstats updated api cluster support.
  Hopefully fix the SumStats cluster support.
  Fix the SumStats top-k plugin and test.
  Updates for SumStats API to deal with high memory stats.
  Beginning rework of SumStats API.
  Tiny fix to account for missing str field  (not sure how this happens yet)
  Add server samples to SSH bruteforce detection.
  Fix a reporter message in sumstats.
  SumStats changes to how thresholding works to simplify and reduce memory use.
  More adjustments to try and correct SumStats memory use.
  Hopefully fixing a strange error.
  Large update for the SumStats framework.
2013-08-02 22:37:39 -04:00