Commit graph

1189 commits

Author SHA1 Message Date
Jon Siwek
dc2e3d6e04 Fix global opaque val segfault, addresses BIT-1071
The opaque types need to be created before scripts are parsed.
2013-08-29 17:17:40 -05:00
Jon Siwek
fb8b78840b Fix bloom filter memory leaks. 2013-08-29 11:24:24 -05:00
Robin Sommer
6373d817a5 Merge remote-tracking branch 'origin/topic/jsiwek/uid'
Thanks!

* origin/topic/jsiwek/uid:
  Add bits_per_uid unit test, addressing BIT0-1016.
  UID optimizations addressing BIT-1016.

BIT-1016 #merged
2013-08-28 18:57:00 -07:00
Jon Siwek
7a2972d0e4 Add bits_per_uid unit test, addressing BIT0-1016. 2013-08-28 16:12:11 -05:00
Seth Hall
0592b528ef Merge branch 'master' of ssh://git.bro-ids.org/bro 2013-08-28 00:58:03 -04:00
Seth Hall
2b78922af8 Added a $unique_max field to Reducers for the SumStats::UNIQUE calculation
- Using the new option in scan.bro and the FTP bruteforce detection.
2013-08-28 00:57:44 -04:00
Robin Sommer
f46f692185 Merge remote-tracking branch 'origin/topic/bernhard/hexstr'
BIT-1069 #merged

* origin/topic/bernhard/hexstr:
  add hexstr_to_bytestring bif that does exactly the opposite of bytestring_to_hexstr.
2013-08-27 13:30:25 -07:00
Bernhard Amann
8a5a2b5b39 add hexstr_to_bytestring bif that does exactly the opposite of
bytestring_to_hexstr.
2013-08-27 12:20:03 -07:00
Jon Siwek
50dbbab52c Fix UID compiler warning/error & missed baselines. 2013-08-26 16:11:20 -05:00
Jon Siwek
22bf3e1196 Increase UIDs to 96 bits w/ C/F prefix - BIT-1016
- The bit-length is adjustable via redef'ing bits_per_uid.

- Prefix 'C' is used for connection UIDS (including IP tunnels) and
  'F' for files.
2013-08-26 15:36:31 -05:00
Hui Lin
3e3ca1bb74 fixed number of object bug in dnp3-protocol pac; update two base test trases 2013-08-23 18:10:30 -05:00
Jon Siwek
89ae4ffd05 Add options to limit extracted file sizes w/ 100MB default. 2013-08-22 16:37:58 -05:00
Jon Siwek
71704ab002 Merge branch 'master' into topic/jsiwek/misc 2013-08-19 14:40:09 -05:00
Robin Sommer
ab8d13889e Merge remote-tracking branch 'origin/topic/matthias/bloom-filter'
* origin/topic/matthias/bloom-filter:
  Use Bro-style platform-independent integer types.
  Change bloom filter's dependence on size_t.
  Remove debugging code.
  Update baseline with now correct FP tests.
  Add debugging code to find FP inconsistency.

Conflicts:
	src/3rdparty
2013-08-19 11:26:29 -07:00
Robin Sommer
95f74313d0 Merge branch 'master' of https://github.com/anthonykasza/bro
* 'master' of https://github.com/anthonykasza/bro:
  levenshtein distance function unit test
  levenshtein distance

Conflicts:
	src/3rdparty
2013-08-19 11:20:50 -07:00
Matthias Vallentin
8660c1ab55 Merge ssh://git.bro.org/bro into topic/matthias/bloom-filter 2013-08-16 21:10:37 +02:00
Jon Siwek
e202500f5f Fix a unit test. 2013-08-15 15:53:00 -05:00
Jon Siwek
57b193593d Make mem leak tests able to time out.
This is a workaround for what seems to be a deadlock in gperftools
(seen in their 2.0 and 2.1 releases) that happens occasionally.
2013-08-15 14:27:02 -05:00
Jon Siwek
4198414118 Add a diff canonifier to a test in need of it. 2013-08-14 15:10:47 -05:00
Jon Siwek
c18d1100ab Merge branch 'master' into topic/jsiwek/misc 2013-08-14 14:19:54 -05:00
Jon Siwek
85084f2493 Input framework unit test adjustments.
- Generally increased the time allowed before they timeout.

- For tests w/ a clear termination condition (most of them), made
  timeouts result in a test failure.

- Seemed to be a race in some cases between tests generating output and
  the input reader stream getting removed/closed, so moved stream removal
  closer to termination time, when all output should be available.
2013-08-14 14:10:04 -05:00
Robin Sommer
a646fde884 Merge remote-tracking branch 'origin/topic/seth/unified2-analyzer' into topic/robin/unified2-analyzer-merge
* origin/topic/seth/unified2-analyzer:
  Fixed a problem where the Unified2 analyzer was attached to every file.
  Fixing intel framework tests.
  Updating submodule(s).
  Add file name support to intel framework.
  Add file support to intel framework and slightly restructure intel http handling.

Conflicts:
	CHANGES
	VERSION
	scripts/base/files/unified2/main.bro
	testing/btest/Baseline/coverage.bare-load-baseline/canonified_loaded_scripts.log
	testing/btest/Baseline/coverage.default-load-baseline/canonified_loaded_scripts.log
2013-08-14 10:27:26 -07:00
Jon Siwek
d3dad31bdc Raw input reader command execution "fixes".
- Primarily working around an issue that occurs when threads
  concurrently create pipes and fork a child process.  See comment in
  code...

- Other minor cleanup of the code:  making sure the child process calls
  _exit() versus exit(), limits itself to few select system calls before
  the exec(), and closes more unused file descriptors.
2013-08-14 11:37:30 -05:00
Seth Hall
f8f465e259 Fixed a problem where the Unified2 analyzer was attached to every file. 2013-08-14 01:01:03 -04:00
Seth Hall
efca3c0840 Merge remote-tracking branch 'origin/master' into topic/seth/unified2-analyzer
Conflicts:
	testing/btest/Baseline/coverage.bare-load-baseline/canonified_loaded_scripts.log
	testing/btest/Baseline/coverage.default-load-baseline/canonified_loaded_scripts.log
2013-08-14 00:54:41 -04:00
Seth Hall
a98c78c0d1 Fixing intel framework tests. 2013-08-13 23:49:39 -04:00
Robin Sommer
83eae53f54 Merge remote-tracking branch 'origin/topic/seth/unified2-analyzer'
BIT-1054 #merged

* origin/topic/seth/unified2-analyzer:
  Fixes in case a packet isn't seen that matches an event.
  Finished work on unified2 analyzer.
  Fixed some tests.
  Working unified2 analyzer.
  Unified2 file analyzer updated to new plugin style.
  Adding the unified2 analyzer.

Conflicts:
	testing/btest/Baseline/coverage.bare-load-baseline/canonified_loaded_scripts.log
	testing/btest/Baseline/coverage.default-load-baseline/canonified_loaded_scripts.log
2013-08-13 18:37:52 -07:00
Seth Hall
f7c6dd7f7e Finished work on unified2 analyzer. 2013-08-13 03:21:43 -04:00
anthonykasza
c9313df382 levenshtein distance function unit test 2013-08-12 21:29:57 -05:00
Seth Hall
95161a920c Fixed some tests. 2013-08-12 15:31:31 -04:00
Robin Sommer
45f1b89f60 Merge branch 'topic/robin/dnp3-merge-v3'
Includes a bit more docs/comments cleanup. We should eventually
document the events further but it should suffice for now.

* topic/robin/dnp3-merge-v3:
  Tiny bit of cleanup and adapting the new test.
  added a test case for dnp3 packets with only link layer
  added condition to check DNP3 packet without app layer data
  Fixing well-known port.
  Pluginizing the DNP3 analyzer, plus a basic script logging requests and replies.
2013-08-12 11:39:03 -07:00
Robin Sommer
a927189bdb Tiny bit of cleanup and adapting the new test. 2013-08-11 16:20:08 -07:00
Hui Lin
bced60f7a8 added a test case for dnp3 packets with only link layer 2013-08-11 16:02:44 -07:00
Robin Sommer
36c2433075 Fixing well-known port.
This fixes the remaining test.
2013-08-11 15:59:32 -07:00
Robin Sommer
0e7f51f78c Merge branch 'master' into topic/robin/dnp3-merge-v3
Conflicts:
	scripts/base/init-default.bro
2013-08-09 17:11:51 -07:00
Robin Sommer
214272b8e3 Merge remote-tracking branch 'origin/topic/jsiwek/load-order-fix'
* origin/topic/jsiwek/load-order-fix:
  Update coverage baselines for canonical load order of scripts.
2013-08-09 15:32:30 -07:00
Jon Siwek
4bdbd1762d Update coverage baselines for canonical load order of scripts. 2013-08-09 11:26:49 -05:00
Jon Siwek
db7b6661fe Fix a unit test outdated by recent sumstats changes. 2013-08-09 09:50:05 -05:00
Robin Sommer
2f0671aeeb Updating tests for DHCP. 2013-08-03 20:50:33 -07:00
Robin Sommer
e7aefcdf36 Merge branch 'master' into topic/robin/dhcp-merge 2013-08-03 19:14:01 -07:00
Robin Sommer
308db797c3 Merge remote-tracking branch 'origin/topic/vladg/dhcp'
* origin/topic/vladg/dhcp:
  DHCP: Adding unit tests.
  DHCP:
  Rework the DHCP analyzer to make it compatible again.
2013-08-03 19:14:00 -07:00
Matthias Vallentin
c526ebcfeb Update baseline with now correct FP tests. 2013-08-03 16:54:47 +02:00
Seth Hall
56de65461e Merge remote-tracking branch 'origin/topic/seth/sumstats-updates'
* origin/topic/seth/sumstats-updates:
  Still fixing bugs in sumstats updated api cluster support.
  Hopefully fix the SumStats cluster support.
  Fix the SumStats top-k plugin and test.
  Updates for SumStats API to deal with high memory stats.
  Beginning rework of SumStats API.
  Tiny fix to account for missing str field  (not sure how this happens yet)
  Add server samples to SSH bruteforce detection.
  Fix a reporter message in sumstats.
  SumStats changes to how thresholding works to simplify and reduce memory use.
  More adjustments to try and correct SumStats memory use.
  Hopefully fixing a strange error.
  Large update for the SumStats framework.
2013-08-02 22:37:39 -04:00
Jon Siwek
1531980f3d Exec module and raw input reader fixes.
- Do stream mode for commands done by exec module, it seems important
  in some cases (e.g. ensure requested stdin is fully written).

- For cases where the raw input reader knows the child process has been
  reaped, set the childpid member to a sentinel value to indicate such
  so we don't later think we should kill it or wait on it anymore.

- More error checking on dup2/close calls.  Set sentinel values when
  closing ends of pipes to prevent double closing a fd.

- Signal flag not set when raw input reader's child exits as a result
  of a signal.  Left out a test for this -- might be portability issues
  (e.g. Ubuntu seems to do things different regarding the exit code and
  also is printing "Killed" to stderr where other platforms don't).
2013-08-02 17:27:10 -05:00
Seth Hall
7b8073556e Fix the SumStats top-k plugin and test. 2013-08-02 16:30:20 -04:00
Seth Hall
d6edbd27b1 Merge remote-tracking branch 'origin/master' into topic/seth/sumstats-updates 2013-08-02 13:17:48 -04:00
Seth Hall
4f8100774c Updates for SumStats API to deal with high memory stats.
- The code is a mess and will need to be cleaned up, but the
    tests do pass.
2013-08-02 12:44:33 -04:00
Robin Sommer
306d4fa6f9 Pluginizing the DNP3 analyzer, plus a basic script logging requests
and replies.

Almost ready, but now 1 test fails again ...

[Note I broke git history by copying over the files into a new branch.]
2013-08-01 17:36:11 -07:00
Robin Sommer
04ccb12183 Merge branch 'topic/robin/topk-merge'
BIT-1048 #merged

I'm reverting the serializer version update for now as that breaks
Broccoli. Let's do that later for 2.2.

* topic/robin/topk-merge:
  update documentation, rename get* to Get* and make hasher persistent
  adapt to new folder structure
  fix opaqueval-related memleak
  synchronize pruned attribute
  potentially found wrong Ref.
  add sum function that can be used to get the number of total observed elements.
  in cluster settings, the resultvals can apparently been uninitialized in some special cases
  fix memory leaks
  fix warnings
  add topk cluster test
  make size of topk-list configureable when using sumstats
  implement merging for top-k.
  add serialization for topk
  make the get function const
  topk for sumstats
  well, a test that works..
  implement topk.
2013-08-01 14:39:16 -07:00
Robin Sommer
f6e5de91fa Merge remote-tracking branch 'origin/topic/bernhard/topk' into topic/robin/topk-merge
* origin/topic/bernhard/topk:
  update documentation, rename get* to Get* and make hasher persistent

Conflicts:
	src/probabilistic/Topk.cc
	src/probabilistic/Topk.h
	src/probabilistic/top-k.bif
2013-08-01 14:13:25 -07:00