Commit graph

52 commits

Author SHA1 Message Date
Vern Paxson
7d00ce0082 speed up ZAM compilation by capping function size when inlining 2022-05-14 14:52:31 -07:00
Vern Paxson
7a9c6c476e ZAM maintenance for recent changes and some newly exercised corner cases 2022-05-01 14:33:58 -07:00
Vern Paxson
b59ee83979 update existing test suite usage of "when" statements to include captures 2022-01-07 14:54:06 -08:00
Vern Paxson
b6e9776a11 updates to test suite tests for compatibility with upcoming ZAM functionality 2021-06-01 09:25:30 -07:00
Johanna Amann
7362f30c3a Sumstats: allow users to manage epoch manually
This change allows users to specify an epoch length of 0, which means
that the user manually has to finish the epochs. A new next_epoch
function is introduced to allow users to manually end epochs.

Addresses GH-348
2020-11-16 15:48:46 +00:00
Jon Siwek
ccf8e271a1 Adjust more btest timings 2020-08-11 11:26:22 -07:00
Jon Siwek
7967a5b0aa General btest cleanup
- Use `-b` most everywhere, it will save time.

- Start some intel tests upon the input file being fully read instead of
  at an arbitrary time.

- Improve termination condition for some sumstats/cluster tests.

- Filter uninteresting output from some supervisor tests.

- Test for `notice_policy.log` is no longer needed.
2020-08-11 11:26:22 -07:00
Jon Siwek
48a243069c Adjust a few btests that were unstable due to time-sensitivity 2020-08-11 11:26:21 -07:00
Jon Siwek
b96b1593d8 Sort output of a few SumStats cluster tests
The order of $epoch_result() function calls among all keys within a
single epoch isn't consequential.
2020-08-11 11:26:21 -07:00
Jon Siwek
4fbcca04e8 Improve btest timeouts
* Generally increase timeouts for tests that have recent transient
  failures

* Change any test that relied on `btest-bg-wait -k` since that's never
  going to play with with CI systems.  Instead, we always need to have
  a well-defined termination condition in the test itself (and most
  already did, so didn't really need the `-k` flag anyway).
2020-02-06 17:50:17 -08:00
Jon Siwek
580822a32c Merge remote-tracking branch 'origin/topic/dnthayer/gh-339'
* origin/topic/dnthayer/gh-339:
  Rename all BRO-prefixed environment variables
2019-05-23 20:23:14 -07:00
Johanna Amann
74bb7716f6 Finish implementation of copy method.
All types (besides EntropyVal) now support a native copy operation,
which uses primitives of the underlying datatypes to perform a quick
copy, without serialization.

EntropyVal is the one exception - since that type is rather complex
(many members) and will probably not be copied a lot, if at all, it
makes sense to just use the serialization function.

This will have to be slightly re-written in the near-term-future to use
the new serialization function for that opaque type.

This change also introduces a new x509_from_der bif, which allows to
parse a der into an opaque of x509.

This change removes the d2i_X509_ wrapper function; this was a remnant
when d2i_X509 took non-const arguments. We directly use d2i_X509 at
several places assuming const-ness, so there does not seem to ba a
reason to keep the wrapper.

This change also exposed a problem in the File cache - cases in which an
object was brought back into the cache, and writing occurred in the
file_open event were never correctly handeled as far as I can tell.
2019-05-22 14:29:37 -07:00
Daniel Thayer
1a74516db1 Rename all BRO-prefixed environment variables
For backward compatibility when reading values, we first check
the ZEEK-prefixed value, and if not set, then check the corresponding
BRO-prefixed value.
2019-05-22 00:12:31 -05:00
Robin Sommer
789cb376fd GH-239: Rename bro to zeek, bro-config to zeek-config, and bro-path-dev to zeek-path-dev.
This also installs symlinks from "zeek" and "bro-config" to a wrapper
script that prints a deprecation warning.

The btests pass, but this is still WIP. broctl renaming is still
missing.

#239
2019-05-01 21:43:45 +00:00
Jon Siwek
a994be9eeb Merge remote-tracking branch 'origin/topic/seth/zeek_init'
* origin/topic/seth/zeek_init:
  Some more testing fixes.
  Update docs and tests for bro_(init|done) -> zeek_(init|done)
  Implement the zeek_init handler.
2019-04-19 11:24:29 -07:00
Johanna Amann
9421ee0293 Merge branch 'topic/jsbarber/fix-topk-merge-core-dump' of https://github.com/jsbarber/bro
* 'topic/jsbarber/fix-topk-merge-core-dump' of https://github.com/jsbarber/bro:
  Prevent topk_merge from crashing when second argument is empty set
2019-04-18 09:36:48 +02:00
Jon Siwek
1e57e3f026 Use .zeek file suffix in unit tests 2019-04-16 16:08:57 -07:00
Seth Hall
5db766bd88 Update docs and tests for bro_(init|done) -> zeek_(init|done) 2019-04-14 08:49:12 -04:00
Jeff Barber
0d61df30b7 Prevent topk_merge from crashing when second argument is empty set 2019-04-10 15:10:29 -07:00
Jon Siwek
fd63168171 Merge remote-tracking branch 'origin/topic/jsiwek/parallelize-comm-tests'
* origin/topic/jsiwek/parallelize-comm-tests:
  Parallelize communication tests using btest TEST-PORT
2018-11-29 17:06:30 -06:00
Jon Siwek
01e17b5ea0 Fix SumStats "last" plugin in cluster mode 2018-11-08 12:27:37 -06:00
Jon Siwek
907297ba59 Parallelize communication tests using btest TEST-PORT 2018-11-04 15:29:59 -06:00
Jon Siwek
da9f91fc19 Add env. variables to override Broker listen/connect retry intervals
And use them to default retries to 1sec for all unit tests.
2018-08-16 12:16:03 -05:00
Vern Paxson
88fd7510c6 reap the fruits of v += e 2018-07-26 12:51:36 -07:00
Jon Siwek
15d74ac081 BIT-1941: improve unit test stability
Mostly trying to standardize the way tests sleep for arbitrary amounts
of time to make it easier to tell at which particular point the
unit test actually may need the timeout interval increased (or else
debugged further).
2018-07-03 15:00:52 -05:00
Robin Sommer
fe7e1ee7f0 Merge topic/actor-system throug a squashed commit. 2018-05-18 22:39:23 +00:00
Bernhard Amann
b283883997 define empty request_key method for sumstats in cluster mode.
This prevents the worker nodes from crashing, when request_key is used
in cluster mode and called on the worker and the manager nodes (i.e. when
a non-cluster-aware script is used).

Addresses BIT-1177
2014-04-18 16:29:51 -07:00
Seth Hall
a07a40dc86 Now the standalone sumstats works even if Bro is shutting down by blocking. 2014-03-17 16:45:36 -04:00
Seth Hall
af36915186 Change the standalone sumstats mode to work incrementally. 2014-03-17 16:06:14 -04:00
Robin Sommer
a8ac7d8a5b Updating tests for HLL merge. 2013-08-31 11:17:49 -07:00
Robin Sommer
4dcf8fc0db Merge remote-tracking branch 'origin/topic/bernhard/hyperloglog'
* origin/topic/bernhard/hyperloglog: (32 commits)
  add clustered leak test for hll. No issues.
  make gcc happy
  (hopefully) fix refcounting problem in hll/bloom-filter opaque vals. Thanks Robin.
  re-use same hash class for all add operations
  get hll ready for merging
  and forgot a file...
  adapt to new structure
  fix opaqueval-related memleak.
  make it compile on case-sensitive file systems and fix warnings
  make error rate configureable
  add persistence test not using predetermined random seeds.
  update cluster test to also use hll
  persistence really works.
  well, with this commit synchronizing the data structure should work.. ...if we had consistent hashing.
  and also serialize the other things we need
  ok, this bug was hard to find.
  serialization compiles.
  change plugin after feedback of seth
  Forgot a file. Again. Like always. Basically.
  do away with old file.
  ...
2013-08-30 11:30:05 -07:00
Bernhard Amann
d83edf8068 Merge remote-tracking branch 'origin/master' into topic/bernhard/hyperloglog
Conflicts:
	src/NetVar.cc
	src/NetVar.h
	src/SerialTypes.h
	src/probabilistic/CMakeLists.txt
	testing/btest/scripts/base/frameworks/sumstats/basic-cluster.bro
	testing/btest/scripts/base/frameworks/sumstats/basic.bro
2013-08-12 09:47:53 -07:00
Seth Hall
7b8073556e Fix the SumStats top-k plugin and test. 2013-08-02 16:30:20 -04:00
Seth Hall
d6edbd27b1 Merge remote-tracking branch 'origin/master' into topic/seth/sumstats-updates 2013-08-02 13:17:48 -04:00
Seth Hall
4f8100774c Updates for SumStats API to deal with high memory stats.
- The code is a mess and will need to be cleaned up, but the
    tests do pass.
2013-08-02 12:44:33 -04:00
Bernhard Amann
9e0fd963e0 Merge remote-tracking branch 'origin/topic/robin/bloom-filter-merge' into topic/bernhard/hyperloglog
Conflicts:
	scripts/base/frameworks/sumstats/plugins/__load__.bro
	src/CMakeLists.txt
	src/NetVar.cc
	src/NetVar.h
	src/OpaqueVal.h
	src/SerialTypes.h
	src/bro.bif
2013-07-23 21:31:05 -07:00
Bernhard Amann
03b584c34a Merge remote-tracking branch 'origin/master' into topic/bernhard/topk 2013-07-09 14:56:05 -07:00
Seth Hall
0a18b62d12 Merge remote-tracking branch 'origin/master' into topic/seth/sumstats-updates
Conflicts:
	scripts/base/frameworks/sumstats/cluster.bro
	scripts/base/frameworks/sumstats/plugins/average.bro
	scripts/base/frameworks/sumstats/plugins/max.bro
	scripts/base/frameworks/sumstats/plugins/min.bro
	scripts/base/frameworks/sumstats/plugins/sample.bro
	scripts/base/frameworks/sumstats/plugins/std-dev.bro
	scripts/base/frameworks/sumstats/plugins/sum.bro
	scripts/base/frameworks/sumstats/plugins/unique.bro
	scripts/base/frameworks/sumstats/plugins/variance.bro
	scripts/policy/protocols/http/detect-sqli.bro
	testing/btest/scripts/base/frameworks/sumstats/cluster-intermediate-update.bro
2013-05-21 22:33:16 -04:00
Seth Hall
bec965b66f Large update for the SumStats framework.
- On-demand access to sumstats results through "return from"
   functions named SumStats::request and Sumstats::request_key.
   Both functions are tested in standalone and clustered modes.

 - $name field has returned to SumStats which simplifies cluster
   code and makes the on-demand access stuff possible.

 - Clustered results can only be collected for 1 minute from their
   time of creation now instead of time of last read.

 - Thresholds use doubles instead of counts everywhere now.

 - Calculation dependency resolution occurs at start up time now
   instead of doing it at observation time which provide a minor
   cpu performance improvement.  A new plugin registration mechanism
   was created to support this change.

 - AppStats now has a minimal doc string and is broken into hook-based
   plugins.

 - AppStats and traceroute detection added to local.bro
2013-05-21 15:52:59 -04:00
Bernhard Amann
ab6d5b08a8 finishing touches, make test more robust, rename function in last again 2013-05-15 11:33:25 -07:00
Bernhard Amann
80962ad74b change names of data structures after talking with seth 2013-05-15 09:44:43 -07:00
Bernhard Amann
d939c2bdfc add tests for sampler 2013-05-13 22:11:17 -07:00
Bernhard Amann
663082e2d5 reservoir sampler. untested. 2013-05-05 11:19:53 -07:00
Bernhard Amann
6acbbe0231 Merge remote-tracking branch 'origin/master' into topic/bernhard/topk 2013-05-03 23:04:22 -07:00
Bernhard Amann
3e74cdc6e0 Merge remote-tracking branch 'origin/master' into topic/bernhard/hyperloglog 2013-05-03 22:58:02 -07:00
Bernhard Amann
d984243a77 duct-tape fix of values not propagating after intermediate check in cluster environments. 2013-05-02 11:34:33 -07:00
Bernhard Amann
12cbf20ce0 add topk cluster test 2013-04-24 15:30:24 -07:00
Bernhard Amann
de5769a88f topk for sumstats 2013-04-23 15:19:01 -07:00
Bernhard Amann
6e532e8960 update cluster test to also use hll 2013-04-19 09:58:57 -07:00
Bernhard Amann
dc18a6d6e3 Merge remote-tracking branch 'origin/topic/seth/metrics-merge' into topic/bernhard/hyperloglog-with-measurement
and fix up the hll scripts for it.

Conflicts:
	scripts/base/frameworks/sumstats/plugins/__load__.bro
	testing/btest/scripts/base/frameworks/measurement/basic.bro
2013-04-16 05:25:10 -07:00