Commit graph

4428 commits

Author SHA1 Message Date
Robin Sommer
f4fc9dd947 Merge remote-tracking branch 'origin/topic/bernhard/sqlite' into topic/robin/sqlite-merge
* origin/topic/bernhard/sqlite:
  and there is no has-reader.
  make sqlite3 executable required and add test-cases for errors
2013-05-15 15:24:02 -07:00
Robin Sommer
e050648621 Merge branch 'topic/robin/file-analysis-merge'
Closes #982.

* topic/robin/file-analysis-merge: (64 commits)
  A few more small tweaks.
  Various smalle tweaks in preparation for merging.
  FileAnalysis: load custom mime magic database just once.
  Improve a libmagic-related error message.
  FileAnalysis: add is_orig field to fa_file & Info.
  FileAnalysis: inlined doc fixes.
  FileAnalysis: optimizate connection set updating.
  FileAnalysis: optimize file handle construction.
  FileAnalysis: workarounds for older libmagics.
  FileAnalysis: add custom libmagic database.
  FileAnalysis: change terminology s/action/analyzer
  FileAnalysis: libmagic tweaks.
  FileAnalysis: add bif for setting timeout interval
  FileAnalysis: add more params to some events.
  FileAnalysis: insert explicit event queue flush points.
  FileAnalysis: remove some file events.
  FileAnalysis: finish switching hooks to events.
  FileAnalysis: checkpoint in middle of big reorganization.
  FileAnalysis: fix file type canonification for file_analysis.log
  Revert "FileAnalysis: optimize get_file_handle event queueing."
  ...

Conflicts:
	NEWS
2013-05-15 15:14:21 -07:00
Bernhard Amann
ab6d5b08a8 finishing touches, make test more robust, rename function in last again 2013-05-15 11:33:25 -07:00
Bernhard Amann
80962ad74b change names of data structures after talking with seth 2013-05-15 09:44:43 -07:00
Bernhard Amann
2b009939f5 and there is no has-reader. 2013-05-14 22:12:23 -07:00
Bernhard Amann
2ad67643c0 make sqlite3 executable required and add test-cases for errors
logging: table does not contain all required columns (when extending
data structures)

input: table does not contain all required columns (when extending
data structure), wrong sql statement
2013-05-14 22:09:46 -07:00
Robin Sommer
de88645d05 Merge remote-tracking branch 'origin/topic/bernhard/sqlite'
* origin/topic/bernhard/sqlite:
  fix a few small rough edges (mostly comments that do no longer apply)
  fix bug in input-manager regarding enums that a writer reads without 0-terminating the string
  actually make sqlite work again (tests passed because the writer was not actually defined because of the define.)
  add sqlite distribution.
  fix warnings, update baselines, handle rotation
  add sqlite tests and fix small vector/set escaping bugs
  fix small bug with vectors and sets.
  make work with newer AsciiFormatter.
  start adding a different text for empty records for the sqlite writer.
  no, you will never guess from where I copied this file...
  make sqlite support more or less work for logging and input
  make sqlite-writer more stable.
  make it compile with new version of AsciiInputOutput
  and adapt to AsciiInputOutput - seems to work...
  make it compile
  add SQLite reader.
  ...adapt to new api...
  now the writer supports tables and vectors.
  basic sqlite writer seems to work.
2013-05-14 17:11:09 -07:00
Bernhard Amann
d939c2bdfc add tests for sampler 2013-05-13 22:11:17 -07:00
Bernhard Amann
fa58e26aa0 Merge remote-tracking branch 'origin/master' into topic/bernhard/metrics-samples 2013-05-13 21:20:25 -07:00
Bernhard Amann
56ab9285a4 Merge remote-tracking branch 'origin/master' into topic/bernhard/topk 2013-05-13 21:03:23 -07:00
Bernhard Amann
8f7619971f fix bug in input-manager regarding enums that a writer reads without
0-terminating the string
2013-05-13 20:26:41 -07:00
Bernhard Amann
6c99df508c actually make sqlite work again (tests passed because the writer
was not actually defined because of the define.)
2013-05-13 19:27:11 -07:00
Robin Sommer
1459216b20 Merge branch 'topic/robin/file-analysis-merge' of git.bro-ids.org:bro into topic/robin/file-analysis-merge
Conflicts:
	testing/btest/Baseline/coverage.bare-load-baseline/canonified_loaded_scripts.log
	testing/btest/Baseline/coverage.default-load-baseline/canonified_loaded_scripts.log
2013-05-13 17:01:55 -07:00
Bernhard Amann
6392acecd2 fix warnings, update baselines, handle rotation 2013-05-12 20:48:17 -07:00
Bernhard Amann
747ba68030 Merge remote branch 'origin/master' into topic/bernhard/sqlite 2013-05-12 20:47:55 -07:00
Jon Siwek
e2a1d4a233 Allow default function/hook/event parameters. Addresses #972.
And changed the endianness parameter of bytestring_to_count() BIF to
default to false (big endian), mostly just to prove that the BIF parser
doesn't choke on default parameters.
2013-05-07 14:32:22 -05:00
Scott Runnels
4aa51e7608 Basic cross-referencing UIDs between files, btests, and baselines.
Also includes appropriate btest-rst-cmd directives with titles.
2013-05-07 13:33:38 -04:00
Scott Runnels
a93a6535e7 Using redirection with bro-cut. Include initial btests for this document. 2013-05-07 05:56:45 -04:00
Jon Siwek
ec50cad9db Merge branch 'master' into topic/jsiwek/file-analysis
Conflicts:
	scripts/base/protocols/ftp/main.bro
	src/OpaqueVal.h
	testing/btest/Baseline/coverage.bare-load-baseline/canonified_loaded_scripts.log
	testing/btest/Baseline/coverage.default-load-baseline/canonified_loaded_scripts.log
2013-05-06 10:21:16 -05:00
Bernhard Amann
663082e2d5 reservoir sampler. untested. 2013-05-05 11:19:53 -07:00
Bernhard Amann
6acbbe0231 Merge remote-tracking branch 'origin/master' into topic/bernhard/topk 2013-05-03 23:04:22 -07:00
Bernhard Amann
3e74cdc6e0 Merge remote-tracking branch 'origin/master' into topic/bernhard/hyperloglog 2013-05-03 22:58:02 -07:00
Robin Sommer
a1bae68140 Merge remote-tracking branch 'origin/master' into topic/robin/file-analysis-merge
Conflicts:
	scripts/base/protocols/ftp/main.bro
	testing/btest/Baseline/coverage.bare-load-baseline/canonified_loaded_scripts.log
	testing/btest/Baseline/coverage.default-load-baseline/canonified_loaded_scripts.log
2013-05-03 13:52:26 -07:00
Jon Siwek
6a7a242db9 Table lookups return copy of non-const &default vals (addresses #981).
This prevents unintentional modifications to the &default value itself.
2013-05-03 11:22:15 -05:00
Robin Sommer
8992dc6cff Merge remote-tracking branch 'origin/topic/bernhard/metrics-bug'
* origin/topic/bernhard/metrics-bug:
  add comment for seth to make us not forget about the copy statements
  fix the fix (thanks seth)
  duct-tape fix of values not propagating after intermediate check in cluster environments.
  Fixing coverage.bare-mode-errors test.
2013-05-02 12:47:36 -07:00
Bernhard Amann
d984243a77 duct-tape fix of values not propagating after intermediate check in cluster environments. 2013-05-02 11:34:33 -07:00
Bernhard Amann
5e01c34943 Merge remote-tracking branch 'origin/master' into topic/bernhard/hyperloglog 2013-05-01 18:04:39 -07:00
Robin Sommer
1603da5af3 Always apply tcp_connection_attempt.
Before this change it was only applied when a connection_attempt()
event handler was defined.
2013-05-01 18:03:52 -07:00
Robin Sommer
9d483b7e74 Fixing coverage.bare-mode-errors test. 2013-05-01 17:52:16 -07:00
Robin Sommer
9ea5a470e6 Fixing coverage.bare-mode-errors test. 2013-05-01 15:28:45 -07:00
Bernhard Amann
8680e2da00 Merge remote-tracking branch 'origin/topic/robin/metrics-merge' into topic/bernhard/topk 2013-04-29 22:09:46 -07:00
Robin Sommer
7f0e25bdef Replacing cluster leak test.
The former one used the old metrics framework, now switching to
sumstats.
2013-04-29 21:14:11 -07:00
Bernhard Amann
321dfadaab Merge remote-tracking branch 'origin/topic/robin/metrics-merge' into topic/bernhard/topk 2013-04-29 14:08:17 -07:00
Bernhard Amann
b968103c92 Merge remote-tracking branch 'origin/master' into topic/bernhard/sqlite 2013-04-28 22:06:34 -07:00
Bernhard Amann
160da6f1a6 add sum function that can be used to get the number of total
observed elements.

Add methods to merge with and without pruning (before only merge
method was with pruning, which invalidates the number of total
observed elements)
2013-04-28 21:55:06 -07:00
Robin Sommer
c1f08cc435 Updating test for removed metric_* log fields. 2013-04-28 15:36:49 -07:00
Robin Sommer
1e40a2f88c Merge remote-tracking branch 'origin/topic/seth/metrics-merge'
* origin/topic/seth/metrics-merge: (70 commits)
  Added protocol to the traceroute detection script.
  Added an automatic state limiter for threshold based SumStats.
  Removed some dead code in scan.bro
  Renamed a plugin hook in sumstats framework.
  Move loading variance back to where it should be alphabetically.
  Fix a bug with path building in FTP.  Came up when changing the path utils.
  Fix a few tests.
  SumStats test checkpoint.
  SumStats tests pass.
  Checkpoint for SumStats rename.
  Fix another occasional reporter error.
  Small updates to hopefully correct reporter errors leading to lost memory.
  Trying to fix a state maintenance issue.
  Updating DocSourcesList
  Updated FTP bruteforce detection and a few other small changes.
  Test updates and cleanup.
  Fixed the measurement "sample" plugin.
  Fix path compression to include removing "/./".
  Removed the example metrics scripts. Better real world examples exist now.
  Measurement framework is ready for testing.
  ...
2013-04-28 13:21:46 -07:00
Scott Runnels
cb3e05edd4 Include Notice Policy shortcuts in the Scripting User Manual.
Include two tests and baselines for the RST output.
2013-04-28 15:48:44 -04:00
Bernhard Amann
9802e2332d Merge branch 'topic/bernhard/hyperloglog-with-measurement' into topic/bernhard/hyperloglog 2013-04-25 13:46:36 -07:00
Bernhard Amann
32620952d0 Merge remote-tracking branch 'origin/topic/seth/metrics-merge' into topic/bernhard/hyperloglog 2013-04-25 13:45:30 -07:00
Seth Hall
4bddcd2379 Fixed a bug in the vulnerable software script and added a test. 2013-04-25 14:56:14 -04:00
Bernhard Amann
f2967f485b add persistence test not using predetermined random seeds.
This is failing at the moment.
2013-04-24 16:03:40 -07:00
Bernhard Amann
f69db71f57 Merge remote-tracking branch 'origin/master' into topic/bernhard/hyperloglog 2013-04-24 16:01:05 -07:00
Bernhard Amann
12cbf20ce0 add topk cluster test 2013-04-24 15:30:24 -07:00
Bernhard Amann
dbd53a09a6 Merge remote-tracking branch 'origin/master' into topic/bernhard/topk 2013-04-24 15:02:19 -07:00
Bernhard Amann
2f48008c42 implement merging for top-k.
I am not (entirely) sure that this is mathematically correct, but
I am (more and more) getting the feeling that it... might be.

In any case - this was the last step and now it should work
in cluster settings.
2013-04-24 06:17:51 -07:00
Bernhard Amann
6f863d2259 add serialization for topk 2013-04-23 23:24:02 -07:00
Robin Sommer
e986247ff2 Merge remote-tracking branch 'origin/topic/jsiwek/974'
Closes #974.

* origin/topic/jsiwek/974:
  Fix schedule statements used outside event handlers (addresses #974).
2013-04-23 20:38:21 -07:00
Robin Sommer
f6f00924fc Merge remote-tracking branch 'origin/topic/jsiwek/973'
Closes #973.

* origin/topic/jsiwek/973:
  Fix record coercion for default inner record fields (addresses #973).
2013-04-23 20:37:08 -07:00
Robin Sommer
71591d706e Small tweaks for bytestring_to_count().
Closes #968.
2013-04-23 20:32:57 -07:00