Commit graph

13262 commits

Author SHA1 Message Date
Robin Sommer
8e7ef001b3 A work-around for supporting plugins compiled as *.a.
This is for older CMakes which don't understand object libraries yet.
We auto-generate a file at the top-level that references all the
global plugin objects. Once they are referenced, the linker includes
them into the final executable and their ctors run to register them
with the plugin manager.

By default this is on. Once we can require more recent CMakes, we can
turn it off (and remove) in src/CMakeLists.txt.
2013-06-02 19:36:25 -07:00
Robin Sommer
c19779ae88 More analyzer framework tests. 2013-06-02 18:22:08 -07:00
Robin Sommer
c6ad731562 More smaller cleanup. 2013-06-02 18:21:45 -07:00
Robin Sommer
d3d14e10cf Merge remote-tracking branch 'origin/topic/jsiwek/983'
Closes #983.

* origin/topic/jsiwek/983:
  Add named constructor examples to docs.
  Allow named vector constructors. Addresses #983.
  Allow named table constructors.  Addresses #983.
  Improve set constructor argument coercion.
  Allow named set constructors. Addresses #983.
  Allow named record constructors. Addresses #983.
2013-06-02 17:41:33 -07:00
Matthias Vallentin
d873db03ce Add draft of Bloom filter type hierarchy. 2013-05-31 18:31:14 -07:00
Robin Sommer
2fd0b36057 Merge remote-tracking branch 'origin/master' into topic/robin/plugins
Conflicts:
	aux/broctl
2013-05-31 17:47:05 -07:00
Robin Sommer
45f6f11e51 Adding Makefile target test-all that also runs the BroControl test
suite.

Eventually "make test" should do this but let's trigger it separately
for now.
2013-05-31 17:32:27 -07:00
Robin Sommer
6d478bea57 Updating submodule(s).
[nomail]
2013-05-31 17:30:43 -07:00
Robin Sommer
58ac7c80cd Switching back to object libs for now. 2013-05-30 19:14:04 -07:00
Robin Sommer
23463d064c Little fixes. 2013-05-30 19:13:08 -07:00
Robin Sommer
b3370584c7 Updating submodule(s).
[nomail]
2013-05-30 17:44:09 -07:00
Robin Sommer
c049c758c3 Merge remote-tracking branch 'origin/master' into topic/robin/plugins
Conflicts:
	aux/bro-aux
	aux/broctl
	src/DPM.cc
2013-05-30 17:43:50 -07:00
Robin Sommer
2e9de30bcc New CMake variable bro_HAVE_OBJECT_LIBRARIES that switches between
object and static libraries for sub directories.

Default is static to support old CMakes.
2013-05-30 17:41:20 -07:00
Robin Sommer
e3a7e0301b Cleanup and more API docs. 2013-05-30 16:45:14 -07:00
Jon Siwek
3c0578d009 Merge branch 'master' into topic/jsiwek/983 2013-05-30 11:43:02 -05:00
Jon Siwek
4301002f1a Add named constructor examples to docs. 2013-05-30 11:41:22 -05:00
Jon Siwek
a66b7380b6 Allow named vector constructors. Addresses #983. 2013-05-30 10:57:28 -05:00
Jon Siwek
bcf5c41786 Allow named table constructors. Addresses #983. 2013-05-30 10:21:15 -05:00
Jon Siwek
29740d3d6e Improve set constructor argument coercion. 2013-05-29 16:49:12 -05:00
Jon Siwek
b256642f27 Allow named set constructors. Addresses #983. 2013-05-29 15:11:44 -05:00
Jon Siwek
a0ad87b4c2 Allow named record constructors. Addresses #983. 2013-05-29 12:48:15 -05:00
Robin Sommer
965a26e447 Merge remote-tracking branch 'origin/fastpath'
Closes #1004. Great job tracking this down!

* origin/fastpath:
  Dangling pointer fix.  Addresses #1004.
  Sorry, that libmagic version actually might have some problems - at least on the linux distribution I have access to. So... it was a bad idea.
  accept libmagic starting from 5.03
2013-05-29 07:25:19 -07:00
Matthias Vallentin
9e32eaad6d Make bitvectors serializable. 2013-05-28 20:58:01 -07:00
Jon Siwek
22a4113ac3 Dangling pointer fix. Addresses #1004. 2013-05-28 16:21:29 -05:00
Bernhard Amann
f1745ff488 fix stderr test. ls behaves differently on errors on linux... 2013-05-27 23:07:37 -07:00
Bernhard Amann
08656c976b small fixes. 2013-05-27 22:59:27 -07:00
Bernhard Amann
d61973a92d linux does not have strnstr 2013-05-27 21:14:07 -07:00
Bernhard Amann
bcc81a1a14 Sorry, that libmagic version actually might have some problems - at
least on the linux distribution I have access to. So... it was a
bad idea.

Revert "accept libmagic starting from 5.03"

This reverts commit 04dd363279.
2013-05-27 21:10:51 -07:00
Bernhard Amann
3719524a6a Merge remote branch 'origin/master' into topic/bernhard/input-update 2013-05-27 20:32:50 -07:00
Bernhard Amann
04dd363279 accept libmagic starting from 5.03 2013-05-27 20:30:03 -07:00
Robin Sommer
d67123d0c3 Updating submodule(s).
[nomail]
2013-05-24 18:07:36 -07:00
Robin Sommer
7e02339b39 Merge remote-tracking branch 'origin/fastpath'
* origin/fastpath:
  Fix broken/missing documentation.
2013-05-24 17:24:42 -07:00
Matthias Vallentin
4d275522c7 Add abstraction for vector of bits.
A bitvector is a vector of bits with underlying block storage. Since C++ has no
notion of lvalues in the context of bits, we use a small wrapper class
Reference that masks the desired bit in the corresponding block.
2013-05-23 16:03:26 -07:00
Jon Siwek
e45933562e Fix broken/missing documentation. 2013-05-23 16:53:42 -05:00
Jon Siwek
9c86a3ee0e Add a general file analysis overview/how-to document. 2013-05-23 14:29:13 -05:00
Jon Siwek
f82167d067 Improve file analysis doxygen comments. 2013-05-23 10:22:49 -05:00
Seth Hall
4f4ef99a6b SumStats changes to how thresholding works to simplify and reduce memory use. 2013-05-23 10:12:17 -04:00
Seth Hall
6bd9ab3bd6 More adjustments to try and correct SumStats memory use. 2013-05-22 16:41:46 -04:00
Seth Hall
c4a1f30a87 Hopefully fixing a strange error. 2013-05-22 14:59:31 -04:00
Seth Hall
0a18b62d12 Merge remote-tracking branch 'origin/master' into topic/seth/sumstats-updates
Conflicts:
	scripts/base/frameworks/sumstats/cluster.bro
	scripts/base/frameworks/sumstats/plugins/average.bro
	scripts/base/frameworks/sumstats/plugins/max.bro
	scripts/base/frameworks/sumstats/plugins/min.bro
	scripts/base/frameworks/sumstats/plugins/sample.bro
	scripts/base/frameworks/sumstats/plugins/std-dev.bro
	scripts/base/frameworks/sumstats/plugins/sum.bro
	scripts/base/frameworks/sumstats/plugins/unique.bro
	scripts/base/frameworks/sumstats/plugins/variance.bro
	scripts/policy/protocols/http/detect-sqli.bro
	testing/btest/scripts/base/frameworks/sumstats/cluster-intermediate-update.bro
2013-05-21 22:33:16 -04:00
Jon Siwek
705a84d688 Improve tracking of HTTP file extraction (addresses #988).
http.log now has files taken from request and response bodies in
different fields for each, and can now track multiple files per body.
That is, the "extraction_file" field is now "extracted_request_files"
and "extracted_response_files".
2013-05-21 16:42:35 -05:00
Jon Siwek
3cbef60f57 Fix HTTP multipart body file analysis.
Each part now gets assigned a different file handle/id.
2013-05-21 15:35:22 -05:00
Seth Hall
bec965b66f Large update for the SumStats framework.
- On-demand access to sumstats results through "return from"
   functions named SumStats::request and Sumstats::request_key.
   Both functions are tested in standalone and clustered modes.

 - $name field has returned to SumStats which simplifies cluster
   code and makes the on-demand access stuff possible.

 - Clustered results can only be collected for 1 minute from their
   time of creation now instead of time of last read.

 - Thresholds use doubles instead of counts everywhere now.

 - Calculation dependency resolution occurs at start up time now
   instead of doing it at observation time which provide a minor
   cpu performance improvement.  A new plugin registration mechanism
   was created to support this change.

 - AppStats now has a minimal doc string and is broken into hook-based
   plugins.

 - AppStats and traceroute detection added to local.bro
2013-05-21 15:52:59 -04:00
Jon Siwek
38ac03d558 Remove logging of analyzers field of FileAnalysis::Info.
It was mostly redundant when logged, but still can be useful to
inspect at runtime.  In the future, a better field for logging
will be available which will be similar to the "service" field
for connection records (there's not any file-format-specific
analyzers that would currently make use of such a thing).
2013-05-21 12:01:40 -05:00
Jon Siwek
28f51a9a22 Remove extraction counter in default file extraction scripts. 2013-05-21 11:12:00 -05:00
Jon Siwek
16f924c2c0 Remove FileAnalysis::postpone_timeout.
FileAnalysis::set_timeout_interval can now perform same function.
2013-05-21 10:50:07 -05:00
Jon Siwek
bc5cd3acc8 Make default get_file_handle handlers &priority=5.
So they're easier to override (just provide a new handler without
specifying a priority).
2013-05-21 10:34:19 -05:00
Jon Siwek
0ef074594d Add input interface to forward data for file analysis.
The new Input::add_analysis function is used to automatically forward
input data on to the file analysis framework.
2013-05-21 10:29:22 -05:00
Jon Siwek
90fa331279 File analysis framework interface simplifications.
- Remove script-layer data input interface (will be managed directly
  by input framework later).

- Only track files internally by file id hash.  Chance of collision
  too small to justify also tracking unique file string.
2013-05-20 12:02:48 -05:00
Robin Sommer
e46300a724 Fixing test that would fail without ES/curl support.
It used to special-case an error message produced in the case that ES
isn't available, however with scripts/test-all-policy.bro now
explicitly disabling ES output, that doesn't seem necessary anymore.
2013-05-18 16:47:04 -07:00