Commit graph

6667 commits

Author SHA1 Message Date
Seth Hall
1e098bae8d Moving the PE analyzer to the new plugin structure. 2013-07-27 00:07:47 -04:00
Seth Hall
998cedb3b8 Merge remote-tracking branch 'origin/master' into topic/seth/file-analysis-exe-analyzer
Conflicts:
	src/CMakeLists.txt
	src/binpac_bro.h
	src/event.bif
	src/file_analysis.bif
	src/file_analysis/AnalyzerSet.cc
2013-07-27 00:04:40 -04:00
Jon Siwek
939619889d File analysis fixes and test updates.
- Several places were just using old variable names or not loading
  scripts correctly after they'd been renamed/moved.

- Revert/adjust a change in how HTTP file handles are generated that
  broke partial content responses.

- Turn some libmagic builtin checks back on; seems some are actually
  useful (e.g. text detection seems to be a builtin).  The rule going
  forward probably will be only to turn off a builtin if we confirm it
  causes issues.

- Removed some tests that are redundant or not necessary anymore because
  the generic file analysis tests cover them.

- A couple FTP tests still fail that I think need an actual solution via
  script changes.
2013-07-25 16:51:16 -05:00
Bernhard Amann
32c2885742 Merge remote-tracking branch 'origin/master' into topic/bernhard/hyperloglog
Conflicts:
	src/Func.cc
	src/probabilistic/CMakeLists.txt
2013-07-25 14:46:38 -07:00
Robin Sommer
7dd5771384 Protection about broken traces with empty pcap headers. 2013-07-25 12:32:01 -07:00
Robin Sommer
c11bf3d922 Fixing serialization bug introduced during earlier merge. 2013-07-25 11:29:13 -07:00
Robin Sommer
b97e045c9a Merge branch 'master' into topic/robin/bloom-filter-merge 2013-07-25 10:18:46 -07:00
Robin Sommer
4a7046848c bif files declared with bif_target() are now automatically compiled
in.

No more manual includes to pull them in.

(It doesn't quite work fully automatically yet for some bifs that need
script-level types defined, like the input and logging frameworks.
They still do a manual "@load foo.bif" in their main.bro to get the
order right. It's a bit tricky to fix that and would probably need
splitting main.bro into two parts; not sure that's worth it.)
2013-07-25 10:12:52 -07:00
Matthias Vallentin
2fc5ca53ff Make hashers serializable.
There exists still a small bug that I could not find; the unit test
istate/opaque.bro fails. If someone sees why, please chime in.
2013-07-25 17:35:35 +02:00
Jon Siwek
ac6b12b5d4 Merge branch 'master' into topic/jsiwek/faf-updates
Conflicts:
	magic
2013-07-25 10:02:26 -05:00
Matthias Vallentin
e482897f88 Add docs and use default value for hasher names. 2013-07-25 15:16:53 +02:00
Robin Sommer
d8226169b8 Fixing random number generation so that it returns same numbers as
before.

That broke a lot of tests.
2013-07-24 16:34:52 -07:00
Robin Sommer
599dadf30b Merge branch 'topic/robin/bloom-filter-merge'
* topic/robin/bloom-filter-merge: (50 commits)
  Support emptiness check on Bloom filters.
  Refactor Bloom filter merging.
  Add bloomfilter_clear() BiF.
  Updating NEWS.
  Broifying the code.
  Implement and test Bloom filter merging.
  Make hash functions equality comparable.
  Make counter vectors mergeable.
  Use half adder for bitwise addition and subtraction.
  Fix and test counting Bloom filter.
  Implement missing CounterVector functions.
  Tweak hasher interface.
  Add missing include for GCC.
  Fixing for unserializion error.
  Small fixes and style tweaks.
  Only serialize Bloom filter type if available.
  Create hash policies through factory.
  Remove lingering debug code.
  Factor implementation and change interface.
  Expose Bro's linear congruence PRNG as utility function.
  ...
2013-07-24 15:51:10 -07:00
Robin Sommer
23b352b702 Merge remote-tracking branch 'origin/topic/matthias/bloom-filter' into topic/robin/bloom-filter-merge
* origin/topic/matthias/bloom-filter:
  Support emptiness check on Bloom filters.
  Refactor Bloom filter merging.
  Add bloomfilter_clear() BiF.
2013-07-24 15:39:50 -07:00
Bernhard Amann
efdffaec9e and forgot a file... 2013-07-24 12:51:31 -07:00
Bernhard Amann
b7cdfc0e6e adapt to new structure 2013-07-24 12:50:01 -07:00
Matthias Vallentin
5769c32f1e Support emptiness check on Bloom filters. 2013-07-24 13:18:19 +02:00
Matthias Vallentin
5736aef440 Refactor Bloom filter merging. 2013-07-24 13:05:38 +02:00
Matthias Vallentin
5383e8f75b Add bloomfilter_clear() BiF. 2013-07-24 11:21:10 +02:00
Seth Hall
75814e58e4 Fix a bug with getting analyzer tags. 2013-07-24 00:35:46 -04:00
Bernhard Amann
9e0fd963e0 Merge remote-tracking branch 'origin/topic/robin/bloom-filter-merge' into topic/bernhard/hyperloglog
Conflicts:
	scripts/base/frameworks/sumstats/plugins/__load__.bro
	src/CMakeLists.txt
	src/NetVar.cc
	src/NetVar.h
	src/OpaqueVal.h
	src/SerialTypes.h
	src/bro.bif
2013-07-23 21:31:05 -07:00
Robin Sommer
474107fe40 Broifying the code.
Also extending API documentation a bit more and fixing a memory leak.
2013-07-23 20:10:32 -07:00
Robin Sommer
21685d2529 Merge remote-tracking branch 'origin/topic/matthias/bloom-filter'
I'm moving the new files into a subdirectory probabilistic, and into a
corresponding namespace. We can later put code for the other
probabilistic data structures there as well.

* origin/topic/matthias/bloom-filter: (45 commits)
  Implement and test Bloom filter merging.
  Make hash functions equality comparable.
  Make counter vectors mergeable.
  Use half adder for bitwise addition and subtraction.
  Fix and test counting Bloom filter.
  Implement missing CounterVector functions.
  Tweak hasher interface.
  Add missing include for GCC.
  Fixing for unserializion error.
  Small fixes and style tweaks.
  Only serialize Bloom filter type if available.
  Create hash policies through factory.
  Remove lingering debug code.
  Factor implementation and change interface.
  Expose Bro's linear congruence PRNG as utility function.
  H3 does not check for zero length input.
  Support seeding for hashers.
  Add utility function to access first random seed.
  Update H3 documentation (and minor style nits.)
  Make H3 seed configurable.
  ...
2013-07-23 16:40:56 -07:00
Matthias Vallentin
69a7dd03bc Merge remote-tracking branch 'origin/master' into topic/matthias/bloom-filter 2013-07-22 22:26:15 +02:00
Matthias Vallentin
a39f980cd4 Implement and test Bloom filter merging. 2013-07-22 18:11:12 +02:00
Matthias Vallentin
eb64f5f961 Make hash functions equality comparable. 2013-07-22 18:03:55 +02:00
Matthias Vallentin
9c2f57a9d9 Make counter vectors mergeable. 2013-07-22 16:36:54 +02:00
Matthias Vallentin
a3c61fe7eb Use half adder for bitwise addition and subtraction. 2013-07-22 15:55:51 +02:00
Matthias Vallentin
7a0240694e Fix and test counting Bloom filter. 2013-07-22 14:09:32 +02:00
Vlad Grigorescu
deeb5ec38e Rework the DHCP analyzer to make it compatible again. 2013-07-21 20:31:23 -04:00
Matthias Vallentin
79a2e4b5d5 Implement missing CounterVector functions. 2013-07-21 22:41:48 +02:00
Matthias Vallentin
fd2e155d1a Tweak hasher interface. 2013-07-21 17:59:03 +02:00
Vlad Grigorescu
4b81030e3f Merge topic/seth/faf-updates. 2013-07-18 19:40:34 -04:00
Robin Sommer
d8801bb9c4 Canonifying internal order for plugins and their components to make it
deterministic.
2013-07-17 21:57:13 -07:00
Robin Sommer
57b05a2989 Small raw reader tweaks that I forgot to commit earlier. 2013-07-17 17:30:35 -07:00
Seth Hall
7838113dc2 Merge remote-tracking branch 'origin/master' into topic/seth/faf-updates
Conflicts:
	magic
2013-07-16 12:09:53 -04:00
Seth Hall
0bfdcc1fbc Added protocol description functions that provide a super compressed log representation. 2013-07-16 12:01:50 -04:00
Bernhard Amann
7427ce511b Small raw reader fixes
* crash when accessing nonexistant file.
* memory leak when reading from file.

Addresses #1038.
2013-07-15 13:50:40 -07:00
Robin Sommer
50357ec47a Merge remote-tracking branch 'origin/topic/bernhard/sqlite-update'
* origin/topic/bernhard/sqlite-update:
  yep, freebsd still needs this fix
  bump sqlite to 3.7.17.

Closes #1037.
2013-07-14 08:04:19 -07:00
Bernhard Amann
e01678d132 yep, freebsd still needs this fix 2013-07-12 21:09:13 +02:00
Jon Siwek
1a60fae41c Clean up queued but unused file_over_new_connections event args. 2013-07-11 11:36:49 -05:00
Seth Hall
2e0912b543 Merge remote-tracking branch 'origin/topic/seth/bittorrent-fix-and-dpd-sig-breakout' into topic/seth/faf-updates
Conflicts:
	magic
	scripts/base/protocols/http/__load__.bro
	scripts/base/protocols/irc/__load__.bro
	scripts/base/protocols/smtp/__load__.bro
2013-07-10 16:28:38 -04:00
Robin Sommer
cb09bd6358 Merge remote-tracking branch 'origin/topic/seth/bittorrent-fix-and-dpd-sig-breakout'
Closes #1035.

* origin/topic/seth/bittorrent-fix-and-dpd-sig-breakout:
  Small test fixes.
  Added a missing curly brace in smtp/dpd.sig
  Fix a bug where the same analyzer tag was reused for two different analyzers.
  Moved DPD signatures into script specific directories.
2013-07-10 11:37:57 -07:00
Jon Siwek
0394493fac const adjustment
And fixes compiler warning about overloaded virtual function hiding.
2013-07-10 11:55:40 -05:00
Matthias Vallentin
446344ae99 Add missing include for GCC. 2013-07-10 01:32:59 -07:00
Matthias Vallentin
21a07ced82 Merge branch 'topic/matthias/bloom-filter' of ssh://git.bro.org/bro into topic/matthias/bloom-filter 2013-07-10 01:29:06 -07:00
Robin Sommer
40201a180e Fixing for unserializion error.
Because BloomFilter is a base class, with other classes derived from
it, it needs special treatment.
2013-07-09 21:00:53 -07:00
Seth Hall
4dda9cd3ba Fix a bug where the same analyzer tag was reused for two different analyzers. 2013-07-09 22:45:21 -04:00
Bernhard Amann
03b584c34a Merge remote-tracking branch 'origin/master' into topic/bernhard/topk 2013-07-09 14:56:05 -07:00
Jon Siwek
73155c321b Add an is_orig parameter to file_over_new_connection event. 2013-07-09 15:58:28 -05:00