Mirror/zeek
1
0
Fork 0
mirror of https://github.com/zeek/zeek.git synced 2025-10-05 16:18:19 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 1
11634 commits 386 branches 195 tags 258 MiB
Commit graph

3495 commits

Author SHA1 Message Date
Robin Sommer
efd343af8d Extending external canonifier to remove fractional values from 
capture_loss.log.
 2013-07-17 21:57:17 -07:00
Robin Sommer
d8801bb9c4 Canonifying internal order for plugins and their components to make it 
deterministic.
 2013-07-17 21:57:13 -07:00
Seth Hall
7838113dc2 Merge remote-tracking branch 'origin/master' into topic/seth/faf-updates 
Conflicts:
	magic
 2013-07-16 12:09:53 -04:00
Robin Sommer
06287966a1 Bringing the DPD POP3 signature back. 
This also avoids the need for updating the external test suite.
 2013-07-10 14:19:00 -07:00
Seth Hall
2e0912b543 Merge remote-tracking branch 'origin/topic/seth/bittorrent-fix-and-dpd-sig-breakout' into topic/seth/faf-updates 
Conflicts:
	magic
	scripts/base/protocols/http/__load__.bro
	scripts/base/protocols/irc/__load__.bro
	scripts/base/protocols/smtp/__load__.bro
 2013-07-10 16:28:38 -04:00
Seth Hall
8322bbfd62 Small test fixes. 2013-07-09 23:28:09 -04:00
Bernhard Amann
03b584c34a Merge remote-tracking branch 'origin/master' into topic/bernhard/topk 2013-07-09 14:56:05 -07:00
Jon Siwek
73155c321b Add an is_orig parameter to file_over_new_connection event. 2013-07-09 15:58:28 -05:00
Seth Hall
5dbc354898 extract_filename_from_content_disposition is still hacky but more closely aligns with RFC5987 2013-07-09 14:05:36 -04:00
Robin Sommer
2ea1f483db Bringing back test for enable_auto_protocol_capture_filters (formerly 
all_packets).
 2013-07-08 13:06:03 -07:00
Seth Hall
58d133e764 Merge remote-tracking branch 'origin/master' into topic/seth/faf-updates 
Conflicts:
	scripts/base/frameworks/files/main.bro
	scripts/base/init-bare.bro
	scripts/base/protocols/ftp/file-analysis.bro
	scripts/base/protocols/http/file-analysis.bro
	scripts/base/protocols/irc/file-analysis.bro
	scripts/base/protocols/smtp/file-analysis.bro
	src/const.bif
	src/event.bif
	src/file_analysis/Analyzer.h
	src/file_analysis/file_analysis.bif
 2013-07-05 02:13:27 -04:00
Seth Hall
df2841458d Large overhaul in name and appearance for file analysis. 2013-07-05 02:00:14 -04:00
Seth Hall
af87126521 Updating test baselines. 2013-07-05 01:27:59 -04:00
Seth Hall
5f8ee93ef0 Merge remote-tracking branch 'origin/master' into topic/seth/analyzer-framework 
Conflicts:
	scripts/base/init-default.bro
	scripts/base/protocols/dns/main.bro
	scripts/base/protocols/ftp/main.bro
	scripts/base/protocols/http/main.bro
	scripts/base/protocols/irc/main.bro
	scripts/base/protocols/smtp/main.bro
	scripts/base/protocols/ssh/main.bro
	scripts/base/protocols/ssl/main.bro
	scripts/base/protocols/syslog/main.bro
	src/main.cc
	testing/btest/Baseline/coverage.default-load-baseline/canonified_loaded_scripts.log
 2013-07-04 23:07:52 -04:00
Robin Sommer
96fe05633a Merge remote-tracking branch 'origin/topic/bernhard/input-update' 
Closes #1021.

* origin/topic/bernhard/input-update:
  this event handler fails the unused-event-handlers test because it is a bit of a special case.
  ...and fix the event ordering issue. Dispatch != QueueEvent
  add Terminate to input framework to prevent potential shutdown race-conditions.
  fix warning.
  fix stderr test. ls behaves differently on errors on linux...
  small fixes.
  linux does not have strnstr
  and close only fds that are currently open (the logging framework really did not like that :) )
  A bunch of more changes for the raw reader
  make reading from stdout and stderr simultaneously work.
  allow sending data to stdin of child process
  Streaming reads from external commands work without blocking anything.
  replace popen with fork and exec.
  change raw reader to use basic c io instead of fdstream encapsulation class.
 2013-07-03 16:52:28 -07:00
Robin Sommer
ba4f03bc98 Merge remote-tracking branch 'origin/topic/seth/tls-1.2-fix' 
Closes #1020.

* origin/topic/seth/tls-1.2-fix:
  Single character fix to correct support for TLS 1.2 (my bad).
 2013-07-03 16:34:21 -07:00
Robin Sommer
ed45a6ea60 Merge remote-tracking branch 'origin/topic/jsiwek/1013' 
Closes #1013.

* origin/topic/jsiwek/1013:
  Fix redef of table index from clearing table.  Addresses #1013.
 2013-07-03 16:28:33 -07:00
Robin Sommer
d8b05af7e5 Merge remote-tracking branch 'origin/topic/jsiwek/faf-cleanup' 
Closes #1002.

* origin/topic/jsiwek/faf-cleanup:
  Move file analyzers to new plugin infrastructure.
  Add a general file analysis overview/how-to document.
  Improve file analysis doxygen comments.
  Improve tracking of HTTP file extraction (addresses #988).
  Fix HTTP multipart body file analysis.
  Remove logging of analyzers field of FileAnalysis::Info.
  Remove extraction counter in default file extraction scripts.
  Remove FileAnalysis::postpone_timeout.
  Make default get_file_handle handlers &priority=5.
  Add input interface to forward data for file analysis.
  File analysis framework interface simplifications.
 2013-07-03 16:27:16 -07:00
Seth Hall
030564a710 Single character fix to correct support for TLS 1.2 (my bad). 
- Thanks for help from Rafal Lesniak in nailing down the location
   of the bug and supplying test traffic.

 - Test traffic with a TLS 1.2 connection.

 - Addresses ticket #1020
 2013-07-02 14:49:36 -04:00
Matthias Vallentin
532fbfb4d2 Factor implementation and change interface. 
When constructing a Bloom filter, one now has to pass a HashPolicy instance to
it. This separates more clearly the concerns of hashing and Bloom filter
management.

This commit also changes the interface to initialize Bloom filters: there exist
now two initialization functions, one for each type:

  (1) bloomfilter_basic_init(fp: double,
                             capacity: count,
                             name: string &default=""): opaque of bloomfilter

  (2) bloomfilter_counting_init(k: count,
                                cells: count,
                                max: count,
                                name: string &default=""): opaque of bloomfilter

The BiFs for adding elements and performing lookups remain the same. This
essentially gives us "BiF polymorphism" at script land, where the
initialization BiF constructs the most derived type while subsequent BiFs
adhere to the same interface.

The reason why we split up the constructor in this case is that we have not yet
derived the math that computes the optimal number of hash functions for
counting Bloom filters---users have to explicitly parameterize them for now.
 2013-06-17 16:14:11 -07:00
Jon Siwek
ae5a75bad9 Fix redef of table index from clearing table. Addresses #1013. 
`redef foo["x"] = 1` now acts like `redef foo += { ["x"] = 1 }`
instead of `redef foo = { ["x"] = 1 }`.
 2013-06-12 15:18:58 -05:00
Matthias Vallentin
4c21576c12 Add Bloomfilter serialization test code. 2013-06-10 20:14:34 -07:00
Jon Siwek
7c7b6214a6 Move file analyzers to new plugin infrastructure. 2013-06-10 15:50:18 -05:00
Matthias Vallentin
d25984ba45 Update baseline for unit tests. 2013-06-10 12:55:03 -07:00
Bernhard Amann
ebb7af1483 this event handler fails the unused-event-handlers test because 
it is a bit of a special case.

It is only called via the SendEvent function from a reader. The reader
does (at least with the current interface) however not provide
the function pointer, but looks up the name of the event dynamically.

Hence, internal_handler is never called for the event.

Even if resolving the event in the reader, e.g. in an initialization
function, this would not solve the issue - the initialization function
is only called when the first Raw reader is initialized - and in the
base configuration the raw reader will never be used (hence, internal_handler
also won't be called).

Calling it once in the manager seems like a really dirty hack. So - now
it is the second exception in the testcase, unless anyone has a better
idea :)
 2013-06-09 16:18:17 -04:00
Bernhard Amann
a32bb59770 fix warning. 
Update baseline of stderr test to what it should be. There still is
a message ordering issue there (which is the last issue in the new
Raw reader I know of).

One message that sidesteps a bit of the usual processing does
not always arrive at the correct time (meaning it pops up from the
event queue too early). Even though it sidesteps a bit of the usual
processing that should not happen in my opinion (which clearly
does not matter). And I have not yet fully grasped how this can happen.
 2013-06-08 05:57:56 -07:00
Bernhard Amann
b39bffd9aa Merge remote-tracking branch 'origin/master' into topic/bernhard/input-update 2013-06-08 05:43:21 -07:00
Jon Siwek
f2574636b6 Merge branch 'master' into topic/jsiwek/faf-cleanup 
Conflicts:
	scripts/base/protocols/ftp/file-analysis.bro
	scripts/base/protocols/http/file-analysis.bro
	scripts/base/protocols/irc/file-analysis.bro
	scripts/base/protocols/smtp/file-analysis.bro
	src/file_analysis/File.cc
	src/file_analysis/File.h
	src/file_analysis/Manager.cc
	src/file_analysis/Manager.h
	testing/btest/Baseline/scripts.base.frameworks.file-analysis.logging/file_analysis.log
	testing/btest/Baseline/scripts.base.protocols.ftp.ftp-extract/ftp-item-0.dat
	testing/btest/Baseline/scripts.base.protocols.ftp.ftp-extract/ftp-item-1.dat
	testing/btest/Baseline/scripts.base.protocols.ftp.ftp-extract/ftp-item-2.dat
	testing/btest/Baseline/scripts.base.protocols.ftp.ftp-extract/ftp-item-3.dat
	testing/btest/Baseline/scripts.base.protocols.ftp.ftp-extract/ftp-item-BTsa70Ua9x7-1.dat
	testing/btest/Baseline/scripts.base.protocols.ftp.ftp-extract/ftp-item-BTsa70Ua9x7.dat
	testing/btest/Baseline/scripts.base.protocols.ftp.ftp-extract/ftp-item-Rqjkzoroau4-0.dat
	testing/btest/Baseline/scripts.base.protocols.ftp.ftp-extract/ftp-item-Rqjkzoroau4.dat
	testing/btest/Baseline/scripts.base.protocols.ftp.ftp-extract/ftp-item-VLQvJybrm38-2.dat
	testing/btest/Baseline/scripts.base.protocols.ftp.ftp-extract/ftp-item-VLQvJybrm38.dat
	testing/btest/Baseline/scripts.base.protocols.ftp.ftp-extract/ftp-item-zrfwSs9K1yk-3.dat
	testing/btest/Baseline/scripts.base.protocols.ftp.ftp-extract/ftp-item-zrfwSs9K1yk.dat
	testing/btest/Baseline/scripts.base.protocols.ftp.ftp-extract/ftp.log
	testing/btest/Baseline/scripts.base.protocols.http.http-extract-files/http-item-BFymS6bFgT3-0.dat
	testing/btest/Baseline/scripts.base.protocols.http.http-extract-files/http-item-BFymS6bFgT3.dat
	testing/btest/Baseline/scripts.base.protocols.http.http-extract-files/http-item.dat
	testing/btest/Baseline/scripts.base.protocols.http.http-extract-files/http.log
	testing/btest/Baseline/scripts.base.protocols.irc.dcc-extract/irc-dcc-item-wqKMAamJVSb-0.dat
	testing/btest/Baseline/scripts.base.protocols.irc.dcc-extract/irc-dcc-item-wqKMAamJVSb.dat
	testing/btest/Baseline/scripts.base.protocols.irc.dcc-extract/irc-dcc-item.dat
	testing/btest/Baseline/scripts.base.protocols.irc.dcc-extract/irc.log
	testing/btest/Baseline/scripts.base.protocols.smtp.mime-extract/smtp-entity-0.dat
	testing/btest/Baseline/scripts.base.protocols.smtp.mime-extract/smtp-entity-1.dat
	testing/btest/Baseline/scripts.base.protocols.smtp.mime-extract/smtp-entity-Ltd7QO7jEv3-1.dat
	testing/btest/Baseline/scripts.base.protocols.smtp.mime-extract/smtp-entity-Ltd7QO7jEv3.dat
	testing/btest/Baseline/scripts.base.protocols.smtp.mime-extract/smtp-entity-cwR7l6Zctxb-0.dat
	testing/btest/Baseline/scripts.base.protocols.smtp.mime-extract/smtp-entity-cwR7l6Zctxb.dat
	testing/btest/Baseline/scripts.base.protocols.smtp.mime-extract/smtp_entities.log
	testing/btest/scripts/base/protocols/ftp/ftp-extract.bro
	testing/btest/scripts/base/protocols/http/http-extract-files.bro
	testing/btest/scripts/base/protocols/irc/dcc-extract.test
	testing/btest/scripts/base/protocols/smtp/mime-extract.test
 2013-06-07 15:44:36 -05:00
Matthias Vallentin
86becdd6e4 Add tests. 2013-06-06 15:08:24 -07:00
Robin Sommer
203df4fa6b Merge remote-tracking branch 'origin/topic/jsiwek/869' 
* origin/topic/jsiwek/869:
  Change @PATH to @DIR for clarity.  Add @FILENAME.  Addresses #869.
  Make @PATH always return absolute path.  Addresses #869.
  Add @PATH bro script macro.  Addresses #869.

Closes #869.
 2013-06-06 12:42:18 -07:00
Jon Siwek
022ce2505f Change @PATH to @DIR for clarity. Add @FILENAME. Addresses #869. 
@DIR expands to directory path of the script, @FILENAME expands to just
the script file name without path.
 2013-06-05 11:01:11 -05:00
Robin Sommer
74e99a27db Ignoring file ids in external tests. 
They can change occasionally, and we likewise ignore uids already.
 2013-06-04 21:23:50 -07:00
Robin Sommer
da3eb2d3e2 Merge remote-tracking branch 'origin/topic/robin/plugins' 2013-06-04 20:58:16 -07:00
Jon Siwek
7e8b504305 Make @PATH always return absolute path. Addresses #869. 2013-06-04 14:16:56 -05:00
Jon Siwek
307fc187c0 Add @PATH bro script macro. Addresses #869. 
The macro expands to a string value containing the file system path
in which the script lives.
 2013-06-04 10:53:10 -05:00
Robin Sommer
a5cb605b1d Fixing test that was accidentally broken. 2013-06-03 20:10:48 -07:00
Seth Hall
caf61f619b Merge remote-tracking branch 'origin/topic/jsiwek/faf-cleanup' into topic/seth/faf-updates 2013-06-03 10:51:55 -04:00
Robin Sommer
c19779ae88 More analyzer framework tests. 2013-06-02 18:22:08 -07:00
Robin Sommer
c049c758c3 Merge remote-tracking branch 'origin/master' into topic/robin/plugins 
Conflicts:
	aux/bro-aux
	aux/broctl
	src/DPM.cc
 2013-05-30 17:43:50 -07:00
Jon Siwek
a66b7380b6 Allow named vector constructors. Addresses #983. 2013-05-30 10:57:28 -05:00
Jon Siwek
bcf5c41786 Allow named table constructors. Addresses #983. 2013-05-30 10:21:15 -05:00
Jon Siwek
29740d3d6e Improve set constructor argument coercion. 2013-05-29 16:49:12 -05:00
Jon Siwek
b256642f27 Allow named set constructors. Addresses #983. 2013-05-29 15:11:44 -05:00
Jon Siwek
a0ad87b4c2 Allow named record constructors. Addresses #983. 2013-05-29 12:48:15 -05:00
Bernhard Amann
f1745ff488 fix stderr test. ls behaves differently on errors on linux... 2013-05-27 23:07:37 -07:00
Bernhard Amann
08656c976b small fixes. 2013-05-27 22:59:27 -07:00
Bernhard Amann
3719524a6a Merge remote branch 'origin/master' into topic/bernhard/input-update 2013-05-27 20:32:50 -07:00
Seth Hall
0a18b62d12 Merge remote-tracking branch 'origin/master' into topic/seth/sumstats-updates 
Conflicts:
	scripts/base/frameworks/sumstats/cluster.bro
	scripts/base/frameworks/sumstats/plugins/average.bro
	scripts/base/frameworks/sumstats/plugins/max.bro
	scripts/base/frameworks/sumstats/plugins/min.bro
	scripts/base/frameworks/sumstats/plugins/sample.bro
	scripts/base/frameworks/sumstats/plugins/std-dev.bro
	scripts/base/frameworks/sumstats/plugins/sum.bro
	scripts/base/frameworks/sumstats/plugins/unique.bro
	scripts/base/frameworks/sumstats/plugins/variance.bro
	scripts/policy/protocols/http/detect-sqli.bro
	testing/btest/scripts/base/frameworks/sumstats/cluster-intermediate-update.bro
 2013-05-21 22:33:16 -04:00
Jon Siwek
705a84d688 Improve tracking of HTTP file extraction (addresses #988). 
http.log now has files taken from request and response bodies in
different fields for each, and can now track multiple files per body.
That is, the "extraction_file" field is now "extracted_request_files"
and "extracted_response_files".
 2013-05-21 16:42:35 -05:00
Jon Siwek
3cbef60f57 Fix HTTP multipart body file analysis. 
Each part now gets assigned a different file handle/id.
 2013-05-21 15:35:22 -05:00