Commit graph

436 commits

Author SHA1 Message Date
Jon Siwek
580822a32c Merge remote-tracking branch 'origin/topic/dnthayer/gh-339'
* origin/topic/dnthayer/gh-339:
  Rename all BRO-prefixed environment variables
2019-05-23 20:23:14 -07:00
Johanna Amann
63e6921895 Add leak-checks for new copy operations 2019-05-22 14:57:55 -07:00
Johanna Amann
74bb7716f6 Finish implementation of copy method.
All types (besides EntropyVal) now support a native copy operation,
which uses primitives of the underlying datatypes to perform a quick
copy, without serialization.

EntropyVal is the one exception - since that type is rather complex
(many members) and will probably not be copied a lot, if at all, it
makes sense to just use the serialization function.

This will have to be slightly re-written in the near-term-future to use
the new serialization function for that opaque type.

This change also introduces a new x509_from_der bif, which allows to
parse a der into an opaque of x509.

This change removes the d2i_X509_ wrapper function; this was a remnant
when d2i_X509 took non-const arguments. We directly use d2i_X509 at
several places assuming const-ness, so there does not seem to ba a
reason to keep the wrapper.

This change also exposed a problem in the File cache - cases in which an
object was brought back into the cache, and writing occurred in the
file_open event were never correctly handeled as far as I can tell.
2019-05-22 14:29:37 -07:00
Daniel Thayer
1a74516db1 Rename all BRO-prefixed environment variables
For backward compatibility when reading values, we first check
the ZEEK-prefixed value, and if not set, then check the corresponding
BRO-prefixed value.
2019-05-22 00:12:31 -05:00
Daniel Thayer
3f9e7138bd More bro-to-zeek renaming in the unit tests 2019-05-16 02:27:54 -05:00
Jon Siwek
6ad7099f7e Merge remote-tracking branch 'origin/topic/robin/gh-239'
* origin/topic/robin/gh-239:
  Undo a change to btest.cfg from a recent commit
  Updating submodule.
  Fix zeek-wrapper
  Update for renaming BroControl to ZeekControl.
  Updating submodule.
  GH-239: Rename bro to zeek, bro-config to zeek-config, and bro-path-dev to zeek-path-dev.
2019-05-14 13:27:40 -07:00
Johanna Amann
5d44735209 Remove deprecated functions/events
This commit removed functions/events that have been deprecated in Bro
2.6. It also removes the detection code that checks if the old
communication framework is used (since all the functions that are
checked were removed).

Addresses parts of GH-243
2019-05-02 12:06:39 -07:00
Robin Sommer
789cb376fd GH-239: Rename bro to zeek, bro-config to zeek-config, and bro-path-dev to zeek-path-dev.
This also installs symlinks from "zeek" and "bro-config" to a wrapper
script that prints a deprecation warning.

The btests pass, but this is still WIP. broctl renaming is still
missing.

#239
2019-05-01 21:43:45 +00:00
Jon Siwek
a994be9eeb Merge remote-tracking branch 'origin/topic/seth/zeek_init'
* origin/topic/seth/zeek_init:
  Some more testing fixes.
  Update docs and tests for bro_(init|done) -> zeek_(init|done)
  Implement the zeek_init handler.
2019-04-19 11:24:29 -07:00
Jon Siwek
f21e11d811 GH-237: add @load foo.bro -> foo.zeek fallback
When failing to locate a script with explicit .bro suffix, check for
whether one with a .zeek suffix exists and use it instead.
2019-04-16 17:49:37 -07:00
Jon Siwek
1e57e3f026 Use .zeek file suffix in unit tests 2019-04-16 16:08:57 -07:00
Seth Hall
9d676d368b Some more testing fixes. 2019-04-14 09:58:30 -04:00
Seth Hall
5db766bd88 Update docs and tests for bro_(init|done) -> zeek_(init|done) 2019-04-14 08:49:12 -04:00
Daniel Thayer
4e0c1997a0 Update tests and baselines due to renaming all scripts 2019-04-11 23:32:58 -05:00
Daniel Thayer
438fe27ce4 Add test cases to verify new file extension is recognized
Added tests to verify that scripts with the new ".zeek" file extension
are recognized and that ".bro" is used as a fallback.
2019-04-11 12:26:50 -05:00
Daniel Thayer
7c2477d87a Fix the core/load-duplicates.bro test
The load-duplicates.bro test would never fail because loading
the provided script code twice wouldn't trigger an error.
Fixed this by changing the sample script content.  Also added a
test case to verify that an error occurs as expected
when two scripts with the same content are loaded.
2019-04-11 02:38:53 -05:00
Jon Siwek
a262ed8d9c Merge branch 'master' of https://github.com/rdenniston/zeek
* 'master' of https://github.com/rdenniston/zeek:
  Add linux netfilter NFLOG capture functionality initial commit

I made modifications:

  - Formatting / code style

  - More error handling and validity checks

  - The Type and Length value of TLVs is technically host order

  - Changed / fixed the Length value padding check: it's generally
    32-bit alignment, not just aligning any TLV less than 8 bytes.
2019-03-19 19:19:02 -07:00
Jon Siwek
41c7b229d3 Merge branch 'master' of https://github.com/ZekeMedley/zeek
* 'master' of https://github.com/ZekeMedley/zeek:
  Add key-value for loop

Fixes GH-154
2019-03-15 19:45:07 -07:00
Jon Siwek
09ae539ea8 GH-250: Improve/cleanup VXLAN decapsulation support
* Better parsing/error-checking of VXLAN and encapsulated packet headers

* Add/implement the "vxlan_packet" event

* Add "Tunnel::vxlan_ports" option to tune the set of VXLAN ports to
  analyze/decapsulate

* Add "Tunnel::validate_vxlan_checksums" option to allow for tuning of how
  checksums associated with the outer UDP header of a possible VXLAN
  tunnel are handled

Fixes GH-250
2019-03-12 18:15:34 -07:00
Henrik Lund Kramshoej
f4088be8a6 Initial VXLAN support, need -C flag to work when running bro 2019-03-11 14:15:39 -07:00
Jon Siwek
0790c1c559 Merge remote-tracking branch 'origin/topic/jsiwek/gh-211'
* origin/topic/jsiwek/gh-211:
  GH-208: change invalid subnet expressions to a runtime error
  GH-211: improve consistency of how scripting errors are handled

Removed the 'allow_init_errors' option.
2019-02-07 10:33:52 -06:00
Jon Siwek
67484a90fa GH-211: improve consistency of how scripting errors are handled
Scripting errors/mistakes now consistently generate a runtime error
which have the behavior of unwinding the call stack all the way out of
the current event handler.

Before, such errors were not treated consistently and either aborted
the process entirely or emitted a message while continuing to execute
subsequent statements without well-defined behavior (possibly causing
a cascade of errors).

The previous behavior also would only unwind out of the current
function (if within a function body), not out the current event
handler, which is especially problematic for functions that return
a value: the caller is essentially left a mess with no way to deal
with it.

This also changes the behavior of the startup/initialization process
to abort if there's errors during bro_init() rather than continue one
to the main run loop.  The `allow_init_errors` option may change this
new, default behavior.
2019-01-30 11:20:09 -06:00
Stu H
5ef70c4df1 Added ERSPAN III testing 2019-01-24 14:05:13 +00:00
Jon Siwek
5618b21cca Improve ERSPAN Type III support
Added check for the optional sub-header in ERSPAN Type III as well
as additional truncation checks to the GRE parsing logic in general.
Also added a unit test for ERSPAN Type II.
2019-01-17 18:06:10 -06:00
Jon Siwek
12f3ab303b Improve unit test for reporter shutdown order 2019-01-15 10:32:13 -06:00
Jon Siwek
38cb3dbf2e GH-188: fix crash when shutting down with pending reporter errors
This fixes some use-after-free issues in the shutdown order of
various systems: I/O, plugin, logging, and reporter systems may
interact during shutdown if there's errors emitted (or maybe just
still pending) during the shutdown.

Fixes GH-188
2019-01-14 15:40:17 -06:00
Jon Siwek
fd63168171 Merge remote-tracking branch 'origin/topic/jsiwek/parallelize-comm-tests'
* origin/topic/jsiwek/parallelize-comm-tests:
  Parallelize communication tests using btest TEST-PORT
2018-11-29 17:06:30 -06:00
Jon Siwek
907297ba59 Parallelize communication tests using btest TEST-PORT 2018-11-04 15:29:59 -06:00
Jon Siwek
43aab5be7b Merge remote-tracking branch 'origin/master' into dev/2.7 2018-09-25 16:47:01 -05:00
Daniel Thayer
5ebed9158f Add some missing @TEST-REQUIRES to a few tests 2018-09-25 15:52:19 -05:00
Jon Siwek
71ef5c8428 Merge branch 'fatemabw/bro' of https://github.com/fatemabw/bro into dev/2.7
* 'fatemabw/bro' of https://github.com/fatemabw/bro:
  DNSSEC support in Bro

I made several changes:

  - renamed event/record types
  - reformatted the info added to dns.log
  - removed the "addl" scripts that added extended dnssec info to dns.log
  - simplifications/improvements to the internal parsing logic
2018-09-21 16:40:41 -05:00
Daniel Thayer
5161165c8a Fix some broken @TEST-REQUIRES
Some tests were always being skipped because config.h was renamed to
bro-config.h a long time ago.
2018-09-21 12:35:13 -05:00
Jon Siwek
34d0cf886c Fix potential memory leak in Kerberos scripts
Reported by Maksim Shudrak.
2018-09-10 18:06:07 -05:00
Johanna Amann
5c68093bc3 Permit weird sampling rate of 0.
This change allows a weird sampling rate of 0, which completely suppresses
all notifications (previously this crashed Bro). If also fixes the sampling
threshold to work with sampling rates of 0.
2018-09-05 13:12:23 -07:00
Jon Siwek
f5848f0279 Increase timeout for a memleak test 2018-08-22 19:22:08 -05:00
Jon Siwek
b679a51376 Merge remote-tracking branch 'origin/topic/jazoff/fix-snaplen'
* origin/topic/jazoff/fix-snaplen:
  problem: default snaplen is too small for jumbo frames
2018-08-21 14:54:55 -05:00
Jon Siwek
bcf97f70ea Merge remote-tracking branch 'origin/topic/jsiwek/empty-lines'
* origin/topic/jsiwek/empty-lines:
  Add 'smtp_excessive_pending_cmds' weird
  Fix SMTP command string comparisons
  Improve handling of empty lines in several text protocol analyzers
  Add rate-limiting sampling mechanism for weird events
  Teach timestamp canonifier about timestamps before ~2001
2018-08-20 15:35:16 -05:00
Jon Siwek
da9f91fc19 Add env. variables to override Broker listen/connect retry intervals
And use them to default retries to 1sec for all unit tests.
2018-08-16 12:16:03 -05:00
Jon Siwek
0e6913fba0 BIT-1798: fix PPTP GRE tunnel decapsulation 2018-08-14 16:48:04 -05:00
Johanna Amann
7b44a64994 Fix test that fails now that options are automatically redefable. 2018-08-10 14:28:17 -07:00
Johanna Amann
c34fbee0d1 Make options redef-able by default. 2018-08-10 11:53:36 -07:00
Jon Siwek
599af26496 Merge remote-tracking branch 'origin/topic/vern/vec-append'
* origin/topic/vern/vec-append:
  d'oh, still have a (deprecated) string_array rather than string_vector
  forgot to update test suite results for v += e
  reap the fruits of v += e
  test case for v += e
  documentation of v += e
  v += e implemented

Fixed a mistake in find_ip_addresses()
2018-08-03 11:38:40 -05:00
Jon Siwek
06c6e1188a Merge remote-tracking branch 'origin/topic/vern/set-ops2'
* origin/topic/vern/set-ops2:
  documentation, test suite update
  implemented set relationals
  bug fix for set intersection
  set intersection implemented
  mirroring previous topic/vern/set-ops to get branch up to date, since I'm a n00b

Fixed a couple memory leaks and added a leak test
2018-08-02 10:43:41 -05:00
Damani Wade
3710ff936f Add Cisco FabricPath support 2018-07-27 16:00:54 -05:00
Jon Siwek
35827eeb31 Add rate-limiting sampling mechanism for weird events
The generation of weird events, by default, are now rate-limited
according to these tunable options:

  - Weird::sampling_whitelist
  - Weird::sampling_threshold
  - Weird::sampling_rate
  - Weird::sampling_duration

The new get_reporter_stats() BIF also allows one to query the
total number of weirds generated (pre-sampling) which the new
policy/misc/weird-stats.bro script uses periodically to populate
a weird_stats.log.

There's also new reporter BIFs to allow generating weirds from the
script-layer such that they go through the same, internal
rate-limiting/sampling mechanisms:

  - Reporter::conn_weird
  - Reporter::flow_weird
  - Reporter::net_weird

Some of the code was adapted from previous work by Johanna Amann.
2018-07-26 19:57:36 -05:00
Vern Paxson
88fd7510c6 reap the fruits of v += e 2018-07-26 12:51:36 -07:00
Robin Sommer
8ac17d99a1 Merge remote-tracking branch 'origin/topic/jsiwek/bit-1950'
BIT-1950 #merged

* origin/topic/jsiwek/bit-1950:
  BIT-1950: support PPPoE over QinQ
2018-07-24 15:05:48 +00:00
Johanna Amann
12add53131 Fix special-case-bug for vectors in UnaryExpr.
In some cases one can get the Type() of unaryexpr to be ANY. Vectors so
far did not deal gracefully with this and crashed because trying to
convert any to a vectortype.

This patch fixes this by just using the original vector-type in this
case.
2018-07-20 13:36:38 -07:00
Jon Siwek
463e540c9b Merge remote-tracking branch 'origin/topic/vern/case-insensitive-patterns'
* origin/topic/vern/case-insensitive-patterns:
  use PCRE syntax instead of the beautiful new (?i ...) syntax
  nitlet in NEWS entry
  test suite update for case-insensitive patterns
  document use of double quotes to escape case-insensitivity
  bug fix for recent memory leak patch
  documentation updates for case-insensitive patterns
  d'oh there's isalpha.  I looked earlier for isletter :-P
  fix for handling [:(lower|upper):] in case-insensitive patterns
  implemented /re/i for case-insensitive patterns
2018-07-16 16:04:38 -05:00
Jon Siwek
ad9abd4c9b BIT-1950: support PPPoE over QinQ 2018-07-06 08:04:02 -05:00