Commit graph

2615 commits

Author SHA1 Message Date
Johanna Amann
cb47b37215 Be more liberal with whitespaces for ip-addresses, subnets, etc.
This change ignores leading/trailing whitespaces for a couple of
data-types (bool, port, subnet, addr) and just parses them as if the
whitespace was not present.
2019-02-19 15:06:06 -08:00
Jon Siwek
aff3f4b3fd Skip autogenerated doc coverage test for Travis pull requests 2019-02-15 17:53:53 -06:00
Jon Siwek
e6e328efb4 Merge branch 'master' of https://github.com/ZekeMedley/zeek
* 'master' of https://github.com/ZekeMedley/zeek:
  lstrip test output cleanup
  implemented rstrip
  add rstrip tests
  cleanup of lstrip function
  added implementation of lstrip
  added tests for lstrip function
2019-02-15 17:51:29 -06:00
Zeke Medley
ffbe366646 lstrip test output cleanup 2019-02-14 00:02:41 -08:00
Zeke Medley
8ac53a5fae add rstrip tests 2019-02-13 23:51:27 -08:00
Zeke Medley
f9191d439b added tests for lstrip function 2019-02-13 23:19:31 -08:00
Jon Siwek
0790c1c559 Merge remote-tracking branch 'origin/topic/jsiwek/gh-211'
* origin/topic/jsiwek/gh-211:
  GH-208: change invalid subnet expressions to a runtime error
  GH-211: improve consistency of how scripting errors are handled

Removed the 'allow_init_errors' option.
2019-02-07 10:33:52 -06:00
Jon Siwek
01a8418d79 GH-208: change invalid subnet expressions to a runtime error
Rather than abort.
2019-01-30 12:01:46 -06:00
Jon Siwek
67484a90fa GH-211: improve consistency of how scripting errors are handled
Scripting errors/mistakes now consistently generate a runtime error
which have the behavior of unwinding the call stack all the way out of
the current event handler.

Before, such errors were not treated consistently and either aborted
the process entirely or emitted a message while continuing to execute
subsequent statements without well-defined behavior (possibly causing
a cascade of errors).

The previous behavior also would only unwind out of the current
function (if within a function body), not out the current event
handler, which is especially problematic for functions that return
a value: the caller is essentially left a mess with no way to deal
with it.

This also changes the behavior of the startup/initialization process
to abort if there's errors during bro_init() rather than continue one
to the main run loop.  The `allow_init_errors` option may change this
new, default behavior.
2019-01-30 11:20:09 -06:00
Jon Siwek
49a30d61cf Fix memory leak due to enum type/val circular references
Introduced in 2982765128
2019-01-29 18:28:13 -06:00
Jon Siwek
3e2c2a6140 GH-210: improve call stack tracking
Adds tracking of arguments
2019-01-28 14:17:04 -06:00
Johanna Amann
7465bceb7e Merge remote-tracking branch 'origin/topic/jsiwek/gh-167'
* origin/topic/jsiwek/gh-167:
  GH-167: improve error message for unclosed function at EOF
2019-01-24 15:22:42 -08:00
Jon Siwek
7a3ecd76b4 Merge remote-tracking branch 'origin/topic/johanna/md5-fips'
* origin/topic/johanna/md5-fips:
  A few more updates to the digest functions.
  Tell OpenSSL that MD5 is not used for security in order to allow bro to work properly on a FIPS system

I changed a couple places that looked like memory management pitfalls:
moved some cleanup code into the dtors of HashVal derived classes
(seemed like it got stuck in ctors by accident) and also added a
cautionary cleanup in the MIME code.

Plus minor formatting changes.
2019-01-24 17:20:06 -06:00
Jon Siwek
9b43f1d5f4 GH-167: improve error message for unclosed function at EOF 2019-01-24 14:54:39 -06:00
Johanna Amann
ec2d98b382 Merge remote-tracking branch 'origin/master' into topic/johanna/md5-fips 2019-01-24 12:19:17 -08:00
Johanna Amann
86161c85c4 A few more updates to the digest functions.
This builds upon the previous commit to make Zeek compile on FIPS
systems.

This patch makes the changes a bit more aggressive. Instead of having a
number of different hash functions with different return values, we now
standardize on EVP_MD_CTX and just have one set of functions, to which
the hash algorithm that is desired is passed.

On the positive side, this enables us to support a wider range of hash
algorithm (and to easily add to them in the future).

I reimplemented the internal_md5 function - we don't support ebdic
systems in any case.

The md5/sha1 serialization functions are now also tested (I don't think
they were before).
2019-01-24 10:44:28 -08:00
Stu H
5ef70c4df1 Added ERSPAN III testing 2019-01-24 14:05:13 +00:00
Jon Siwek
3256ac7c49 GH-219: fix |x| operator int overflow / floating point type inconsistency
When 'x' is an integral arithmetic expression, it's now coerced to
yield a signed integer before taking the absolute value of it to
prevent the common issue of unsigned integer overflow/wraparound for
values below zero.

Using a time or interval value/expression for 'x' now also yields a
time or interval, respective, from the |x| operation instead of
a double.
2019-01-22 16:42:40 -06:00
Johanna Amann
7c892ef7d4 Merge remote-tracking branch 'origin/topic/jsiwek/gh-151'
* origin/topic/jsiwek/gh-151:
  GH-151: fix hash calculation for nested sets
2019-01-22 09:08:14 -08:00
Jon Siwek
d86fb9f87a GH-151: fix hash calculation for nested sets
Hash key construction of nested sets depended on the order in
which their elements are iterated, which varied even between sets
containing equivalent elements.  The iteration order is now sorted
by each element's hash value (or, on collision, by full key) such
that equivalent sets no longer hash differently.
2019-01-18 20:54:22 -06:00
Johanna Amann
d4f7dae768 Merge remote-tracking branch 'origin/topic/jsiwek/val_mgr'
* origin/topic/jsiwek/val_mgr:
  Pre-allocate and re-use Vals for bool, int, count, enum and empty string
  Preallocate booleans and small counts

I added a tiny change to CompHash to make sure that nothing messes this
up in the future.
2019-01-18 15:17:34 -08:00
Jon Siwek
5618b21cca Improve ERSPAN Type III support
Added check for the optional sub-header in ERSPAN Type III as well
as additional truncation checks to the GRE parsing logic in general.
Also added a unit test for ERSPAN Type II.
2019-01-17 18:06:10 -06:00
Jon Siwek
2ff746fea7 Change doc/ subdir into a git submodule
The docs now live at https://github.com/zeek/zeek-docs
2019-01-17 14:15:45 -06:00
Jon Siwek
0d685efbf5 Add Broker::peer_counts_as_iosource option
Disabling this option allows one to read pcaps, but still initiate
Broker peerings and automatically exit when done processing the pcap
file.  The default behavior would normally cause Broker::peer() to
prevent shutting the process down even after done reading the pcap.
2019-01-16 19:03:35 -06:00
Jon Siwek
12f3ab303b Improve unit test for reporter shutdown order 2019-01-15 10:32:13 -06:00
Jon Siwek
ed1a50ec5b GH-213: change type of vector for-loop index to a count 2019-01-14 18:04:30 -06:00
Jon Siwek
38cb3dbf2e GH-188: fix crash when shutting down with pending reporter errors
This fixes some use-after-free issues in the shutdown order of
various systems: I/O, plugin, logging, and reporter systems may
interact during shutdown if there's errors emitted (or maybe just
still pending) during the shutdown.

Fixes GH-188
2019-01-14 15:40:17 -06:00
Jon Siwek
cada0d5e6e GH-161: fix segfault in &default type checking for sets
Fixes GH-161
2019-01-14 14:03:15 -06:00
Jon Siwek
b1b7027982 Merge remote-tracking branch 'origin/rtd-test'
* origin/rtd-test:
  Disable RTD pdf format due to exceeded capacity
  Add RTD pdf format
  Add RTD yaml config file
  Remove some Bro usages in main TOC entries
  Remove "contents" Sphinx directive usages
  Add a `make livehtml` target
  Use sourcecode Sphinx directive more widely
  Use Sphinx RTD theme for user manual
  Replace some code-block Sphinx directives
  Remove unused Sphinx extensions
  Remove broxygen Sphinx integration
  Remove Sphinx btest integrations and tests
  Fix a Sphinx deprecation

These are all changes required to build documentation from a static
Sphinx tree (e.g. on Read the Docs)
2019-01-10 14:08:40 -06:00
Johanna Amann
60fd0e1b8d Baseline update for catch-and-release change. 2019-01-10 09:10:08 -08:00
Jon Siwek
2982765128 Pre-allocate and re-use Vals for bool, int, count, enum and empty string 2019-01-09 18:29:23 -06:00
Jon Siwek
e0b8b4b6b1 Replace some bro.org usages with zeek.org 2019-01-04 17:51:25 -06:00
Jon Siwek
7e9d48f532 Remove broxygen Sphinx integration
The broxygen-generated files now live in the git repo, have tests
that check that they are up-to-date, and a script to re-generate
them on-demand.
2018-12-18 10:15:22 -06:00
Jon Siwek
9e5e9d04b7 Remove Sphinx btest integrations and tests 2018-12-14 16:32:35 -06:00
Jon Siwek
9736182fbc Update github/download links 2018-12-07 16:36:51 -06:00
Jon Siwek
f287b2bdf9 Fix Travis git clone command 2018-12-07 07:27:42 -06:00
Jon Siwek
34e577c4f0 Update external test suite locations 2018-12-06 20:11:57 -06:00
Jon Siwek
fd63168171 Merge remote-tracking branch 'origin/topic/jsiwek/parallelize-comm-tests'
* origin/topic/jsiwek/parallelize-comm-tests:
  Parallelize communication tests using btest TEST-PORT
2018-11-29 17:06:30 -06:00
Jon Siwek
28a2964e38 Merge remote-tracking branch 'origin/dev/2.7'
* origin/dev/2.7:
  Improve introspection of Record and TypeType values
  Bro plugins should support a patch version (x.y.z)
  GH-148: add priority to DNSSEC event handlers
  DNSSEC support in Bro
2018-11-29 16:44:50 -06:00
Jon Siwek
635e030be2 Merge remote-tracking branch 'origin/master' into dev/2.7
* origin/master:
  Fix SumStats "last" plugin in cluster mode
  Remove unnessary check
  Support appending to vector of any
  Fix coding conventions nits/typos
  Updating submodule(s).
  Revert "Switch frag_timeout from redef to option"
  Improve error handling in x509_ocsp_verify function
  Updating submodule(s).
  Updating submodule(s).
  Update baseline for plugins.hooks for redef -> option changes
  Switch frag_timeout from redef to option
  Switch GridFTP options from redef to option
  Updating submodule(s).
  Fix a unit test relying on a bash-ism
2018-11-08 12:43:22 -06:00
Jon Siwek
01e17b5ea0 Fix SumStats "last" plugin in cluster mode 2018-11-08 12:27:37 -06:00
Jon Siwek
ca014d6e35 Support appending to vector of any
For consistency, since it already works to assign to an index within
vector of any.
2018-11-07 11:48:43 -06:00
Jon Siwek
c1f02aaa61 Revert "Switch frag_timeout from redef to option"
This reverts commit 38259afa8d.

The change to an option wasn't yet incomplete -- that would
require dynamic lookup of the value at the C++ layer.
2018-11-06 09:32:58 -06:00
Vlad Grigorescu
4f6db2f960 Update baseline for plugins.hooks for redef -> option changes 2018-11-05 14:00:44 -06:00
Jon Siwek
907297ba59 Parallelize communication tests using btest TEST-PORT 2018-11-04 15:29:59 -06:00
Jon Siwek
de640d651f Fix a unit test relying on a bash-ism 2018-11-02 18:31:45 -05:00
Jon Siwek
0249ecba4b Merge remote-tracking branch 'origin/master' into dev/2.7
* origin/master:
  Add script-layer call stack to internal errors messages that abort
2018-11-02 17:45:28 -05:00
Jon Siwek
b2560384c4 Add script-layer call stack to internal errors messages that abort 2018-11-02 17:43:34 -05:00
Jon Siwek
b54570aa98 Merge remote-tracking branch 'origin/master' into dev/2.7
* origin/master:
  Improve error message of index assignment expression failures
2018-11-02 16:42:00 -05:00
Jon Siwek
a7ba44089b Improve error message of index assignment expression failures 2018-11-02 16:40:28 -05:00