Commit graph

11634 commits

Author SHA1 Message Date
Tim Wojtulewicz
ac57377a94 Merge remote-tracking branch 'origin/topic/jsiwek/gh-1068-zeek-include-dir'
* origin/topic/jsiwek/gh-1068-zeek-include-dir:
  GH-1068: Add zeek symlink to allow "zeek/Foo.h" include style
2020-07-23 08:32:20 -07:00
Jon Siwek
b17627fa09 Deprecate bro_srandom(), replace with zeek::seed_random().
Avoiding zeek::srandom() to avoid potential for confusion with srandom()
2020-07-22 14:01:33 -07:00
Jon Siwek
d486af06b1 Add zeek::max_random() & fix misuse of RAND_MAX w/ zeek::random_number()
In deterministic mode, RAND_MAX is not related to the result of
zeek::random_number() (formerly bro_random()), but some logic was
using RAND_MAX as indication of the possible range of values.  The
new zeek::max_random() will give the correct upper-bound regardless
of whether deterministic-mode is used.
2020-07-22 14:01:33 -07:00
Jon Siwek
bde38893ce Deprecate bro_random(), replace with zeek::random_number()
Avoiding the use of zeek::random() due to potential for confusion
with random().
2020-07-22 14:01:33 -07:00
Jon Siwek
6bbb0a6b48 Deprecate bro_prng(), replace with zeek::prng()
The type used for storing the state of the RNG is changed from
`unsigned int` to `long int` since the former has a minimal range
of [0, 65,535] while the RNG function itself has a range of
[1, 2147483646].  A `long int` must be capable of
[−2147483647, +2147483647] and is also the return type of `random()`,
which is what zeek::prng() aims to roughly parity.
2020-07-22 14:01:33 -07:00
Jon Siwek
887b53b7f3 GH-1076: Fix bro_srandom() to replace 0 seeds with 1
The bro_prng() implementation cannot generate 0 as a result since it
causes every subsequent number from the PRNG to also be 0, so use the
number 1 instead of 0.
2020-07-22 14:01:33 -07:00
Jon Siwek
0f4eb9af02 GH-1076: Fix bro_prng() implementation
The intermediate result of the PRNG used unsigned storage, preventing
the ( result < 0 ) branch from ever being evaluated.  This could cause
return values to exceed the modulus as well as RAND_MAX.

One interesting effect of this is potential for the rand() BIF to
return values outside the requested maximum limit.

Another interesting effect of this is that a PacketFilter may start
randomly dropping packets even if it was not configured for
random-packet-drops.
2020-07-22 14:01:33 -07:00
Jon Siwek
dba764386b GH-1076: Fix use of getrandom()
The availability and use of getrandom() actually caused unrandom and
deterministic results in terms of Zeek's random number generation.
2020-07-22 14:01:33 -07:00
Jon Siwek
d7425b90d7 Update submodule(s)
[nomail]
2020-07-21 12:58:28 -07:00
Tim Wojtulewicz
ebf0f65f55 Merge remote-tracking branch 'origin/topic/jsiwek/add-zeek-archiver-submodule'
* origin/topic/jsiwek/add-zeek-archiver-submodule:
  Update zeek-archiver submodule
  Add zeek-archiver tool as submodule
2020-07-21 12:40:13 -07:00
Jon Siwek
b912dc74df Update zeek-archiver submodule 2020-07-21 12:32:11 -07:00
Robin Sommer
c3f4971eb2 Merge remote-tracking branch 'origin/topic/johanna/table-changes'
* origin/topic/johanna/table-changes: (26 commits)
  TableSync: try to make test more robust & add debug output
  Increase timeouts to see if FreeBSD will be happy with this.
  Try to make FreeBSD test happy with larger timeout.
  TableSync: refactor common functionality into function
  TableSync: don't raise &on_change, smaller fixes
  TableSync: rename auto_store -> table_store
  SyncTables: address feedback part 1 - naming (broker and zeek)
  BrokerStore <-> Zeek Tables: cleanup and bug workaround
  Zeek Table<->Brokerstore: cleanup, documentation, small fixes
  BrokerStore<->Zeek table: adopt to recent Zeek API changes
  BrokerStore<->Zeek Tables Fix a few small test failures.
  BrokerStore<->Zeek tables: allow setting storage location & tests
  BrokerStore<->Zeek tables: &backend works for in-memory stores.
  BrokerStore<->Zeek table - introdude &backend attribute
  BrokerStore<->Zeek tables: test for clones synchronizing to a master
  BrokerStore<->Zeek tables: load persistent tables on startup.
  Brokerstore<->Tables: attribute conflicts
  Zeek/Brokerstore updates: expiration
  Zeek/Brokerstore updates: add test that includes updates from clones
  Zeek/Brokerstore updates: first working end-to-end test
  ...
2020-07-21 15:39:39 +00:00
Jon Siwek
9642feeaf2 GH-1068: Add zeek symlink to allow "zeek/Foo.h" include style
Also changes `zeek-config --include_dir` to now output
`$prefix/include:$prefix/include/zeek` instead of only
`$prefix/include/zeek`
2020-07-20 17:54:09 -07:00
Jon Siwek
1a4990fc7e Add zeek-archiver tool as submodule
This is added to the default Zeek build as a convenience since it's the
new suggested method for archiving the logs produced by a Supervised
Zeek Cluster.
2020-07-20 16:06:44 -07:00
Jon Siwek
abba6fd3d0 Fix potential NetbiosSSN analyzer stack overflow
The Contents_NetbiosSSN analyzer used a recursive message parsing
function that determined the size of the next message from the input
packet-data itself.  A packet containing a sequence of many small
messages could cause a stack overflow since a recursion happened after
processing each message.
2020-07-20 13:40:25 -07:00
Jon Siwek
a66533f41a Fix potential DNS analyzer stack overflow
The Contents_DNS analyzer used a recursive message parsing function that
determined the size of the next message from the input packet-data
itself.  A packet containing a sequence of many small messages could
cause a stack overflow since a recursion happenened after processing
each message.

Credit to OSS-Fuzz for discovery
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=24272
2020-07-20 11:48:23 -07:00
Johanna Amann
a505ed4bfe TableSync: try to make test more robust & add debug output 2020-07-17 18:59:52 -07:00
Johanna Amann
8e70ff653f Merge remote-tracking branch 'origin/topic/johanna/update-the-ca-lists-once-again'
* origin/topic/johanna/update-the-ca-lists-once-again:
  Update Mozilla CA & Google CT lists
2020-07-18 01:43:36 +00:00
Johanna Amann
095491711e Increase timeouts to see if FreeBSD will be happy with this. 2020-07-17 17:14:44 -07:00
Jon Siwek
bd5108022f Merge remote-tracking branch 'origin/topic/timw/deprecation-fixes-for-spicy'
* origin/topic/timw/deprecation-fixes-for-spicy:
  Use namespaced version of Location to silence warnings
  Various deprecation fixes, reported by failed Spicy builds
  Add deprecated version of EnumType::GetVal() to returns EnumVal*, rename IntrusivePtr version to GetEnumVal
2020-07-17 16:37:30 -07:00
Johanna Amann
66f586c02c Update Mozilla CA & Google CT lists 2020-07-17 23:26:37 +00:00
Johanna Amann
42b566935e Try to make FreeBSD test happy with larger timeout. 2020-07-17 16:26:02 -07:00
Jon Siwek
54c9f4a0e1 Merge remote-tracking branch 'origin/topic/timw/deprecation-fixes-for-spicy'
Merge adjustments:

- Revert unneeded changes in mmdb_dir lookups to fix build
- Fix deprecated EnumType::GetVal() to return +1 ref-count

* origin/topic/timw/deprecation-fixes-for-spicy:
  Various deprecation fixes, reported by failed Spicy builds
  Add deprecated version of EnumType::GetVal() to returns EnumVal*, rename IntrusivePtr version to GetEnumVal
2020-07-17 16:14:49 -07:00
Tim Wojtulewicz
e1c339c962 Use namespaced version of Location to silence warnings 2020-07-17 19:12:08 -04:00
Tim Wojtulewicz
9fc48df7fd Various deprecation fixes, reported by failed Spicy builds
- Add deprecated version of ID::SetType() that takes Type*
- Add deprecated versions of zeek::set_location in the global namespace
- Fix global namespace version of lookup_ID to return ID*
2020-07-17 19:12:08 -04:00
Tim Wojtulewicz
964e956cac Add deprecated version of EnumType::GetVal() to returns EnumVal*, rename IntrusivePtr version to GetEnumVal 2020-07-17 19:12:08 -04:00
Johanna Amann
6ed5987124 Update submodules
[nomail]
2020-07-17 23:02:05 +00:00
Johanna Amann
a0c0236fb1 Merge remote-tracking branch 'origin/topic/jsiwek/util-fixes'
* origin/topic/jsiwek/util-fixes:
  Fix race condition in ensure_dir()
  Fix tokenize_string() to work with delimiters of length > 1
2020-07-17 22:34:48 +00:00
Johanna Amann
5982b1e4b2 TableSync: refactor common functionality into function
This addresses feedback and puts the common update and insert
functionality into its own function.
2020-07-17 15:27:01 -07:00
Johanna Amann
36db9d8369 TableSync: don't raise &on_change, smaller fixes
This addresses PR feedback. The main component in this commit is to
disable &on_change notifications when &backend loads a table from sqlite
on startup.
2020-07-17 14:21:27 -07:00
Johanna Amann
930a5c8ebd TableSync: rename auto_store -> table_store 2020-07-17 11:40:59 -07:00
Johanna Amann
6d2aa84952 SyncTables: address feedback part 1 - naming (broker and zeek)
This commit fixes capitalization issues.
2020-07-17 10:56:28 -07:00
Robin Sommer
651831ad31 Merge remote-tracking branch 'origin/topic/jsiwek/gh-734-fix-lambda-escaping'
* origin/topic/jsiwek/gh-734-fix-lambda-escaping:
  GH-734: Improve handling of lambdas that escape enclosing frame
2020-07-17 10:19:18 +00:00
Robin Sommer
edaeb2f308 Merge remote-tracking branch 'origin/topic/jsiwek/gh-837-improve-broker-mismatch-errors'
* origin/topic/jsiwek/gh-837-improve-broker-mismatch-errors:
  GH-837: Add test cases for mismatched Broker SSL configs
  GH-837: emit Reporter errors for Broker errors
  Fix incorrect/missing Broker error status code numbers
2020-07-17 10:09:05 +00:00
Jon Siwek
85fbdaf429 GH-837: Add test cases for mismatched Broker SSL configs 2020-07-16 18:32:06 -07:00
Jon Siwek
c84a51ac09 GH-837: emit Reporter errors for Broker errors
Instead of only writing them in broker.log, which may be easy to
overlook.
2020-07-16 18:07:00 -07:00
Jon Siwek
7e03233d55 Fix incorrect/missing Broker error status code numbers 2020-07-16 18:03:41 -07:00
Jon Siwek
d60f16c229 Fix race condition in ensure_dir()
If something else created the dir between the stat() and mkdir(),
it previously reported that as a failure.
2020-07-16 12:32:10 -07:00
Jon Siwek
dfc34563b5 Fix tokenize_string() to work with delimiters of length > 1 2020-07-16 11:51:40 -07:00
Tim Wojtulewicz
4d96b7f1cc Various deprecation fixes, reported by failed Spicy builds
- Add deprecated version of ID::SetType() that takes Type*
- Add deprecated versions of zeek::set_location in the global namespace
- Fix global namespace version of lookup_ID to return ID*
2020-07-15 15:24:19 -07:00
Tim Wojtulewicz
118605f4ac Add deprecated version of EnumType::GetVal() to returns EnumVal*, rename IntrusivePtr version to GetEnumVal 2020-07-15 14:56:05 -07:00
Jon Siwek
eb826bd337 GH-734: Improve handling of lambdas that escape enclosing frame 2020-07-14 22:38:43 -07:00
Jon Siwek
fd8db24daf Merge remote-tracking branch 'origin/topic/jsiwek/gh-1062-fix-tag-subtypes'
* origin/topic/jsiwek/gh-1062-fix-tag-subtypes:
  GH-1062: fix integer conversion errors related to Tag subtypes
2020-07-14 19:11:09 -07:00
Jon Siwek
7f2aae1721 GH-1062: fix integer conversion errors related to Tag subtypes 2020-07-14 15:55:28 -07:00
Jon Siwek
da105b3bf9 Update submodule(s)
[nomail]
2020-07-14 14:58:50 -07:00
Johanna Amann
1888d6acae BrokerStore <-> Zeek Tables: cleanup and bug workaround 2020-07-13 17:23:35 -07:00
Johanna Amann
7c37226eaa Merge remote-tracking branch 'origin/master' into topic/johanna/table-changes 2020-07-13 17:11:55 -07:00
Johanna Amann
da2c968ba0 Merge remote-tracking branch 'origin/topic/jsiwek/gh-1024-broker-store-handle-type-checks'
* origin/topic/jsiwek/gh-1024-broker-store-handle-type-checks:
  Improve Broker store API's handling of invalid arguments
  Add builtin_exception() functions
  GH-1024: fix crash on passing wrong types to Broker store API

This was a bit of a bigger merge since Zeek changed inbetween the time
of the PR and me actually merging it.

I put the new functions into the zeek::detail namespace -- since it
seems unlikely that those will be used by something external.

I also renamed them to fit better with the naming scheme of the new
error functions.

Fixes GH-1024
2020-07-14 00:05:09 +00:00
Tim Wojtulewicz
37dbca965e Update submodule(s)
[nomail]
2020-07-13 12:39:32 -07:00
Tim Wojtulewicz
75cb35e2d9 Merge remote-tracking branch 'origin/topic/jsiwek/gh-1019-deprecate-icmp-conn'
* origin/topic/jsiwek/gh-1019-deprecate-icmp-conn:
  GH-1019: deprecate icmp_conn params for ICMP events
2020-07-13 12:12:44 -07:00