Commit graph

10798 commits

Author SHA1 Message Date
Jon Siwek
249ef5adfc Update submodule(s)
[nomail]
2020-07-24 15:29:26 -07:00
Jon Siwek
b62fd71181 Misc/minor cleanup after reviewing new Broker table store code
Just some naming/style + minor memory management tweaks (avoid
some copies; use move semantics).
2020-07-24 14:00:24 -07:00
Jon Siwek
823d4e6220 Merge remote-tracking branch 'origin/topic/vladg/gh-1084'
* origin/topic/vladg/gh-1084:
  Add btest for GH-1084
  Update baselines
  MySQL: Fix parsing logic bug. We were correctly NOT expecting an EOF, but because we were parsing the header and then not parsing the rest, we would get out of sync
2020-07-24 11:41:16 -07:00
Jon Siwek
6662c9353a Update submodule(s)
[nomail]
2020-07-24 11:31:03 -07:00
Tim Wojtulewicz
3eab3e600e Merge remote-tracking branch 'ronwellman/parse_ecs'
* ronwellman/parse_ecs:
  Avoid typecast to int& in EDNS parsing.
  Validate option_len in EDNS packets.
  Adjust for zeek namespace.
  Implement EDNS Client Subnet Option
2020-07-24 09:26:06 -07:00
Vlad Grigorescu
f3c656ef17 Add btest for GH-1084 2020-07-24 09:28:11 -05:00
ronwellman
7f130e9e16 Avoid typecast to int& in EDNS parsing. 2020-07-24 10:21:42 -04:00
ronwellman
19e91292e8 Validate option_len in EDNS packets. 2020-07-24 09:26:09 -04:00
Jon Siwek
c765fd20fd GH-1083: Fix Input Framework 'change' events for 'set' destinations 2020-07-23 23:29:09 -07:00
Tim Wojtulewicz
ebc073ba92 Merge remote-tracking branch 'origin/topic/jsiwek/gh-1076-fix-random'
* origin/topic/jsiwek/gh-1076-fix-random:
  Deprecate bro_srandom(), replace with zeek::seed_random().
  Add zeek::max_random() & fix misuse of RAND_MAX w/ zeek::random_number()
  Deprecate bro_random(), replace with zeek::random_number()
  Deprecate bro_prng(), replace with zeek::prng()
  GH-1076: Fix bro_srandom() to replace 0 seeds with 1
  GH-1076: Fix bro_prng() implementation
  GH-1076: Fix use of getrandom()
2020-07-23 13:02:46 -07:00
Jon Siwek
d279f3467e Update submodule(s)
[nomail]
2020-07-23 10:13:00 -07:00
Vlad Grigorescu
58800a4e93 Merge remote-tracking branch 'origin/master' into topic/vladg/gh-1084 2020-07-23 12:00:17 -05:00
Vlad Grigorescu
e9768ccb18 Update baselines 2020-07-23 11:59:30 -05:00
Vlad Grigorescu
bfeaecd0aa MySQL: Fix parsing logic bug. We were correctly NOT expecting an EOF, but because we were parsing the header and then not parsing the rest, we would get out of sync 2020-07-23 11:51:18 -05:00
Tim Wojtulewicz
ac57377a94 Merge remote-tracking branch 'origin/topic/jsiwek/gh-1068-zeek-include-dir'
* origin/topic/jsiwek/gh-1068-zeek-include-dir:
  GH-1068: Add zeek symlink to allow "zeek/Foo.h" include style
2020-07-23 08:32:20 -07:00
Jon Siwek
b17627fa09 Deprecate bro_srandom(), replace with zeek::seed_random().
Avoiding zeek::srandom() to avoid potential for confusion with srandom()
2020-07-22 14:01:33 -07:00
Jon Siwek
d486af06b1 Add zeek::max_random() & fix misuse of RAND_MAX w/ zeek::random_number()
In deterministic mode, RAND_MAX is not related to the result of
zeek::random_number() (formerly bro_random()), but some logic was
using RAND_MAX as indication of the possible range of values.  The
new zeek::max_random() will give the correct upper-bound regardless
of whether deterministic-mode is used.
2020-07-22 14:01:33 -07:00
Jon Siwek
bde38893ce Deprecate bro_random(), replace with zeek::random_number()
Avoiding the use of zeek::random() due to potential for confusion
with random().
2020-07-22 14:01:33 -07:00
Jon Siwek
6bbb0a6b48 Deprecate bro_prng(), replace with zeek::prng()
The type used for storing the state of the RNG is changed from
`unsigned int` to `long int` since the former has a minimal range
of [0, 65,535] while the RNG function itself has a range of
[1, 2147483646].  A `long int` must be capable of
[−2147483647, +2147483647] and is also the return type of `random()`,
which is what zeek::prng() aims to roughly parity.
2020-07-22 14:01:33 -07:00
Jon Siwek
887b53b7f3 GH-1076: Fix bro_srandom() to replace 0 seeds with 1
The bro_prng() implementation cannot generate 0 as a result since it
causes every subsequent number from the PRNG to also be 0, so use the
number 1 instead of 0.
2020-07-22 14:01:33 -07:00
Jon Siwek
0f4eb9af02 GH-1076: Fix bro_prng() implementation
The intermediate result of the PRNG used unsigned storage, preventing
the ( result < 0 ) branch from ever being evaluated.  This could cause
return values to exceed the modulus as well as RAND_MAX.

One interesting effect of this is potential for the rand() BIF to
return values outside the requested maximum limit.

Another interesting effect of this is that a PacketFilter may start
randomly dropping packets even if it was not configured for
random-packet-drops.
2020-07-22 14:01:33 -07:00
Jon Siwek
dba764386b GH-1076: Fix use of getrandom()
The availability and use of getrandom() actually caused unrandom and
deterministic results in terms of Zeek's random number generation.
2020-07-22 14:01:33 -07:00
Jon Siwek
d7425b90d7 Update submodule(s)
[nomail]
2020-07-21 12:58:28 -07:00
Tim Wojtulewicz
ebf0f65f55 Merge remote-tracking branch 'origin/topic/jsiwek/add-zeek-archiver-submodule'
* origin/topic/jsiwek/add-zeek-archiver-submodule:
  Update zeek-archiver submodule
  Add zeek-archiver tool as submodule
2020-07-21 12:40:13 -07:00
Jon Siwek
b912dc74df Update zeek-archiver submodule 2020-07-21 12:32:11 -07:00
Robin Sommer
c3f4971eb2 Merge remote-tracking branch 'origin/topic/johanna/table-changes'
* origin/topic/johanna/table-changes: (26 commits)
  TableSync: try to make test more robust & add debug output
  Increase timeouts to see if FreeBSD will be happy with this.
  Try to make FreeBSD test happy with larger timeout.
  TableSync: refactor common functionality into function
  TableSync: don't raise &on_change, smaller fixes
  TableSync: rename auto_store -> table_store
  SyncTables: address feedback part 1 - naming (broker and zeek)
  BrokerStore <-> Zeek Tables: cleanup and bug workaround
  Zeek Table<->Brokerstore: cleanup, documentation, small fixes
  BrokerStore<->Zeek table: adopt to recent Zeek API changes
  BrokerStore<->Zeek Tables Fix a few small test failures.
  BrokerStore<->Zeek tables: allow setting storage location & tests
  BrokerStore<->Zeek tables: &backend works for in-memory stores.
  BrokerStore<->Zeek table - introdude &backend attribute
  BrokerStore<->Zeek tables: test for clones synchronizing to a master
  BrokerStore<->Zeek tables: load persistent tables on startup.
  Brokerstore<->Tables: attribute conflicts
  Zeek/Brokerstore updates: expiration
  Zeek/Brokerstore updates: add test that includes updates from clones
  Zeek/Brokerstore updates: first working end-to-end test
  ...
2020-07-21 15:39:39 +00:00
Jon Siwek
9642feeaf2 GH-1068: Add zeek symlink to allow "zeek/Foo.h" include style
Also changes `zeek-config --include_dir` to now output
`$prefix/include:$prefix/include/zeek` instead of only
`$prefix/include/zeek`
2020-07-20 17:54:09 -07:00
Jon Siwek
1a4990fc7e Add zeek-archiver tool as submodule
This is added to the default Zeek build as a convenience since it's the
new suggested method for archiving the logs produced by a Supervised
Zeek Cluster.
2020-07-20 16:06:44 -07:00
Jon Siwek
abba6fd3d0 Fix potential NetbiosSSN analyzer stack overflow
The Contents_NetbiosSSN analyzer used a recursive message parsing
function that determined the size of the next message from the input
packet-data itself.  A packet containing a sequence of many small
messages could cause a stack overflow since a recursion happened after
processing each message.
2020-07-20 13:40:25 -07:00
Jon Siwek
a66533f41a Fix potential DNS analyzer stack overflow
The Contents_DNS analyzer used a recursive message parsing function that
determined the size of the next message from the input packet-data
itself.  A packet containing a sequence of many small messages could
cause a stack overflow since a recursion happenened after processing
each message.

Credit to OSS-Fuzz for discovery
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=24272
2020-07-20 11:48:23 -07:00
Johanna Amann
a505ed4bfe TableSync: try to make test more robust & add debug output 2020-07-17 18:59:52 -07:00
Johanna Amann
8e70ff653f Merge remote-tracking branch 'origin/topic/johanna/update-the-ca-lists-once-again'
* origin/topic/johanna/update-the-ca-lists-once-again:
  Update Mozilla CA & Google CT lists
2020-07-18 01:43:36 +00:00
Johanna Amann
095491711e Increase timeouts to see if FreeBSD will be happy with this. 2020-07-17 17:14:44 -07:00
Jon Siwek
bd5108022f Merge remote-tracking branch 'origin/topic/timw/deprecation-fixes-for-spicy'
* origin/topic/timw/deprecation-fixes-for-spicy:
  Use namespaced version of Location to silence warnings
  Various deprecation fixes, reported by failed Spicy builds
  Add deprecated version of EnumType::GetVal() to returns EnumVal*, rename IntrusivePtr version to GetEnumVal
2020-07-17 16:37:30 -07:00
Johanna Amann
66f586c02c Update Mozilla CA & Google CT lists 2020-07-17 23:26:37 +00:00
Johanna Amann
42b566935e Try to make FreeBSD test happy with larger timeout. 2020-07-17 16:26:02 -07:00
Jon Siwek
54c9f4a0e1 Merge remote-tracking branch 'origin/topic/timw/deprecation-fixes-for-spicy'
Merge adjustments:

- Revert unneeded changes in mmdb_dir lookups to fix build
- Fix deprecated EnumType::GetVal() to return +1 ref-count

* origin/topic/timw/deprecation-fixes-for-spicy:
  Various deprecation fixes, reported by failed Spicy builds
  Add deprecated version of EnumType::GetVal() to returns EnumVal*, rename IntrusivePtr version to GetEnumVal
2020-07-17 16:14:49 -07:00
Tim Wojtulewicz
e1c339c962 Use namespaced version of Location to silence warnings 2020-07-17 19:12:08 -04:00
Tim Wojtulewicz
9fc48df7fd Various deprecation fixes, reported by failed Spicy builds
- Add deprecated version of ID::SetType() that takes Type*
- Add deprecated versions of zeek::set_location in the global namespace
- Fix global namespace version of lookup_ID to return ID*
2020-07-17 19:12:08 -04:00
Tim Wojtulewicz
964e956cac Add deprecated version of EnumType::GetVal() to returns EnumVal*, rename IntrusivePtr version to GetEnumVal 2020-07-17 19:12:08 -04:00
Johanna Amann
6ed5987124 Update submodules
[nomail]
2020-07-17 23:02:05 +00:00
Johanna Amann
a0c0236fb1 Merge remote-tracking branch 'origin/topic/jsiwek/util-fixes'
* origin/topic/jsiwek/util-fixes:
  Fix race condition in ensure_dir()
  Fix tokenize_string() to work with delimiters of length > 1
2020-07-17 22:34:48 +00:00
Johanna Amann
5982b1e4b2 TableSync: refactor common functionality into function
This addresses feedback and puts the common update and insert
functionality into its own function.
2020-07-17 15:27:01 -07:00
Johanna Amann
36db9d8369 TableSync: don't raise &on_change, smaller fixes
This addresses PR feedback. The main component in this commit is to
disable &on_change notifications when &backend loads a table from sqlite
on startup.
2020-07-17 14:21:27 -07:00
Johanna Amann
930a5c8ebd TableSync: rename auto_store -> table_store 2020-07-17 11:40:59 -07:00
Johanna Amann
6d2aa84952 SyncTables: address feedback part 1 - naming (broker and zeek)
This commit fixes capitalization issues.
2020-07-17 10:56:28 -07:00
Robin Sommer
651831ad31 Merge remote-tracking branch 'origin/topic/jsiwek/gh-734-fix-lambda-escaping'
* origin/topic/jsiwek/gh-734-fix-lambda-escaping:
  GH-734: Improve handling of lambdas that escape enclosing frame
2020-07-17 10:19:18 +00:00
Robin Sommer
edaeb2f308 Merge remote-tracking branch 'origin/topic/jsiwek/gh-837-improve-broker-mismatch-errors'
* origin/topic/jsiwek/gh-837-improve-broker-mismatch-errors:
  GH-837: Add test cases for mismatched Broker SSL configs
  GH-837: emit Reporter errors for Broker errors
  Fix incorrect/missing Broker error status code numbers
2020-07-17 10:09:05 +00:00
Jon Siwek
85fbdaf429 GH-837: Add test cases for mismatched Broker SSL configs 2020-07-16 18:32:06 -07:00
Jon Siwek
c84a51ac09 GH-837: emit Reporter errors for Broker errors
Instead of only writing them in broker.log, which may be easy to
overlook.
2020-07-16 18:07:00 -07:00