Commit graph

12374 commits

Author SHA1 Message Date
Christian Kreibich
2bcaa33563 Merge branch 'topic/christian/gh-1731-broker-peers-bug'
* topic/christian/gh-1731-broker-peers-bug:
  Fix addr/string type confusion in Broker::peers()
2021-08-19 15:13:37 -07:00
Christian Kreibich
7d5ca89860 Merge branch 'topic/christian/broker_listen_default'
* topic/christian/broker_listen_default:
  Simplify the supervisor's listen() on default address/port
2021-08-19 15:13:27 -07:00
Tim Wojtulewicz
8d9b06a12e Add template to NEWS for 4.2.0 2021-08-19 09:42:47 -07:00
Tim Wojtulewicz
fe22d7cbb4 Merge remote-tracking branch 'origin/topic/timw/1581-utf8-escaping'
* origin/topic/timw/1581-utf8-escaping:
  Return fully-escaped string if utf8 conversion fails
2021-08-19 09:39:35 -07:00
Tim Wojtulewicz
f442893c98 Return fully-escaped string if utf8 conversion fails
This adds a new function for validating UTF-8 sequences by converting to
UTF-32. This allows us to also check for various blocks of codepointsi
that we consider invalid while checking for valid sequences in general.
2021-08-19 08:56:27 -07:00
Christian Kreibich
fa38d81a7b Fix addr/string type confusion in Broker::peers()
Broker::NetworkInfo$address is of type string. The Broker::peers() BiF treated
it as an address, which caused ZVal's type validation to flag a
mismatch and abort.

Includes a small tweak to the broker.unpeer test that makes it invoke
Broker::peers(), which breaks in absence of the fix.
2021-08-18 15:16:58 -07:00
Christian Kreibich
90b7c6961e Simplify the supervisor's listen() on default address/port 2021-08-18 12:35:49 -07:00
Tim Wojtulewicz
b28fc77054 Merge remote-tracking branch 'origin/topic/bbannier/issue-1726'
* origin/topic/bbannier/issue-1726:
  Fix option length computation in Geneve analyzer.
2021-08-18 08:40:57 -07:00
Benjamin Bannier
28e8abbf19 Fix option length computation in Geneve analyzer.
We previously computed the length of the Geneve options field
incorrectly which lead to us passing data at an incorrect offset to
inner analyzers. With this patch we now interpret the length field
correctly, according the the spec
https://datatracker.ietf.org/doc/html/rfc8926#section-3.4.

Closes #1726.
2021-08-18 09:19:49 +02:00
Christian Kreibich
d60f85a868 Update submodule(s) [nomail] [skip ci] 2021-08-16 14:14:30 -07:00
Tim Wojtulewicz
7a4d3400d6 Merge remote-tracking branch 'origin/topic/timw/1713-less-realloc'
* origin/topic/timw/1713-less-realloc:
  GH-1713: Avoid unneeded reallocs in SerializationFormat::WriteData
2021-08-12 09:58:08 -07:00
Tim Wojtulewicz
9c29c8ddfc GH-1713: Avoid unneeded reallocs in SerializationFormat::WriteData 2021-08-11 15:40:09 -07:00
zeek-bot
2476e36634 Update doc submodule [nomail] [skip ci] 2021-08-07 00:39:20 +00:00
Tim Wojtulewicz
cdfa50ddec Merge remote-tracking branch 'origin/topic/johanna/ignore-checksums-nets'
* origin/topic/johanna/ignore-checksums-nets:
  Do not lookup ignore_checksums_nets for every packet
2021-08-06 13:29:46 -07:00
Johanna Amann
8192ad581d Do not lookup ignore_checksums_nets for every packet
This could lead to a noticeable (single-percent) performance
improvement.

Most of the functionality for this is in the packet analyzers that now
cache ignore_chesksums_nets.

Based on a patch by Arne Welzel (Corelight).
2021-08-06 10:32:53 +01:00
Johanna Amann
d24cecf268 Merge remote-tracking branch 'origin/topic/timw/sessions-unordered-map'
* origin/topic/timw/sessions-unordered-map:
  Only sort sessions during Drain() if a random seed is set
  Use unordered_map to store sessions for performance reasons
2021-08-04 10:01:37 +01:00
Tim Wojtulewicz
ae39225f82 Update broker submodule [nomail] 2021-08-03 17:42:15 -07:00
Tim Wojtulewicz
201617540d Only sort sessions during Drain() if a random seed is set 2021-08-03 10:07:47 -07:00
Johanna Amann
ec6b954499 Merge branch 'master' of https://github.com/sowmyaramapatruni/zeek
Fixes GH-1689

* 'master' of https://github.com/sowmyaramapatruni/zeek:
  Fix issue-1689
2021-08-03 10:25:26 +01:00
Johanna Amann
8b506ca113 Merge branch 'topic/jgvt/issue-1598' of https://github.com/ThalesGroup/zeek
* 'topic/jgvt/issue-1598' of https://github.com/ThalesGroup/zeek:
  Fix when HTTP header are on several packet
  Fix HTTP evasion     - Happen when there is no CRLF at the end of HTTP     - Fix by adding CRLF when packet is complete (in relation to content-length in header)
2021-08-03 10:21:41 +01:00
Sowmya Ramapatruni
58fae22708 Fix issue-1689 2021-08-02 13:52:43 -07:00
Tim Wojtulewicz
d343dbbd76 Merge remote-tracking branch 'origin/topic/timw/patricia-rebase'
* origin/topic/timw/patricia-rebase:
  Fix failing test due to patricia change
  Rebase patricia code on upstream version
2021-08-02 11:10:15 -07:00
Tim Wojtulewicz
e89a743901 Merge remote-tracking branch 'origin/topic/timw/1654-exclude-cmake-dot-git'
* origin/topic/timw/1654-exclude-cmake-dot-git:
  GH-1654: Exclude the .git directory when installing cmake files
2021-07-30 08:52:56 -07:00
Tim Wojtulewicz
a9deb1bfd2 GH-1654: Exclude the .git directory when installing cmake files 2021-07-29 15:30:21 -07:00
Tim Wojtulewicz
e2dc6df8a2 Use unordered_map to store sessions for performance reasons 2021-07-29 13:26:46 -07:00
Tim Wojtulewicz
2a717e05cc Merge remote-tracking branch 'origin/topic/timw/1692-binary-to-int64-shifting'
* origin/topic/timw/1692-binary-to-int64-shifting:
  GH-1692: Add some safety to ASN's binary_to_int64 to avoid bad shifts
2021-07-29 13:24:02 -07:00
Tim Wojtulewicz
b5841d0ab8 GH-1692: Add some safety to ASN's binary_to_int64 to avoid bad shifts 2021-07-29 10:11:50 -07:00
Tim Wojtulewicz
bbfce7aaf7 Update broker submodule 2021-07-28 19:10:24 -07:00
zeek-bot
9e23eb4309 Update doc submodule [nomail] [skip ci] 2021-07-28 00:31:19 +00:00
Tim Wojtulewicz
56ee21b97b Merge remote-tracking branch 'origin/topic/timw/shutdown-use-after-free'
* origin/topic/timw/shutdown-use-after-free:
  Fix a use-after-free during shutdown
2021-07-27 09:36:26 -07:00
Tim Wojtulewicz
cea9194650 Fix a use-after-free during shutdown 2021-07-26 14:26:22 -07:00
Tim Wojtulewicz
6acc3418e0 Merge remote-tracking branch 'origin/topic/timw/1693-robust-iterator-invalidation'
* origin/topic/timw/1693-robust-iterator-invalidation:
  GH-1693: Fix potential crash with elements being modified during robust iteration
2021-07-26 13:03:20 -07:00
Tim Wojtulewicz
41273afad8 GH-1693: Fix potential crash with elements being modified during robust iteration 2021-07-26 11:22:38 -07:00
jerome Grandvalet
83f4903250 Fix when HTTP header are on several packet 2021-07-26 15:58:14 +02:00
Tim Wojtulewicz
2fda808302 Update HMAC key used for benchmarking service 2021-07-23 16:28:36 -07:00
Tim Wojtulewicz
802dfd80c1 Fix failing test due to patricia change 2021-07-23 12:26:09 -07:00
Tim Wojtulewicz
083e0c5bdc Rebase patricia code on upstream version 2021-07-23 09:57:58 -07:00
Tim Wojtulewicz
36972ba9e1 Merge remote-tracking branch 'origin/topic/timw/1684-run-state-time-updates'
* origin/topic/timw/1684-run-state-time-updates:
  GH-1684: Ensure that the time gets updated every pass if we're reading live traffic
2021-07-23 09:29:00 -07:00
jerome Grandvalet
8cabecec40 Fix HTTP evasion
- Happen when there is no CRLF at the end of HTTP
    - Fix by adding CRLF when packet is complete (in relation to content-length in header)
2021-07-23 09:28:29 +02:00
Tim Wojtulewicz
5fab986ccb GH-1684: Ensure that the time gets updated every pass if we're reading live traffic
This is necessary for e.g. packet sources that don't have a selectable
file descriptor. They'll always be ready on a very short timeout, but
won't necessarily have a packet to process. In these case, sometimes
the time won't get updated for a long time and timers don't function
correctly.
2021-07-22 14:08:21 -07:00
Tim Wojtulewicz
9383c926ad Merge remote-tracking branch 'origin/topic/johanna/gh-1687'
* origin/topic/johanna/gh-1687:
  Fix handling of timers when cloning TableVals
2021-07-21 11:41:34 -07:00
Johanna Amann
13cfa4c938 Fix handling of timers when cloning TableVals
When cloning TableVals, a new timer was created for the wrong object
(the existing TableVal, not the clone). This lead to the already
existing timer being no longer accessible. Which, in turn, leads to an
abandoned timer reading into no longer allocated data when the original
TableVal is deleted.

Fixes GH-1687
2021-07-21 12:52:22 +01:00
Tim Wojtulewicz
c58e4ec445 Update doc submodule [nomail] [skip ci] 2021-07-20 10:38:20 -07:00
Tim Wojtulewicz
f4016f34e8 Merge remote-tracking branch 'origin/topic/christian/btest-fodder-news'
* origin/topic/christian/btest-fodder-news:
  Cover in NEWS the inclusion of btest tooling in the installation
2021-07-20 09:58:27 -07:00
Tim Wojtulewicz
2d95c38148 Merge remote-tracking branch 'origin/topic/timw/send-email-on-docs-action-failure'
* origin/topic/timw/send-email-on-docs-action-failure:
  Fix generate-docs github action to send email when it fails
2021-07-20 09:56:17 -07:00
Tim Wojtulewicz
0209958475 Fix generate-docs github action to send email when it fails 2021-07-20 09:55:17 -07:00
Christian Kreibich
8ed5d462cc Merge branch 'topic/christian/lets-be-greedy'
* topic/christian/lets-be-greedy:
  Use Cirrus's new greedy mode for parallelizing builds and tests
2021-07-19 16:55:13 -07:00
Christian Kreibich
a6daa80730 Cover in NEWS the inclusion of btest tooling in the installation
[skip ci]
2021-07-19 16:47:21 -07:00
Christian Kreibich
959de22349 Use Cirrus's new greedy mode for parallelizing builds and tests
This oversubscribes our cores 2x, which testing shows we actually
run with at times: speedup is around a third on average for builds,
and a bit more than that for testing.

Also some light Bashification in ci/build.sh, for consistency.
2021-07-19 13:44:16 -07:00
Tim Wojtulewicz
d8b0ee0853 Merge remote-tracking branch 'origin/topic/christian/bump-highwayhash'
* origin/topic/christian/bump-highwayhash:
  Bump highwayhash to pull in FreeBSD 14 fix
2021-07-19 08:56:59 -07:00