Commit graph

15378 commits

Author SHA1 Message Date
zeek-bot
e56c7c9ac2 Update doc submodule [nomail] [skip ci] 2023-05-12 00:32:20 +00:00
Christian Kreibich
d305e09c9e Updating auxil/zeek-client submodule [nomail] 2023-05-11 13:26:31 -07:00
Johanna Amann
3d257e52e2 Merge branch 'topic/johanna/dtls13'
* topic/johanna/dtls13:
  DTLS1.3 - address review feedback
  DTLS 1.3: Update test baseline
  DTLS 1.3: finish implementation, add connection_id extension
  Add basic DTLSv1.3 support
  SSL: Fix logging of Hello Retry Requests
2023-05-11 15:04:26 +01:00
Arne Welzel
03be610684 remote_event_ts_compat: Fix NETWORK_TIMESTAMP to NetworkTimestamp 2023-05-11 14:58:37 +02:00
Arne Welzel
11776d60e0 Merge remote-tracking branch 'jgras/topic/jgras/event-ts'
* jgras/topic/jgras/event-ts:
  Add compatibility tests for timestamped events.
  Add timestamps to auto published broker events.
  Add timestamps to manually published broker events.
  Annotate scheduled events with intended timestamp.
  Add timestamp to events.

One timestamp to ts rename during the merge.
2023-05-11 14:02:08 +02:00
Jan Grashoefer
1780d4cc2f Add compatibility tests for timestamped events.
This adds compatibility tests for receiving non-timestamped events as
well as providing timestamps via broker websockets.

Co-authored-by: Arne Welzel <arne.welzel@corelight.com>
2023-05-11 12:54:11 +02:00
Jan Grashoefer
1e807a9f0a Add timestamps to auto published broker events. 2023-05-11 12:54:11 +02:00
Jan Grashoefer
c12640b048 Add timestamps to manually published broker events. 2023-05-11 12:53:52 +02:00
Jan Grashoefer
ae152f1777 Annotate scheduled events with intended timestamp. 2023-05-11 12:51:06 +02:00
Jan Grashoefer
fa84d2896a Add timestamp to events. 2023-05-11 12:51:06 +02:00
Johanna Amann
67f6abf908 DTLS1.3 - address review feedback
See GH-3036
2023-05-11 11:19:35 +01:00
zeek-bot
4d97316a6d Update doc submodule [nomail] [skip ci] 2023-05-11 00:44:56 +00:00
Arne Welzel
31d5c15573 Merge remote-tracking branch 'origin/topic/awelzel/community-id-logging-connection-state-remove'
* origin/topic/awelzel/community-id-logging-connection-state-remove:
  Bump external test suites
  policy/community-id: Do not use new_connection()
2023-05-11 00:01:58 +02:00
Arne Welzel
8c0e7cd8bc Merge remote-tracking branch 'origin/topic/neverlord/version-header'
* origin/topic/neverlord/version-header:
  testing/zeek-version-link: Assume nm is there
  Drop dependency for zeek_dynamic_plugin_base
  Fixup ifdef check in Plugin.h
  Update cmake_minimum_required() in test plugins
  testing: Add zeek-version-link tests
  Clean up ZEEK_CONFIG_SKIP_VERSION_H issues

cmake-format during merge
2023-05-10 23:33:11 +02:00
Arne Welzel
19a2ba6986 Bump external test suites 2023-05-10 19:24:01 +02:00
Arne Welzel
e82a7455d7 policy/community-id: Do not use new_connection()
Issue #3028 tracks how a flipped connections reset a connection's value
including any state set during new_connection(). For the time being,
update community-id functionality back to the original connection_state_remove()
approach to avoid missing community_ids on flipped connections.
2023-05-10 17:48:33 +02:00
Arne Welzel
ca75834c83 testing/zeek-version-link: Assume nm is there 2023-05-10 17:37:32 +02:00
Dominik Charousset
879f605407 Drop dependency for zeek_dynamic_plugin_base 2023-05-10 16:59:07 +02:00
Arne Welzel
8f45313672 Fixup ifdef check in Plugin.h 2023-05-10 15:53:13 +02:00
Arne Welzel
85934ec632 Update cmake_minimum_required() in test plugins 2023-05-10 15:53:13 +02:00
Arne Welzel
460dc16d32 testing: Add zeek-version-link tests 2023-05-10 15:53:10 +02:00
Johanna Amann
33ee2bffa9 DTLS 1.3: Update test baseline 2023-05-10 12:27:27 +00:00
Johanna Amann
527c0dc09f Merge remote-tracking branch 'origin/master' into topic/johanna/dtls13
* origin/master: (35 commits)
  Update doc submodule [nomail] [skip ci]
  Updating submodule(s) [nomail]
  zeek.bif: Add log2() and ceil()
  Use the same rules as cmake submodule to reformat Zeek
  Update cmake submodule after reformat
  Fixup Val.h/Val.cc: Actually move ValFromJSON into zeek::detail
  Implement from_json bif
  Revert "Skip version.h by default for Zeek sources"
  BTest baseline updates for -O gen-C++
  updates to C++ maintenance scripts to better handle uncompilable BTests
  added ZEEK_REPORT_UNCOMPILABLE environment variable for "-O report-uncompilable"
  Skip version.h by default for Zeek sources
  core.network_time.broker: Test reliability improvement
  cluster/supervisor: Multi-logger awareness
  Bump zeek-archiver submodule
  ci: Add public-ecr-vacuum.sh
  Update doc submodule [nomail] [skip ci]
  generate-docs: Only update submodule pointer during scheduled builds
  BTest baseline updates for ZAM
  NTP: Detect out-of-order packets
  ...
2023-05-10 13:02:08 +01:00
Johanna Amann
a8e84c6192 DTLS 1.3: finish implementation, add connection_id extension
This commit adds support for the connection_id extension, adds a trace
that uses DTLS 1.3 connection IDs, and adds parsing for the DTLS 1.3
unified header, in case connection IDs are not used.

In case connection IDs are used, parsing of the DTLS 1.3 unified header
is skipped. This is due to the fact, that the header then contains a
variable length element, with the length of the element not given in the
header. Instead, the length is given in the client/server hello message
of the opposite side of the connection (which we might have missed).

Furthermore, parsing is not of a high importance, since we are not
passing the connection ID, or any of the other parsed values of the
unified header into scriptland.
2023-05-10 11:17:24 +01:00
mAsk°
e08e4a5fee Fix memory leak caused by pattern compilation failure 2023-05-10 05:42:03 +00:00
zeek-bot
c543387ce0 Update doc submodule [nomail] [skip ci] 2023-05-10 00:37:23 +00:00
Tim Wojtulewicz
04d23a8b5b Merge remote-tracking branch 'origin/topic/awelzel/3013-btest-core-network-time-broker'
* origin/topic/awelzel/3013-btest-core-network-time-broker:
  core.network_time.broker: Test reliability improvement
2023-05-09 13:38:52 -07:00
Tim Wojtulewicz
9d46db484f Merge remote-tracking branch 'origin/topic/awelzel/3005-public-ecr-vacuum'
* origin/topic/awelzel/3005-public-ecr-vacuum:
  ci: Add public-ecr-vacuum.sh
2023-05-09 13:38:25 -07:00
Tim Wojtulewicz
51b7a81926 Merge remote-tracking branch 'origin/topic/timw/update-submodule-pointers'
* origin/topic/timw/update-submodule-pointers:
  Updating submodule(s) [nomail]
2023-05-09 12:54:36 -07:00
Tim Wojtulewicz
2a587ad9c9 Updating submodule(s) [nomail] 2023-05-09 12:52:28 -07:00
Arne Welzel
ef2b30ba34 Merge remote-tracking branch 'origin/topic/awelzel/2930-2931-add-log2-and-ceil'
* origin/topic/awelzel/2930-2931-add-log2-and-ceil:
  zeek.bif: Add log2() and ceil()
2023-05-09 19:11:35 +02:00
Arne Welzel
6cb391f9a2 zeek.bif: Add log2() and ceil()
Closes #2930, #2931.
2023-05-09 18:57:57 +02:00
Tim Wojtulewicz
c0d15d6d0e Merge remote-tracking branch 'origin/topic/timw/cmake-format'
* origin/topic/timw/cmake-format:
  Use the same rules as cmake submodule to reformat Zeek
  Update cmake submodule after reformat
2023-05-09 09:06:00 -07:00
Tim Wojtulewicz
84e3f414a7 Use the same rules as cmake submodule to reformat Zeek 2023-05-09 08:31:43 -07:00
Tim Wojtulewicz
fd72d81bad Update cmake submodule after reformat 2023-05-09 08:31:43 -07:00
Arne Welzel
9330a74fe1 Merge remote-tracking branch 'origin/topic/awelzel/zeek-archiver-multiple-loggers'
* origin/topic/awelzel/zeek-archiver-multiple-loggers:
  cluster/supervisor: Multi-logger awareness
  Bump zeek-archiver submodule
2023-05-09 15:20:53 +02:00
Arne Welzel
1facc34e09 Fixup Val.h/Val.cc: Actually move ValFromJSON into zeek::detail
Lost during merge..
2023-05-09 11:23:32 +02:00
Arne Welzel
264284150b Merge remote-tracking branch 'amazing-pp/topic/fupeng/from_json_bif'
* amazing-pp/topic/fupeng/from_json_bif:
  Implement from_json bif

Minor updates during merge: Moved ValFromJSON into zeek::detail for the
time being, removed gotos, normalized some error messages to lower case,
minimal test extension and added a raw reader input framework test reading
"json lines" as a demo, adding notes about the implicit type
conversions.
2023-05-09 10:36:58 +02:00
Dominik Charousset
986834dce7 Clean up ZEEK_CONFIG_SKIP_VERSION_H issues 2023-05-08 18:05:56 +02:00
Fupeng Zhao
584e68434d Implement from_json bif 2023-05-06 00:42:46 +00:00
Tim Wojtulewicz
49b01c3867 Merge remote-tracking branch 'origin/topic/vern/script-opt-maint.May23'
* origin/topic/vern/script-opt-maint.May23:
  BTest baseline updates for -O gen-C++
  updates to C++ maintenance scripts to better handle uncompilable BTests
  added ZEEK_REPORT_UNCOMPILABLE environment variable for "-O report-uncompilable"
  BTest baseline updates for ZAM
2023-05-05 14:08:28 -07:00
Arne Welzel
02b7439d44 Revert "Skip version.h by default for Zeek sources"
This reverts commit 8246baf25c.

Actually fails the include_plugins CI test and I had just
opened another related issue.
2023-05-05 21:05:17 +02:00
Vern Paxson
832f48382c BTest baseline updates for -O gen-C++ 2023-05-05 12:03:00 -07:00
Vern Paxson
7d7a293c04 updates to C++ maintenance scripts to better handle uncompilable BTests 2023-05-05 12:01:31 -07:00
Vern Paxson
c173cfdbe7 added ZEEK_REPORT_UNCOMPILABLE environment variable for "-O report-uncompilable" 2023-05-05 12:00:55 -07:00
Arne Welzel
ab4f231232 Merge remote-tracking branch 'origin/topic/neverlord/gh-3019'
* origin/topic/neverlord/gh-3019:
  Skip version.h by default for Zeek sources
2023-05-05 20:15:12 +02:00
Tim Wojtulewicz
152f7aae40 Merge remote-tracking branch 'origin/topic/timw/docs-action-only-for-master'
* origin/topic/timw/docs-action-only-for-master:
  generate-docs: Only update submodule pointer during scheduled builds
2023-05-05 08:48:28 -07:00
Dominik Charousset
8246baf25c Skip version.h by default for Zeek sources 2023-05-05 17:12:10 +02:00
Arne Welzel
979f2df57d core.network_time.broker: Test reliability improvement
I wasn't able to reproduce this locally, but after looking at
-B main-loop,tm for a bit it dawned that if the manager is sending
ticks too fast, the Broker IO source may consume two ticks in one go
before expiring timers and that would explain the observed baseline
differences.

Solve this by removing the reliance on realtime delays and switch to
a request-reply pattern instead.

Also fix indentation and bogus messages.

Fixes #3013
2023-05-05 14:08:29 +02:00
Arne Welzel
c813872915 cluster/supervisor: Multi-logger awareness
When multiple loggers are configured in a Supervisor controlled cluster
configuration, encode extra information into the rotated filename to
identify which logger produced the log.

This is similar to the approach taken for ZeekControl, re-using the
log_suffix terminology, but as there's only a single zeek-archiver
process and no postprocessors and no other side-channel for additional
information, we encode extra metadata into the filename. zeek-archiver
is extended to recognize the special metadata part of the filename.

This also solves the issue that multiple loggers in a supervisor setup
overwrite each others log files within a single log-queue directory.
2023-05-05 12:27:25 +02:00