Commit graph

15389 commits

Author SHA1 Message Date
Arne Welzel
16bdcd27bd Merge remote-tracking branch 'origin/topic/awelzel/next-break-no-error-but-warning'
* origin/topic/awelzel/next-break-no-error-but-warning:
  ScriptValidation: Make break/next a warning until Zeek 6.1
2023-03-10 09:05:27 +01:00
Johanna Amann
b56b856da9 SSL/TLS: Parse CertificateRequest message
This commit introduces parsing of the CertificateRequest message in the
TLS handshake. It introduces a new event ssl_certificate_request, as
well as a new function parse_distinguished_name, which can be used to
parse part of the ssl_certificate_request event parameters.

This commit also introduces a new policy script, which appends
information about the CAs a TLS server requests in the
CertificateRequest message, if it sends it.
2023-03-09 09:12:29 +01:00
Arne Welzel
1912ba7002 cirrus: Add smoke testing for builtin plugins
This adds two example plugins within testing/builtin-plugins/Files:

* protocol-plugin copied over from testing/btest/plugins/protocol-plugin

* py-lib-plugin that embeds Python to have a dependency on an external
  shared library which was already available in CI and fun to use, too.

Closes #2837
2023-03-08 22:32:29 +01:00
Arne Welzel
42c1fc3e7d ci/collect-repo-info: Make plugin VERSION reading more robust
The zeek-kafak plugin's VERSION file contains a commented license
header followed by an empty line, followed by the actual version.

Attempt to deal with this by using the first non-commented non-empty
line.
2023-03-08 20:57:26 +01:00
Arne Welzel
bfd182f583 configure: Quote --include-plugins argument
This is allowed and documented to use `;` for separation. The newer
eval functionality then sees that as a command terminator.
2023-03-08 20:57:26 +01:00
Arne Welzel
1146c62728 Fix --no-install-recommends typo 2023-03-08 20:57:26 +01:00
Arne Welzel
b73dda5cff Merge remote-tracking branch 'origin/topic/awelzel/bump-spicy-latest-main'
* origin/topic/awelzel/bump-spicy-latest-main:
  Bump Spicy to latest main
2023-03-08 13:21:53 +01:00
Arne Welzel
b58a2d3822 Revert "Merge branch 'topic/jgras/iosource-offline-fd' of https://github.com/J-Gras/zeek"
This reverts commit 957825441a, reversing
changes made to c8cdc75f2b.

Caused spurious CI failures in the external testing baselines. See zeek/zeek#2842.
2023-03-08 12:32:14 +01:00
Arne Welzel
957825441a Merge branch 'topic/jgras/iosource-offline-fd' of https://github.com/J-Gras/zeek
* 'topic/jgras/iosource-offline-fd' of https://github.com/J-Gras/zeek:
  Allow offline packet sources to register FDs.
2023-03-08 11:05:32 +01:00
Arne Welzel
c8cdc75f2b Merge remote-tracking branch 'origin/topic/vern/Mar23-script-opt-maint'
* origin/topic/vern/Mar23-script-opt-maint:
  extend BTest "path" canonicalization to include compiled-to-C++ variable names
  use a subclass of TableType for incremental construction of compiled scripts
  script_opt/CPP: errors, recursive type fixes, fix embedded comments
  Fix for EnumVal's returning their underlying value
2023-03-08 10:43:28 +01:00
Arne Welzel
ec1933ddaa Bump Spicy to latest main
To fix compile warnings.
2023-03-08 10:25:37 +01:00
Vern Paxson
cdcd3f79aa extend BTest "path" canonicalization to include compiled-to-C++ variable names 2023-03-08 10:19:21 +01:00
Vern Paxson
a934acc4f1 use a subclass of TableType for incremental construction of compiled scripts 2023-03-08 10:19:21 +01:00
Vern Paxson
c0dd2b4e81 script_opt/CPP: errors, recursive type fixes, fix embedded comments
better (than nothing) run-time errors for compiled scripts
fixes for dealing with recursive types in compiled scripts
fix for values in compiled scripts containing embedded comment markers
2023-03-08 10:19:17 +01:00
Vern Paxson
b7f7d32bf7 Fix for EnumVal's returning their underlying value
Change EnumVal()->AsEnum() to zeek_int_t.
2023-03-08 10:10:24 +01:00
Arne Welzel
c3ef42a2e3 Merge remote-tracking branch 'origin/topic/bbannier/ccache-manual-pruning'
* origin/topic/bbannier/ccache-manual-pruning:
  Manually prune CI ccache.
2023-03-08 09:40:49 +01:00
Tim Wojtulewicz
e9ff389912 Merge remote-tracking branch 'origin/topic/timw/2846-dns-mgr-crash'
* origin/topic/timw/2846-dns-mgr-crash:
  Add trigger_mgr to iosource_mgr later during startup
2023-03-07 09:28:47 -07:00
Benjamin Bannier
b3cd1c5215 Manually prune CI ccache.
Since ccache in Cirrus currently does not preserve timestamps the
timestamp-based cache pruning implemented by ccache likely would just
remove random files. To work around this, this patch implements a manual
pruning step. This step runs after the build so that at least the files
used in the build should have their timestamps updated. We can then
force eviction of some of the unused files from the cache by cleaning
the cache with a size less than the maximum cache size.
2023-03-07 12:02:57 +01:00
Tim Wojtulewicz
b26f4a83b3 Add trigger_mgr to iosource_mgr later during startup
This fixes a potential crash due to trigger_mgr getting shutdown earlier
than dns_mgr, and dns_mgr then trying to use it after it's been deleted.
This change forces the order of initialization/destruction in
iosource_mgr to cause dns_mgr to be deleted first.
2023-03-06 13:12:45 -07:00
Arne Welzel
117d25dcd8 Update cmake submodule [nomail] 2023-03-06 20:18:37 +01:00
Jan Grashoefer
99f16282c0 Allow offline packet sources to register FDs. 2023-03-05 13:02:00 +01:00
zeek-bot
8cf9e5b374 Update doc submodule [nomail] [skip ci] 2023-03-03 02:01:24 +00:00
Tim Wojtulewicz
2735109ad7 Update external testing commit hashes for telemetry changes 2023-03-02 13:41:09 -07:00
Tim Wojtulewicz
e47980143c Merge remote-tracking branch 'origin/topic/timw/default-known-stores-to-false'
* origin/topic/timw/default-known-stores-to-false:
  Default known stores to false to avoid Broker stores
2023-03-02 12:49:22 -07:00
Tim Wojtulewicz
f6d3ed2395 Default known stores to false to avoid Broker stores 2023-03-02 12:48:53 -07:00
Arne Welzel
56c76e5949 ScriptValidation: Make break/next a warning until Zeek 6.1
The ja3 package uses next instead of return and triggers the new
errors with Zeek 5.2. That seems somewhat bad.

In case we want to do a 5.2.1 that makes this a warning, this
would be change to do so.
2023-03-02 18:46:10 +01:00
Arne Welzel
802d24cad7 Merge remote-tracking branch 'origin/topic/awelzel/try-log-stream-metrics'
* origin/topic/awelzel/try-log-stream-metrics:
  Bump external test suites, filter some telemetry.log entries
  logging/Manager: Remove unused variable
  logging: Add telemetry for streams and log writers
2023-03-02 10:45:54 +01:00
zeek-bot
97f675d35b Update doc submodule [nomail] [skip ci] 2023-03-02 00:19:00 +00:00
Johanna Amann
3be5e4de46 Merge remote-tracking branch 'origin/topic/johanna/tsb-sig-alg'
* origin/topic/johanna/tsb-sig-alg:
  X.509: expose the signature type inside the tbs certificate
2023-03-01 10:42:15 +00:00
Arne Welzel
fd0d2e2657 Merge remote-tracking branch 'origin/topic/awelzel/zeekctl-ci'
* origin/topic/awelzel/zeekctl-ci:
  Bump zeekctl submodule
  cirrus: Add zeekctl CI task
  cirrus: Do not run tasks for builds on cron triggered jobs
  cirrus: Rename BRANCH_WHITELIST
2023-03-01 10:21:33 +01:00
Arne Welzel
209c226e48 Bump zeekctl submodule 2023-03-01 10:19:55 +01:00
Johanna Amann
989e9c29d2 X.509: expose the signature type inside the tbs certificate
This change exposes the signature tyope inside the signed portion of an
X.509 certificate. In the past, we only exposed the signature type that
is contained inside the signature, which is outside the signed portion
of the X.509 certificate.

In theory, both signature fields should have the same value; it is,
however, possible to encode differing values in both fields. The new
field is not logged by default.
2023-02-28 19:24:16 +00:00
Arne Welzel
e03ea4c211 cirrus: Add zeekctl CI task
Run the zeekctl testing suite within a Debian 11 container. This runs on
pushes to master and release branches. Also, tagging PRs with fullci or
zeekctlci triggers it.

Closes zeek/zeekctl#49
2023-02-28 18:33:49 +01:00
Arne Welzel
c9bc187791 cirrus: Do not run tasks for builds on cron triggered jobs
...at least not right now. The outcome should not be different compared
to running for master/release upon merge, anyway.
2023-02-28 18:33:49 +01:00
Arne Welzel
4ffcbc7710 cirrus: Rename BRANCH_WHITELIST 2023-02-28 18:33:49 +01:00
Tim Wojtulewicz
3febe56dd1 Merge remote-tracking branch 'origin/topic/timw/badges'
* origin/topic/timw/badges:
  Add build status, slack, and discourse badges to readme
2023-02-28 09:24:43 -07:00
Tim Wojtulewicz
144c40e835 Add build status, slack, and discourse badges to readme 2023-02-27 15:25:36 -07:00
Tim Wojtulewicz
097771229c Merge remote-tracking branch 'origin/topic/timw/debian-mirror'
* origin/topic/timw/debian-mirror:
  Configure apt to retry a few times on Docker image builder tasks
2023-02-27 14:48:27 -07:00
Tim Wojtulewicz
62049f5e89 Configure apt to retry a few times on Docker image builder tasks 2023-02-27 14:29:56 -07:00
Tim Wojtulewicz
13d5753e49 Merge remote-tracking branch 'origin/topic/awelzel/af-packet-zeekctl-5.2'
* origin/topic/awelzel/af-packet-zeekctl-5.2:
  Bump zeekctl and add NEWS entries
2023-02-27 13:47:53 -07:00
Tim Wojtulewicz
21a78816e9 Merge remote-tracking branch 'origin/topic/timw/bifcl-cleanup'
* origin/topic/timw/bifcl-cleanup:
  Update bifcl submodule [nomail]
2023-02-27 09:36:40 -07:00
Tim Wojtulewicz
6bb93a5ef8 Update bifcl submodule [nomail] 2023-02-27 09:34:03 -07:00
Arne Welzel
d32ef970a9 Bump zeekctl and add NEWS entries
These are added to the 5.2 section of NEWS. This is for backporting
into release/5.2.
2023-02-27 16:26:30 +01:00
Arne Welzel
bfc2374c8f Bump external test suites, filter some telemetry.log entries 2023-02-27 15:42:18 +01:00
Arne Welzel
545b867ddd logging/Manager: Remove unused variable 2023-02-27 12:51:03 +01:00
Arne Welzel
69a98e2cbb logging: Add telemetry for streams and log writers
This adds one metric per log stream and one metric per log writer (path based)
to track the number of writes on a stream level as well as on a writer level.

    $ curl -sSf localhost:8181/metrics | grep Conn
    zeek_log_writer_writes_total{endpoint="",filter-name="default",module="HTTP",path="http",stream="HTTP::LOG",writer="Log::WRITER_SQLITE"} 1 1677497572770
    zeek_log_stream_writes_total{endpoint="",module="HTTP",stream="HTTP::LOG"} 1 1677497572770

The initial version of this change also included metrics around log
write vetoes, but given no log policies exist in the default configuration
and they are mostly interesting for a few streams/writers only, skip this
for now. These can always be added by the script writer, too.

The difference between the stream level writes and concrete writers can
be used to deduce the number of vetoes (or errors) as a starting point.
2023-02-27 12:51:03 +01:00
Robin Sommer
a41e282a7d
Merge remote-tracking branch 'origin/topic/robin/gh-2801-removed-child-analyzers'
* origin/topic/robin/gh-2801-removed-child-analyzers:
  Fix re-instantiation of analyzers that were just removed.
2023-02-27 12:36:30 +01:00
Arne Welzel
2b4aa38315 Merge remote-tracking branch 'origin/topic/awelzel/bump-cmake-plugin-dereference-symlinks'
* origin/topic/awelzel/bump-cmake-plugin-dereference-symlinks:
  tests: Add regression tests for zeek/cmake#61
  tests/bifs-and-scripts-install: Remove unused activate.zeek
  Update cmake submodule
2023-02-27 09:53:28 +01:00
Arne Welzel
15be6b5011 Merge remote-tracking branch 'origin/topic/awelzel/cap-ccache-sizes'
* origin/topic/awelzel/cap-ccache-sizes:
  cirrus: Also add ccache limits to container builds
  cirrus: Add an epoch to the ccache fingerprint, bump to 1000M
  cirrus: Cap ccache max size to 500M and max files to 20000
2023-02-26 17:29:29 +01:00
Arne Welzel
3be4712163 tests: Add regression tests for zeek/cmake#61 2023-02-24 19:37:52 +01:00