Commit graph

13346 commits

Author SHA1 Message Date
Christian Kreibich
d40bb6e85f Management framework: simplify agent and controller stdout/stderr files
Moving to a model in which every Zeek process runs out of its own working
directory simplifies the handling of those files.
2022-05-25 13:56:23 -07:00
Christian Kreibich
f8f7fd97e8 Management framework: prefix the management logs with "management-"
These were still using "cluster-", a leftover from earlier days of the
framework.
2022-05-25 13:56:23 -07:00
Christian Kreibich
bd6c1683a2 Management framework: comment and layouting tweaks, no functional change
Also remove additional instances of the term "data cluster".
2022-05-25 13:56:23 -07:00
Christian Kreibich
d4d6f10299 Management framework: rename env var that labels agents/controllers
Just a consistency tweak to avoid confusion with "cluster".
2022-05-25 13:56:23 -07:00
Christian Kreibich
d2903bb645 Management framework: increase robustness of agent/controller naming
The fallback mechanism when no explicit agent/controller names are configured
didn't work properly, because many places in the code relied on accessing the
name via the variables meant for explicit configuration, such as
Management::Agent::name. Agent and controller now offer functions for computing
the correct effective name, and we use that throughout.
2022-05-25 13:56:23 -07:00
Tim Wojtulewicz
2f453cd0cd Merge remote-tracking branch 'origin/topic/timw/1995-pcap-filter-error-messages'
* origin/topic/timw/1995-pcap-filter-error-messages:
  Propagate BPF_Program error message to script land
  Allow pcap pktsrc to use other BPF_Program::Compile method
2022-05-25 12:09:52 -07:00
Tim Wojtulewicz
f8bc23d3e1 Propagate BPF_Program error message to script land 2022-05-25 09:41:35 -07:00
Tim Wojtulewicz
b30d5702f6 Allow pcap pktsrc to use other BPF_Program::Compile method 2022-05-25 09:41:16 -07:00
Tim Wojtulewicz
515e5bf8a3 Merge remote-tracking branch 'origin/topic/bbannier/spicy-bump'
* origin/topic/bbannier/spicy-bump:
  Bump Spicy and spicy-plugin.
  Clean up test `spicy.spicyz-jit`.
2022-05-25 09:40:26 -07:00
Tim Wojtulewicz
7c56b6923b Merge remote-tracking branch 'origin/topic/timw/2125-scriptprofile-warnings'
* origin/topic/timw/2125-scriptprofile-warnings:
  GH-2125: Fix compiler warnings in ScriptProfile
2022-05-25 09:38:30 -07:00
Tim Wojtulewicz
97e8f9d268 Merge remote-tracking branch 'origin/topic/timw/dict-debug-fixes'
* origin/topic/timw/dict-debug-fixes:
  Use ZEEK_DICT_DEBUG more in Dict.cc instead of DEBUG
2022-05-25 09:38:06 -07:00
Tim Wojtulewicz
0b1c8575cc Merge remote-tracking branch 'origin/topic/timw/avoid-needing-flex-arguments'
* origin/topic/timw/avoid-needing-flex-arguments:
  Move fuzzer setup in CMake below builtin plugin setup
  Remove flex/bison flags from CI build
  Update cmake submodule [nomail]
2022-05-25 09:37:31 -07:00
Benjamin Bannier
8672c953e1 Bump Spicy and spicy-plugin. 2022-05-25 15:01:46 +02:00
Benjamin Bannier
3b7adde599 Clean up test spicy.spicyz-jit.
This test was created from a copy of `spicy.spicyz-aot`, but updated
incompletly. This patch removes the last ahead-of-time compilation bits.
2022-05-25 15:01:46 +02:00
zeek-bot
269d3be974 Update doc submodule [nomail] [skip ci] 2022-05-25 00:39:30 +00:00
Christian Kreibich
d4ecfa0a67 Merge branch 'topic/christian/installation-dirs-in-scriptland'
* topic/christian/installation-dirs-in-scriptland:
  Add scripts.base.misc.installation btest
  Add base/misc/installation.zeek, with Zeek installation directories
  Ensure presence of Zeek-related directories in toplevel CMakeLists.txt
2022-05-24 12:12:05 -07:00
Tim Wojtulewicz
772ea5b71c GH-2125: Fix compiler warnings in ScriptProfile 2022-05-24 11:42:18 -07:00
Tim Wojtulewicz
a61bc16b10 Merge remote-tracking branch 'origin/topic/timw/dockerfile-versions'
* origin/topic/timw/dockerfile-versions:
  Add DOCKERFILE_VERSION variable to all CI dockerfiles
2022-05-23 17:47:20 -07:00
Tim Wojtulewicz
69e6554f10 Update broker submodule [nomail] 2022-05-23 14:20:01 -07:00
Christian Kreibich
04c91736a1 Add scripts.base.misc.installation btest 2022-05-23 14:16:59 -07:00
Christian Kreibich
84a09debe3 Add base/misc/installation.zeek, with Zeek installation directories
This makes several of the installation's main directories available to the
script layer.
2022-05-23 14:16:59 -07:00
Christian Kreibich
9860b5686d Ensure presence of Zeek-related directories in toplevel CMakeLists.txt
This establishes several of the Zeek installation's directories in
CMakeLists.txt, and adds a new ZEEK_STATE_DIR (via `configure --state-dir`) for
arbitrary Zeek-related variable state. It defaults to ZEEK_ROOT/var/lib.

This leaves the existing, zeekctl-related ZEEK_LOCAL_STATE_DIR unchanged and
does not use it. We'll likely deprecate it when removal of zeekctl approaches.
2022-05-23 14:16:59 -07:00
Tim Wojtulewicz
3397beabf7 Use ZEEK_DICT_DEBUG more in Dict.cc instead of DEBUG 2022-05-23 14:12:39 -07:00
Tim Wojtulewicz
d8993aac0e Move fuzzer setup in CMake below builtin plugin setup 2022-05-23 12:50:03 -07:00
Tim Wojtulewicz
6b4c565151 Remove flex/bison flags from CI build 2022-05-23 12:06:04 -07:00
Tim Wojtulewicz
6f2640a4da Update cmake submodule [nomail] 2022-05-23 11:55:01 -07:00
Tim Wojtulewicz
4f11a57159 Add DOCKERFILE_VERSION variable to all CI dockerfiles 2022-05-23 08:47:11 -07:00
zeek-bot
eeebf8a429 Update doc submodule [nomail] [skip ci] 2022-05-20 00:37:20 +00:00
Tim Wojtulewicz
73273fc87b Merge remote-tracking branch 'origin/topic/vern/ZAM-inlining'
* origin/topic/vern/ZAM-inlining:
  speed up ZAM compilation by capping function size when inlining
2022-05-19 11:45:50 -07:00
Christian Kreibich
7b43153785 Merge branch 'topic/christian/build-path-tweak'
* topic/christian/build-path-tweak:
  Zeekify the scripts.base.utils.paths test
  Expand build_path() function to handle empty dir arguments gracefully
2022-05-19 11:25:08 -07:00
Tim Wojtulewicz
48c978ae17 Merge remote-tracking branch 'origin/topic/timw/cirrus-ccache'
* origin/topic/timw/cirrus-ccache:
  Rework .cirrus.yml a bit to commonize some parts
  Enable 'greedy' mode for other CI builds
  Enable ccache for Cirrus builds
  Fix SSL tests on Centos Stream 9
2022-05-19 10:52:11 -07:00
Christian Kreibich
0641d032c3 Zeekify the scripts.base.utils.paths test 2022-05-19 09:51:36 -07:00
Christian Kreibich
9d59a48ae2 Expand build_path() function to handle empty dir arguments gracefully
When passing an empty string as a directory, the function would produce
filenames starting with a slash even when the given file_name is not an absolute
path. Defaulting to the root directory is likely never intended and might
conveivably be dangerous. The middle "/" is now skipped also if dir is an empty
string.
2022-05-19 09:45:52 -07:00
Tim Wojtulewicz
b8f745ca84 Rework .cirrus.yml a bit to commonize some parts 2022-05-19 09:21:55 -07:00
Tim Wojtulewicz
7990986c01 Enable 'greedy' mode for other CI builds 2022-05-19 09:21:55 -07:00
Tim Wojtulewicz
d48a1b520f Enable ccache for Cirrus builds 2022-05-19 09:21:55 -07:00
Johanna Amann
4d1b9f4a08 Merge remote-tracking branch 'origin/topic/johanna/centos-9-stream-openssl'
* origin/topic/johanna/centos-9-stream-openssl:
  Fix SSL tests on Centos Stream 9
2022-05-19 17:14:04 +01:00
Tim Wojtulewicz
ee85595e48 Merge remote-tracking branch 'origin/topic/timw/disable-spicy-for-coverity-builds'
* origin/topic/timw/disable-spicy-for-coverity-builds:
  Disable Spicy in Zeek Coverity builds
2022-05-19 09:13:25 -07:00
Johanna Amann
4d66e4e16c Fix SSL tests on Centos Stream 9
SHA1 signatures are only allowed in the legacy policy. I have no idea
where this is documented - I found it in a patch for the OpenSSL source
package.

Also adding tar as a specific dependency, since we have a test that
requires it, and at least for me it was not present in a minimal
installation.

Fixes #2115
2022-05-19 15:00:12 +01:00
Tim Wojtulewicz
1223386fd3 Merge remote-tracking branch 'origin/topic/bbannier/spicy-helper-scripts'
* origin/topic/bbannier/spicy-helper-scripts:
  Always generate helper scripts for Spicy.
2022-05-18 11:12:08 -07:00
Tim Wojtulewicz
840a57b458 Merge remote-tracking branch 'origin/topic/johanna/openssl-3-fips'
* origin/topic/johanna/openssl-3-fips:
  Partially revert OpenSSL hashing changes
2022-05-18 11:07:40 -07:00
Tim Wojtulewicz
327eca32fd Merge remote-tracking branch 'origin/topic/timw/1672-table-expiration-deletion'
* origin/topic/timw/1672-table-expiration-deletion:
  GH-1672: Keep a reference to a table being expired, just in case
2022-05-18 11:07:13 -07:00
Tim Wojtulewicz
2416f832a2 Disable Spicy in Zeek Coverity builds 2022-05-18 11:06:27 -07:00
Benjamin Bannier
c68cfd8d8e Always generate helper scripts for Spicy.
We previously only generated these files if configuring a build which
included Spicy. This meant that e.g., `zeek-path-dev.*` scripts would
emit errors for non-existing paths if Spicy was not included.

We now always generate the helper scripts so we can reference them
regardless of build configuration. If Spicy is not included the files
will still be sourced, but have no actual effect.
2022-05-18 09:49:09 +02:00
zeek-bot
da9148f535 Update doc submodule [nomail] [skip ci] 2022-05-18 00:36:09 +00:00
Tim Wojtulewicz
92a7773dd5 Merge remote-tracking branch 'origin/topic/timw/unneeded-fmt'
* origin/topic/timw/unneeded-fmt:
  Remove unnecessary uses of util::fmt()
2022-05-17 09:15:37 -07:00
Tim Wojtulewicz
aaaca5d5a6 Merge remote-tracking branch 'origin/topic/bbannier/spicy'
* origin/topic/bbannier/spicy:
  Add NEWS item for new requirements due to default-enabled Spicy support.
  Include headers in binary dir before other headers.
  Increase memory in CI.
  Add tests for bundled Spicy infrastructure.
  Include spicy in build.
  Add `bare_mode` bif.
  Change test so included plugins can load notice framework.
  Do not log function arguments in test.
2022-05-17 08:50:33 -07:00
Johanna Amann
30167d659a Partially revert OpenSSL hashing changes
This commit partially reverts the changes that we made in
6217851d6d. It turns out that reverting to
the legacy OpenSSL API means that Zeek can no longer be run in FIPS mode
for old (but still used) versions of OpenSSL.

This commit thus uses a bunch of #ifdefs to support both implementations
simultaneously.
2022-05-17 08:52:06 +00:00
Tim Wojtulewicz
3233400930 Merge remote-tracking branch 'origin/topic/timw/1857-input-log-messages'
* origin/topic/timw/1857-input-log-messages:
  Move some repetitive code into a separate method
  Store location during Ascii input parsing for use in error messages
  Include file information in input reader error messages
2022-05-16 17:00:34 -07:00
Tim Wojtulewicz
3d133ccb0a Merge remote-tracking branch 'origin/topic/vern/CPP-feature-complete'
* origin/topic/vern/CPP-feature-complete:
  NEWS items for script profiling, "footprint" BiFs, and Trigger method deprecations
  tweaks per reviewing feedback
  make deprecation advice a bit more helpful
  format shell scripts per style guide
  add deprecation for Trigger Cache/Lookup interfaces
  helper scripts for -O C++ maintenance
  "-O use-C++" now fails if no C++ bodies wound up being applicable
  update environemnt variable names to match command-line arguments
  final changes for -O C++ feature completeness, mainly "when" statement support
  Change "when" statements that don't require closures to use new implementation. Provide hooks for script optimization access to "when" statements. Regularize treatment of naming and timeouts for Triggers.
  switch cached Trigger values to be opaque-and-generic, rather than assuming CallExpr's
  tracking of when statements/expressions occur in a "when" context
  fix bug that sometimes associated the wrong location with global statements
  fix incomplete AST traversal of "when" statements
  add some missing "when" closures
  regularize test suite names to avoid custom/outdated suffices
2022-05-16 16:59:59 -07:00