Commit graph

146 commits

Author SHA1 Message Date
Elad Solomon
3f349b8a37 Optimize initial memory consumption 2022-11-09 18:15:34 +02:00
Elad Solomon
3a80b79497 Compile Zeek with MSVC
Allow Zeek to be embedded in another project
2022-11-09 18:15:30 +02:00
Arne Welzel
d34167b2c4 zeek-setup: Load builtin-plugins/__preload__.zeek before initializing bifs
Prevent errors as follows with the bro-http2 plugin.

    error in /mitrecnd_HTTP2.events.bif.zeek, line 95: identifier not defined: http2_stream_stat
    error in /mitrecnd_HTTP2.events.bif.zeek, line 363: identifier not defined: http2_settings
    internal error in /mitrecnd_HTTP2.events.bif.zeek, line 460: Failed to find type named: http2_settings_unrecognized_table
2022-11-02 12:21:44 +01:00
Christian Kreibich
48486b4156 Merge branch 'topic/christian/gh-2239-stdin-ctrl-c'
* topic/christian/gh-2239-stdin-ctrl-c:
  Stop signal-masking upon running unit tests
  Pause signal-masking during script parsing
  Add btests to verify Zeek's handling of SIGTERM and reading stdin
  Add procps/procps-ng to several CI Docker images
2022-07-13 11:57:52 -07:00
Tim Wojtulewicz
4d4c6280e9 Miscellaneous deprecations and renaming 2022-07-12 12:01:23 -07:00
Christian Kreibich
9607deeae0 Stop signal-masking upon running unit tests
It helps to be able to ctrl-c these.
2022-07-10 21:50:47 -07:00
Christian Kreibich
9138d5c64e Pause signal-masking during script parsing
Script parsing includes reading script content from stdin, which in turn
includes interactive Zeek sessions. Keeping the signals masked there broke
ctrl-c behavior.
2022-07-09 11:19:41 -07:00
Arne Welzel
d703033ae6 zeek-setup: Validate plugin debug streams during startup
Providing an unknown plugin debug stream with -B was previously silently
accepted. This caused user confusing as the behavior is "no output, but
seems to work".

Check the enabled debug streams once all plugins have been loaded and
exit early on for invalid streams.

    $ ZEEK_PLUGIN_PATH=./build zeek -B plugin-zeek-myplugin -e 'print zeek_version();'
    error in <command line>, line 3: No plugin debug stream 'plugin-zeek-myplugin' found

Closes #913.
2022-07-07 17:56:19 +02:00
Tim Wojtulewicz
7c4fd382d9 Code modernization: Convert from deprecated C standard library headers 2022-06-27 09:47:31 -07:00
Tim Wojtulewicz
b77ede4bed Merge PQ_Timer into base TimerMgr class 2022-06-14 12:59:14 -07:00
Tim Wojtulewicz
5ca0bb79c8 Fixes from review, post-rebase 2022-05-26 18:39:01 -07:00
Vern Paxson
d9479c0502 --no-usage-warnings flag to suppress analysis 2022-05-26 17:39:15 -07:00
Benjamin Bannier
e83c4c834d Add bare_mode bif. 2022-05-16 09:07:11 +02:00
Dominik Charousset
327f5e76f9 Fix formatting 2022-05-12 07:17:41 +02:00
Dominik Charousset
6b1e796df7 Fix UB during early shutdown on OpenSSL state 2022-05-12 07:15:37 +02:00
Vern Paxson
2bb4e696e1 fix for script coverage missing on-exit activity 2022-05-03 10:44:30 -07:00
Vern Paxson
fbe33d4ee9 memory management fixes for loggers 2022-05-03 10:42:25 -07:00
Dominik Charousset
6182acd198 Fix build on CentOS 7 2022-04-28 19:37:11 +02:00
Dominik Charousset
8786f5ab1d Initialize OpenSSL on startup 2022-04-28 08:35:36 +02:00
Johanna Amann
521539d4f9 Disable OpenSSL initialization starting with 1.1.0
Starting with OpenSSL 1.1.0, library initialization is no longer
required - and might even be harmful.

See https://wiki.openssl.org/index.php/Library_Initialization for
details.
2022-04-25 10:18:27 +01:00
Tim Wojtulewicz
9f197aa458 Rework DNS_Mgr API to be more consistent and to support more request types 2022-04-22 13:10:37 -07:00
Tim Wojtulewicz
336c6ae5c2 Replace nb_dns library with C-Ares 2022-04-22 13:10:37 -07:00
Christian Kreibich
9a11df808b Mask our signal handlers' triggering signals around thread creation
This avoids tsan complaints around race conditions in signal handlers running
from threads other than the main one.
2022-04-08 18:05:49 -07:00
Vern Paxson
ecd273c3ca --event-trace / -E option to generate event trace 2022-03-23 10:03:13 -07:00
Vern Paxson
39cbd7a9dc hooks to support event tracing 2022-03-23 10:03:13 -07:00
Vern Paxson
475063804e low-level naming tweaks / comments / const-ified accessor 2022-03-23 10:03:13 -07:00
Robin Sommer
5f1f005142
Merge remote-tracking branch 'origin/topic/neverlord/hide-caf'
* origin/topic/neverlord/hide-caf:
  Fix GCC builds and string output for Broker errors
  Update to latest Broker without public CAF dep
2022-02-07 16:25:22 +01:00
Vern Paxson
16e9af137f fixes for ZAM profiling, which didn't get fully integrated originally 2022-02-03 08:59:39 -08:00
Vern Paxson
d758585e42 updated Bro->Zeek in comments in the source tree 2022-01-24 14:26:20 -08:00
Dominik Charousset
56f30b500a Update to latest Broker without public CAF dep 2021-12-20 08:16:21 +01:00
Christian Kreibich
ea20b8e55e Merge branch 'topic/christian/doctest-in-packages'
* topic/christian/doctest-in-packages:
  Update cmake and auxil/zeek-aux submodules
  Remove the --enable-cpp-tests configure flag in Cirrus CI
  Add testcases for plugin with doctest usage
  Flip unit tests to being enabled by default
  Support for unit tests in plugins
  Alphabetically sort configure's enable/disable/with options
2021-12-06 17:08:27 -08:00
Christian Kreibich
f20f8ad4a8 Support for unit tests in plugins
This pushes the unit test kickoff down in the Zeek startup sequence, to give
plugins a chance to register. It also enforces deterministic mode for unit
testing, since without it some unit tests start to have nondeterministic results
at that stage.
2021-12-06 12:17:22 -08:00
Tim Wojtulewicz
acb5e5dd26 Remove separate Tag types, note breaking change in NEWS 2021-12-06 11:25:13 -07:00
Tim Wojtulewicz
92b84a00f9 Add command-line option to write unprocessed packets to a file
This commit also changes the PcapDumper to automatically flush after
every called to Dump(). This is because pcap_dump has an internal buffer
of some sort that only writes to the file after a set amount of bytes.
When using the new option on a low-traffic network, it might be a while
before you see any packets written since it has to overcome that buffer
limit first.
2021-11-12 09:30:26 -07:00
Robin Sommer
1bb21bfcb4 Merge remote-tracking branch 'origin/topic/robin/gh-1757-loadfile2'
* origin/topic/robin/gh-1757-loadfile2:
  Tweaking a couple of debug message.
  Add new hook `HookLoadFileExtended` that allows plugins to supply Zeek script code to parse.
  Move logic to execute `HookLoadFile` for signatures into rule matcher code.
2021-11-10 11:28:53 +01:00
Benjamin Bannier
72cbc7cd13 Move 3rdparty source files to 3rdparty/
This patch moves in-tree 3rdparty source files to `3rdparty/`. With that
we can remove special treatment of these files for `run-clang-format`.
2021-11-09 07:20:18 +01:00
Robin Sommer
34eaf42b92 Add new hook HookLoadFileExtended that allows plugins to supply Zeek script code to parse.
The new hooks works similar to the existing `HookLoadFile` but,
additionally, allows the plugin to return a string that contains the
code to be used for the file being loaded. If the plugin does so, the
content of any actual file on disk will be ignored (in fact, there
doesn't even need to be a file on disk in that case). This works for
both Zeek scripts and signatures.

There's a new test that covers the new functionality, testing loading
both scripts and signatures from memory. I also manually tested that the
debugger integration works, but I don't see much of a way to add a
regression test for that part.

We keep the existing hook as well for backwards compatibility. We could
decide to deprecate it, but not sure that buys us much, so left that
out.

Closes #1757.
2021-11-05 13:01:19 +01:00
Robin Sommer
1efaf8d7a4 Move logic to execute HookLoadFile for signatures into rule matcher code.
This (1) fixes an issue where signature files supplied on the command
line wouldn't pass through the hooks, and (2) prepares for allowing
hooks to supply the content of a signature file directly.
2021-11-05 12:58:38 +01:00
Tim Wojtulewicz
9af6b2f48d clang-format: Set penalty for breaking after assignment operator 2021-09-27 10:49:48 -07:00
Tim Wojtulewicz
9cb54f5d44 clang-format: Force zeek-config.h to be earlier in the config ordering 2021-09-25 11:52:55 -07:00
Tim Wojtulewicz
b2f171ec69 Reformat the world 2021-09-16 15:35:39 -07:00
Vern Paxson
16bd121ea0 for parse-only, only do script analysis if looking for usage issues 2021-09-08 10:23:38 -07:00
Tim Wojtulewicz
cea9194650 Fix a use-after-free during shutdown 2021-07-26 14:26:22 -07:00
Robin Sommer
a7343ee019 Fix registration of protocol analyzers from inside plugins.
With the recent packet manager work, it broke to register a protocol
analyzer for a specific port from inside a plugin's initialization code.
That's because that registration now depends on the packet manager being
set up, which isn't case at that time a plugin's `InitPostInit()` runs.
This fix contains two parts:

    - Initialize the packet manager before the analyzer manager, so that
      the latter's `InitPostScript()` can rely on the former being
      ready.

    - Change the analyzer manager to (only) record port registrations
      happening before it's fully initialized. Its `InitPostScript()`
      then performs the actual registrations, knowing it can use the
      packet manager now.

This comes with a `cmake/` to add a missing include directory.
2021-07-18 10:00:49 +02:00
Christian Kreibich
a3623bfb2d Establish a separate init script when using the supervisor
The supervisor does not require the full weight of scripts that
init-default.zeek brings with it. The new file, init-supervisor.zeek, contains
only what's required by the supervisor in addition to the other always-loaded
init files.
2021-07-08 13:12:53 -07:00
Tim Wojtulewicz
0b342b7bfa Merge remote-tracking branch 'origin/topic/vern/CPP-workflow2'
* origin/topic/vern/CPP-workflow2:
  low-level coding style fixes
  support for standalone compiled scripts to export globals with module qualifiers
  updates for documentation of functionality for compiling scripts to C++
  fixes for standalone C++ scripts making types & variables/functions available
  fixed bug limiting availability of load_CPP() BiF
  updates to development helper scripts to support new workflow
  simpler workflow for -O gen-C++ ; also some hooks for -O gen-standalone-C++
  ReplaceBody now deletes a body if the replacement is nil
  removal of can't-actually-be-executed code
2021-07-01 08:46:53 -07:00
Tim Wojtulewicz
3c0bb8f1dd Merge remote-tracking branch 'origin/topic/seth/small-builtin-plugin-fixes'
* origin/topic/seth/small-builtin-plugin-fixes:
  Fix tests
  Removed a non-functional builtin plugin preload loading mechanism
  Load the builtin-plugin preload files and fix the dev path.
2021-06-30 14:11:10 -07:00
Christian Kreibich
712a28a813 Remove remaining vestigial IDMEF support code 2021-06-28 15:18:16 -07:00
Seth Hall
0470b06537 Load the builtin-plugin preload files and fix the dev path. 2021-06-23 10:12:57 +00:00
Seth Hall
8222193525 Fixes for the builtin plugin functionality 2021-06-11 18:52:03 +00:00