Commit graph

269 commits

Author SHA1 Message Date
Arne Welzel
f0b9c59adb Add experimental JavaScript support when libnode is available
zeek.on('zeek_init', () => {
        console.log('Hello, Zeek!');
    });

For interaction with external systems and HTTP APIs, JavaScript and the
Node.js ecosystem beat Zeek script. Make it more easily accessible by
including ZeekJS with Zeek directly.

When a recent enough libnode version is found on the build system, ZeekJS is
added as a builtin plugin. This behavior can be disabled via
``--disable-javascript``. Linux distributions providing such a package are
Ubuntu (22.10) and Debian (testing/bookworm) as libnode-dev.
Fedora provides it as nodejs-devel.

This plugin takes over loading of .js or .cjs files. When no such files
are provided to Zeek, Node and the V8 engine are not initialized and
should not get into the way.

This should be considered experimental.
2023-04-14 11:26:41 +02:00
Dominik Charousset
9aeed5284d Redesign subdir-libs and plugin scaffolding 2023-04-13 22:23:46 +02:00
Arne Welzel
d23b1331e5 zeek-config: Move ZEEK_VERSION* out of zeek-config.h
When bumping the VERSION file, everything that includes the auto-generated
zeek/zeek-config.h needs to rebuild and ccache usage is voided due the file
changing. Split the mutable version information into a new zeek-version.h
file to avoid this.

Further, do not include zeek-version.h within Plugin.h and avoid the reference
to ZEEK_VERSION_FUNCTION unless we're building an external plugin.

Closes #2776.
2023-02-22 16:48:39 +01:00
Tim Wojtulewicz
d34e4fb29b Call python explicitly from cmake for collecting repo info on Windows 2023-02-15 10:47:43 -07:00
Arne Welzel
3284259561 Add zeek -V/--build-info
This adds a new utility called ci/collect-repo-info.py to produce a JSON
document that is then baked into the Zeek executable file. Further, when
creating a tarball via `make dist`, put a top-level repo-info.json file
in place that is picked when no .git directory exists.

Closes #1405
2023-02-13 12:23:29 +01:00
Robin Sommer
bc252c63dc
Add BIF have_spicy_analyzers().
We previously used the Spicy plugin's `Spicy::available` to test for
Spicy support. However, having Spicy support does not necessarily mean that we
have built Zeek with its in-tree Spicy analyzers: the Spicy plugin
could have been pulled in from external. The new BIF now reliably
tells us whether the Spicy analyzers are available; its result
corresponds to what `zeek-config --have-spicy-analyzers` returns as
well.

We also move the two current checks over to use this BIF.

(Note: I refrained from renaming the CMake-side `USE_SPICY_ANALYERS`
to `HAVE_SPICY_ANALYZERS`. We should do this eventually for
consistency, but I didn't want to make more changes than necessary
right now.)
2023-02-03 13:47:26 +01:00
Robin Sommer
04a1ead978
Provide infrastructure to migrate legacy analyzers to Spicy.
As initial examples, this branch ports the Syslog and Finger analyzers
over. We leave the old analyzers in place for now and activate them
iff we compile without any Spicy.

Needs `zeek-spicy-infra` branches in `spicy/`, `spicy-plugin/`,
`CMake/`, and `zeek/zeek-testing-private`.

Note that the analyzer events remain associated with the Spicy plugin
for now: that's where they will show up with `-NN`, and also inside
the Zeekygen documentation.

We switch CMake over to linking the runtime library into the plugin,
vs. at the top-level through object libraries.
2023-02-01 11:33:48 +01:00
Tim Wojtulewicz
1f0860b5e2 Make sure the DEBUG flag is defined when building in debug mode
MSVC uses _DEBUG, unlike all of the other compilers, so we aren't
properly enabling/disabling varying bits of code that are dependent
on DEBUG being defined.
2023-01-19 09:13:33 -07:00
Tim Wojtulewicz
e44eb3522e Fix zeek-path-dev.sh creation to better support Windows 2023-01-19 09:13:33 -07:00
Tim Wojtulewicz
58f4ff91d8 Fixes to support the Npcap library on Windows
- Ignore conan libpcap if PCAP_ROOT_DIR is passed
- Update the cmake submodule to pick up changes for finding the right
  paths to npcap
- Add lazy-loading of npcap so the library path gets set correctly
  at startup
2023-01-11 11:42:58 -07:00
Tim Wojtulewicz
dd6e4a65ee GH-2637: Re-remove the bro symlink wrapper 2022-12-16 08:16:11 -07:00
Tim Wojtulewicz
a0659994f0 Merge remote-tracking branch 'origin/topic/bbannier/issue-2586'
* origin/topic/bbannier/issue-2586:
  Make dependency of `zeek` on spicy-plugin's driver object file explicit.
2022-11-17 11:34:55 -07:00
Josh Soref
4b7eb50892 Spelling NEWS & CMakeLists.txt
* analyzer
* calling
* can
* compatibility
* configurable
* conjunction
* connection
* corresponding
* currently
* evaluated
* framework
* frequently
* functionality
* handshake
* information
* more
* necessary
* occurred
* ocsp
* potentially
* preexisting
* serialized
* superseded
* synchronized
* timer
* unnecessary
* workarounds

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>
2022-11-16 20:21:38 -05:00
Benjamin Bannier
b94c8bc91b Make dependency of zeek on spicy-plugin's driver object file explicit.
For generators like Makefile the implicit dependency would not have been
apparent to `make` which could have lead to build failures (depending on
the way targets were scheduled). This patch makes the dependency
explicit so it can be enforced.

Closes #2586.
2022-11-16 09:23:47 +01:00
voidbar
2e3425000f Apply suggestions from code review
Co-authored-by: Tim Wojtulewicz <timwoj@gmail.com>
2022-11-09 18:56:00 +02:00
Tomer Lev
12494aac45 Fixed initial CR comments 2022-11-09 18:54:42 +02:00
Tim Wojtulewicz
d3baa0d1f7 Temporarily move cmake directive into MSVC section 2022-11-09 18:16:13 +02:00
Tim Wojtulewicz
bf06cc9c2f Turn on unit tests by default, and ifdef out a few of them 2022-11-09 18:16:13 +02:00
Tim Wojtulewicz
77c555a3a8 Fixing some issues from rebasing 2022-11-09 18:16:13 +02:00
Tomer Lev
0c61bcbfcd Fixed extern C issue with windows headers, fixed a GEN-ZAM regex issue, fixd compilation with c-ares 2022-11-09 18:16:04 +02:00
Elad Solomon
dbaf6696ef Fixed arec package linkage 2022-11-09 18:15:34 +02:00
Elad Solomon
f39a29435c Fixed CARES and SUPERVISOR cmake scripts 2022-11-09 18:15:34 +02:00
Elad Solomon
af656a235b Fixed cmake scripts 2022-11-09 18:15:34 +02:00
Tomer Lev
b7cbd946d6 Updating submodule to support new doctest version. Compiling without allowing doctest's multithreading which causes crashes on runtime 2022-11-09 18:15:34 +02:00
Elad Solomon
1b3719cf29 Fix libpcap usage 2022-11-09 18:15:34 +02:00
Elad Solomon
f500c4faf4 Added Compilation flags for vpack 2022-11-09 18:15:34 +02:00
Tim Wojtulewicz
216c00d7e9 Use multithreaded runtime library under MSVC 2022-11-09 18:15:34 +02:00
Tim Wojtulewicz
d352ace4d2 Add conanfile for windows builds 2022-11-09 18:15:34 +02:00
Elad Solomon
3f349b8a37 Optimize initial memory consumption 2022-11-09 18:15:34 +02:00
Elad Solomon
fde3a6b73f Added kqueue as conan package 2022-11-09 18:15:33 +02:00
Elad Solomon
3a80b79497 Compile Zeek with MSVC
Allow Zeek to be embedded in another project
2022-11-09 18:15:30 +02:00
Christian Kreibich
3237ad0ba2 Merge remote-tracking branch 'origin/topic/awelzel/builtin-af-packet'
* origin/topic/awelzel/builtin-af-packet:
  Include in Jan's AF_PACKET plugin as builtin plugin
2022-10-26 16:48:37 -07:00
Tim Wojtulewicz
0ba53de80e Surface a better CMake error if the user passes an invalid sanitizer name 2022-10-17 08:47:50 -07:00
Arne Welzel
0bc7d0905e Include in Jan's AF_PACKET plugin as builtin plugin
This has come up a few times and the motivation is mainly better "first timer"
experience with Zeek. Concretely, if one wants to run a Zeek cluster with
multiple workers and reasonable load balancing on Linux, AF_PACKET is a decent
start. Without AF_PACKET support being built into Zeek, however, a new user's
next experience is that of setting up a development environment in order
to compile an external plugin (think compiler, kernel headers, zkg, ...).
Only to get what could be termed basic functionality.

This is using the ZEEK_INCLUDE_PLUGINS infrastructure. I've used the all
upper case spelling of AF_PACKET in the help output because it seems everyone
else references/writes it like that. I think we should also write it
like that in the docs.
2022-10-13 13:29:27 +02:00
Benjamin Bannier
7774a548ea Add toplevel build dir to PATH for dev helper scripts.
We already added the toplevel build dir to the paths exposed by
`build/zeek-path-dev`, but never made the matching change for
`build/zeek-path-dev.[sh,csh]`. Due to that a shell would have never
found `zeek-config` from the build env like it would have been found for
installations, and could potentially even have picked up a different
one.
2022-10-11 14:32:35 +02:00
Benjamin Bannier
ace80793a8 Remove stray / at the end of directory name.
This variable is used to construct a path where the trailing `/` lead to
full paths like `/foo/spicy-plugin//spicyz`.
2022-10-04 11:03:24 +02:00
Benjamin Bannier
30ee2d703f Remove commented out code. 2022-10-04 11:02:10 +02:00
Benjamin Bannier
13d011da66 Use correct variable to pass existing Spicy root to spicy-plugin.
spicy-plugin can find Spicy in paths given by `SPICY_ROOT_DIR` while
`./configure` instead set `SPICY_ROOT`. With this patch we now set the
correct variable.

We also adjust variations of the previous variable name with different
capitalization which caused us to not properly configure spicy-plugin
(which triggers finding Spicy in its given prefix).

Closes #2363.
2022-08-23 09:10:57 +02:00
Tim Wojtulewicz
99bc16073a Deprecate BRO_PLUGIN_API_VERSION, create zeek::PLUGIN_API_VERSION 2022-07-12 12:01:23 -07:00
Arne Welzel
b77f5fa14c Add gulrak/filesystem to auxil, expose via zeek::filesystem
This adds https://github.com/gulrak/filesystem as a submodule into auxil
as a compiler-independent std::filesystem replacement.

The ghc::filesystem namespace is exposed as zeek::filesystem in util.h.

In the build directory, we add 3rdparty/ghc as a symlink to auxil in
order to support building from the build tree.

    <build_dir>/src/3rdparty/ghc -> /path/to/zeek/src/auxil/filesystem/include/ghc

In the installation tree, the headers are installed into include/zeek/3rdparty:

    <install_dir>/include/zeek/3rdparty/ghc

Note, this differs from how we approached rapidjson which isn't included
using a zeek/3rdparty and instead requires an additional include path of
the following form for external plugins to find and use it.

    <install_dir>/include/zeek/3rdparty/rapidjson/include/

We diverge from this approach. Placing ghc directly into 3rdparty appears
nicer and avoids changing external components (DynamicPlugin.cmake / spicyc)
2022-07-04 13:32:18 +02:00
Christian Kreibich
0be74a0aa8 Provide zeek-client by default
The user so far had to configure with --enable-zeek-client to trigger
installation of the client (from auxil/zeek-client). This flips it around to
allow disabling the installation, and removes --enable-zeek-client from the
Docker build in CI, where we've already been using it to allow the cluster
testsuite to run tests with that image.
2022-06-21 18:17:51 -07:00
Christian Kreibich
9860b5686d Ensure presence of Zeek-related directories in toplevel CMakeLists.txt
This establishes several of the Zeek installation's directories in
CMakeLists.txt, and adds a new ZEEK_STATE_DIR (via `configure --state-dir`) for
arbitrary Zeek-related variable state. It defaults to ZEEK_ROOT/var/lib.

This leaves the existing, zeekctl-related ZEEK_LOCAL_STATE_DIR unchanged and
does not use it. We'll likely deprecate it when removal of zeekctl approaches.
2022-05-23 14:16:59 -07:00
Benjamin Bannier
c68cfd8d8e Always generate helper scripts for Spicy.
We previously only generated these files if configuring a build which
included Spicy. This meant that e.g., `zeek-path-dev.*` scripts would
emit errors for non-existing paths if Spicy was not included.

We now always generate the helper scripts so we can reference them
regardless of build configuration. If Spicy is not included the files
will still be sourced, but have no actual effect.
2022-05-18 09:49:09 +02:00
Benjamin Bannier
83b36dfc54 Include headers in binary dir before other headers.
If e.g., optional dependencies are installed in the same prefix as a
previous Zeek installation, the build would have previously found
`zeek-config.h` from the external prefix instead of from the current
build.

With this patch we now include the current binary dir before any
previously configured include directories (here in particular: optional
dependencies). With that the correct `zeek-config.h` is found.
2022-05-16 09:07:11 +02:00
Benjamin Bannier
95aff9a1e3 Include spicy in build. 2022-05-16 09:07:11 +02:00
Tim Wojtulewicz
c2bf602d94 Add out_ptr, use for c-ares interface calls 2022-04-22 13:10:37 -07:00
Tim Wojtulewicz
336c6ae5c2 Replace nb_dns library with C-Ares 2022-04-22 13:10:37 -07:00
Tim Wojtulewicz
c6023774a3 Enable vptr undefined behavior check 2022-04-20 17:18:57 -07:00
Tim Wojtulewicz
1941fd80f2 Disable object-size analysis if optimization set to -O0 2022-04-14 08:59:05 -07:00
Christian Kreibich
145c7063c6 Add cmake-time reporting of bifcl, binpac, and gen-zam used for build
Cross-compilation requires providing those pre-built for the build host, so it
helps to see confirmation of the three --with-... overrides.
2022-03-21 15:30:10 -07:00