Commit graph

153 commits

Author SHA1 Message Date
Arne Welzel
f0b9c59adb Add experimental JavaScript support when libnode is available
zeek.on('zeek_init', () => {
        console.log('Hello, Zeek!');
    });

For interaction with external systems and HTTP APIs, JavaScript and the
Node.js ecosystem beat Zeek script. Make it more easily accessible by
including ZeekJS with Zeek directly.

When a recent enough libnode version is found on the build system, ZeekJS is
added as a builtin plugin. This behavior can be disabled via
``--disable-javascript``. Linux distributions providing such a package are
Ubuntu (22.10) and Debian (testing/bookworm) as libnode-dev.
Fedora provides it as nodejs-devel.

This plugin takes over loading of .js or .cjs files. When no such files
are provided to Zeek, Node and the V8 engine are not initialized and
should not get into the way.

This should be considered experimental.
2023-04-14 11:26:41 +02:00
Vern Paxson
cdcd3f79aa extend BTest "path" canonicalization to include compiled-to-C++ variable names 2023-03-08 10:19:21 +01:00
Robin Sommer
04a1ead978
Provide infrastructure to migrate legacy analyzers to Spicy.
As initial examples, this branch ports the Syslog and Finger analyzers
over. We leave the old analyzers in place for now and activate them
iff we compile without any Spicy.

Needs `zeek-spicy-infra` branches in `spicy/`, `spicy-plugin/`,
`CMake/`, and `zeek/zeek-testing-private`.

Note that the analyzer events remain associated with the Spicy plugin
for now: that's where they will show up with `-NN`, and also inside
the Zeekygen documentation.

We switch CMake over to linking the runtime library into the plugin,
vs. at the top-level through object libraries.
2023-02-01 11:33:48 +01:00
Tim Wojtulewicz
d6ce5894a7 Fix diff-remove-abspath to ignore Windows drive letters 2023-01-19 09:13:33 -07:00
Josh Soref
186fe346ad spelling: repetitions
Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>
2022-11-18 10:26:33 -05:00
Arne Welzel
0bc7d0905e Include in Jan's AF_PACKET plugin as builtin plugin
This has come up a few times and the motivation is mainly better "first timer"
experience with Zeek. Concretely, if one wants to run a Zeek cluster with
multiple workers and reasonable load balancing on Linux, AF_PACKET is a decent
start. Without AF_PACKET support being built into Zeek, however, a new user's
next experience is that of setting up a development environment in order
to compile an external plugin (think compiler, kernel headers, zkg, ...).
Only to get what could be termed basic functionality.

This is using the ZEEK_INCLUDE_PLUGINS infrastructure. I've used the all
upper case spelling of AF_PACKET in the help output because it seems everyone
else references/writes it like that. I think we should also write it
like that in the docs.
2022-10-13 13:29:27 +02:00
Tim Wojtulewicz
f624c18383 Deprecate bro_int_t and bro_uint_t 2022-07-12 12:01:23 -07:00
Vern Paxson
a9f67f52a5 baseline & btest updates 2022-05-26 17:39:47 -07:00
Benjamin Bannier
e40aa0f6d2 Add tests for bundled Spicy infrastructure. 2022-05-16 09:07:11 +02:00
Dominik Charousset
a8196cf4fb Fix formatting 2022-04-27 23:02:27 +02:00
Dominik Charousset
02608df99c Canonify intel.log in read-file-dist-cluster test 2022-04-27 23:02:27 +02:00
Christian Kreibich
78803b04fc Add testcases for plugin with doctest usage 2021-12-06 12:17:22 -08:00
Benjamin Bannier
fd56eddcfb Format Python scripts with yapf.
We also add a very basic yapf configuration file. Most of the changes in
this patch were performed automatically, but we broke one overly long
string into multiple components on `src/make_dbg_constants.py`.
2021-11-24 23:13:24 +01:00
Benjamin Bannier
1f388e3f40 Format shell scripts with shfmt.
All changes in this patch were performed automatically with `shfmt` with
configuration flags specified in `.pre-commit-config.yaml`.

In addition to fixing whitespace the roundtrip through shfmt's AST also
transforms command substitutions

    `cmd`
    # becomes
    $(cmd)

and some redirects

    >&2 echo "msg"
    # becomes
    echo >&2 "msg"
2021-11-24 23:13:02 +01:00
Vlad Grigorescu
ac720a1313 Add tests for e-mail actions, and cleanup the new logic a bit. 2021-04-26 23:01:34 -05:00
Jon Siwek
8e47494dff Remove newline-eof canonification attempt in diff-remove-timestamps
On Alpine (BusyBox `sed`), the previous `sed -e '$a\'` invocation always
added a newline, breaking most every diff.  There doesn't seem to be a
need to attempt normalizing EOF newlines at the moment and doing it that
way doesn't seem to be portable anyway.  If canonifiers need to be
portable, `sed` should be treated as a text-processing tool and POSIX
definition of text-file is zero or more newline-terminated
character-sequences, so if canonification of Baselines via `sed` is
required, those Baselines should always end with a newline to be
considered text files.  I.e. that's not the job of this canonifier, and
changing it also doesn't necessarily generalize since it could be
considered coincidental that diff-remove-timestamps in particular is the
default canonifier that's commonly used while there's still others that
also make use of `sed`.
2021-03-30 16:04:34 -07:00
Johanna Amann
442cb40db8 Fix binary baseline & line-end problem
By default all baslines are run through diff-remove-timestamp. On a BSD
sed implementation, this means that a newline is added to the end of the
file, if no newline was there originally. This behavior differs from GNU
sed, which does not add a newline.

In this commit we unify this behavior by always adding a newline, even
when using GNU sed. This commit also disables the canonifier for a bunch
of binary baselines, so we do not have to change them.
2020-12-06 20:19:52 -08:00
Johanna Amann
7040e30431 Fix diff-remove-abspath on OSX
\t does not work on OS-X and just matches the letter t. This commit
replaces it with a literal tab instead.
2020-12-06 20:19:52 -08:00
Christian Kreibich
c0bd02e18f Canonifier improvements for the scripts.base.frameworks.logging.ascii-double test
This now more surgically applies canonifiers so that the double-format
numerical output isn't itself canonified.
2020-12-06 20:19:52 -08:00
Christian Kreibich
3984d99a86 Make diff-remove-abspath canonifier match on non-whitespace paths only
This avoids swallowing multiple separate paths separated by unrelated
content into one substitution, like here:

orig_p=59856<...>/tcp] -> orig_p=59856/tcp, resp_h=192.150.187.43, resp_p=80/tcp]
2020-12-06 18:19:06 -08:00
Christian Kreibich
3ca804421e Harden diff-remove-timestamps canonifier
This pattern got mislead by matching suffixes of other numbers, and
noramlizing exact 0-timestamps isn't really required.

- Remove eplicit "0.000000" number pattern from timestamp normalization

- Require beginning of line or non-numeric character before the
  beginning of the number replacement
2020-12-06 18:19:06 -08:00
Jon Siwek
7d68f2ce9d Remove Python 2 compatibility logic in httpd test script 2020-11-25 14:57:09 -08:00
Jon Siwek
9717c623c2 Update Python invocations to explicit python3 2020-11-25 14:57:09 -08:00
Jon Siwek
a7b905d389 GH-1149: Add GitHub Action to automate generation of zeek-docs 2020-09-01 17:52:00 -07:00
Jon Siwek
7967a5b0aa General btest cleanup
- Use `-b` most everywhere, it will save time.

- Start some intel tests upon the input file being fully read instead of
  at an arbitrary time.

- Improve termination condition for some sumstats/cluster tests.

- Filter uninteresting output from some supervisor tests.

- Test for `notice_policy.log` is no longer needed.
2020-08-11 11:26:22 -07:00
Jon Siwek
67835a6520 Extend external test suite canonifier with set-sorting logic
Two new canonifiers: one to sort the contents of conn.log "service"
field and another to sort the contents of any field of type "set".
2020-08-11 11:26:21 -07:00
Jon Siwek
cd9fec7bdb Add Cirrus CI config 2019-12-17 15:28:25 -08:00
Jon Siwek
9941537481 Run doctest unit tests in Travis CI 2019-11-15 16:14:43 -08:00
Jon Siwek
9e83abc5d9 Fix indents/whitespace in Travis CI script 2019-11-15 16:14:43 -08:00
Jon Siwek
8d31baf3b7 Fix Travis CI script to run Coverity build in container
It was previously running on the host and didn't have adequate compiler
to do C++17.
2019-10-31 12:09:18 -07:00
Jon Siwek
36d7628bcb Install cmake3 from EPEL on CentOS CI system 2019-10-28 20:18:45 -07:00
Jon Siwek
08e2f97393 Update Travis CI config for C++17
Several platforms didn't have a C++17 compiler in their default repos.

Also moved from Fedora 28 to Fedora 30 since that will soon be the
oldest, still-supported version.
2019-10-24 22:27:54 -07:00
Tim Wojtulewicz
b9f6992c0e fixup! Add new distro to Travis CI configuration for running leak tests 2019-08-09 11:01:16 -07:00
Tim Wojtulewicz
80e3c42526 Add new distro to Travis CI configuration for running leak tests 2019-08-08 16:36:17 -07:00
Jon Siwek
9698d8d7cc Remove --disable-perftools from coverity builds 2019-07-23 09:38:16 -07:00
Jon Siwek
50e4da74cf Fix canonification of timestamps with a decisecond multiple
Unit tests sometimes use current_time(), which occassionally lands on
a perfect tenth of a second and end up fooling the old canonification
regexp and resulting in output which differs from what's in the
test baseline.
2019-07-12 18:24:56 -07:00
Jon Siwek
580822a32c Merge remote-tracking branch 'origin/topic/dnthayer/gh-339'
* origin/topic/dnthayer/gh-339:
  Rename all BRO-prefixed environment variables
2019-05-23 20:23:14 -07:00
Daniel Thayer
1a74516db1 Rename all BRO-prefixed environment variables
For backward compatibility when reading values, we first check
the ZEEK-prefixed value, and if not set, then check the corresponding
BRO-prefixed value.
2019-05-22 00:12:31 -05:00
Daniel Thayer
3f9e7138bd More bro-to-zeek renaming in the unit tests 2019-05-16 02:27:54 -05:00
Jon Siwek
6ad7099f7e Merge remote-tracking branch 'origin/topic/robin/gh-239'
* origin/topic/robin/gh-239:
  Undo a change to btest.cfg from a recent commit
  Updating submodule.
  Fix zeek-wrapper
  Update for renaming BroControl to ZeekControl.
  Updating submodule.
  GH-239: Rename bro to zeek, bro-config to zeek-config, and bro-path-dev to zeek-path-dev.
2019-05-14 13:27:40 -07:00
Robin Sommer
89b8d6e7ba Update for renaming BroControl to ZeekControl. 2019-05-06 21:22:19 +00:00
Jon Siwek
84ca12fdb4 Rename Zeexygen to Zeekygen 2019-05-02 21:39:21 -07:00
Robin Sommer
789cb376fd GH-239: Rename bro to zeek, bro-config to zeek-config, and bro-path-dev to zeek-path-dev.
This also installs symlinks from "zeek" and "bro-config" to a wrapper
script that prints a deprecation warning.

The btests pass, but this is still WIP. broctl renaming is still
missing.

#239
2019-05-01 21:43:45 +00:00
Jon Siwek
aebcb1415d GH-234: rename Broxygen to Zeexygen along with roles/directives
* All "Broxygen" usages have been replaced in
  code, documentation, filenames, etc.

* Sphinx roles/directives like ":bro:see" are now ":zeek:see"

* The "--broxygen" command-line option is now "--zeexygen"
2019-04-22 19:45:50 -07:00
Jon Siwek
a994be9eeb Merge remote-tracking branch 'origin/topic/seth/zeek_init'
* origin/topic/seth/zeek_init:
  Some more testing fixes.
  Update docs and tests for bro_(init|done) -> zeek_(init|done)
  Implement the zeek_init handler.
2019-04-19 11:24:29 -07:00
Jon Siwek
1e57e3f026 Use .zeek file suffix in unit tests 2019-04-16 16:08:57 -07:00
Seth Hall
5db766bd88 Update docs and tests for bro_(init|done) -> zeek_(init|done) 2019-04-14 08:49:12 -04:00
Jon Siwek
9c843a7d83 Add script to update external test repo commit pointers
It will prompt to update the file storing the external test repo
commit hash when a change is detected upon running update-changes.
2019-04-05 17:09:01 -07:00
Jon Siwek
67484a90fa GH-211: improve consistency of how scripting errors are handled
Scripting errors/mistakes now consistently generate a runtime error
which have the behavior of unwinding the call stack all the way out of
the current event handler.

Before, such errors were not treated consistently and either aborted
the process entirely or emitted a message while continuing to execute
subsequent statements without well-defined behavior (possibly causing
a cascade of errors).

The previous behavior also would only unwind out of the current
function (if within a function body), not out the current event
handler, which is especially problematic for functions that return
a value: the caller is essentially left a mess with no way to deal
with it.

This also changes the behavior of the startup/initialization process
to abort if there's errors during bro_init() rather than continue one
to the main run loop.  The `allow_init_errors` option may change this
new, default behavior.
2019-01-30 11:20:09 -06:00
Jon Siwek
2ff746fea7 Change doc/ subdir into a git submodule
The docs now live at https://github.com/zeek/zeek-docs
2019-01-17 14:15:45 -06:00