No description
Find a file
Arne Welzel f0e67022fd logging: Introduce Log::delay() and Log::delay_finish()
This is a verbose, opinionated and fairly restrictive version of the log delay idea.
Main drivers are explicitly, foot-gun-avoidance and implementation simplicity.

Calling the new Log::delay() function is only allowed within the execution
of a Log::log_stream_policy() hook for the currently active log write.

Conceptually, the delay is placed between the execution of the global stream
policy hook and the individual filter policy hooks. A post delay callback
can be registered with every Log::delay() invocation. Post delay callbacks
can (1) modify a log record as they see fit, (2) veto the forwarding of the
log record to the log filters and (3) extend the delay duration by calling
Log::delay() again. The last point allows to delay a record by an indefinite
amount of time, rather than a fixed maximum amount. This should be rare and
is therefore explicit.

Log::delay() increases an internal reference count and returns an opaque
token value to be passed to Log::delay_finish() to release a delay reference.
Once all references are released, the record is forwarded to all filters
attached to a stream when the delay completes.

This functionality separates Log::log_stream_policy() and individual filter
policy hooks. One consequence is that a common use-case of filter policy hooks,
removing unproductive log records, may run after a record was delayed. Users
can lift their filtering logic to the stream level (or replicate the condition
before the delay decision). The main motivation here is that deciding on a
stream-level delay in per-filter hooks is too late. Attaching multiple filters
to a stream can additionally result in hard to understand behavior.

On the flip side, filter policy hooks are guaranteed to run after the delay
and can be used for further mangling or filtering of a delayed record.
2023-11-29 11:53:11 +01:00
.github/workflows generate-docs: Bump ccache from 500M to 2000M 2023-09-27 18:20:36 +02:00
auxil Update zeekjs submodule 2023-11-29 11:52:44 +01:00
ci CI: Install missing packages on opensuse 2023-11-17 09:24:15 -07:00
cmake@e16c48f380 Bump cmake submodule for INCLUDE_DIRS fix 2023-11-22 11:29:00 +01:00
cmake_templates zeek-config.h: Drop NEED_KRB5_H 2023-08-28 11:31:05 +02:00
doc@39b10c9f28 Update doc submodule [nomail] [skip ci] 2023-11-29 00:21:56 +00:00
docker Add jq to final.Dockerfile 2023-11-21 22:23:37 +09:00
man Use the same rules as cmake submodule to reformat Zeek 2023-05-09 08:31:43 -07:00
scripts logging: Introduce Log::delay() and Log::delay_finish() 2023-11-29 11:53:11 +01:00
src logging: Introduce Log::delay() and Log::delay_finish() 2023-11-29 11:53:11 +01:00
testing logging: Introduce Log::delay() and Log::delay_finish() 2023-11-29 11:53:11 +01:00
.cirrus.yml CI: Add macOS Sonoma build, remove macOS Monterey build 2023-11-06 11:09:44 -07:00
.clang-format Format JSON with clang-format 2023-10-30 09:41:13 +01:00
.clang-tidy Disable annoying bugprone-easily-swappable-parameters clang-tidy check [skip ci] 2022-10-07 16:15:47 -07:00
.cmake-format.json Format JSON with clang-format 2023-10-30 09:41:13 +01:00
.dockerignore Add .dockerignore to suppress btest artifacts 2021-09-24 17:04:26 -07:00
.git-blame-ignore-revs Update .git-blame-ignore-revs 2023-10-30 09:42:39 +01:00
.gitattributes GH-1497: Support CRLF line-endings in Zeek scripts and signature files 2021-04-08 20:32:30 -07:00
.gitignore Cirrus configuration for Windows builds 2022-11-09 18:16:13 +02:00
.gitmodules Move Spicy submodule a layer up. 2023-05-16 12:09:12 +02:00
.pre-commit-config.yaml Format JSON with clang-format 2023-10-30 09:41:13 +01:00
.style.yapf Format Python scripts with yapf. 2021-11-24 23:13:24 +01:00
.typos.toml retention of superseded AST elements to prevent pointer mis-aliasing 2023-11-10 11:06:16 +01:00
.update-changes.cfg Add script to update external test repo commit pointers 2019-04-05 17:09:01 -07:00
CHANGES Update zeekjs submodule 2023-11-29 11:52:44 +01:00
CMakeLists.txt Merge remote-tracking branch 'origin/topic/neverlord/avoid-openssl-header-deps' 2023-11-22 10:35:27 +01:00
configure configure/CMakelists: Use Python_EXECUTABLE as hint for interpreter 2023-09-07 12:48:37 +02:00
COPYING Update COPYING to 2023 2023-01-03 12:10:03 -07:00
COPYING-3rdparty Rename COPYING.3rdparty to COPYING-3rdparty 2023-01-03 12:10:03 -07:00
INSTALL Update documentation to include "Book of Zeek" revisions 2021-02-01 15:54:36 -08:00
Makefile Fix usage of realpath on macOS, instead preferring grealpath 2023-06-13 15:51:47 -07:00
NEWS Merge remote-tracking branch 'origin/topic/neverlord/avoid-openssl-header-deps' 2023-11-22 10:35:27 +01:00
README Add tooling section to README 2023-01-27 13:03:52 -07:00
README.md Update link to slack in README.md 2023-06-01 14:21:44 +02:00
VERSION Update zeekjs submodule 2023-11-29 11:52:44 +01:00
zeek-path-dev.in Move CMake template files to separate directory 2023-06-26 13:39:59 -07:00

Zeek Logo

The Zeek Network Security Monitor

A powerful framework for network traffic analysis and security monitoring.

Key FeaturesDocumentationGetting StartedDevelopmentLicense

Follow us on Twitter at @zeekurity.

Coverage Status Build Status

Slack Discourse

Key Features

  • In-depth Analysis Zeek ships with analyzers for many protocols, enabling high-level semantic analysis at the application layer.

  • Adaptable and Flexible Zeek's domain-specific scripting language enables site-specific monitoring policies and means that it is not restricted to any particular detection approach.

  • Efficient Zeek targets high-performance networks and is used operationally at a variety of large sites.

  • Highly Stateful Zeek keeps extensive application-layer state about the network it monitors and provides a high-level archive of a network's activity.

Getting Started

The best place to find information about getting started with Zeek is our web site www.zeek.org, specifically the documentation section there. On the web site you can also find downloads for stable releases, tutorials on getting Zeek set up, and many other useful resources.

You can find release notes in NEWS, and a complete record of all changes in CHANGES.

To work with the most recent code from the development branch of Zeek, clone the master git repository:

git clone --recursive https://github.com/zeek/zeek

With all dependencies in place, build and install:

./configure && make && sudo make install

Write your first Zeek script:

# File "hello.zeek"

event zeek_init()
    {
    print "Hello World!";
    }

And run it:

zeek hello.zeek

For learning more about the Zeek scripting language, try.zeek.org is a great resource.

Development

Zeek is developed on GitHub by its community. We welcome contributions. Working on an open source project like Zeek can be an incredibly rewarding experience and, packet by packet, makes the Internet a little safer. Today, as a result of countless contributions, Zeek is used operationally around the world by major companies and educational and scientific institutions alike for securing their cyber infrastructure.

If you're interested in getting involved, we collect feature requests and issues on GitHub here and you might find these to be a good place to get started. More information on Zeek's development can be found here, and information about its community and mailing lists (which are fairly active) can be found here.

License

Zeek comes with a BSD license, allowing for free use with virtually no restrictions. You can find it here.

Tooling

We use the following tooling to help discover issues to fix, amongst a number of others.