While it seems interesting functionality, this hasn't been documented,
maintained or knowingly leveraged for many years.
There are various other approaches today, too:
* We track the number of event handler invocations regardless of
profiling. It's possible to approximate a load_sample event by
comparing the result of two get_event_stats() calls. Or, visualize
the corresponding counters in a Prometheus setup to get an idea of
event/s broken down by event names.
* HookCallFunction() allows to intercept script execution, including
measuring the time execution takes.
* The global call_stack and g_frame_stack can be used from plugins
(and even external processes) to walk the Zeek script stack at certain
points to implement a sampling profiler.
* USDT probes or more plugin hooks will likely be preferred over Zeek
builtin functionality in the future.
Relates to #3458
With custom events for signatures, Reporter::error() may be invoked
while loading them. Early exit in case that happens. We could continue
and either disable the signatures or fallback to the default
signature_match() event, but not sure that would be obviously better.
It can be visible overhead to call write() on the underlying pipe of the
EventMgr's flare whenever the first event is enqueued during an IO loop
iteration. Particularly in scenarios where there's about 1 event per packet
for long lived connections and script-side event processing is fast.
Given the event manager is drained anyhow at the end of the main loop, this
shouldn't be needed. In fact, the EventMgr.Process() method is basically
a stub. The one reason it is needed is when more events are enqueued during
a drain. That, however, can be dealt with by implementing GetNextTimeout()
to return 0.0 when there's more events queued. This way the main-loop's poll
timeout is 0.0 and it'll continue immediately.
This also allows to removes some extra code and drop the recently introduced
InitPostFork() addition: Without a pipe, there's no need to recreate it.
With a bit of tweaking in the JavaScript plugin to support opaque types, this
will allow the delay functionality to work there, too.
Making the LogDelayToken an actual opaque seems reasonable, too. It's not
supposed to be user inspected.
and check_for_unused_event_handlers: UsageAnalyzer is more thorough
and the previous ones weren't extended to work with &is_used and
should probably be considered superseded by the UsageAnalyzer even
if that currently does not provide a public API and just prints
out deprecation warnings.
I'm also tempted to deprecate SetUsed() and Used() of EventHandler
for the same reason.
Closes#3187.
Add a new overload to `copy_string` that takes the input characters plus
size. The new overload avoids inefficient scanning of the input for the
null terminator in cases where we know the size beforehand. Furthermore,
this overload *must* be used when dealing with input character sequences
that may have no null terminator, e.g., when the input is from a
`std::string_view` object.
This largely copies over Spicy's `.clang-format` configuration file. The
one place where we deviate is header include order since Zeek depends on
headers being included in a certain order.
Because EventMgr is defined globally as an object (rather than a global
pointer to an EventMgr object), its pipe is created even before main()
is entered. This further means that in the fork-based supervisor setup,
all Zeek processes created from the top-level supervisor process share
the same pipe object for the EventMgr. In turn, whenever any of the
processes enqueued an event, the flare was fired and ready for reading
on all other processes in the cluster, causing much contention and
unneeded overhead.
Closes#3190
When there are errors reported during yyparse(), Zeek still continued
running initialization functions like init_general_global_var(), init_net_var()
and run_bif_initializers(). These usually call abort() in unexpected
situations causing misleading and confusing errors. This patch prevents
this by exiting earlier.
Closes#3316
Put RecordFieldInit instances into creation_inits during parsing and
determine their deferrability in an InitPostScript step. Any
RecordFieldInits can be deferred are moved into deferred_inits.
Closes#3260
This reflects the `spicy-plugin` code as of `d8c296b81cc2a11`.
In addition to moving the code into Zeek's source tree, this comes
with a couple small functional changes:
- `spicyz` no longer tries to infer if it's running from the build
directory. Instead `ZEEK_SPICY_LIBRARY` can be set to a custom
location. `zeek-set-path.sh` does that now.
- ZEEK_CONFIG can be set to change what `spicyz -z` print out. This is
primarily for backwards compatibility.
Some further notes on specifics:
- We raise the minimum Spicy version to 1.8 (i.e., current `main`
branch).
- Renamed the `compiler/` subdirectory to `spicyz` to avoid
include-path conflicts with the Spicy headers.
- In `cmake/`, the corresponding PR brings a new/extended version of
`FindZeek`, which Spicy analyzer packages need. We also now install
some of the files that the Spicy plugin used to bring for testing,
so that existing packages keep working.
- For now, this all remains backwards compatible with the current
`zkg` analyzer templates so that they work with both external and
integrated Spicy support. Later, once we don't need to support any
external Spicy plugin versions anymore, we can clean up the
templates as well.
- All the plugin's tests have moved into the standard test suite. They
are skipped if configure with `--disable-spicy`.
This holds off on adapting the new code further to Zeek's coding
conventions, so that it remains easier to maintain it in parallel to
the (now legacy) external plugin. We'll make a pass over the
formatting for (presumable) Zeek 6.1.
For "individually different but deterministic" runs specifying Zeek's
seed as an environment variable eases setups as one can avoid creating
extra seed files for each of the individual processes.
It is an error to specify the new ZEEK_SEED_VALUES variable together
with the existing ZEEK_SEED_FILE and -G. ZEEK_SEED takes precedence over
deterministic mode (-D) like ZEEK_SEED_FILE does today already.
It turns out that for every ListVal we construct, we also allocate
and construct a new TypeList instance, even though they are all the
same. Pre-create and cache the type instances in a new TypeManager.
The following script runs ~10% faster for me after this change.
global tbl: table[string] of string;
global i = 0;
while ( ++i < 10000000 )
tbl["a"] = "a";
It is currently not possible to call a->Conn()->GetVal() or construct a
zeek/file_analysis/File object from within doctests, as these quickly
reference the unpopulated zeek::id namespace to construct Val objects
of various types, making it hard write basic tests without completely
re-organizing.
Move running of the unit tests after parsing the scripts, so it is possible
for some basic exercising of File objects within tests.
Add a central place where the decision when it's okay to update network time
to the current time (wallclock) is. It checks for pseudo_realtime and packet
source existence as well as packet source idleness.
A new const &redef allows to completely disable forwarding of network time.
This fixes a potential crash due to trigger_mgr getting shutdown earlier
than dns_mgr, and dns_mgr then trying to use it after it's been deleted.
This change forces the order of initialization/destruction in
iosource_mgr to cause dns_mgr to be deleted first.
* origin/topic/timw/fix-windows-build:
Fix linking of zeek_build_info on Windows
CI: Enable Windows builds for PRs
Call python explicitly from cmake for collecting repo info on Windows
Rework zeek-inet-ntop snprintf return value handling
This adds a new utility called ci/collect-repo-info.py to produce a JSON
document that is then baked into the Zeek executable file. Further, when
creating a tarball via `make dist`, put a top-level repo-info.json file
in place that is picked when no .git directory exists.
Closes#1405
Prevent errors as follows with the bro-http2 plugin.
error in /mitrecnd_HTTP2.events.bif.zeek, line 95: identifier not defined: http2_stream_stat
error in /mitrecnd_HTTP2.events.bif.zeek, line 363: identifier not defined: http2_settings
internal error in /mitrecnd_HTTP2.events.bif.zeek, line 460: Failed to find type named: http2_settings_unrecognized_table
* topic/christian/gh-2239-stdin-ctrl-c:
Stop signal-masking upon running unit tests
Pause signal-masking during script parsing
Add btests to verify Zeek's handling of SIGTERM and reading stdin
Add procps/procps-ng to several CI Docker images
Script parsing includes reading script content from stdin, which in turn
includes interactive Zeek sessions. Keeping the signals masked there broke
ctrl-c behavior.
Providing an unknown plugin debug stream with -B was previously silently
accepted. This caused user confusing as the behavior is "no output, but
seems to work".
Check the enabled debug streams once all plugins have been loaded and
exit early on for invalid streams.
$ ZEEK_PLUGIN_PATH=./build zeek -B plugin-zeek-myplugin -e 'print zeek_version();'
error in <command line>, line 3: No plugin debug stream 'plugin-zeek-myplugin' found
Closes#913.
