"Community ID" has become an established flow hash for connection correlation
across different monitoring and storage systems. Other NSMs have had native
and built-in support for Community ID since late 2018. And even though the
roots of "Community ID" are very close to Zeek, Zeek itself has never provided
out-of-the-box support and instead required users to install an external plugin.
While we try to make that installation as easy as possible, an external plugin
always sets the bar higher for an initial setup and can be intimidating.
It also requires a rebuild operation of the plugin during upgrades. Nothing
overly complicated, but somewhat unnecessary for such popular functionality.
This isn't a 1:1 import. The options are parameters and the "verbose"
functionality has been removed. Further, instead of a `connection`
record, the new bif works with `conn_id`, allowing computation of the
hash with little effort on the command line:
$ zeek -e 'print community_id_v1([$orig_h=1.2.3.4, $orig_p=1024/tcp, $resp_h=5.6.7.8, $resp_p=80/tcp])'
1:RcCrCS5fwYUeIzgDDx64EN3+okU
Reference: https://github.com/corelight/zeek-community-id/
This set contains the topics to reach all cluster nodes. Due to broker's
forwarding mechanism, we cannot define a single broadcast topic, as it
would create routing loops.
* origin/topic/awelzel/2935-publish-hrw-rr-type-check:
zeek.bif: Remove cat_sep() fully var_arg changes
broker/messaging: Runtime type checks for pool
bifcl: Bump for runtime-type checks in var_arg bifs
publish_hrw() and publish_rr() are excluded from type checking due to their
variadic nature. Passing a wrong type for the pool argument previously triggered
an abort, now the result is runtime errors. This isn't great, but it's
better than crashing Zeek.
Closes#2935
The file_analysis/Manager.h header only needs run_state::terminating, so
declare just that similar to what Val.h does. This breaks ZBody compilation
due to OP_READING_LIVE_TRAFFIC_V and OP_READING_TRACES_V accessing
run-state::reading_live and run_state::reading_traces. Add the include to
ZBody.cc as a fix.
This may break external plugins, too, but then they didn't have the
right includes setup to begin with.
* origin/topic/vern/record-optimizations.Apr23B:
different fix for MSVC compiler issues
more general approach for addressing MSVC compiler issues with IntrusivePtr
restored RecordType::Create, now marked as deprecated tidying of namespaces and private class members simplification of flagging record field initializations that should be skipped address peculiar MSVC compilation complaint for IntrusivePtr's
clarifications and tidying for record field initializations
optimize record construction by deferring initializations of aggregates
compile-scripts-to-C++ speedups by switching to raw record access
logging speedup by switching to raw record access
remove redundant record coercions
Removed the `#if 0` hunk during merging: Probably could have gone with a
doctest instead.
* origin/topic/neverlord/gh-2953:
Improve CMake variables, update cmake submodule
Fix builds with plugins that use zeek-config
Bumped cmake and used update-changes with explicit -p because
it wasn't detecting the right commit to start with. Suspect
something went off with the last bump.
tidying of namespaces and private class members
simplification of flagging record field initializations that should be skipped
address peculiar MSVC compilation complaint for IntrusivePtr's
The known_log_certs table is populated asynchronously via broker after a
Broker::peer_added. It may take a variable amount of time depending on where
we run this test and it has been observed flaky specifically for the
arm_debian11 task. Instead of racing, give worker-2 3 seconds for receiving
the expected table content before continuing.
Fixes#2885
This new table provides a mechanism to add environment variables to the
postprocessor execution. Use case is from ZeekControl to inject a suffix
to be used when running with multiple logger.
When building plugins externally with `zeek-config` (as opposed to using
`ZEEK_DIST`), they point into the install prefix. There, we add a new
file `ZeekPluginBootstrap.cmake` that helps `ZeekPlugin.cmake` to find
everything else it needs from there.
Our template for plugins generates a `configure` script that sets
various variables with values from `zeek-config`. We only need
`BROKER_ROOT_DIR` with the new bootstrapping logic. Everything else, we
can get from the new bootstrapping file and from the CMake package file
for Zeek.
* origin/topic/awelzel/javascript:
ci/debian-11: Install libnode-dev, too
CMakeLists: Convert string append to list append
Add experimental JavaScript support when libnode is available
zeek.on('zeek_init', () => {
console.log('Hello, Zeek!');
});
For interaction with external systems and HTTP APIs, JavaScript and the
Node.js ecosystem beat Zeek script. Make it more easily accessible by
including ZeekJS with Zeek directly.
When a recent enough libnode version is found on the build system, ZeekJS is
added as a builtin plugin. This behavior can be disabled via
``--disable-javascript``. Linux distributions providing such a package are
Ubuntu (22.10) and Debian (testing/bookworm) as libnode-dev.
Fedora provides it as nodejs-devel.
This plugin takes over loading of .js or .cjs files. When no such files
are provided to Zeek, Node and the V8 engine are not initialized and
should not get into the way.
This should be considered experimental.
