"Community ID" has become an established flow hash for connection correlation
across different monitoring and storage systems. Other NSMs have had native
and built-in support for Community ID since late 2018. And even though the
roots of "Community ID" are very close to Zeek, Zeek itself has never provided
out-of-the-box support and instead required users to install an external plugin.
While we try to make that installation as easy as possible, an external plugin
always sets the bar higher for an initial setup and can be intimidating.
It also requires a rebuild operation of the plugin during upgrades. Nothing
overly complicated, but somewhat unnecessary for such popular functionality.
This isn't a 1:1 import. The options are parameters and the "verbose"
functionality has been removed. Further, instead of a `connection`
record, the new bif works with `conn_id`, allowing computation of the
hash with little effort on the command line:
$ zeek -e 'print community_id_v1([$orig_h=1.2.3.4, $orig_p=1024/tcp, $resp_h=5.6.7.8, $resp_p=80/tcp])'
1:RcCrCS5fwYUeIzgDDx64EN3+okU
Reference: https://github.com/corelight/zeek-community-id/
* origin/topic/awelzel/2935-publish-hrw-rr-type-check:
zeek.bif: Remove cat_sep() fully var_arg changes
broker/messaging: Runtime type checks for pool
bifcl: Bump for runtime-type checks in var_arg bifs
publish_hrw() and publish_rr() are excluded from type checking due to their
variadic nature. Passing a wrong type for the pool argument previously triggered
an abort, now the result is runtime errors. This isn't great, but it's
better than crashing Zeek.
Closes#2935
* origin/topic/vern/record-optimizations.Apr23B:
different fix for MSVC compiler issues
more general approach for addressing MSVC compiler issues with IntrusivePtr
restored RecordType::Create, now marked as deprecated tidying of namespaces and private class members simplification of flagging record field initializations that should be skipped address peculiar MSVC compilation complaint for IntrusivePtr's
clarifications and tidying for record field initializations
optimize record construction by deferring initializations of aggregates
compile-scripts-to-C++ speedups by switching to raw record access
logging speedup by switching to raw record access
remove redundant record coercions
Removed the `#if 0` hunk during merging: Probably could have gone with a
doctest instead.
* origin/topic/neverlord/gh-2953:
Improve CMake variables, update cmake submodule
Fix builds with plugins that use zeek-config
Bumped cmake and used update-changes with explicit -p because
it wasn't detecting the right commit to start with. Suspect
something went off with the last bump.
tidying of namespaces and private class members
simplification of flagging record field initializations that should be skipped
address peculiar MSVC compilation complaint for IntrusivePtr's
The known_log_certs table is populated asynchronously via broker after a
Broker::peer_added. It may take a variable amount of time depending on where
we run this test and it has been observed flaky specifically for the
arm_debian11 task. Instead of racing, give worker-2 3 seconds for receiving
the expected table content before continuing.
Fixes#2885
When building plugins externally with `zeek-config` (as opposed to using
`ZEEK_DIST`), they point into the install prefix. There, we add a new
file `ZeekPluginBootstrap.cmake` that helps `ZeekPlugin.cmake` to find
everything else it needs from there.
Our template for plugins generates a `configure` script that sets
various variables with values from `zeek-config`. We only need
`BROKER_ROOT_DIR` with the new bootstrapping logic. Everything else, we
can get from the new bootstrapping file and from the CMake package file
for Zeek.
* origin/topic/awelzel/javascript:
ci/debian-11: Install libnode-dev, too
CMakeLists: Convert string append to list append
Add experimental JavaScript support when libnode is available
zeek.on('zeek_init', () => {
console.log('Hello, Zeek!');
});
For interaction with external systems and HTTP APIs, JavaScript and the
Node.js ecosystem beat Zeek script. Make it more easily accessible by
including ZeekJS with Zeek directly.
When a recent enough libnode version is found on the build system, ZeekJS is
added as a builtin plugin. This behavior can be disabled via
``--disable-javascript``. Linux distributions providing such a package are
Ubuntu (22.10) and Debian (testing/bookworm) as libnode-dev.
Fedora provides it as nodejs-devel.
This plugin takes over loading of .js or .cjs files. When no such files
are provided to Zeek, Node and the V8 engine are not initialized and
should not get into the way.
This should be considered experimental.
* origin/topic/timw/2947-input-config-regex:
Reimplement fixes to handle commas at the end of config parser lines
Revert "Convert config framework to use std::regex"
Revert "GH-636: Fix regex to handle commas at the end of config parser lines"
While working on a rotation format function, ran into Zeek crashing
when not returning a value from it, fix and recover the same way as
for scripting errors.
* security/topic/timw/154-rdp-timeout:
RDP: Instantiate SSL analyzer instead of PIA
RDP: add some enforcement to required values based on MS-RDPBCGR docs
* security/topic/awelzel/152-smtp-validate-mail-transactions:
smtp: Validate mail transaction and disable SMTP analyzer if excessive
generic-analyzer-fuzzer: Detect disable_analyzer() from scripts
* security/topic/awelzel/148-ftp-skip-get-pending-commands-multi-line-response:
ftp/main: Special case for intermediate reply lines
ftp/main: Skip get_pending_command() for intermediate reply lines
* origin/topic/timw/update-broker-for-gcc-13:
Add cstdint to WeirdState.h to fix compilation error on gcc13
Ignore -Wchanges-meaning warning in Spicy
Update broker submodule to pull in GCC 13 fix
CI: Force rebuild of OpenSUSE Tumbleweed VM to pick up GCC 13
