Copy docs into Zeek repo directly

This is based on commit 99e6942efec5feff50523f6b2a1f5868f19ab638 from the
zeek-docs repo.

doc/devel/plugins/connkey-plugin.rst

@@ -0,0 +1,205 @@
+.. _connkey-plugin:
+
+===============================
+Writing a Connection Key Plugin
+===============================
+
+.. versionadded:: 8.0
+
+By default, Zeek looks up internal connection state using the classic five-tuple
+of originator and responder IP addresses, ports, and the numeric protocol
+identifier (for TCP, UDP, etc). Zeek's data structure driving this is called a
+connection key, or ``ConnKey``.
+
+In certain environments the classic five-tuple does not sufficiently distinguish
+connections. Consider traffic mirrored from multiple VLANs with overlapping IP
+address ranges. Concretely, a connection between 10.0.0.1 and 10.0.0.2 in one
+VLAN is distinct from a connection between the same IPs in another VLAN.  Here,
+Zeek should include the VLAN identifier into the connection key, and you can
+instruct Zeek to do so by loading the
+:doc:`/scripts/policy/frameworks/conn_key/vlan_fivetuple.zeek` policy script.
+
+Zeek's plugin API allows adding support for additional custom connection keys.
+This section provides a tutorial on how to do so, using the example of VXLAN-enabled
+flow tuples. If you're not familiar with plugin development, head over to the
+:ref:`Writing Plugins <writing-plugins>` section.
+
+Our goal is to implement a custom connection key to scope connections
+transported within a `VXLAN <https://datatracker.ietf.org/doc/html/rfc7348/index.html>`_
+tunnel by the VXLAN Network Identifier (VNI).
+
+As a test case, we have encapsulated the `HTTP GET trace <https://github.com/zeek/zeek/raw/refs/heads/master/testing/btest/Traces/http/get.trace>`_
+from the Zeek repository twice with VXLAN using VNIs 4711 and 4242, respectively,
+and merged the resulting two PCAP files with the original PCAP.
+The :download:`resulting PCAP <connkey-vxlan-fivetuple-plugin-src/Traces/vxlan-overlapping-http-get.pcap>`
+contains three HTTP connections, two of which are VXLAN-encapsulated.
+
+By default, Zeek will create the same connection key for the original and
+encapsulated HTTP connections, since they have identical inner five-tuples.
+Therefore, Zeek creates only a single ``http.log`` entry, and two entries
+in ``conn.log``.
+
+.. code-block:: shell
+
+    $ zeek -C -r Traces/vxlan-overlapping-http-get.pcap
+    $ zeek-cut -m uid method host uri < http.log
+    uid     method  host    uri
+    CpWF5etn1l2rpaLu3       GET     bro.org /download/CHANGES.bro-aux.txt
+
+    $ zeek-cut -m uid service history orig_pkts resp_pkts < conn.log
+    uid     service history orig_pkts       resp_pkts
+    Cq2CY245oGGbibJ8k9      http    ShADTadtFf      21      21
+    CMleDu4xANIMzePYd7      vxlan   D       28      0
+
+Note that just two of the HTTP connections are encapsulated.
+That is why the VXLAN connection shows only 28 packets.
+Each HTTP connection has 14 packets total, 7 in each direction. Zeek aggregates
+all packets into the single HTTP connection, but only 28 of them were
+transported within the VXLAN tunnel connection. Note also the ``t`` and ``T``
+flags in the :zeek:field:`Conn::Info$history` field. These stand for retransmissions,
+caused by Zeek not discriminating between the different HTTP connections.
+
+The plugin we'll develop below adds the VXLAN VNI to the connection key.
+As a result, Zeek will correctly report three HTTP connections, tracked
+and logged separately. We'll add the VNI as
+:zeek:field:`vxlan_vni` to the :zeek:see:`conn_id_ctx` record, making it available
+in ``http.log`` and ``conn.log`` via the ``id.ctx.vxlan_vni`` column.
+
+After activating the plugin Zeek tracks each HTTP connection individually and
+the logs will look as follows:
+
+.. code-block:: shell
+
+    $ zeek-cut -m uid method host uri id.ctx.vxlan_vni < http.log
+    uid     method  host    uri     id.ctx.vxlan_vni
+    CBifsS2vqGEg8Fa5ac      GET     bro.org /download/CHANGES.bro-aux.txt   4711
+    CEllEz13txeSrbGqBe      GET     bro.org /download/CHANGES.bro-aux.txt   4242
+    CRfbJw1kBBvHDQQBta      GET     bro.org /download/CHANGES.bro-aux.txt   -
+
+    $ zeek-cut -m uid service history orig_pkts resp_pkts id.ctx.vxlan_vni < conn.log
+    uid     service history orig_pkts       resp_pkts       id.ctx.vxlan_vni
+    CRfbJw1kBBvHDQQBta      http    ShADadFf        7       7       -
+    CEllEz13txeSrbGqBe      http    ShADadFf        7       7       4242
+    CBifsS2vqGEg8Fa5ac      http    ShADadFf        7       7       4711
+    CC6Ald2LejCS1qcDy4      vxlan   D       28      0       -
+
+
+Implementation
+==============
+
+Adding alternative connection keys involves implementing two classes.
+First, a factory class producing ``zeek::ConnKey`` instances. This
+is the class created through the added ``zeek::conn_key::Component``.
+Second, a custom connection key class derived from ``zeek::ConnKey``.
+Instances of this class are created by the factory. This is a typical
+abstract factory pattern.
+
+Our plugin's ``Configure()`` method follows the standard pattern of setting up
+basic information about the plugin and registering our own ``ConnKey`` component.
+
+.. literalinclude:: connkey-vxlan-fivetuple-plugin-src/src/Plugin.cc
+   :caption: Plugin.cc
+   :language: cpp
+   :lines: 16-
+   :linenos:
+   :tab-width: 4
+
+
+Next, in the ``Factory.cc`` file, we're implementing a custom ``zeek::ConnKey`` class.
+This class is named ``VxlanVniConnKey`` and inherits from ``zeek::IPBasedConnKey``.
+While ``zeek::ConnKey`` is technically the base class, in this tutorial we'll
+derive from ``zeek::IPBasedConnKey``.
+Currently, Zeek only supports IP-based connection tracking via the
+``IPBasedAnalyzer`` analyzer. This analyzer requires ``zeek::IPBasedConnKey``
+instances.
+
+.. literalinclude:: connkey-vxlan-fivetuple-plugin-src/src/Factory.cc
+   :caption: VxlanVniConnKey class in Factory.cc
+   :language: cpp
+   :linenos:
+   :lines: 18-78
+   :tab-width: 4
+
+The current pattern for custom connection keys is to embed the bytes used for
+the ``zeek::session::detail::Key`` as a packed struct within a ``ConnKey`` instance.
+We override ``DoPopulateConnIdVal()`` to set the :zeek:field:`vxlan_vni` field
+of the :zeek:see:`conn_id_ctx` record value to the extracted VXLAN VNI. A small trick
+employed is that we default the most significant byte of ``key.vxlan_vni`` to 0xFF.
+As a VNI has only 24 bits, this allows us to determine if a VNI was actually
+extracted, or whether it remained unset.
+
+The ``DoInit()`` implementation is the actual place for connection key customization.
+This is where we extract the VXLAN VNI from packet data. To do so, we're using the relatively
+new ``GetAnalyzerData()`` API of the packet analysis manager.
+This API allows generic access to the raw data layers analyzed by a give packet analyzer.
+For our use-case, we take the most outer VXLAN layer, if any, and extract the VNI
+into ``key.vxlan_vni``.
+
+There's no requirement to use the ``GetAnalyzerData()`` API. If the ``zeek::Packet``
+instance passed to ``DoInit()`` contains the needed information, e.g. VLAN identifiers
+or information from the packet's raw bytes, you can use them directly.
+Specifically, ``GetAnalyzerData()`` may introduce additional overhead into the
+packet path that you can avoid if the information is readily available
+elsewhere.
+Using other Zeek APIs to determine connection key information is of course
+also possible.
+
+The next part shown concerns the ``Factory`` class itself. The
+``DoConnKeyFromVal()`` method contains logic to produce a ``VxlanVniConnKey``
+instance from an existing :zeek:see:`conn_id` record.
+This is needed in order for the :zeek:see:`lookup_connection` builtin function to work properly.
+The implementation re-uses the ``DoConnKeyFromVal()`` implementation of the
+default ``fivetuple::Factory`` that our factory inherits from to extract the
+classic five-tuple information.
+
+.. literalinclude:: connkey-vxlan-fivetuple-plugin-src/src/Factory.cc
+   :caption: Factory class in Factory.cc
+   :language: cpp
+   :linenos:
+   :lines: 80-103
+   :tab-width: 4
+
+Calling the ``fivetuple::Factory::DoConnKeyFromVal()`` in turn calls our
+own factory's ``DoNewConnKey()`` method through virtual dispatch.  Since our
+factory overrides this method to always return a ``VxlanVniConnKey`` instance,
+the static cast later is safe.
+
+Last, the plugin's ``__load__.zeek`` file is shown. It includes the extension
+of the :zeek:see:`conn_id_ctx` identifier by the :zeek:field:`vxlan_vni` field.
+
+.. literalinclude:: connkey-vxlan-fivetuple-plugin-src/scripts/__load__.zeek
+   :caption: The conn_id redefinition in __load__.zeek
+   :language: zeek
+   :linenos:
+   :tab-width: 4
+
+
+Using the custom Connection Key
+===============================
+
+After installing the plugin, the new connection key implementation can be
+selected by redefining the script-level :zeek:see:`ConnKey::factory` variable.
+This can either be done in a separate script, but we do it directly on the
+command-line for simplicity. The ``ConnKey::CONNKEY_VXLAN_VNI_FIVETUPLE`` is
+registered in Zeek during the plugin's ``AddComponent()`` call during
+``Configure()``, where the component has the name ``VXLAN_VNI_FIVETUPLE``.
+
+.. code-block:: shell
+
+    $ zeek -C -r Traces/vxlan-overlapping-http-get.pcap  ConnKey::factory=ConnKey::CONNKEY_VXLAN_VNI_FIVETUPLE
+
+
+Viewing the ``conn.log`` now shows three separate HTTP connections,
+two of which have a ``vxlan_vni`` value set in their logs.
+
+
+.. code-block:: shell
+
+    $ zeek-cut -m uid service history orig_pkts resp_pkts id.ctx.vxlan_vni < conn.log
+    uid     service history orig_pkts       resp_pkts       id.ctx.vxlan_vni
+    CRfbJw1kBBvHDQQBta      http    ShADadFf        7       7       -
+    CEllEz13txeSrbGqBe      http    ShADadFf        7       7       4242
+    CBifsS2vqGEg8Fa5ac      http    ShADadFf        7       7       4711
+    CC6Ald2LejCS1qcDy4      vxlan   D       28      0       -
+
+Pretty cool, isn't it?

doc/devel/plugins/connkey-vxlan-fivetuple-plugin-src/CMakeLists.txt

@@ -0,0 +1,14 @@
+cmake_minimum_required(VERSION 3.15 FATAL_ERROR)
+
+project(ZeekPluginConnKeyVxlanVniFivetuple)
+
+include(ZeekPlugin)
+
+zeek_add_plugin(
+    Zeek
+    ConnKey_Vxlan_Vni_Fivetuple
+    SOURCES
+    src/Factory.cc
+    src/Plugin.cc
+    SCRIPT_FILES scripts/__load__.zeek
+)

doc/devel/plugins/connkey-vxlan-fivetuple-plugin-src/COPYING

@@ -0,0 +1,26 @@
+Copyright (c) 2025 by the Zeek Project. All rights reserved.
+
+Redistribution and use in source and binary forms, with or without modification,
+are permitted provided that the following conditions are met:
+
+1. Redistributions of source code must retain the above copyright notice, this
+   list of conditions and the following disclaimer.
+
+2. Redistributions in binary form must reproduce the above copyright notice,
+   this list of conditions and the following disclaimer in the documentation
+   and/or other materials provided with the distribution.
+
+3. Neither the name of the copyright holder nor the names of its contributors
+   may be used to endorse or promote products derived from this software
+   without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
+ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR
+ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
+ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

doc/devel/plugins/connkey-vxlan-fivetuple-plugin-src/Makefile

@@ -0,0 +1,23 @@
+#
+# Convenience Makefile providing a few common top-level targets.
+#
+
+cmake_build_dir=build
+arch=`uname -s | tr A-Z a-z`-`uname -m`
+
+all: build-it
+
+build-it:
+	( cd $(cmake_build_dir) && make )
+
+install:
+	( cd $(cmake_build_dir) && make install )
+
+clean:
+	( cd $(cmake_build_dir) && make clean )
+
+distclean:
+	rm -rf $(cmake_build_dir)
+
+test:
+	make -C tests

doc/devel/plugins/connkey-vxlan-fivetuple-plugin-src/VERSION

@@ -0,0 +1 @@
+0.1.0

doc/devel/plugins/connkey-vxlan-fivetuple-plugin-src/configure

@@ -0,0 +1,193 @@
+#!/bin/sh
+#
+# Wrapper for viewing/setting options that the plugin's CMake
+# scripts will recognize.
+#
+# Don't edit this. Edit configure.plugin to add plugin-specific options.
+#
+
+set -e
+command="$0 $*"
+
+if [ -e $(dirname $0)/configure.plugin ]; then
+    # Include custom additions.
+    . $(dirname $0)/configure.plugin
+fi
+
+usage() {
+
+    cat 1>&2 <<EOF
+Usage: $0 [OPTIONS]
+
+  Plugin Options:
+    --cmake=PATH               Path to CMake binary
+    --zeek-dist=DIR            Path to Zeek source tree
+    --install-root=DIR         Path where to install plugin into
+    --with-binpac=DIR          Path to BinPAC installation root
+    --with-broker=DIR          Path to Broker installation root
+    --with-bifcl=PATH          Path to bifcl executable
+    --enable-debug             Compile in debugging mode
+    --disable-cpp-tests        Don't build C++ unit tests
+EOF
+
+    if type plugin_usage >/dev/null 2>&1; then
+        plugin_usage 1>&2
+    fi
+
+    echo
+
+    exit 1
+}
+
+# Function to append a CMake cache entry definition to the
+# CMakeCacheEntries variable
+#   $1 is the cache entry variable name
+#   $2 is the cache entry variable type
+#   $3 is the cache entry variable value
+append_cache_entry() {
+    CMakeCacheEntries="$CMakeCacheEntries -D $1:$2=$3"
+}
+
+# set defaults
+builddir=build
+zeekdist=""
+installroot="default"
+zeek_plugin_begin_opts=""
+CMakeCacheEntries=""
+
+while [ $# -ne 0 ]; do
+    case "$1" in
+        -*=*) optarg=$(echo "$1" | sed 's/[-_a-zA-Z0-9]*=//') ;;
+        *) optarg= ;;
+    esac
+
+    case "$1" in
+        --help | -h)
+            usage
+            ;;
+
+        --cmake=*)
+            CMakeCommand=$optarg
+            ;;
+
+        --zeek-dist=*)
+            zeekdist=$(cd $optarg && pwd)
+            ;;
+
+        --install-root=*)
+            installroot=$optarg
+            ;;
+
+        --with-binpac=*)
+            append_cache_entry BinPAC_ROOT_DIR PATH $optarg
+            binpac_root=$optarg
+            ;;
+
+        --with-broker=*)
+            append_cache_entry BROKER_ROOT_DIR PATH $optarg
+            broker_root=$optarg
+            ;;
+
+        --with-bifcl=*)
+            append_cache_entry BifCl_EXE PATH $optarg
+            ;;
+
+        --enable-debug)
+            append_cache_entry BRO_PLUGIN_ENABLE_DEBUG BOOL true
+            ;;
+
+        --disable-cpp-tests)
+            zeek_plugin_begin_opts="DISABLE_CPP_TESTS;$zeek_plugin_begin_opts"
+            ;;
+
+        *)
+            if type plugin_option >/dev/null 2>&1; then
+                plugin_option $1 && shift && continue
+            fi
+
+            echo "Invalid option '$1'.  Try $0 --help to see available options."
+            exit 1
+            ;;
+    esac
+    shift
+done
+
+if [ -z "$CMakeCommand" ]; then
+    # prefer cmake3 over "regular" cmake (cmake == cmake2 on RHEL)
+    if command -v cmake3 >/dev/null 2>&1; then
+        CMakeCommand="cmake3"
+    elif command -v cmake >/dev/null 2>&1; then
+        CMakeCommand="cmake"
+    else
+        echo "This plugin requires CMake, please install it first."
+        echo "Then you may use this script to configure the CMake build."
+        echo "Note: pass --cmake=PATH to use cmake in non-standard locations."
+        exit 1
+    fi
+fi
+
+if [ -z "$zeekdist" ]; then
+    if type zeek-config >/dev/null 2>&1; then
+        zeek_config="zeek-config"
+    else
+        echo "Either 'zeek-config' must be in PATH or '--zeek-dist=<path>' used"
+        exit 1
+    fi
+
+    append_cache_entry BRO_CONFIG_PREFIX PATH $(${zeek_config} --prefix)
+    append_cache_entry BRO_CONFIG_INCLUDE_DIR PATH $(${zeek_config} --include_dir)
+    append_cache_entry BRO_CONFIG_PLUGIN_DIR PATH $(${zeek_config} --plugin_dir)
+    append_cache_entry BRO_CONFIG_LIB_DIR PATH $(${zeek_config} --lib_dir)
+    append_cache_entry BRO_CONFIG_CMAKE_DIR PATH $(${zeek_config} --cmake_dir)
+    append_cache_entry CMAKE_MODULE_PATH PATH $(${zeek_config} --cmake_dir)
+
+    build_type=$(${zeek_config} --build_type)
+
+    if [ "$build_type" = "debug" ]; then
+        append_cache_entry BRO_PLUGIN_ENABLE_DEBUG BOOL true
+    fi
+
+    if [ -z "$binpac_root" ]; then
+        append_cache_entry BinPAC_ROOT_DIR PATH $(${zeek_config} --binpac_root)
+    fi
+
+    if [ -z "$broker_root" ]; then
+        append_cache_entry BROKER_ROOT_DIR PATH $(${zeek_config} --broker_root)
+    fi
+else
+    if [ ! -e "$zeekdist/zeek-path-dev.in" ]; then
+        echo "$zeekdist does not appear to be a valid Zeek source tree."
+        exit 1
+    fi
+
+    # BRO_DIST is the canonical/historical name used by plugin CMake scripts
+    # ZEEK_DIST doesn't serve a function at the moment, but set/provided anyway
+    append_cache_entry BRO_DIST PATH $zeekdist
+    append_cache_entry ZEEK_DIST PATH $zeekdist
+    append_cache_entry CMAKE_MODULE_PATH PATH $zeekdist/cmake
+fi
+
+if [ "$installroot" != "default" ]; then
+    mkdir -p $installroot
+    append_cache_entry BRO_PLUGIN_INSTALL_ROOT PATH $installroot
+fi
+
+if [ -n "$zeek_plugin_begin_opts" ]; then
+    append_cache_entry ZEEK_PLUGIN_BEGIN_OPTS STRING "$zeek_plugin_begin_opts"
+fi
+
+if type plugin_addl >/dev/null 2>&1; then
+    plugin_addl
+fi
+
+echo "Build Directory        : $builddir"
+echo "Zeek Source Directory   : $zeekdist"
+
+mkdir -p $builddir
+cd $builddir
+
+"$CMakeCommand" $CMakeCacheEntries ..
+
+echo "# This is the command used to configure this build" >config.status
+echo $command >>config.status
+chmod u+x config.status

doc/devel/plugins/connkey-vxlan-fivetuple-plugin-src/scripts/__load__.zeek

@@ -0,0 +1,3 @@
+redef record conn_id_ctx += {
+	vxlan_vni: count &log &optional;
+};

doc/devel/plugins/connkey-vxlan-fivetuple-plugin-src/scripts/__preload__.zeek

@@ -0,0 +1 @@
+# Empty

doc/devel/plugins/connkey-vxlan-fivetuple-plugin-src/src/Factory.cc

@@ -0,0 +1,105 @@
+// See the file "COPYING" in the main distribution directory for copyright.
+
+#include "Factory.h"
+
+#include <memory>
+
+#include "zeek/ID.h"
+#include "zeek/Val.h"
+#include "zeek/iosource/Packet.h"
+#include "zeek/packet_analysis/Analyzer.h"
+#include "zeek/packet_analysis/Manager.h"
+#include "zeek/packet_analysis/protocol/ip/conn_key/IPBasedConnKey.h"
+#include "zeek/packet_analysis/protocol/ip/conn_key/fivetuple/Factory.h"
+#include "zeek/util-types.h"
+
+namespace zeek::conn_key::vxlan_vni_fivetuple {
+
+class VxlanVniConnKey : public zeek::IPBasedConnKey {
+public:
+    VxlanVniConnKey() {
+        // Ensure padding holes in the key struct are filled with zeroes.
+        memset(static_cast<void*>(&key), 0, sizeof(key));
+    }
+
+    detail::PackedConnTuple& PackedTuple() override { return key.tuple; }
+
+    const detail::PackedConnTuple& PackedTuple() const override { return key.tuple; }
+
+protected:
+    zeek::session::detail::Key DoSessionKey() const override {
+        return {reinterpret_cast<const void*>(&key), sizeof(key), session::detail::Key::CONNECTION_KEY_TYPE};
+    }
+
+    void DoPopulateConnIdVal(zeek::RecordVal& conn_id, zeek::RecordVal& ctx) override {
+        // Base class populates conn_id fields (orig_h, orig_p, resp_h, resp_p)
+        zeek::IPBasedConnKey::DoPopulateConnIdVal(conn_id, ctx);
+
+        if ( conn_id.GetType() != id::conn_id )
+            return;
+
+        if ( (key.vxlan_vni & 0xFF000000) == 0 ) // High-bits unset: Have VNI
+            ctx.Assign(GetVxlanVniOffset(), static_cast<zeek_uint_t>(key.vxlan_vni));
+        else
+            ctx.Remove(GetVxlanVniOffset());
+    }
+
+    // Extract VNI from most outer VXLAN layer.
+    void DoInit(const Packet& pkt) override {
+        static const auto& analyzer = zeek::packet_mgr->GetAnalyzer("VXLAN");
+
+        // Set the high-bits: This is needed because keys can get reused.
+        key.vxlan_vni = 0xFF000000;
+
+        if ( ! analyzer || ! analyzer->IsEnabled() )
+            return;
+
+        auto spans = zeek::packet_mgr->GetAnalyzerData(analyzer);
+
+        if ( spans.empty() || spans[0].size() < 8 )
+            return;
+
+        key.vxlan_vni = spans[0][4] << 16 | spans[0][5] << 8 | spans[0][6];
+    }
+
+    static int GetVxlanVniOffset() {
+        static const auto& conn_id_ctx = zeek::id::find_type<zeek::RecordType>("conn_id_ctx");
+        static int vxlan_vni_offset = conn_id_ctx->FieldOffset("vxlan_vni");
+        return vxlan_vni_offset;
+    }
+
+private:
+    friend class Factory;
+
+    struct {
+        struct detail::PackedConnTuple tuple;
+        uint32_t vxlan_vni;
+    } __attribute__((packed, aligned)) key; // packed and aligned due to usage for hashing
+};
+
+zeek::ConnKeyPtr Factory::DoNewConnKey() const { return std::make_unique<VxlanVniConnKey>(); }
+
+zeek::expected<zeek::ConnKeyPtr, std::string> Factory::DoConnKeyFromVal(const zeek::Val& v) const {
+    if ( v.GetType() != id::conn_id )
+        return zeek::unexpected<std::string>{"unexpected value type"};
+
+    auto ck = zeek::conn_key::fivetuple::Factory::DoConnKeyFromVal(v);
+    if ( ! ck.has_value() )
+        return ck;
+
+    int vxlan_vni_offset = VxlanVniConnKey::GetVxlanVniOffset();
+    static int ctx_offset = id::conn_id->FieldOffset("ctx");
+
+    auto* k = static_cast<VxlanVniConnKey*>(ck.value().get());
+    auto* ctx = v.AsRecordVal()->GetFieldAs<zeek::RecordVal>(ctx_offset);
+
+    if ( vxlan_vni_offset < 0 )
+        return zeek::unexpected<std::string>{"missing vlxan_vni field"};
+
+    if ( ctx->HasField(vxlan_vni_offset) )
+        k->key.vxlan_vni = ctx->GetFieldAs<zeek::CountVal>(vxlan_vni_offset);
+
+    return ck;
+}
+
+} // namespace zeek::conn_key::vxlan_vni_fivetuple

doc/devel/plugins/connkey-vxlan-fivetuple-plugin-src/src/Factory.h

@@ -0,0 +1,18 @@
+#pragma once
+
+#include "zeek/ConnKey.h"
+#include "zeek/packet_analysis/protocol/ip/conn_key/fivetuple/Factory.h"
+
+namespace zeek::conn_key::vxlan_vni_fivetuple {
+
+class Factory : public zeek::conn_key::fivetuple::Factory {
+public:
+    static zeek::conn_key::FactoryPtr Instantiate() { return std::make_unique<Factory>(); }
+
+private:
+    // Returns a VxlanVniConnKey instance.
+    zeek::ConnKeyPtr DoNewConnKey() const override;
+    zeek::expected<zeek::ConnKeyPtr, std::string> DoConnKeyFromVal(const zeek::Val& v) const override;
+};
+
+} // namespace zeek::conn_key::vxlan_vni_fivetuple

doc/devel/plugins/connkey-vxlan-fivetuple-plugin-src/src/Plugin.cc

@@ -0,0 +1,26 @@
+
+#include "Plugin.h"
+
+#include <zeek/conn_key/Component.h>
+
+#include "Factory.h"
+
+namespace plugin {
+namespace Zeek_ConnKey_Vxlan_Vni_Fivetuple {
+Plugin plugin;
+}
+} // namespace plugin
+
+using namespace plugin::Zeek_ConnKey_Vxlan_Vni_Fivetuple;
+
+zeek::plugin::Configuration Plugin::Configure() {
+    zeek::plugin::Configuration config;
+    config.name = "Zeek::ConnKey_Vxlan_Vni_Fivetuple";
+    config.description = "ConnKey implementation using the most outer VXLAN VNI";
+    config.version = {0, 1, 0};
+
+    AddComponent(new zeek::conn_key::Component("VXLAN_VNI_FIVETUPLE",
+                                               zeek::conn_key::vxlan_vni_fivetuple::Factory::Instantiate));
+
+    return config;
+}

doc/devel/plugins/connkey-vxlan-fivetuple-plugin-src/src/Plugin.h

@@ -0,0 +1,17 @@
+
+#pragma once
+
+#include <zeek/plugin/Plugin.h>
+
+namespace plugin {
+namespace Zeek_ConnKey_Vxlan_Vni_Fivetuple {
+
+class Plugin : public zeek::plugin::Plugin {
+protected:
+    zeek::plugin::Configuration Configure() override;
+};
+
+extern Plugin plugin;
+
+} // namespace Zeek_ConnKey_Vxlan_Vni_Fivetuple
+} // namespace plugin

doc/devel/plugins/event-metadata-plugin-src/.gitignore

@@ -0,0 +1,3 @@
+build
+*.log
+.state

doc/devel/plugins/event-metadata-plugin-src/CMakeLists.txt

@@ -0,0 +1,13 @@
+cmake_minimum_required(VERSION 3.15 FATAL_ERROR)
+
+project(ZeekPluginEventLatency)
+
+include(ZeekPlugin)
+
+zeek_add_plugin(
+    Zeek
+    EventLatency
+    SOURCES
+    src/Plugin.cc
+    SCRIPT_FILES scripts/__load__.zeek
+)

doc/devel/plugins/event-metadata-plugin-src/COPYING

@@ -0,0 +1,26 @@
+Copyright (c) 2025 by the Zeek Project. All rights reserved.
+
+Redistribution and use in source and binary forms, with or without modification,
+are permitted provided that the following conditions are met:
+
+1. Redistributions of source code must retain the above copyright notice, this
+   list of conditions and the following disclaimer.
+
+2. Redistributions in binary form must reproduce the above copyright notice,
+   this list of conditions and the following disclaimer in the documentation
+   and/or other materials provided with the distribution.
+
+3. Neither the name of the copyright holder nor the names of its contributors
+   may be used to endorse or promote products derived from this software
+   without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
+ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR
+ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
+ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

doc/devel/plugins/event-metadata-plugin-src/Makefile

@@ -0,0 +1,23 @@
+#
+# Convenience Makefile providing a few common top-level targets.
+#
+
+cmake_build_dir=build
+arch=`uname -s | tr A-Z a-z`-`uname -m`
+
+all: build-it
+
+build-it:
+	( cd $(cmake_build_dir) && make )
+
+install:
+	( cd $(cmake_build_dir) && make install )
+
+clean:
+	( cd $(cmake_build_dir) && make clean )
+
+distclean:
+	rm -rf $(cmake_build_dir)
+
+test:
+	make -C tests

doc/devel/plugins/event-metadata-plugin-src/VERSION

@@ -0,0 +1 @@
+0.1.0

doc/devel/plugins/event-metadata-plugin-src/configure

@@ -0,0 +1,193 @@
+#!/bin/sh
+#
+# Wrapper for viewing/setting options that the plugin's CMake
+# scripts will recognize.
+#
+# Don't edit this. Edit configure.plugin to add plugin-specific options.
+#
+
+set -e
+command="$0 $*"
+
+if [ -e $(dirname $0)/configure.plugin ]; then
+    # Include custom additions.
+    . $(dirname $0)/configure.plugin
+fi
+
+usage() {
+
+    cat 1>&2 <<EOF
+Usage: $0 [OPTIONS]
+
+  Plugin Options:
+    --cmake=PATH               Path to CMake binary
+    --zeek-dist=DIR            Path to Zeek source tree
+    --install-root=DIR         Path where to install plugin into
+    --with-binpac=DIR          Path to BinPAC installation root
+    --with-broker=DIR          Path to Broker installation root
+    --with-bifcl=PATH          Path to bifcl executable
+    --enable-debug             Compile in debugging mode
+    --disable-cpp-tests        Don't build C++ unit tests
+EOF
+
+    if type plugin_usage >/dev/null 2>&1; then
+        plugin_usage 1>&2
+    fi
+
+    echo
+
+    exit 1
+}
+
+# Function to append a CMake cache entry definition to the
+# CMakeCacheEntries variable
+#   $1 is the cache entry variable name
+#   $2 is the cache entry variable type
+#   $3 is the cache entry variable value
+append_cache_entry() {
+    CMakeCacheEntries="$CMakeCacheEntries -D $1:$2=$3"
+}
+
+# set defaults
+builddir=build
+zeekdist=""
+installroot="default"
+zeek_plugin_begin_opts=""
+CMakeCacheEntries=""
+
+while [ $# -ne 0 ]; do
+    case "$1" in
+        -*=*) optarg=$(echo "$1" | sed 's/[-_a-zA-Z0-9]*=//') ;;
+        *) optarg= ;;
+    esac
+
+    case "$1" in
+        --help | -h)
+            usage
+            ;;
+
+        --cmake=*)
+            CMakeCommand=$optarg
+            ;;
+
+        --zeek-dist=*)
+            zeekdist=$(cd $optarg && pwd)
+            ;;
+
+        --install-root=*)
+            installroot=$optarg
+            ;;
+
+        --with-binpac=*)
+            append_cache_entry BinPAC_ROOT_DIR PATH $optarg
+            binpac_root=$optarg
+            ;;
+
+        --with-broker=*)
+            append_cache_entry BROKER_ROOT_DIR PATH $optarg
+            broker_root=$optarg
+            ;;
+
+        --with-bifcl=*)
+            append_cache_entry BifCl_EXE PATH $optarg
+            ;;
+
+        --enable-debug)
+            append_cache_entry BRO_PLUGIN_ENABLE_DEBUG BOOL true
+            ;;
+
+        --disable-cpp-tests)
+            zeek_plugin_begin_opts="DISABLE_CPP_TESTS;$zeek_plugin_begin_opts"
+            ;;
+
+        *)
+            if type plugin_option >/dev/null 2>&1; then
+                plugin_option $1 && shift && continue
+            fi
+
+            echo "Invalid option '$1'.  Try $0 --help to see available options."
+            exit 1
+            ;;
+    esac
+    shift
+done
+
+if [ -z "$CMakeCommand" ]; then
+    # prefer cmake3 over "regular" cmake (cmake == cmake2 on RHEL)
+    if command -v cmake3 >/dev/null 2>&1; then
+        CMakeCommand="cmake3"
+    elif command -v cmake >/dev/null 2>&1; then
+        CMakeCommand="cmake"
+    else
+        echo "This plugin requires CMake, please install it first."
+        echo "Then you may use this script to configure the CMake build."
+        echo "Note: pass --cmake=PATH to use cmake in non-standard locations."
+        exit 1
+    fi
+fi
+
+if [ -z "$zeekdist" ]; then
+    if type zeek-config >/dev/null 2>&1; then
+        zeek_config="zeek-config"
+    else
+        echo "Either 'zeek-config' must be in PATH or '--zeek-dist=<path>' used"
+        exit 1
+    fi
+
+    append_cache_entry BRO_CONFIG_PREFIX PATH $(${zeek_config} --prefix)
+    append_cache_entry BRO_CONFIG_INCLUDE_DIR PATH $(${zeek_config} --include_dir)
+    append_cache_entry BRO_CONFIG_PLUGIN_DIR PATH $(${zeek_config} --plugin_dir)
+    append_cache_entry BRO_CONFIG_LIB_DIR PATH $(${zeek_config} --lib_dir)
+    append_cache_entry BRO_CONFIG_CMAKE_DIR PATH $(${zeek_config} --cmake_dir)
+    append_cache_entry CMAKE_MODULE_PATH PATH $(${zeek_config} --cmake_dir)
+
+    build_type=$(${zeek_config} --build_type)
+
+    if [ "$build_type" = "debug" ]; then
+        append_cache_entry BRO_PLUGIN_ENABLE_DEBUG BOOL true
+    fi
+
+    if [ -z "$binpac_root" ]; then
+        append_cache_entry BinPAC_ROOT_DIR PATH $(${zeek_config} --binpac_root)
+    fi
+
+    if [ -z "$broker_root" ]; then
+        append_cache_entry BROKER_ROOT_DIR PATH $(${zeek_config} --broker_root)
+    fi
+else
+    if [ ! -e "$zeekdist/zeek-path-dev.in" ]; then
+        echo "$zeekdist does not appear to be a valid Zeek source tree."
+        exit 1
+    fi
+
+    # BRO_DIST is the canonical/historical name used by plugin CMake scripts
+    # ZEEK_DIST doesn't serve a function at the moment, but set/provided anyway
+    append_cache_entry BRO_DIST PATH $zeekdist
+    append_cache_entry ZEEK_DIST PATH $zeekdist
+    append_cache_entry CMAKE_MODULE_PATH PATH $zeekdist/cmake
+fi
+
+if [ "$installroot" != "default" ]; then
+    mkdir -p $installroot
+    append_cache_entry BRO_PLUGIN_INSTALL_ROOT PATH $installroot
+fi
+
+if [ -n "$zeek_plugin_begin_opts" ]; then
+    append_cache_entry ZEEK_PLUGIN_BEGIN_OPTS STRING "$zeek_plugin_begin_opts"
+fi
+
+if type plugin_addl >/dev/null 2>&1; then
+    plugin_addl
+fi
+
+echo "Build Directory        : $builddir"
+echo "Zeek Source Directory   : $zeekdist"
+
+mkdir -p $builddir
+cd $builddir
+
+"$CMakeCommand" $CMakeCacheEntries ..
+
+echo "# This is the command used to configure this build" >config.status
+echo $command >>config.status
+chmod u+x config.status

doc/devel/plugins/event-metadata-plugin-src/scripts/__load__.zeek

@@ -0,0 +1,11 @@
+module EventLatency;
+
+redef enum EventMetadata::ID += {
+	## Identifier for the absolute time at which Zeek published this event.
+	WALLCLOCK_TIMESTAMP = 10001000,
+};
+
+event zeek_init()
+	{
+	assert EventMetadata::register(WALLCLOCK_TIMESTAMP, time);
+	}

doc/devel/plugins/event-metadata-plugin-src/scripts/__preload__.zeek

@@ -0,0 +1 @@
+# Empty

doc/devel/plugins/event-metadata-plugin-src/src/Plugin.cc

@@ -0,0 +1,65 @@
+
+#include "Plugin.h"
+
+#include <zeek/Event.h>
+#include <zeek/Val.h>
+#include <zeek/cluster/Backend.h>
+#include <zeek/plugin/Plugin.h>
+#include <zeek/telemetry/Manager.h>
+
+namespace plugin {
+namespace Zeek_EventLatency {
+Plugin plugin;
+}
+} // namespace plugin
+
+using namespace plugin::Zeek_EventLatency;
+
+zeek::plugin::Configuration Plugin::Configure() {
+    zeek::plugin::Configuration config;
+    config.name = "Zeek::EventLatency";
+    config.description = "Track remote event latencies";
+    config.version = {0, 1, 0};
+    EnableHook(zeek::plugin::HOOK_PUBLISH_EVENT);
+    EnableHook(zeek::plugin::HOOK_QUEUE_EVENT);
+    return config;
+}
+
+void Plugin::InitPostScript() {
+    double bounds[] = {0.0002, 0.0004, 0.0006, 0.0008, 0.0010, 0.0012, 0.0014, 0.0016, 0.0018, 0.0020};
+    histogram =
+        zeek::telemetry_mgr->HistogramInstance("zeek", "cluster_event_latency_seconds", {}, bounds, "event latency");
+}
+
+bool Plugin::HookPublishEvent(zeek::cluster::Backend& backend, const std::string& topic,
+                              zeek::cluster::detail::Event& event) {
+    static const auto& wallclock_id = zeek::id::find_val<zeek::EnumVal>("EventLatency::WALLCLOCK_TIMESTAMP");
+
+    auto now_val = zeek::make_intrusive<zeek::TimeVal>(zeek::util::current_time(/*real=*/true));
+
+    if ( ! event.AddMetadata(wallclock_id, now_val) )
+        zeek::reporter->FatalError("failed to add wallclock timestamp metadata");
+
+    return true;
+}
+
+bool Plugin::HookQueueEvent(zeek::Event* event) {
+    static const auto& wallclock_id = zeek::id::find_val<zeek::EnumVal>("EventLatency::WALLCLOCK_TIMESTAMP");
+
+    if ( event->Source() == zeek::util::detail::SOURCE_LOCAL )
+        return false;
+
+    auto timestamps = event->MetadataValues(wallclock_id);
+
+    if ( timestamps->Size() > 0 ) {
+        double remote_ts = timestamps->ValAt(0)->AsTime();
+        auto now = zeek::util::current_time(/*real=*/true);
+        auto latency = std::max(0.0, now - remote_ts);
+
+        histogram->Observe(latency);
+    }
+    else
+        zeek::reporter->Warning("missing wallclock timestamp metadata");
+
+    return false;
+}

doc/devel/plugins/event-metadata-plugin-src/src/Plugin.h

@@ -0,0 +1,29 @@
+
+#pragma once
+
+#include <zeek/plugin/Plugin.h>
+#include <zeek/telemetry/Histogram.h>
+
+namespace plugin {
+namespace Zeek_EventLatency {
+
+class Plugin : public zeek::plugin::Plugin {
+protected:
+    // Overridden from zeek::plugin::Plugin.
+    zeek::plugin::Configuration Configure() override;
+
+    void InitPostScript() override;
+
+    bool HookPublishEvent(zeek::cluster::Backend& backend, const std::string& topic,
+                          zeek::cluster::detail::Event& event) override;
+
+    bool HookQueueEvent(zeek::Event* event) override;
+
+private:
+    zeek::telemetry::HistogramPtr histogram;
+};
+
+extern Plugin plugin;
+
+} // namespace Zeek_EventLatency
+} // namespace plugin

doc/devel/plugins/event-metadata-plugin.rst

@@ -0,0 +1,103 @@
+.. _event-metadata-plugin:
+
+=====================
+Event Metadata Plugin
+=====================
+
+.. versionadded:: 8.0
+
+
+Zeek's plugin API allows adding metadata to Zeek events. In the Zeek-script
+layer, the :zeek:see:`EventMetadata::current` and :zeek:see:`EventMetadata::current_all`
+functions can be used to introspect metadata attached to events. In a Zeek cluster,
+metadata is transported via remote events for consumption by other Zeek nodes.
+This section describes the functionality in form of a tutorial. We'll
+be using custom event metadata to track the latency of Zeek events in a
+cluster and expose them as a Prometheus histogram.
+
+If you're unfamiliar with plugin development, head over to the
+:ref:`Writing Plugins <writing-plugins>` section. For more information
+about telemetry and Prometheus, see also the :ref:`Telemetry framework's <framework-telemetry>`
+documentation.
+
+
+Registering Metadata
+====================
+
+Initially, we make Zeek's core aware of the metadata to attach to events. This
+requires two steps.
+First, redefining the :zeek:see:`EventMetadata::ID` enumeration with our
+custom enumeration value ``WALLCLOCK_TIMESTAMP``. This is our metadata identifier.
+Its value represents the Unix timestamps when an event was published.
+Second, registering the metadata identifier with Zeek's :zeek:see:`time` type
+by calling :zeek:see:`EventMetadata::register` in a :zeek:see:`zeek_init` handler.
+This instructs Zeek to convert metadata items in received remote events with
+identifier ``10001000`` to a :zeek:see:`time` value.
+
+For simplicity, the second step is done in the plugin's ``scripts/__init__.zeek`` file
+that's loaded automatically when Zeek loads the plugin.
+
+.. literalinclude:: event-metadata-plugin-src/scripts/__load__.zeek
+   :caption: main.zeek
+   :language: zeek
+   :linenos:
+   :tab-width: 4
+
+The ``10001000`` represents the metadata identifier for serialization purposes. It
+needs to be unique and have a defined meaning and consistent type for a given Zeek
+deployment. Metadata identifiers below ``200`` are reserved for Zeek's internal use.
+Users are free to choose any other value. Zeek will fail to start or fail to
+register the type in the case of conflicting identifiers in third-party packages.
+
+
+Implementing the Plugin
+=======================
+
+Next, we implement the ``InitPostScript()``, ``HookPublishEvent()`` and
+``HookQueueEvent()`` methods in our plugin.
+In the ``InitPostScript()`` method, a histogram instance is initialized using
+Zeek's telemetry manager with hard-coded bounds. These define buckets for latency
+monitoring.
+The ``HookPublishEvent()`` method adds ``WALLCLOCK_TIMESTAMP`` metadata with
+the current time to the event, while the ``HookQueueEvent()`` method extracts
+the sender's timestamp and computes the latency based on its own local time.
+Finally, the latency is recorded with the histogram by calling ``Observe()``.
+
+
+.. literalinclude:: event-metadata-plugin-src/src/Plugin.cc
+   :caption: main.zeek
+   :language: zeek
+   :linenos:
+   :lines: 28-
+   :tab-width: 4
+
+
+Resulting Prometheus Metrics
+============================
+
+Deploying the plugin outlined above in a cluster and querying the manager's
+metrics endpoint presents the following result::
+
+    $ curl -s localhost:10001/metrics | grep '^zeek_cluster_event_latency'
+    zeek_cluster_event_latency_seconds_count{endpoint="manager"} 11281
+    zeek_cluster_event_latency_seconds_sum{endpoint="manager"} 7.960928916931152
+    zeek_cluster_event_latency_seconds_bucket{endpoint="manager",le="0.0002"} 37
+    zeek_cluster_event_latency_seconds_bucket{endpoint="manager",le="0.0004"} 583
+    zeek_cluster_event_latency_seconds_bucket{endpoint="manager",le="0.0005999999999999999"} 3858
+    zeek_cluster_event_latency_seconds_bucket{endpoint="manager",le="0.0008"} 7960
+    zeek_cluster_event_latency_seconds_bucket{endpoint="manager",le="0.001"} 10185
+    zeek_cluster_event_latency_seconds_bucket{endpoint="manager",le="0.0012"} 10957
+    zeek_cluster_event_latency_seconds_bucket{endpoint="manager",le="0.0014"} 11239
+    zeek_cluster_event_latency_seconds_bucket{endpoint="manager",le="0.0016"} 11269
+    zeek_cluster_event_latency_seconds_bucket{endpoint="manager",le="0.0018"} 11279
+    zeek_cluster_event_latency_seconds_bucket{endpoint="manager",le="0.002"} 11281
+    zeek_cluster_event_latency_seconds_bucket{endpoint="manager",le="+Inf"} 11281
+
+
+This example indicates that there were a total of 11281 latencies observed,
+the summed up latency was around 8 seconds, 37 events had a latency less or equal
+to 0.2 milliseconds, 583 with less or equal than 0.4 milliseconds and none
+that took more than 2 milliseconds.
+
+This sort of data is usually scraped and ingested by a `Prometheus server <https://prometheus.io/>`_  and
+then visualized using `Grafana <https://grafana.com/>`_.