Copy docs into Zeek repo directly

This is based on commit 2731def9159247e6da8a3191783c89683363689c from the
zeek-docs repo.

doc/devel/plugins/event-metadata-plugin.rst

@@ -0,0 +1,103 @@
+.. _event-metadata-plugin:
+
+=====================
+Event Metadata Plugin
+=====================
+
+.. versionadded:: 8.0
+
+
+Zeek's plugin API allows adding metadata to Zeek events. In the Zeek-script
+layer, the :zeek:see:`EventMetadata::current` and :zeek:see:`EventMetadata::current_all`
+functions can be used to introspect metadata attached to events. In a Zeek cluster,
+metadata is transported via remote events for consumption by other Zeek nodes.
+This section describes the functionality in form of a tutorial. We'll
+be using custom event metadata to track the latency of Zeek events in a
+cluster and expose them as a Prometheus histogram.
+
+If you're unfamiliar with plugin development, head over to the
+:ref:`Writing Plugins <writing-plugins>` section. For more information
+about telemetry and Prometheus, see also the :ref:`Telemetry framework's <framework-telemetry>`
+documentation.
+
+
+Registering Metadata
+====================
+
+Initially, we make Zeek's core aware of the metadata to attach to events. This
+requires two steps.
+First, redefining the :zeek:see:`EventMetadata::ID` enumeration with our
+custom enumeration value ``WALLCLOCK_TIMESTAMP``. This is our metadata identifier.
+Its value represents the Unix timestamps when an event was published.
+Second, registering the metadata identifier with Zeek's :zeek:see:`time` type
+by calling :zeek:see:`EventMetadata::register` in a :zeek:see:`zeek_init` handler.
+This instructs Zeek to convert metadata items in received remote events with
+identifier ``10001000`` to a :zeek:see:`time` value.
+
+For simplicity, the second step is done in the plugin's ``scripts/__init__.zeek`` file
+that's loaded automatically when Zeek loads the plugin.
+
+.. literalinclude:: event-metadata-plugin-src/scripts/__load__.zeek
+   :caption: main.zeek
+   :language: zeek
+   :linenos:
+   :tab-width: 4
+
+The ``10001000`` represents the metadata identifier for serialization purposes. It
+needs to be unique and have a defined meaning and consistent type for a given Zeek
+deployment. Metadata identifiers below ``200`` are reserved for Zeek's internal use.
+Users are free to choose any other value. Zeek will fail to start or fail to
+register the type in the case of conflicting identifiers in third-party packages.
+
+
+Implementing the Plugin
+=======================
+
+Next, we implement the ``InitPostScript()``, ``HookPublishEvent()`` and
+``HookQueueEvent()`` methods in our plugin.
+In the ``InitPostScript()`` method, a histogram instance is initialized using
+Zeek's telemetry manager with hard-coded bounds. These define buckets for latency
+monitoring.
+The ``HookPublishEvent()`` method adds ``WALLCLOCK_TIMESTAMP`` metadata with
+the current time to the event, while the ``HookQueueEvent()`` method extracts
+the sender's timestamp and computes the latency based on its own local time.
+Finally, the latency is recorded with the histogram by calling ``Observe()``.
+
+
+.. literalinclude:: event-metadata-plugin-src/src/Plugin.cc
+   :caption: main.zeek
+   :language: zeek
+   :linenos:
+   :lines: 28-
+   :tab-width: 4
+
+
+Resulting Prometheus Metrics
+============================
+
+Deploying the plugin outlined above in a cluster and querying the manager's
+metrics endpoint presents the following result::
+
+    $ curl -s localhost:10001/metrics | grep '^zeek_cluster_event_latency'
+    zeek_cluster_event_latency_seconds_count{endpoint="manager"} 11281
+    zeek_cluster_event_latency_seconds_sum{endpoint="manager"} 7.960928916931152
+    zeek_cluster_event_latency_seconds_bucket{endpoint="manager",le="0.0002"} 37
+    zeek_cluster_event_latency_seconds_bucket{endpoint="manager",le="0.0004"} 583
+    zeek_cluster_event_latency_seconds_bucket{endpoint="manager",le="0.0005999999999999999"} 3858
+    zeek_cluster_event_latency_seconds_bucket{endpoint="manager",le="0.0008"} 7960
+    zeek_cluster_event_latency_seconds_bucket{endpoint="manager",le="0.001"} 10185
+    zeek_cluster_event_latency_seconds_bucket{endpoint="manager",le="0.0012"} 10957
+    zeek_cluster_event_latency_seconds_bucket{endpoint="manager",le="0.0014"} 11239
+    zeek_cluster_event_latency_seconds_bucket{endpoint="manager",le="0.0016"} 11269
+    zeek_cluster_event_latency_seconds_bucket{endpoint="manager",le="0.0018"} 11279
+    zeek_cluster_event_latency_seconds_bucket{endpoint="manager",le="0.002"} 11281
+    zeek_cluster_event_latency_seconds_bucket{endpoint="manager",le="+Inf"} 11281
+
+
+This example indicates that there were a total of 11281 latencies observed,
+the summed up latency was around 8 seconds, 37 events had a latency less or equal
+to 0.2 milliseconds, 583 with less or equal than 0.4 milliseconds and none
+that took more than 2 milliseconds.
+
+This sort of data is usually scraped and ingested by a `Prometheus server <https://prometheus.io/>`_  and
+then visualized using `Grafana <https://grafana.com/>`_.