Telemetry framework: move BIFs to the primary-bif stage

This moves the Telemetry framework's BIF-defined functionalit from the
secondary-BIFs stage to the primary one. That is, this functionality is now
available from the end of init-bare.zeek, not only after the end of
init-frameworks-and-bifs.zeek.

This allows us to use script-layer telemetry in our Zeek's own code that get
pulled in during init-frameworks-and-bifs.

This change splits up the BIF features into functions, constants, and types,
because that's the granularity most workable in Func.cc and NetVar. It also now
defines the Telemetry::MetricsType enum once, not redundantly in BIFs and script
layer.

Due to subtle load ordering issues between the telemetry and cluster frameworks
this pushes the redef stage of Telemetry::metrics_port and address into
base/frameworks/telemetry/options.zeek, which is loaded sufficiently late in
init-frameworks-and-bifs.zeek to sidestep those issues. (When not doing this,
the effect is that the redef in telemetry/main.zeek doesn't yet find the
cluster-provided values, and Zeek does not end up listening on these ports.)

The need to add basic Zeek headers in script_opt/ZAM/ZBody.cc as a side-effect
of this is curious, but looks harmless.

Also includes baseline updates for the usual btests and adds a few doc strings.

(cherry picked from commit 71f7e89974)
This commit is contained in:
Christian Kreibich 2024-10-17 15:25:24 -07:00
parent 5503688758
commit 2ad80f8fb2
20 changed files with 99 additions and 63 deletions

View file

@ -5,28 +5,10 @@
##! enabled by setting :zeek:see:`Telemetry::metrics_port`.
@load base/misc/version
@load base/frameworks/cluster
@load base/frameworks/telemetry/options
@load base/bif/telemetry_functions.bif
module Telemetry;
# In a cluster configuration, open the port number for metrics
# from the cluster node configuration for exporting data to
# Prometheus.
#
# The manager node will also provide a ``/services.json`` endpoint
# for the HTTP Service Discovery system in Prometheus to use for
# configuration. This endpoint will include information for all of
# the other nodes in the cluster.
@if ( Cluster::is_enabled() )
redef Telemetry::metrics_endpoint_name = Cluster::node;
@if ( Cluster::local_node_metrics_port() != 0/unknown )
redef Telemetry::metrics_port = Cluster::local_node_metrics_port();
@endif
@endif
export {
## Alias for a vector of label values.
type labels_vector: vector of string;

View file

@ -1,9 +1,10 @@
module Telemetry;
##! Configurable settings for the Telemetry framework.
##!
##! These reside separately from the main framework so that they can be loaded
##! in bare mode without all of the framework. This allows things like the
##! plugins.hooks test to see the options without needing the rest.
# This file contains the options for the Telemetry framework. These are kept
# separately so that they can be loaded in bare mode without loading all of
# the rest of the framework. This allows things like the plugins.hooks test
# to see the options without needing the rest.
module Telemetry;
export {
## Address used to make metric data available to Prometheus scrapers via
@ -19,3 +20,23 @@ export {
## defaults to the name of the node in the cluster configuration.
const metrics_endpoint_name = "" &redef;
}
# When running a cluster, use the metrics port from the cluster node
# configuration for exporting data to Prometheus.
#
# The manager node will also provide a ``/services.json`` endpoint
# for the HTTP Service Discovery system in Prometheus to use for
# configuration. This endpoint will include information for all of
# the other nodes in the cluster.
# We do this here, and not in main.zeek, to avoid ordering issues when loading
# the telemetry and cluster frameworks. This applies even in bare mode, per
# init-frameworks-and-bifs.zeek: the cluster's metrics ports need to be available
# for the redefs to assign the correct values.
@if ( Cluster::is_enabled() )
redef Telemetry::metrics_endpoint_name = Cluster::node;
@if ( Cluster::local_node_metrics_port() != 0/unknown )
redef Telemetry::metrics_port = Cluster::local_node_metrics_port();
@endif
@endif

View file

@ -5787,15 +5787,11 @@ export {
const flowbuffer_contract_threshold = 2 * 1024 * 1024 &redef;
}
@load base/bif/telemetry_functions.bif
@load base/bif/telemetry_types.bif
module Telemetry;
export {
type MetricType: enum {
COUNTER,
GAUGE,
HISTOGRAM,
};
## Type that captures options used to create metrics.
type MetricOpts: record {
## The prefix (namespace) of the metric. Zeek uses the ``zeek``