mirror of
https://github.com/zeek/zeek.git
synced 2025-10-17 14:08:20 +00:00
Copy docs into Zeek repo directly
This is based on commit 2731def9159247e6da8a3191783c89683363689c from the zeek-docs repo.
This commit is contained in:
Tim Wojtulewicz
parent
83f1e74643
commit
ded98cd373
1074 changed files with 169319 additions and 0 deletions
173
doc/scripts/policy/frameworks/analyzer/debug-logging.zeek.rst
Normal file
173
doc/scripts/policy/frameworks/analyzer/debug-logging.zeek.rst
Normal file
|
|
@ -0,0 +1,173 @@
|
|||
:tocdepth: 3
|
||||
|
||||
policy/frameworks/analyzer/debug-logging.zeek
|
||||
=============================================
|
||||
.. zeek:namespace:: Analyzer::DebugLogging
|
||||
|
||||
Logging analyzer confirmations and violations into analyzer-debug.log
|
||||
|
||||
:Namespace: Analyzer::DebugLogging
|
||||
:Imports: :doc:`base/frameworks/analyzer </scripts/base/frameworks/analyzer/index>`, :doc:`base/frameworks/config </scripts/base/frameworks/config/index>`, :doc:`base/frameworks/logging </scripts/base/frameworks/logging/index>`
|
||||
|
||||
Summary
|
||||
~~~~~~~
|
||||
Runtime Options
|
||||
###############
|
||||
================================================================================================ ======================================================================
|
||||
:zeek:id:`Analyzer::DebugLogging::enable`: :zeek:type:`bool` :zeek:attr:`&redef` Enable logging of analyzer violations and optionally confirmations
|
||||
when :zeek:see:`Analyzer::DebugLogging::include_confirmations` is set.
|
||||
:zeek:id:`Analyzer::DebugLogging::failure_data_max_size`: :zeek:type:`count` :zeek:attr:`&redef` If a violation contains information about the data causing it,
|
||||
include at most this many bytes of it in the log.
|
||||
:zeek:id:`Analyzer::DebugLogging::ignore_analyzers`: :zeek:type:`set` :zeek:attr:`&redef` Set of analyzers for which to not log confirmations or violations.
|
||||
:zeek:id:`Analyzer::DebugLogging::include_confirmations`: :zeek:type:`bool` :zeek:attr:`&redef` Enable analyzer_confirmation.
|
||||
:zeek:id:`Analyzer::DebugLogging::include_disabling`: :zeek:type:`bool` :zeek:attr:`&redef` Enable tracking of analyzers getting disabled.
|
||||
================================================================================================ ======================================================================
|
||||
|
||||
Types
|
||||
#####
|
||||
============================================================== ===========================================================================
|
||||
:zeek:type:`Analyzer::DebugLogging::Info`: :zeek:type:`record` The record type defining the columns to log in the analyzer logging stream.
|
||||
============================================================== ===========================================================================
|
||||
|
||||
Redefinitions
|
||||
#############
|
||||
======================================= ===========================================
|
||||
:zeek:type:`Log::ID`: :zeek:type:`enum` Add the analyzer logging stream identifier.
|
||||
|
||||
* :zeek:enum:`Analyzer::DebugLogging::LOG`
|
||||
======================================= ===========================================
|
||||
|
||||
Hooks
|
||||
#####
|
||||
=========================================================================== =============================================
|
||||
:zeek:id:`Analyzer::DebugLogging::log_policy`: :zeek:type:`Log::PolicyHook` A default logging policy hook for the stream.
|
||||
=========================================================================== =============================================
|
||||
|
||||
|
||||
Detailed Interface
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
Runtime Options
|
||||
###############
|
||||
.. zeek:id:: Analyzer::DebugLogging::enable
|
||||
:source-code: policy/frameworks/analyzer/debug-logging.zeek 46 46
|
||||
|
||||
:Type: :zeek:type:`bool`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``T``
|
||||
|
||||
Enable logging of analyzer violations and optionally confirmations
|
||||
when :zeek:see:`Analyzer::DebugLogging::include_confirmations` is set.
|
||||
|
||||
.. zeek:id:: Analyzer::DebugLogging::failure_data_max_size
|
||||
:source-code: policy/frameworks/analyzer/debug-logging.zeek 63 63
|
||||
|
||||
:Type: :zeek:type:`count`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``40``
|
||||
|
||||
If a violation contains information about the data causing it,
|
||||
include at most this many bytes of it in the log.
|
||||
|
||||
.. zeek:id:: Analyzer::DebugLogging::ignore_analyzers
|
||||
:source-code: policy/frameworks/analyzer/debug-logging.zeek 66 66
|
||||
|
||||
:Type: :zeek:type:`set` [:zeek:type:`AllAnalyzers::Tag`]
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``{}``
|
||||
|
||||
Set of analyzers for which to not log confirmations or violations.
|
||||
|
||||
.. zeek:id:: Analyzer::DebugLogging::include_confirmations
|
||||
:source-code: policy/frameworks/analyzer/debug-logging.zeek 53 53
|
||||
|
||||
:Type: :zeek:type:`bool`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``T``
|
||||
|
||||
Enable analyzer_confirmation. They are usually less interesting
|
||||
outside of development of analyzers or troubleshooting scenarios.
|
||||
Setting this option may also generated multiple log entries per
|
||||
connection, minimally one for each conn.log entry with a populated
|
||||
service field.
|
||||
|
||||
.. zeek:id:: Analyzer::DebugLogging::include_disabling
|
||||
:source-code: policy/frameworks/analyzer/debug-logging.zeek 59 59
|
||||
|
||||
:Type: :zeek:type:`bool`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``T``
|
||||
|
||||
Enable tracking of analyzers getting disabled. This is mostly
|
||||
interesting for troubleshooting of analyzers in DPD scenarios.
|
||||
Setting this option may also generated multiple log entries per
|
||||
connection.
|
||||
|
||||
Types
|
||||
#####
|
||||
.. zeek:type:: Analyzer::DebugLogging::Info
|
||||
:source-code: policy/frameworks/analyzer/debug-logging.zeek 17 42
|
||||
|
||||
:Type: :zeek:type:`record`
|
||||
|
||||
|
||||
.. zeek:field:: ts :zeek:type:`time` :zeek:attr:`&log`
|
||||
|
||||
Timestamp of confirmation or violation.
|
||||
|
||||
|
||||
.. zeek:field:: cause :zeek:type:`string` :zeek:attr:`&log`
|
||||
|
||||
What caused this log entry to be produced. This can
|
||||
currently be "violation", "confirmation", or "disabled".
|
||||
|
||||
|
||||
.. zeek:field:: analyzer_kind :zeek:type:`string` :zeek:attr:`&log`
|
||||
|
||||
The kind of analyzer involved. Currently "packet", "file"
|
||||
or "protocol".
|
||||
|
||||
|
||||
.. zeek:field:: analyzer_name :zeek:type:`string` :zeek:attr:`&log`
|
||||
|
||||
The name of the analyzer as produced by :zeek:see:`Analyzer::name`
|
||||
for the analyzer's tag.
|
||||
|
||||
|
||||
.. zeek:field:: uid :zeek:type:`string` :zeek:attr:`&log` :zeek:attr:`&optional`
|
||||
|
||||
Connection UID if available.
|
||||
|
||||
|
||||
.. zeek:field:: fuid :zeek:type:`string` :zeek:attr:`&log` :zeek:attr:`&optional`
|
||||
|
||||
File UID if available.
|
||||
|
||||
|
||||
.. zeek:field:: id :zeek:type:`conn_id` :zeek:attr:`&log` :zeek:attr:`&optional`
|
||||
|
||||
Connection identifier if available
|
||||
|
||||
|
||||
.. zeek:field:: failure_reason :zeek:type:`string` :zeek:attr:`&log` :zeek:attr:`&optional`
|
||||
|
||||
Failure or violation reason, if available.
|
||||
|
||||
|
||||
.. zeek:field:: failure_data :zeek:type:`string` :zeek:attr:`&log` :zeek:attr:`&optional`
|
||||
|
||||
Data causing failure or violation if available. Truncated
|
||||
to :zeek:see:`Analyzer::DebugLogging::failure_data_max_size`.
|
||||
|
||||
|
||||
The record type defining the columns to log in the analyzer logging stream.
|
||||
|
||||
Hooks
|
||||
#####
|
||||
.. zeek:id:: Analyzer::DebugLogging::log_policy
|
||||
:source-code: policy/frameworks/analyzer/debug-logging.zeek 14 14
|
||||
|
||||
:Type: :zeek:type:`Log::PolicyHook`
|
||||
|
||||
A default logging policy hook for the stream.
|
||||
|
||||
|
||||
152
doc/scripts/policy/frameworks/analyzer/detect-protocols.zeek.rst
Normal file
152
doc/scripts/policy/frameworks/analyzer/detect-protocols.zeek.rst
Normal file
|
|
@ -0,0 +1,152 @@
|
|||
:tocdepth: 3
|
||||
|
||||
policy/frameworks/analyzer/detect-protocols.zeek
|
||||
================================================
|
||||
.. zeek:namespace:: ProtocolDetector
|
||||
|
||||
Finds connections with protocols on non-standard ports with DPD.
|
||||
|
||||
:Namespace: ProtocolDetector
|
||||
:Imports: :doc:`base/frameworks/notice </scripts/base/frameworks/notice/index>`, :doc:`base/protocols/conn/removal-hooks.zeek </scripts/base/protocols/conn/removal-hooks.zeek>`, :doc:`base/utils/conn-ids.zeek </scripts/base/utils/conn-ids.zeek>`, :doc:`base/utils/site.zeek </scripts/base/utils/site.zeek>`
|
||||
|
||||
Summary
|
||||
~~~~~~~
|
||||
Runtime Options
|
||||
###############
|
||||
======================================================================================== =
|
||||
:zeek:id:`ProtocolDetector::minimum_duration`: :zeek:type:`interval` :zeek:attr:`&redef`
|
||||
:zeek:id:`ProtocolDetector::minimum_volume`: :zeek:type:`double` :zeek:attr:`&redef`
|
||||
:zeek:id:`ProtocolDetector::suppress_servers`: :zeek:type:`set` :zeek:attr:`&redef`
|
||||
:zeek:id:`ProtocolDetector::valids`: :zeek:type:`table` :zeek:attr:`&redef`
|
||||
======================================================================================== =
|
||||
|
||||
Constants
|
||||
#########
|
||||
================================================================== =
|
||||
:zeek:id:`ProtocolDetector::check_interval`: :zeek:type:`interval`
|
||||
================================================================== =
|
||||
|
||||
State Variables
|
||||
###############
|
||||
================================================================================================== =
|
||||
:zeek:id:`ProtocolDetector::servers`: :zeek:type:`table` :zeek:attr:`&read_expire` = ``14.0 days``
|
||||
================================================================================================== =
|
||||
|
||||
Types
|
||||
#####
|
||||
===================================================== =
|
||||
:zeek:type:`ProtocolDetector::dir`: :zeek:type:`enum`
|
||||
===================================================== =
|
||||
|
||||
Redefinitions
|
||||
#############
|
||||
============================================ ===============================================
|
||||
:zeek:type:`Notice::Type`: :zeek:type:`enum`
|
||||
|
||||
* :zeek:enum:`ProtocolDetector::Protocol_Found`
|
||||
|
||||
* :zeek:enum:`ProtocolDetector::Server_Found`
|
||||
============================================ ===============================================
|
||||
|
||||
Hooks
|
||||
#####
|
||||
======================================================================================== =======================================================
|
||||
:zeek:id:`ProtocolDetector::finalize_protocol_detection`: :zeek:type:`Conn::RemovalHook` Non-standard protocol port detection finalization hook.
|
||||
======================================================================================== =======================================================
|
||||
|
||||
Functions
|
||||
#########
|
||||
================================================================== =
|
||||
:zeek:id:`ProtocolDetector::found_protocol`: :zeek:type:`function`
|
||||
================================================================== =
|
||||
|
||||
|
||||
Detailed Interface
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
Runtime Options
|
||||
###############
|
||||
.. zeek:id:: ProtocolDetector::minimum_duration
|
||||
:source-code: policy/frameworks/analyzer/detect-protocols.zeek 56 56
|
||||
|
||||
:Type: :zeek:type:`interval`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``30.0 secs``
|
||||
|
||||
|
||||
.. zeek:id:: ProtocolDetector::minimum_volume
|
||||
:source-code: policy/frameworks/analyzer/detect-protocols.zeek 57 57
|
||||
|
||||
:Type: :zeek:type:`double`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``4000.0``
|
||||
|
||||
|
||||
.. zeek:id:: ProtocolDetector::suppress_servers
|
||||
:source-code: policy/frameworks/analyzer/detect-protocols.zeek 48 48
|
||||
|
||||
:Type: :zeek:type:`set` [:zeek:type:`AllAnalyzers::Tag`]
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``{}``
|
||||
|
||||
|
||||
.. zeek:id:: ProtocolDetector::valids
|
||||
:source-code: policy/frameworks/analyzer/detect-protocols.zeek 25 25
|
||||
|
||||
:Type: :zeek:type:`table` [:zeek:type:`AllAnalyzers::Tag`, :zeek:type:`addr`, :zeek:type:`port`] of :zeek:type:`ProtocolDetector::dir`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``{}``
|
||||
|
||||
|
||||
Constants
|
||||
#########
|
||||
.. zeek:id:: ProtocolDetector::check_interval
|
||||
:source-code: policy/frameworks/analyzer/detect-protocols.zeek 60 60
|
||||
|
||||
:Type: :zeek:type:`interval`
|
||||
:Default: ``5.0 secs``
|
||||
|
||||
|
||||
State Variables
|
||||
###############
|
||||
.. zeek:id:: ProtocolDetector::servers
|
||||
:source-code: policy/frameworks/analyzer/detect-protocols.zeek 69 69
|
||||
|
||||
:Type: :zeek:type:`table` [:zeek:type:`addr`, :zeek:type:`port`, :zeek:type:`string`] of :zeek:type:`set` [:zeek:type:`string`]
|
||||
:Attributes: :zeek:attr:`&read_expire` = ``14.0 days``
|
||||
:Default: ``{}``
|
||||
|
||||
|
||||
Types
|
||||
#####
|
||||
.. zeek:type:: ProtocolDetector::dir
|
||||
:source-code: policy/frameworks/analyzer/detect-protocols.zeek 23 24
|
||||
|
||||
:Type: :zeek:type:`enum`
|
||||
|
||||
.. zeek:enum:: ProtocolDetector::NONE ProtocolDetector::dir
|
||||
|
||||
.. zeek:enum:: ProtocolDetector::INCOMING ProtocolDetector::dir
|
||||
|
||||
.. zeek:enum:: ProtocolDetector::OUTGOING ProtocolDetector::dir
|
||||
|
||||
.. zeek:enum:: ProtocolDetector::BOTH ProtocolDetector::dir
|
||||
|
||||
|
||||
Hooks
|
||||
#####
|
||||
.. zeek:id:: ProtocolDetector::finalize_protocol_detection
|
||||
:source-code: policy/frameworks/analyzer/detect-protocols.zeek 189 199
|
||||
|
||||
:Type: :zeek:type:`Conn::RemovalHook`
|
||||
|
||||
Non-standard protocol port detection finalization hook.
|
||||
|
||||
Functions
|
||||
#########
|
||||
.. zeek:id:: ProtocolDetector::found_protocol
|
||||
:source-code: policy/frameworks/analyzer/detect-protocols.zeek 227 238
|
||||
|
||||
:Type: :zeek:type:`function` (c: :zeek:type:`connection`, atype: :zeek:type:`AllAnalyzers::Tag`, protocol: :zeek:type:`string`) : :zeek:type:`void`
|
||||
|
||||
|
||||
|
||||
|
|
@ -0,0 +1,56 @@
|
|||
:tocdepth: 3
|
||||
|
||||
policy/frameworks/analyzer/packet-segment-logging.zeek
|
||||
======================================================
|
||||
.. zeek:namespace:: Analyzer::Logging
|
||||
|
||||
This script enables logging of packet segment data when a protocol
|
||||
parsing violation is encountered. The amount of data from the
|
||||
packet logged is set by the :zeek:see:`Analyzer::Logging::packet_segment_size` variable.
|
||||
A caveat to logging packet data is that in some cases, the packet may
|
||||
not be the packet that actually caused the protocol violation.
|
||||
|
||||
:Namespace: Analyzer::Logging
|
||||
|
||||
Summary
|
||||
~~~~~~~
|
||||
Runtime Options
|
||||
###############
|
||||
======================================================================================= =====================================================
|
||||
:zeek:id:`Analyzer::Logging::packet_segment_size`: :zeek:type:`int` :zeek:attr:`&redef` Size of the packet segment to display in the DPD log.
|
||||
======================================================================================= =====================================================
|
||||
|
||||
Redefinitions
|
||||
#############
|
||||
========================================================= ==============================================================================
|
||||
:zeek:type:`Analyzer::Logging::Info`: :zeek:type:`record`
|
||||
|
||||
:New Fields: :zeek:type:`Analyzer::Logging::Info`
|
||||
|
||||
packet_segment: :zeek:type:`string` :zeek:attr:`&optional` :zeek:attr:`&log`
|
||||
A chunk of the payload that most likely resulted in the
|
||||
analyzer violation.
|
||||
:zeek:type:`connection`: :zeek:type:`record`
|
||||
|
||||
:New Fields: :zeek:type:`connection`
|
||||
|
||||
packet_segment: :zeek:type:`string` :zeek:attr:`&optional` :zeek:attr:`&log`
|
||||
A chunk of the payload that most likely resulted in a
|
||||
analyzer violation.
|
||||
========================================================= ==============================================================================
|
||||
|
||||
|
||||
Detailed Interface
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
Runtime Options
|
||||
###############
|
||||
.. zeek:id:: Analyzer::Logging::packet_segment_size
|
||||
:source-code: policy/frameworks/analyzer/packet-segment-logging.zeek 23 23
|
||||
|
||||
:Type: :zeek:type:`int`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``255``
|
||||
|
||||
Size of the packet segment to display in the DPD log.
|
||||
|
||||
|
||||
|
|
@ -0,0 +1,14 @@
|
|||
:tocdepth: 3
|
||||
|
||||
policy/frameworks/cluster/backend/zeromq/__load__.zeek
|
||||
======================================================
|
||||
|
||||
|
||||
:Imports: :doc:`policy/frameworks/cluster/backend/zeromq/main.zeek </scripts/policy/frameworks/cluster/backend/zeromq/main.zeek>`
|
||||
|
||||
Summary
|
||||
~~~~~~~
|
||||
|
||||
Detailed Interface
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
|
||||
|
|
@ -0,0 +1,17 @@
|
|||
:tocdepth: 3
|
||||
|
||||
policy/frameworks/cluster/backend/zeromq/connect.zeek
|
||||
=====================================================
|
||||
.. zeek:namespace:: Cluster::Backend::ZeroMQ
|
||||
|
||||
Establish ZeroMQ connectivity with the broker.
|
||||
|
||||
:Namespace: Cluster::Backend::ZeroMQ
|
||||
:Imports: :doc:`policy/frameworks/cluster/backend/zeromq/main.zeek </scripts/policy/frameworks/cluster/backend/zeromq/main.zeek>`
|
||||
|
||||
Summary
|
||||
~~~~~~~
|
||||
|
||||
Detailed Interface
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
|
||||
|
|
@ -0,0 +1,79 @@
|
|||
:orphan:
|
||||
|
||||
Package: policy/frameworks/cluster/backend/zeromq
|
||||
=================================================
|
||||
|
||||
|
||||
:doc:`/scripts/policy/frameworks/cluster/backend/zeromq/__load__.zeek`
|
||||
|
||||
|
||||
:doc:`/scripts/policy/frameworks/cluster/backend/zeromq/main.zeek`
|
||||
|
||||
ZeroMQ cluster backend support.
|
||||
|
||||
Overview
|
||||
|
||||
For publish-subscribe functionality, one node in the Zeek cluster spawns a
|
||||
thread running a central broker listening on a XPUB and XSUB socket.
|
||||
These sockets are connected via `zmq_proxy() <https://libzmq.readthedocs.io/en/latest/zmq_proxy.html>`_.
|
||||
All other nodes connect to this central broker with their own XSUB and
|
||||
XPUB sockets, establishing a global many-to-many publish-subscribe system
|
||||
where each node sees subscriptions and messages from all other nodes in a
|
||||
Zeek cluster. ZeroMQ's `publish-subscribe pattern <http://api.zeromq.org/4-2:zmq-socket#toc9>`_
|
||||
documentation may be a good starting point. Elsewhere in ZeroMQ's documentation,
|
||||
the central broker is also called `forwarder <http://api.zeromq.org/4-2:zmq-proxy#toc5>`_.
|
||||
|
||||
For remote logging functionality, the ZeroMQ `pipeline pattern <http://api.zeromq.org/4-2:zmq-socket#toc14>`_
|
||||
is used. All logger nodes listen on a PULL socket. Other nodes connect
|
||||
via PUSH sockets to all of the loggers. Concretely, remote logging
|
||||
functionality is not publish-subscribe, but instead leverages ZeroMQ's
|
||||
built-in load-balancing functionality provided by PUSH and PULL
|
||||
sockets.
|
||||
|
||||
The ZeroMQ cluster backend technically allows to run a non-Zeek central
|
||||
broker (it only needs to offer XPUB and XSUB sockets). Further, it is
|
||||
possible to run non-Zeek logger nodes. All a logger node needs to do is
|
||||
open a ZeroMQ PULL socket and interpret the format used by Zeek nodes
|
||||
to send their log writes.
|
||||
|
||||
Overload Behavior
|
||||
|
||||
The ZeroMQ cluster backend by default drops outgoing and incoming events
|
||||
when the Zeek cluster is overloaded. Dropping of outgoing events is governed
|
||||
by the :zeek:see:`Cluster::Backend::ZeroMQ::xpub_sndhwm` setting. This
|
||||
is the High Water Mark (HWM) for the local XPUB socket's queue. Once reached,
|
||||
any outgoing events are dropped until there's room in the socket's queue again.
|
||||
The metric ``zeek_cluster_zeromq_xpub_drops_total`` is incremented for every
|
||||
dropped event.
|
||||
|
||||
For incoming events, the :zeek:see:`Cluster::Backend::ZeroMQ::onloop_queue_hwm`
|
||||
setting is used. Remote events received via the local XSUB socket are first
|
||||
enqueued as raw event messages for processing on Zeek's main event loop.
|
||||
When this queue is full due to more remote events incoming than Zeek
|
||||
can possibly process in an event loop iteration, incoming events are dropped
|
||||
and the ``zeek_cluster_zeromq_onloop_drops_total`` metric is incremented.
|
||||
|
||||
Incoming log batches or subscription and unsubscription events are passed
|
||||
through the onloop queue, but the HWM does currently not apply to them. The
|
||||
assumption is that 1) these are not frequent and 2) more important than
|
||||
arbitrary publish-subscribe events.
|
||||
|
||||
To avoid dropping any events (e.g. for performance testing or offline PCAP
|
||||
processing), the recommended strategy is to set both
|
||||
:zeek:see:`Cluster::Backend::ZeroMQ::xpub_sndhwm` and
|
||||
:zeek:see:`Cluster::Backend::ZeroMQ::onloop_queue_hwm` to ``0``,
|
||||
disabling the HWM and dropping logic. It is up to the user to monitor CPU
|
||||
and memory usage of individual nodes to avoid overloading and running into
|
||||
out-of-memory situations.
|
||||
|
||||
As a Zeek operator, you should monitor ``zeek_cluster_zeromq_xpub_drops_total``
|
||||
and ``zeek_cluster_zeromq_onloop_drops_total``. Any non-zero values for these
|
||||
metrics indicate an overloaded Zeek cluster. See the the cluster telemetry
|
||||
options :zeek:see:`Cluster::Telemetry::core_metrics` and
|
||||
:zeek:see:`Cluster::Telemetry::websocket_metrics` for ways to get a better
|
||||
understanding about the events published and received.
|
||||
|
||||
:doc:`/scripts/policy/frameworks/cluster/backend/zeromq/connect.zeek`
|
||||
|
||||
Establish ZeroMQ connectivity with the broker.
|
||||
|
||||
|
|
@ -0,0 +1,608 @@
|
|||
:tocdepth: 3
|
||||
|
||||
policy/frameworks/cluster/backend/zeromq/main.zeek
|
||||
==================================================
|
||||
.. zeek:namespace:: Cluster::Backend::ZeroMQ
|
||||
|
||||
ZeroMQ cluster backend support.
|
||||
|
||||
Overview
|
||||
|
||||
For publish-subscribe functionality, one node in the Zeek cluster spawns a
|
||||
thread running a central broker listening on a XPUB and XSUB socket.
|
||||
These sockets are connected via `zmq_proxy() <https://libzmq.readthedocs.io/en/latest/zmq_proxy.html>`_.
|
||||
All other nodes connect to this central broker with their own XSUB and
|
||||
XPUB sockets, establishing a global many-to-many publish-subscribe system
|
||||
where each node sees subscriptions and messages from all other nodes in a
|
||||
Zeek cluster. ZeroMQ's `publish-subscribe pattern <http://api.zeromq.org/4-2:zmq-socket#toc9>`_
|
||||
documentation may be a good starting point. Elsewhere in ZeroMQ's documentation,
|
||||
the central broker is also called `forwarder <http://api.zeromq.org/4-2:zmq-proxy#toc5>`_.
|
||||
|
||||
For remote logging functionality, the ZeroMQ `pipeline pattern <http://api.zeromq.org/4-2:zmq-socket#toc14>`_
|
||||
is used. All logger nodes listen on a PULL socket. Other nodes connect
|
||||
via PUSH sockets to all of the loggers. Concretely, remote logging
|
||||
functionality is not publish-subscribe, but instead leverages ZeroMQ's
|
||||
built-in load-balancing functionality provided by PUSH and PULL
|
||||
sockets.
|
||||
|
||||
The ZeroMQ cluster backend technically allows to run a non-Zeek central
|
||||
broker (it only needs to offer XPUB and XSUB sockets). Further, it is
|
||||
possible to run non-Zeek logger nodes. All a logger node needs to do is
|
||||
open a ZeroMQ PULL socket and interpret the format used by Zeek nodes
|
||||
to send their log writes.
|
||||
|
||||
Overload Behavior
|
||||
|
||||
The ZeroMQ cluster backend by default drops outgoing and incoming events
|
||||
when the Zeek cluster is overloaded. Dropping of outgoing events is governed
|
||||
by the :zeek:see:`Cluster::Backend::ZeroMQ::xpub_sndhwm` setting. This
|
||||
is the High Water Mark (HWM) for the local XPUB socket's queue. Once reached,
|
||||
any outgoing events are dropped until there's room in the socket's queue again.
|
||||
The metric ``zeek_cluster_zeromq_xpub_drops_total`` is incremented for every
|
||||
dropped event.
|
||||
|
||||
For incoming events, the :zeek:see:`Cluster::Backend::ZeroMQ::onloop_queue_hwm`
|
||||
setting is used. Remote events received via the local XSUB socket are first
|
||||
enqueued as raw event messages for processing on Zeek's main event loop.
|
||||
When this queue is full due to more remote events incoming than Zeek
|
||||
can possibly process in an event loop iteration, incoming events are dropped
|
||||
and the ``zeek_cluster_zeromq_onloop_drops_total`` metric is incremented.
|
||||
|
||||
Incoming log batches or subscription and unsubscription events are passed
|
||||
through the onloop queue, but the HWM does currently not apply to them. The
|
||||
assumption is that 1) these are not frequent and 2) more important than
|
||||
arbitrary publish-subscribe events.
|
||||
|
||||
To avoid dropping any events (e.g. for performance testing or offline PCAP
|
||||
processing), the recommended strategy is to set both
|
||||
:zeek:see:`Cluster::Backend::ZeroMQ::xpub_sndhwm` and
|
||||
:zeek:see:`Cluster::Backend::ZeroMQ::onloop_queue_hwm` to ``0``,
|
||||
disabling the HWM and dropping logic. It is up to the user to monitor CPU
|
||||
and memory usage of individual nodes to avoid overloading and running into
|
||||
out-of-memory situations.
|
||||
|
||||
As a Zeek operator, you should monitor ``zeek_cluster_zeromq_xpub_drops_total``
|
||||
and ``zeek_cluster_zeromq_onloop_drops_total``. Any non-zero values for these
|
||||
metrics indicate an overloaded Zeek cluster. See the the cluster telemetry
|
||||
options :zeek:see:`Cluster::Telemetry::core_metrics` and
|
||||
:zeek:see:`Cluster::Telemetry::websocket_metrics` for ways to get a better
|
||||
understanding about the events published and received.
|
||||
|
||||
:Namespace: Cluster::Backend::ZeroMQ
|
||||
:Imports: :doc:`base/utils/addrs.zeek </scripts/base/utils/addrs.zeek>`
|
||||
|
||||
Summary
|
||||
~~~~~~~
|
||||
Redefinable Options
|
||||
###################
|
||||
=================================================================================================== ==================================================================
|
||||
:zeek:id:`Cluster::Backend::ZeroMQ::connect_log_endpoints`: :zeek:type:`vector` :zeek:attr:`&redef` Vector of ZeroMQ endpoints to connect to for logging.
|
||||
:zeek:id:`Cluster::Backend::ZeroMQ::connect_xpub_endpoint`: :zeek:type:`string` :zeek:attr:`&redef` The central broker's XPUB endpoint to connect to.
|
||||
:zeek:id:`Cluster::Backend::ZeroMQ::connect_xpub_nodrop`: :zeek:type:`bool` :zeek:attr:`&redef` Do not silently drop messages if high-water-mark is reached.
|
||||
:zeek:id:`Cluster::Backend::ZeroMQ::connect_xsub_endpoint`: :zeek:type:`string` :zeek:attr:`&redef` The central broker's XSUB endpoint to connect to.
|
||||
:zeek:id:`Cluster::Backend::ZeroMQ::debug_flags`: :zeek:type:`count` :zeek:attr:`&redef` Bitmask to enable low-level stderr based debug printing.
|
||||
:zeek:id:`Cluster::Backend::ZeroMQ::hello_expiration`: :zeek:type:`interval` :zeek:attr:`&redef` Expiration for hello state.
|
||||
:zeek:id:`Cluster::Backend::ZeroMQ::internal_topic_prefix`: :zeek:type:`string` :zeek:attr:`&redef` The topic prefix used for internal ZeroMQ specific communication.
|
||||
:zeek:id:`Cluster::Backend::ZeroMQ::ipv6`: :zeek:type:`bool` :zeek:attr:`&redef` Set ZMQ_IPV6 option.
|
||||
:zeek:id:`Cluster::Backend::ZeroMQ::linger_ms`: :zeek:type:`int` :zeek:attr:`&redef` Configure the ZeroMQ's sockets linger value.
|
||||
:zeek:id:`Cluster::Backend::ZeroMQ::listen_log_endpoint`: :zeek:type:`string` :zeek:attr:`&redef` PULL socket address to listen on for log messages.
|
||||
:zeek:id:`Cluster::Backend::ZeroMQ::listen_xpub_endpoint`: :zeek:type:`string` :zeek:attr:`&redef` XPUB listen endpoint for the central broker.
|
||||
:zeek:id:`Cluster::Backend::ZeroMQ::listen_xpub_nodrop`: :zeek:type:`bool` :zeek:attr:`&redef` Do not silently drop messages if high-water-mark is reached.
|
||||
:zeek:id:`Cluster::Backend::ZeroMQ::listen_xsub_endpoint`: :zeek:type:`string` :zeek:attr:`&redef` XSUB listen endpoint for the central broker.
|
||||
:zeek:id:`Cluster::Backend::ZeroMQ::log_immediate`: :zeek:type:`bool` :zeek:attr:`&redef` Configure ZeroMQ's immediate setting on PUSH sockets
|
||||
:zeek:id:`Cluster::Backend::ZeroMQ::log_rcvbuf`: :zeek:type:`int` :zeek:attr:`&redef` Kernel receive buffer size for log sockets.
|
||||
:zeek:id:`Cluster::Backend::ZeroMQ::log_rcvhwm`: :zeek:type:`int` :zeek:attr:`&redef` Receive high water mark value for the log PULL sockets.
|
||||
:zeek:id:`Cluster::Backend::ZeroMQ::log_sndbuf`: :zeek:type:`int` :zeek:attr:`&redef` Kernel transmit buffer size for log sockets.
|
||||
:zeek:id:`Cluster::Backend::ZeroMQ::log_sndhwm`: :zeek:type:`int` :zeek:attr:`&redef` Send high water mark value for the log PUSH sockets.
|
||||
:zeek:id:`Cluster::Backend::ZeroMQ::onloop_queue_hwm`: :zeek:type:`count` :zeek:attr:`&redef` Maximum number of incoming events queued for Zeek's event loop.
|
||||
:zeek:id:`Cluster::Backend::ZeroMQ::poll_max_messages`: :zeek:type:`count` :zeek:attr:`&redef` Messages to receive before yielding.
|
||||
:zeek:id:`Cluster::Backend::ZeroMQ::proxy_io_threads`: :zeek:type:`count` :zeek:attr:`&redef` How many IO threads to configure for the ZeroMQ context that
|
||||
acts as a central broker.
|
||||
:zeek:id:`Cluster::Backend::ZeroMQ::run_proxy_thread`: :zeek:type:`bool` :zeek:attr:`&redef` Toggle for running a central ZeroMQ XPUB-XSUB broker on this node.
|
||||
:zeek:id:`Cluster::Backend::ZeroMQ::xpub_sndbuf`: :zeek:type:`int` :zeek:attr:`&redef` Kernel transmit buffer size for the XPUB socket.
|
||||
:zeek:id:`Cluster::Backend::ZeroMQ::xpub_sndhwm`: :zeek:type:`int` :zeek:attr:`&redef` Send high water mark value for the XPUB socket.
|
||||
:zeek:id:`Cluster::Backend::ZeroMQ::xsub_rcvbuf`: :zeek:type:`int` :zeek:attr:`&redef` Kernel receive buffer size for the XSUB socket.
|
||||
:zeek:id:`Cluster::Backend::ZeroMQ::xsub_rcvhwm`: :zeek:type:`int` :zeek:attr:`&redef` Receive high water mark value for the XSUB socket.
|
||||
=================================================================================================== ==================================================================
|
||||
|
||||
State Variables
|
||||
###############
|
||||
================================================================================================= ================================
|
||||
:zeek:id:`Cluster::Backend::ZeroMQ::node_topic_prefix`: :zeek:type:`string` :zeek:attr:`&redef` The node topic prefix to use.
|
||||
:zeek:id:`Cluster::Backend::ZeroMQ::nodeid_topic_prefix`: :zeek:type:`string` :zeek:attr:`&redef` The node_id topic prefix to use.
|
||||
================================================================================================= ================================
|
||||
|
||||
Redefinitions
|
||||
#############
|
||||
================================================================================================================= =
|
||||
:zeek:id:`Cluster::Backend::ZeroMQ::run_proxy_thread`: :zeek:type:`bool` :zeek:attr:`&redef`
|
||||
:zeek:id:`Cluster::Telemetry::topic_normalizations`: :zeek:type:`table` :zeek:attr:`&ordered` :zeek:attr:`&redef`
|
||||
:zeek:id:`Cluster::backend`: :zeek:type:`Cluster::BackendTag` :zeek:attr:`&redef`
|
||||
:zeek:id:`Cluster::logger_pool_spec`: :zeek:type:`Cluster::PoolSpec` :zeek:attr:`&redef`
|
||||
:zeek:id:`Cluster::logger_topic`: :zeek:type:`string` :zeek:attr:`&redef`
|
||||
:zeek:id:`Cluster::manager_topic`: :zeek:type:`string` :zeek:attr:`&redef`
|
||||
:zeek:id:`Cluster::node_id`: :zeek:type:`function` :zeek:attr:`&redef`
|
||||
:zeek:id:`Cluster::node_topic`: :zeek:type:`function` :zeek:attr:`&redef`
|
||||
:zeek:id:`Cluster::nodeid_topic`: :zeek:type:`function` :zeek:attr:`&redef`
|
||||
:zeek:id:`Cluster::proxy_pool_spec`: :zeek:type:`Cluster::PoolSpec` :zeek:attr:`&redef`
|
||||
:zeek:id:`Cluster::proxy_topic`: :zeek:type:`string` :zeek:attr:`&redef`
|
||||
:zeek:id:`Cluster::worker_pool_spec`: :zeek:type:`Cluster::PoolSpec` :zeek:attr:`&redef`
|
||||
:zeek:id:`Cluster::worker_topic`: :zeek:type:`string` :zeek:attr:`&redef`
|
||||
================================================================================================================= =
|
||||
|
||||
Events
|
||||
######
|
||||
======================================================================= =================================================================
|
||||
:zeek:id:`Cluster::Backend::ZeroMQ::hello`: :zeek:type:`event` Low-level event send to a node in response to their subscription.
|
||||
:zeek:id:`Cluster::Backend::ZeroMQ::subscription`: :zeek:type:`event` Low-level event when a subscription is added.
|
||||
:zeek:id:`Cluster::Backend::ZeroMQ::unsubscription`: :zeek:type:`event` Low-level event when a subscription vanishes.
|
||||
======================================================================= =================================================================
|
||||
|
||||
|
||||
Detailed Interface
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
Redefinable Options
|
||||
###################
|
||||
.. zeek:id:: Cluster::Backend::ZeroMQ::connect_log_endpoints
|
||||
:source-code: policy/frameworks/cluster/backend/zeromq/main.zeek 86 86
|
||||
|
||||
:Type: :zeek:type:`vector` of :zeek:type:`string`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default:
|
||||
|
||||
::
|
||||
|
||||
[]
|
||||
|
||||
|
||||
Vector of ZeroMQ endpoints to connect to for logging.
|
||||
|
||||
A node's PUSH socket used for logging connects to each
|
||||
of the ZeroMQ endpoints listed in this vector.
|
||||
|
||||
.. zeek:id:: Cluster::Backend::ZeroMQ::connect_xpub_endpoint
|
||||
:source-code: policy/frameworks/cluster/backend/zeromq/main.zeek 73 73
|
||||
|
||||
:Type: :zeek:type:`string`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``"tcp://127.0.0.1:5556"``
|
||||
|
||||
The central broker's XPUB endpoint to connect to.
|
||||
|
||||
A node connects with its XSUB socket to the XPUB socket
|
||||
of the central broker.
|
||||
|
||||
.. zeek:id:: Cluster::Backend::ZeroMQ::connect_xpub_nodrop
|
||||
:source-code: policy/frameworks/cluster/backend/zeromq/main.zeek 250 250
|
||||
|
||||
:Type: :zeek:type:`bool`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``T``
|
||||
|
||||
Do not silently drop messages if high-water-mark is reached.
|
||||
|
||||
Whether to configure ``ZMQ_XPUB_NODROP`` on the XPUB socket
|
||||
connecting to the proxy to detect when sending a message fails
|
||||
due to reaching the high-water-mark. If you set this to **F**,
|
||||
then the XPUB drops metric will stop working as sending on the
|
||||
XPUB socket will always succeed. Unless you're developing on the
|
||||
ZeroMQ cluster backend, keep this set to **T**.
|
||||
|
||||
See ZeroMQ's `ZMQ_XPUB_NODROP documentation <http://api.zeromq.org/4-2:zmq-setsockopt#toc61>`_
|
||||
for more details.
|
||||
|
||||
.. zeek:id:: Cluster::Backend::ZeroMQ::connect_xsub_endpoint
|
||||
:source-code: policy/frameworks/cluster/backend/zeromq/main.zeek 80 80
|
||||
|
||||
:Type: :zeek:type:`string`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``"tcp://127.0.0.1:5555"``
|
||||
|
||||
The central broker's XSUB endpoint to connect to.
|
||||
|
||||
A node connects with its XPUB socket to the XSUB socket
|
||||
of the central broker.
|
||||
|
||||
.. zeek:id:: Cluster::Backend::ZeroMQ::debug_flags
|
||||
:source-code: policy/frameworks/cluster/backend/zeromq/main.zeek 280 280
|
||||
|
||||
:Type: :zeek:type:`count`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``0``
|
||||
|
||||
Bitmask to enable low-level stderr based debug printing.
|
||||
|
||||
poll: 1 (produce verbose zmq::poll() output)
|
||||
thread: 2 (produce thread related output)
|
||||
|
||||
Or values from the above list together and set debug_flags
|
||||
to the result. E.g. use 7 to select 4, 2 and 1. Only use this
|
||||
in development if something seems off. The thread used internally
|
||||
will produce output on stderr.
|
||||
|
||||
.. zeek:id:: Cluster::Backend::ZeroMQ::hello_expiration
|
||||
:source-code: policy/frameworks/cluster/backend/zeromq/main.zeek 320 320
|
||||
|
||||
:Type: :zeek:type:`interval`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``10.0 secs``
|
||||
|
||||
Expiration for hello state.
|
||||
|
||||
How long to wait before expiring information about
|
||||
subscriptions and hello messages from other
|
||||
nodes. These expirations trigger reporter warnings.
|
||||
|
||||
.. zeek:id:: Cluster::Backend::ZeroMQ::internal_topic_prefix
|
||||
:source-code: policy/frameworks/cluster/backend/zeromq/main.zeek 332 332
|
||||
|
||||
:Type: :zeek:type:`string`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``"zeek.zeromq.internal."``
|
||||
|
||||
The topic prefix used for internal ZeroMQ specific communication.
|
||||
|
||||
This is used for the "ready to publish callback" topics.
|
||||
|
||||
Zeek creates a short-lived subscription for a auto-generated
|
||||
topic name with this prefix and waits for it to be confirmed
|
||||
on its XPUB socket. Once this happens, the XPUB socket should've
|
||||
also received all other active subscriptions of other nodes in a
|
||||
cluster from the central XPUB/XSUB proxy and therefore can be
|
||||
deemed ready for publish operations.
|
||||
|
||||
.. zeek:id:: Cluster::Backend::ZeroMQ::ipv6
|
||||
:source-code: policy/frameworks/cluster/backend/zeromq/main.zeek 237 237
|
||||
|
||||
:Type: :zeek:type:`bool`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``T``
|
||||
|
||||
Set ZMQ_IPV6 option.
|
||||
|
||||
The ZeroMQ library has IPv6 support in ZeroMQ. For Zeek we enable it
|
||||
unconditionally such that listening or connecting with IPv6 just works.
|
||||
|
||||
See ZeroMQ's `ZMQ_IPV6 documentation <http://api.zeromq.org/4-2:zmq-setsockopt#toc23>`_
|
||||
for more details.
|
||||
|
||||
.. zeek:id:: Cluster::Backend::ZeroMQ::linger_ms
|
||||
:source-code: policy/frameworks/cluster/backend/zeromq/main.zeek 136 136
|
||||
|
||||
:Type: :zeek:type:`int`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``500``
|
||||
|
||||
Configure the ZeroMQ's sockets linger value.
|
||||
|
||||
The default used by libzmq is 30 seconds (30 000) which is very long
|
||||
when loggers vanish before workers during a shutdown, so we reduce
|
||||
this to 500 milliseconds by default.
|
||||
|
||||
A value of ``-1`` configures blocking forever, while ``0`` would
|
||||
immediately discard any pending messages.
|
||||
|
||||
See ZeroMQ's `ZMQ_LINGER documentation <http://api.zeromq.org/4-2:zmq-setsockopt#toc24>`_
|
||||
for more details.
|
||||
|
||||
.. zeek:id:: Cluster::Backend::ZeroMQ::listen_log_endpoint
|
||||
:source-code: policy/frameworks/cluster/backend/zeromq/main.zeek 123 123
|
||||
|
||||
:Type: :zeek:type:`string`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``""``
|
||||
|
||||
PULL socket address to listen on for log messages.
|
||||
|
||||
If empty, don't listen for log messages, otherwise
|
||||
a ZeroMQ address to bind to. E.g., ``tcp://127.0.0.1:5555``.
|
||||
|
||||
.. zeek:id:: Cluster::Backend::ZeroMQ::listen_xpub_endpoint
|
||||
:source-code: policy/frameworks/cluster/backend/zeromq/main.zeek 117 117
|
||||
|
||||
:Type: :zeek:type:`string`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``"tcp://127.0.0.1:5555"``
|
||||
|
||||
XPUB listen endpoint for the central broker.
|
||||
|
||||
This setting is used for the XPUB socket of the central broker started
|
||||
when :zeek:see:`Cluster::Backend::ZeroMQ::run_proxy_thread` is ``T``.
|
||||
|
||||
.. zeek:id:: Cluster::Backend::ZeroMQ::listen_xpub_nodrop
|
||||
:source-code: policy/frameworks/cluster/backend/zeromq/main.zeek 263 263
|
||||
|
||||
:Type: :zeek:type:`bool`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``T``
|
||||
|
||||
Do not silently drop messages if high-water-mark is reached.
|
||||
|
||||
Whether to configure ``ZMQ_XPUB_NODROP`` on the XPUB socket
|
||||
to detect when sending a message fails due to reaching
|
||||
the high-water-mark.
|
||||
|
||||
This setting applies to the XPUB/XSUB broker started when
|
||||
:zeek:see:`Cluster::Backend::ZeroMQ::run_proxy_thread` is ``T``.
|
||||
|
||||
See ZeroMQ's `ZMQ_XPUB_NODROP documentation <http://api.zeromq.org/4-2:zmq-setsockopt#toc61>`_
|
||||
for more details.
|
||||
|
||||
.. zeek:id:: Cluster::Backend::ZeroMQ::listen_xsub_endpoint
|
||||
:source-code: policy/frameworks/cluster/backend/zeromq/main.zeek 111 111
|
||||
|
||||
:Type: :zeek:type:`string`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``"tcp://127.0.0.1:5556"``
|
||||
|
||||
XSUB listen endpoint for the central broker.
|
||||
|
||||
This setting is used for the XSUB socket of the central broker started
|
||||
when :zeek:see:`Cluster::Backend::ZeroMQ::run_proxy_thread` is ``T``.
|
||||
|
||||
.. zeek:id:: Cluster::Backend::ZeroMQ::log_immediate
|
||||
:source-code: policy/frameworks/cluster/backend/zeromq/main.zeek 193 193
|
||||
|
||||
:Type: :zeek:type:`bool`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``F``
|
||||
|
||||
Configure ZeroMQ's immediate setting on PUSH sockets
|
||||
|
||||
Setting this to ``T`` will queue log writes only to completed
|
||||
connections. By default, log writes are queued to all potential
|
||||
endpoints listed in :zeek:see:`Cluster::Backend::ZeroMQ::connect_log_endpoints`.
|
||||
|
||||
See ZeroMQ's `ZMQ_IMMEDIATE documentation <http://api.zeromq.org/4-2:zmq-setsockopt#toc21>`_
|
||||
for more details.
|
||||
|
||||
.. zeek:id:: Cluster::Backend::ZeroMQ::log_rcvbuf
|
||||
:source-code: policy/frameworks/cluster/backend/zeromq/main.zeek 228 228
|
||||
|
||||
:Type: :zeek:type:`int`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``-1``
|
||||
|
||||
Kernel receive buffer size for log sockets.
|
||||
|
||||
Using -1 will use the kernel's default.
|
||||
|
||||
See ZeroMQ's `ZMQ_RCVBUF documentation <http://api.zeromq.org/4-2:zmq-setsockopt#toc34>`_
|
||||
for more details.
|
||||
|
||||
.. zeek:id:: Cluster::Backend::ZeroMQ::log_rcvhwm
|
||||
:source-code: policy/frameworks/cluster/backend/zeromq/main.zeek 213 213
|
||||
|
||||
:Type: :zeek:type:`int`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``1000``
|
||||
|
||||
Receive high water mark value for the log PULL sockets.
|
||||
|
||||
If reached, Zeek workers will block or drop messages.
|
||||
|
||||
See ZeroMQ's `ZMQ_RCVHWM documentation <http://api.zeromq.org/4-2:zmq-setsockopt#toc35>`_
|
||||
for more details.
|
||||
|
||||
TODO: Make action configurable (block vs drop)
|
||||
|
||||
.. zeek:id:: Cluster::Backend::ZeroMQ::log_sndbuf
|
||||
:source-code: policy/frameworks/cluster/backend/zeromq/main.zeek 220 220
|
||||
|
||||
:Type: :zeek:type:`int`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``-1``
|
||||
|
||||
Kernel transmit buffer size for log sockets.
|
||||
|
||||
Using -1 will use the kernel's default.
|
||||
|
||||
See ZeroMQ's `ZMQ_SNDBUF documentation <http://api.zeromq.org/4-2:zmq-setsockopt#toc45>`_.
|
||||
|
||||
.. zeek:id:: Cluster::Backend::ZeroMQ::log_sndhwm
|
||||
:source-code: policy/frameworks/cluster/backend/zeromq/main.zeek 203 203
|
||||
|
||||
:Type: :zeek:type:`int`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``1000``
|
||||
|
||||
Send high water mark value for the log PUSH sockets.
|
||||
|
||||
If reached, Zeek nodes will block or drop messages.
|
||||
|
||||
See ZeroMQ's `ZMQ_SNDHWM documentation <http://api.zeromq.org/4-2:zmq-setsockopt#toc46>`_
|
||||
for more details.
|
||||
|
||||
TODO: Make action configurable (block vs drop)
|
||||
|
||||
.. zeek:id:: Cluster::Backend::ZeroMQ::onloop_queue_hwm
|
||||
:source-code: policy/frameworks/cluster/backend/zeromq/main.zeek 183 183
|
||||
|
||||
:Type: :zeek:type:`count`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``10000``
|
||||
|
||||
Maximum number of incoming events queued for Zeek's event loop.
|
||||
|
||||
This constant defines the maximum number of remote events queued
|
||||
by the ZeroMQ cluster backend for Zeek's event loop to drain in
|
||||
one go. If you set this value to 0 (unlimited), consider closely
|
||||
CPU and memory usage of cluster nodes as high remote event rates
|
||||
may starve packet processing.
|
||||
|
||||
If more events are received than can fit the queue, new events will be
|
||||
dropped and the ``zeek_cluster_zeromq_onloop_drops_total`` metric
|
||||
incremented.
|
||||
|
||||
.. zeek:id:: Cluster::Backend::ZeroMQ::poll_max_messages
|
||||
:source-code: policy/frameworks/cluster/backend/zeromq/main.zeek 269 269
|
||||
|
||||
:Type: :zeek:type:`count`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``100``
|
||||
|
||||
Messages to receive before yielding.
|
||||
|
||||
Yield from the receive loop when this many messages have been
|
||||
received from one of the used sockets.
|
||||
|
||||
.. zeek:id:: Cluster::Backend::ZeroMQ::proxy_io_threads
|
||||
:source-code: policy/frameworks/cluster/backend/zeromq/main.zeek 105 105
|
||||
|
||||
:Type: :zeek:type:`count`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``2``
|
||||
|
||||
How many IO threads to configure for the ZeroMQ context that
|
||||
acts as a central broker.
|
||||
See ZeroMQ's `ZMQ_IO_THREADS documentation <http://api.zeromq.org/4-2:zmq-ctx-set#toc4>`_
|
||||
and the `I/O threads <https://zguide.zeromq.org/docs/chapter2/#I-O-Threads>`_
|
||||
section in the ZeroMQ guide for details.
|
||||
|
||||
.. zeek:id:: Cluster::Backend::ZeroMQ::run_proxy_thread
|
||||
:source-code: policy/frameworks/cluster/backend/zeromq/main.zeek 97 97
|
||||
|
||||
:Type: :zeek:type:`bool`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``F``
|
||||
:Redefinition: from :doc:`/scripts/policy/frameworks/cluster/backend/zeromq/main.zeek`
|
||||
|
||||
``=``::
|
||||
|
||||
Cluster::local_node_type() == Cluster::MANAGER
|
||||
|
||||
|
||||
Toggle for running a central ZeroMQ XPUB-XSUB broker on this node.
|
||||
|
||||
If set to ``T``, :zeek:see:`Cluster::Backend::ZeroMQ::spawn_zmq_proxy_thread`
|
||||
is called during :zeek:see:`zeek_init`. The node will listen
|
||||
on :zeek:see:`Cluster::Backend::ZeroMQ::listen_xsub_endpoint` and
|
||||
:zeek:see:`Cluster::Backend::ZeroMQ::listen_xpub_endpoint` and
|
||||
forward subscriptions and messages between nodes.
|
||||
|
||||
By default, this is set to ``T`` on the manager and ``F`` elsewhere.
|
||||
|
||||
.. zeek:id:: Cluster::Backend::ZeroMQ::xpub_sndbuf
|
||||
:source-code: policy/frameworks/cluster/backend/zeromq/main.zeek 153 153
|
||||
|
||||
:Type: :zeek:type:`int`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``-1``
|
||||
|
||||
Kernel transmit buffer size for the XPUB socket.
|
||||
|
||||
Using -1 will use the kernel's default.
|
||||
|
||||
See ZeroMQ's `ZMQ_SNDBUF documentation <http://api.zeromq.org/4-2:zmq-setsockopt#toc45>`_
|
||||
for more details.
|
||||
|
||||
.. zeek:id:: Cluster::Backend::ZeroMQ::xpub_sndhwm
|
||||
:source-code: policy/frameworks/cluster/backend/zeromq/main.zeek 145 145
|
||||
|
||||
:Type: :zeek:type:`int`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``1000``
|
||||
|
||||
Send high water mark value for the XPUB socket.
|
||||
|
||||
Events published when the XPUB queue is full will be dropped and the
|
||||
``zeek_cluster_zeromq_xpub_drops_total`` metric incremented.
|
||||
|
||||
See ZeroMQ's `ZMQ_SNDHWM documentation <http://api.zeromq.org/4-2:zmq-setsockopt#toc46>`_
|
||||
for more details.
|
||||
|
||||
.. zeek:id:: Cluster::Backend::ZeroMQ::xsub_rcvbuf
|
||||
:source-code: policy/frameworks/cluster/backend/zeromq/main.zeek 170 170
|
||||
|
||||
:Type: :zeek:type:`int`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``-1``
|
||||
|
||||
Kernel receive buffer size for the XSUB socket.
|
||||
|
||||
Using -1 will use the kernel's default.
|
||||
|
||||
See ZeroMQ's `ZMQ_RCVBUF documentation <http://api.zeromq.org/4-2:zmq-setsockopt#toc34>`_
|
||||
for more details.
|
||||
|
||||
.. zeek:id:: Cluster::Backend::ZeroMQ::xsub_rcvhwm
|
||||
:source-code: policy/frameworks/cluster/backend/zeromq/main.zeek 162 162
|
||||
|
||||
:Type: :zeek:type:`int`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``1000``
|
||||
|
||||
Receive high water mark value for the XSUB socket.
|
||||
|
||||
If reached, the Zeek node will start reporting back pressure
|
||||
to the central XPUB socket.
|
||||
|
||||
See ZeroMQ's `ZMQ_RCVHWM documentation <http://api.zeromq.org/4-2:zmq-setsockopt#toc35>`_
|
||||
for more details.
|
||||
|
||||
State Variables
|
||||
###############
|
||||
.. zeek:id:: Cluster::Backend::ZeroMQ::node_topic_prefix
|
||||
:source-code: policy/frameworks/cluster/backend/zeromq/main.zeek 283 283
|
||||
|
||||
:Type: :zeek:type:`string`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``"zeek.cluster.node"``
|
||||
|
||||
The node topic prefix to use.
|
||||
|
||||
.. zeek:id:: Cluster::Backend::ZeroMQ::nodeid_topic_prefix
|
||||
:source-code: policy/frameworks/cluster/backend/zeromq/main.zeek 286 286
|
||||
|
||||
:Type: :zeek:type:`string`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``"zeek.cluster.nodeid"``
|
||||
|
||||
The node_id topic prefix to use.
|
||||
|
||||
Events
|
||||
######
|
||||
.. zeek:id:: Cluster::Backend::ZeroMQ::hello
|
||||
:source-code: policy/frameworks/cluster/backend/zeromq/main.zeek 488 525
|
||||
|
||||
:Type: :zeek:type:`event` (name: :zeek:type:`string`, id: :zeek:type:`string`)
|
||||
|
||||
Low-level event send to a node in response to their subscription.
|
||||
|
||||
|
||||
:param name: The sending node's name in :zeek:see:`Cluster::nodes`.
|
||||
|
||||
|
||||
:param id: The sending node's identifier, as generated by :zeek:see:`Cluster::node_id`.
|
||||
|
||||
.. zeek:id:: Cluster::Backend::ZeroMQ::subscription
|
||||
:source-code: policy/frameworks/cluster/backend/zeromq/main.zeek 457 483
|
||||
|
||||
:Type: :zeek:type:`event` (topic: :zeek:type:`string`)
|
||||
|
||||
Low-level event when a subscription is added.
|
||||
|
||||
Every node observes all subscriptions from other nodes
|
||||
in a cluster through its XPUB socket. Whenever a new
|
||||
subscription topic is added, this event is raised with
|
||||
the topic.
|
||||
|
||||
|
||||
:param topic: The topic.
|
||||
|
||||
.. zeek:id:: Cluster::Backend::ZeroMQ::unsubscription
|
||||
:source-code: policy/frameworks/cluster/backend/zeromq/main.zeek 530 549
|
||||
|
||||
:Type: :zeek:type:`event` (topic: :zeek:type:`string`)
|
||||
|
||||
Low-level event when a subscription vanishes.
|
||||
|
||||
Every node observes all subscriptions from other nodes
|
||||
in a cluster through its XPUB socket. Whenever a subscription
|
||||
is removed from the local XPUB socket, this event is raised
|
||||
with the topic set to the removed subscription.
|
||||
|
||||
|
||||
:param topic: The topic.
|
||||
|
||||
|
||||
74
doc/scripts/policy/frameworks/cluster/experimental.zeek.rst
Normal file
74
doc/scripts/policy/frameworks/cluster/experimental.zeek.rst
Normal file
|
|
@ -0,0 +1,74 @@
|
|||
:tocdepth: 3
|
||||
|
||||
policy/frameworks/cluster/experimental.zeek
|
||||
===========================================
|
||||
.. zeek:namespace:: Cluster::Experimental
|
||||
|
||||
Experimental features of the Cluster framework.
|
||||
|
||||
:Namespace: Cluster::Experimental
|
||||
:Imports: :doc:`base/frameworks/cluster </scripts/base/frameworks/cluster/index>`
|
||||
|
||||
Summary
|
||||
~~~~~~~
|
||||
Events
|
||||
######
|
||||
========================================================================== =======================================================================
|
||||
:zeek:id:`Cluster::Experimental::cluster_started`: :zeek:type:`event` When using broker-enabled cluster framework, this event will be
|
||||
broadcasted from the manager once all nodes reported that they have set
|
||||
up all their outgoing connections to other cluster nodes based on the
|
||||
given cluster layout.
|
||||
:zeek:id:`Cluster::Experimental::node_fully_connected`: :zeek:type:`event` When using broker-enabled cluster framework, this event will be sent to
|
||||
the manager and raised locally, once a cluster node has successfully
|
||||
conducted cluster-level handshakes for all its outgoing connections to
|
||||
other cluster nodes based on the given cluster layout.
|
||||
========================================================================== =======================================================================
|
||||
|
||||
|
||||
Detailed Interface
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
Events
|
||||
######
|
||||
.. zeek:id:: Cluster::Experimental::cluster_started
|
||||
:source-code: policy/frameworks/cluster/nodes-experimental/manager.zeek 33 36
|
||||
|
||||
:Type: :zeek:type:`event` ()
|
||||
|
||||
When using broker-enabled cluster framework, this event will be
|
||||
broadcasted from the manager once all nodes reported that they have set
|
||||
up all their outgoing connections to other cluster nodes based on the
|
||||
given cluster layout.
|
||||
|
||||
.. warning::
|
||||
|
||||
There is no tracking of cluster node connectivity. Thus, there is
|
||||
no guarantee that all peerings still exist at the time of this event
|
||||
being raised.
|
||||
|
||||
.. zeek:id:: Cluster::Experimental::node_fully_connected
|
||||
:source-code: policy/frameworks/cluster/nodes-experimental/manager.zeek 16 31
|
||||
|
||||
:Type: :zeek:type:`event` (name: :zeek:type:`string`, id: :zeek:type:`string`, resending: :zeek:type:`bool`)
|
||||
|
||||
When using broker-enabled cluster framework, this event will be sent to
|
||||
the manager and raised locally, once a cluster node has successfully
|
||||
conducted cluster-level handshakes for all its outgoing connections to
|
||||
other cluster nodes based on the given cluster layout.
|
||||
|
||||
|
||||
:param name: The name of the now fully connected node.
|
||||
|
||||
|
||||
:param id: The identifier of the now fully connected node.
|
||||
|
||||
|
||||
:param resending: If true, the node has previously signaled that it is fully
|
||||
connected. This may happen in case the manager restarts.
|
||||
|
||||
.. warning::
|
||||
|
||||
There is no tracking of cluster node connectivity. Thus, there is
|
||||
no guarantee that all peerings still exist at the time of this event
|
||||
being raised.
|
||||
|
||||
|
||||
|
|
@ -0,0 +1,18 @@
|
|||
:tocdepth: 3
|
||||
|
||||
policy/frameworks/cluster/nodes-experimental/manager.zeek
|
||||
=========================================================
|
||||
.. zeek:namespace:: Cluster::Experimental
|
||||
|
||||
This script is loaded on the cluster manager to cover manager-related
|
||||
parts of experimental features.
|
||||
|
||||
:Namespace: Cluster::Experimental
|
||||
:Imports: :doc:`base/frameworks/cluster </scripts/base/frameworks/cluster/index>`, :doc:`policy/frameworks/cluster/experimental.zeek </scripts/policy/frameworks/cluster/experimental.zeek>`
|
||||
|
||||
Summary
|
||||
~~~~~~~
|
||||
|
||||
Detailed Interface
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
|
||||
|
|
@ -0,0 +1,32 @@
|
|||
:tocdepth: 3
|
||||
|
||||
policy/frameworks/conn_key/vlan_fivetuple.zeek
|
||||
==============================================
|
||||
|
||||
This script adapts Zeek's connection key to include 802.1Q VLAN and
|
||||
Q-in-Q tags, when available. Zeek normally ignores VLAN tags for connection
|
||||
lookups; this change makes it factor them in and also makes those VLAN tags
|
||||
part of the :zeek:see:`conn_id` record.
|
||||
|
||||
|
||||
Summary
|
||||
~~~~~~~
|
||||
Redefinitions
|
||||
#############
|
||||
========================================================================== =======================================================================
|
||||
:zeek:id:`ConnKey::factory`: :zeek:type:`ConnKey::Tag` :zeek:attr:`&redef`
|
||||
:zeek:type:`conn_id_ctx`: :zeek:type:`record`
|
||||
|
||||
:New Fields: :zeek:type:`conn_id_ctx`
|
||||
|
||||
vlan: :zeek:type:`int` :zeek:attr:`&log` :zeek:attr:`&optional`
|
||||
The outer VLAN for this connection, if applicable.
|
||||
|
||||
inner_vlan: :zeek:type:`int` :zeek:attr:`&log` :zeek:attr:`&optional`
|
||||
The inner VLAN for this connection, if applicable.
|
||||
========================================================================== =======================================================================
|
||||
|
||||
|
||||
Detailed Interface
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
|
||||
24
doc/scripts/policy/frameworks/control/controllee.zeek.rst
Normal file
24
doc/scripts/policy/frameworks/control/controllee.zeek.rst
Normal file
|
|
@ -0,0 +1,24 @@
|
|||
:tocdepth: 3
|
||||
|
||||
policy/frameworks/control/controllee.zeek
|
||||
=========================================
|
||||
.. zeek:namespace:: Control
|
||||
|
||||
The controllee portion of the control framework. Load this script if remote
|
||||
runtime control of the Zeek process is desired.
|
||||
|
||||
A controllee only needs to load the controllee script in addition
|
||||
to the specific analysis scripts desired. It may also need a node
|
||||
configured as a controller node in the communications nodes configuration::
|
||||
|
||||
zeek <scripts> frameworks/control/controllee
|
||||
|
||||
:Namespace: Control
|
||||
:Imports: :doc:`base/frameworks/broker </scripts/base/frameworks/broker/index>`, :doc:`base/frameworks/control </scripts/base/frameworks/control/index>`
|
||||
|
||||
Summary
|
||||
~~~~~~~
|
||||
|
||||
Detailed Interface
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
|
||||
23
doc/scripts/policy/frameworks/control/controller.zeek.rst
Normal file
23
doc/scripts/policy/frameworks/control/controller.zeek.rst
Normal file
|
|
@ -0,0 +1,23 @@
|
|||
:tocdepth: 3
|
||||
|
||||
policy/frameworks/control/controller.zeek
|
||||
=========================================
|
||||
.. zeek:namespace:: Control
|
||||
|
||||
This is a utility script that implements the controller interface for the
|
||||
control framework. It's intended to be run to control a remote Zeek
|
||||
and then shutdown.
|
||||
|
||||
It's intended to be used from the command line like this::
|
||||
|
||||
zeek <scripts> frameworks/control/controller Control::host=<host_addr> Control::host_port=<host_port> Control::cmd=<command> [Control::arg=<arg>]
|
||||
|
||||
:Namespace: Control
|
||||
:Imports: :doc:`base/frameworks/broker </scripts/base/frameworks/broker/index>`, :doc:`base/frameworks/control </scripts/base/frameworks/control/index>`
|
||||
|
||||
Summary
|
||||
~~~~~~~
|
||||
|
||||
Detailed Interface
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
|
||||
77
doc/scripts/policy/frameworks/files/detect-MHR.zeek.rst
Normal file
77
doc/scripts/policy/frameworks/files/detect-MHR.zeek.rst
Normal file
|
|
@ -0,0 +1,77 @@
|
|||
:tocdepth: 3
|
||||
|
||||
policy/frameworks/files/detect-MHR.zeek
|
||||
=======================================
|
||||
.. zeek:namespace:: TeamCymruMalwareHashRegistry
|
||||
|
||||
Detect file downloads that have hash values matching files in Team
|
||||
Cymru's Malware Hash Registry (https://www.team-cymru.com/mhr.html).
|
||||
|
||||
:Namespace: TeamCymruMalwareHashRegistry
|
||||
:Imports: :doc:`base/frameworks/files </scripts/base/frameworks/files/index>`, :doc:`base/frameworks/notice </scripts/base/frameworks/notice/index>`, :doc:`policy/frameworks/files/hash-all-files.zeek </scripts/policy/frameworks/files/hash-all-files.zeek>`
|
||||
|
||||
Summary
|
||||
~~~~~~~
|
||||
Runtime Options
|
||||
###############
|
||||
=================================================================================================== ====================================================================
|
||||
:zeek:id:`TeamCymruMalwareHashRegistry::match_file_types`: :zeek:type:`pattern` :zeek:attr:`&redef` File types to attempt matching against the Malware Hash Registry.
|
||||
:zeek:id:`TeamCymruMalwareHashRegistry::match_sub_url`: :zeek:type:`string` :zeek:attr:`&redef` The Match notice has a sub message with a URL where you can get more
|
||||
information about the file.
|
||||
:zeek:id:`TeamCymruMalwareHashRegistry::notice_threshold`: :zeek:type:`count` :zeek:attr:`&redef` The malware hash registry runs each malware sample through several
|
||||
A/V engines.
|
||||
=================================================================================================== ====================================================================
|
||||
|
||||
Redefinitions
|
||||
#############
|
||||
============================================ ===============================================================
|
||||
:zeek:type:`Notice::Type`: :zeek:type:`enum`
|
||||
|
||||
* :zeek:enum:`TeamCymruMalwareHashRegistry::Match`:
|
||||
The hash value of a file transferred over HTTP matched in the
|
||||
malware hash registry.
|
||||
============================================ ===============================================================
|
||||
|
||||
|
||||
Detailed Interface
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
Runtime Options
|
||||
###############
|
||||
.. zeek:id:: TeamCymruMalwareHashRegistry::match_file_types
|
||||
:source-code: policy/frameworks/files/detect-MHR.zeek 18 18
|
||||
|
||||
:Type: :zeek:type:`pattern`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default:
|
||||
|
||||
::
|
||||
|
||||
/^?((^?((^?((^?((^?((^?((^?(application\/x-dosexec)$?)|(^?(application\/vnd\.ms-cab-compressed)$?))$?)|(^?(application\/pdf)$?))$?)|(^?(application\/x-shockwave-flash)$?))$?)|(^?(application\/x-java-applet)$?))$?)|(^?(application\/jar)$?))$?)|(^?(video\/mp4)$?))$?/
|
||||
|
||||
|
||||
File types to attempt matching against the Malware Hash Registry.
|
||||
|
||||
.. zeek:id:: TeamCymruMalwareHashRegistry::match_sub_url
|
||||
:source-code: policy/frameworks/files/detect-MHR.zeek 29 29
|
||||
|
||||
:Type: :zeek:type:`string`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``"https://www.virustotal.com/gui/search/%s"``
|
||||
|
||||
The Match notice has a sub message with a URL where you can get more
|
||||
information about the file. The %s will be replaced with the SHA-1
|
||||
hash of the file.
|
||||
|
||||
.. zeek:id:: TeamCymruMalwareHashRegistry::notice_threshold
|
||||
:source-code: policy/frameworks/files/detect-MHR.zeek 35 35
|
||||
|
||||
:Type: :zeek:type:`count`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``10``
|
||||
|
||||
The malware hash registry runs each malware sample through several
|
||||
A/V engines. Team Cymru returns a percentage to indicate how
|
||||
many A/V engines flagged the sample as malicious. This threshold
|
||||
allows you to require a minimum detection rate.
|
||||
|
||||
|
||||
|
|
@ -0,0 +1,27 @@
|
|||
:tocdepth: 3
|
||||
|
||||
policy/frameworks/files/entropy-test-all-files.zeek
|
||||
===================================================
|
||||
.. zeek:namespace:: Files
|
||||
|
||||
|
||||
:Namespace: Files
|
||||
|
||||
Summary
|
||||
~~~~~~~
|
||||
Redefinitions
|
||||
#############
|
||||
================================================================= =======================================================================
|
||||
:zeek:type:`Files::Info`: :zeek:type:`record` :zeek:attr:`&redef`
|
||||
|
||||
:New Fields: :zeek:type:`Files::Info`
|
||||
|
||||
entropy: :zeek:type:`double` :zeek:attr:`&log` :zeek:attr:`&optional`
|
||||
The information density of the contents of the file,
|
||||
expressed as a number of bits per character.
|
||||
================================================================= =======================================================================
|
||||
|
||||
|
||||
Detailed Interface
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
|
||||
|
|
@ -0,0 +1,15 @@
|
|||
:tocdepth: 3
|
||||
|
||||
policy/frameworks/files/extract-all-files.zeek
|
||||
==============================================
|
||||
|
||||
Extract all files to disk.
|
||||
|
||||
:Imports: :doc:`base/files/extract </scripts/base/files/extract/index>`
|
||||
|
||||
Summary
|
||||
~~~~~~~
|
||||
|
||||
Detailed Interface
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
|
||||
15
doc/scripts/policy/frameworks/files/hash-all-files.zeek.rst
Normal file
15
doc/scripts/policy/frameworks/files/hash-all-files.zeek.rst
Normal file
|
|
@ -0,0 +1,15 @@
|
|||
:tocdepth: 3
|
||||
|
||||
policy/frameworks/files/hash-all-files.zeek
|
||||
===========================================
|
||||
|
||||
Perform MD5 and SHA1 hashing on all files.
|
||||
|
||||
:Imports: :doc:`base/files/hash </scripts/base/files/hash/index>`
|
||||
|
||||
Summary
|
||||
~~~~~~~
|
||||
|
||||
Detailed Interface
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
|
||||
23
doc/scripts/policy/frameworks/intel/do_expire.zeek.rst
Normal file
23
doc/scripts/policy/frameworks/intel/do_expire.zeek.rst
Normal file
|
|
@ -0,0 +1,23 @@
|
|||
:tocdepth: 3
|
||||
|
||||
policy/frameworks/intel/do_expire.zeek
|
||||
======================================
|
||||
.. zeek:namespace:: Intel
|
||||
|
||||
This script enables expiration for intelligence items.
|
||||
|
||||
:Namespace: Intel
|
||||
:Imports: :doc:`base/frameworks/intel </scripts/base/frameworks/intel/index>`
|
||||
|
||||
Summary
|
||||
~~~~~~~
|
||||
Redefinitions
|
||||
#############
|
||||
============================================================================ =
|
||||
:zeek:id:`Intel::item_expiration`: :zeek:type:`interval` :zeek:attr:`&redef`
|
||||
============================================================================ =
|
||||
|
||||
|
||||
Detailed Interface
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
|
||||
40
doc/scripts/policy/frameworks/intel/do_notice.zeek.rst
Normal file
40
doc/scripts/policy/frameworks/intel/do_notice.zeek.rst
Normal file
|
|
@ -0,0 +1,40 @@
|
|||
:tocdepth: 3
|
||||
|
||||
policy/frameworks/intel/do_notice.zeek
|
||||
======================================
|
||||
.. zeek:namespace:: Intel
|
||||
|
||||
This script enables notice generation for intelligence matches.
|
||||
|
||||
:Namespace: Intel
|
||||
:Imports: :doc:`base/frameworks/intel </scripts/base/frameworks/intel/index>`, :doc:`base/frameworks/notice </scripts/base/frameworks/notice/index>`
|
||||
|
||||
Summary
|
||||
~~~~~~~
|
||||
Redefinitions
|
||||
#############
|
||||
================================================= ===================================================================================
|
||||
:zeek:type:`Intel::MetaData`: :zeek:type:`record`
|
||||
|
||||
:New Fields: :zeek:type:`Intel::MetaData`
|
||||
|
||||
do_notice: :zeek:type:`bool` :zeek:attr:`&default` = ``F`` :zeek:attr:`&optional`
|
||||
A boolean value to allow the data itself to represent
|
||||
if the indicator that this metadata is attached to
|
||||
is notice worthy.
|
||||
|
||||
if_in: :zeek:type:`Intel::Where` :zeek:attr:`&optional`
|
||||
Restrictions on when notices are created to only create
|
||||
them if the *do_notice* field is T and the notice was
|
||||
seen in the indicated location.
|
||||
:zeek:type:`Notice::Type`: :zeek:type:`enum`
|
||||
|
||||
* :zeek:enum:`Intel::Notice`:
|
||||
This notice is generated when an intelligence
|
||||
indicator is denoted to be notice-worthy.
|
||||
================================================= ===================================================================================
|
||||
|
||||
|
||||
Detailed Interface
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
|
||||
28
doc/scripts/policy/frameworks/intel/removal.zeek.rst
Normal file
28
doc/scripts/policy/frameworks/intel/removal.zeek.rst
Normal file
|
|
@ -0,0 +1,28 @@
|
|||
:tocdepth: 3
|
||||
|
||||
policy/frameworks/intel/removal.zeek
|
||||
====================================
|
||||
.. zeek:namespace:: Intel
|
||||
|
||||
This script enables removal of intelligence items.
|
||||
|
||||
:Namespace: Intel
|
||||
:Imports: :doc:`base/frameworks/intel </scripts/base/frameworks/intel/index>`
|
||||
|
||||
Summary
|
||||
~~~~~~~
|
||||
Redefinitions
|
||||
#############
|
||||
================================================= ================================================================================
|
||||
:zeek:type:`Intel::MetaData`: :zeek:type:`record`
|
||||
|
||||
:New Fields: :zeek:type:`Intel::MetaData`
|
||||
|
||||
remove: :zeek:type:`bool` :zeek:attr:`&default` = ``F`` :zeek:attr:`&optional`
|
||||
A boolean value to indicate whether the item should be removed.
|
||||
================================================= ================================================================================
|
||||
|
||||
|
||||
Detailed Interface
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
|
||||
14
doc/scripts/policy/frameworks/intel/seen/__load__.zeek.rst
Normal file
14
doc/scripts/policy/frameworks/intel/seen/__load__.zeek.rst
Normal file
|
|
@ -0,0 +1,14 @@
|
|||
:tocdepth: 3
|
||||
|
||||
policy/frameworks/intel/seen/__load__.zeek
|
||||
==========================================
|
||||
|
||||
|
||||
:Imports: :doc:`policy/frameworks/intel/seen/conn-established.zeek </scripts/policy/frameworks/intel/seen/conn-established.zeek>`, :doc:`policy/frameworks/intel/seen/dns.zeek </scripts/policy/frameworks/intel/seen/dns.zeek>`, :doc:`policy/frameworks/intel/seen/file-hashes.zeek </scripts/policy/frameworks/intel/seen/file-hashes.zeek>`, :doc:`policy/frameworks/intel/seen/file-names.zeek </scripts/policy/frameworks/intel/seen/file-names.zeek>`, :doc:`policy/frameworks/intel/seen/http-headers.zeek </scripts/policy/frameworks/intel/seen/http-headers.zeek>`, :doc:`policy/frameworks/intel/seen/http-url.zeek </scripts/policy/frameworks/intel/seen/http-url.zeek>`, :doc:`policy/frameworks/intel/seen/pubkey-hashes.zeek </scripts/policy/frameworks/intel/seen/pubkey-hashes.zeek>`, :doc:`policy/frameworks/intel/seen/smb-filenames.zeek </scripts/policy/frameworks/intel/seen/smb-filenames.zeek>`, :doc:`policy/frameworks/intel/seen/smtp-url-extraction.zeek </scripts/policy/frameworks/intel/seen/smtp-url-extraction.zeek>`, :doc:`policy/frameworks/intel/seen/smtp.zeek </scripts/policy/frameworks/intel/seen/smtp.zeek>`, :doc:`policy/frameworks/intel/seen/ssl.zeek </scripts/policy/frameworks/intel/seen/ssl.zeek>`, :doc:`policy/frameworks/intel/seen/x509.zeek </scripts/policy/frameworks/intel/seen/x509.zeek>`
|
||||
|
||||
Summary
|
||||
~~~~~~~
|
||||
|
||||
Detailed Interface
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
|
||||
|
|
@ -0,0 +1,14 @@
|
|||
:tocdepth: 3
|
||||
|
||||
policy/frameworks/intel/seen/conn-established.zeek
|
||||
==================================================
|
||||
|
||||
|
||||
:Imports: :doc:`base/frameworks/intel </scripts/base/frameworks/intel/index>`, :doc:`policy/frameworks/intel/seen/where-locations.zeek </scripts/policy/frameworks/intel/seen/where-locations.zeek>`
|
||||
|
||||
Summary
|
||||
~~~~~~~
|
||||
|
||||
Detailed Interface
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
|
||||
14
doc/scripts/policy/frameworks/intel/seen/dns.zeek.rst
Normal file
14
doc/scripts/policy/frameworks/intel/seen/dns.zeek.rst
Normal file
|
|
@ -0,0 +1,14 @@
|
|||
:tocdepth: 3
|
||||
|
||||
policy/frameworks/intel/seen/dns.zeek
|
||||
=====================================
|
||||
|
||||
|
||||
:Imports: :doc:`base/frameworks/intel </scripts/base/frameworks/intel/index>`, :doc:`policy/frameworks/intel/seen/where-locations.zeek </scripts/policy/frameworks/intel/seen/where-locations.zeek>`
|
||||
|
||||
Summary
|
||||
~~~~~~~
|
||||
|
||||
Detailed Interface
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
|
||||
|
|
@ -0,0 +1,14 @@
|
|||
:tocdepth: 3
|
||||
|
||||
policy/frameworks/intel/seen/file-hashes.zeek
|
||||
=============================================
|
||||
|
||||
|
||||
:Imports: :doc:`base/frameworks/intel </scripts/base/frameworks/intel/index>`, :doc:`policy/frameworks/intel/seen/where-locations.zeek </scripts/policy/frameworks/intel/seen/where-locations.zeek>`
|
||||
|
||||
Summary
|
||||
~~~~~~~
|
||||
|
||||
Detailed Interface
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
|
||||
14
doc/scripts/policy/frameworks/intel/seen/file-names.zeek.rst
Normal file
14
doc/scripts/policy/frameworks/intel/seen/file-names.zeek.rst
Normal file
|
|
@ -0,0 +1,14 @@
|
|||
:tocdepth: 3
|
||||
|
||||
policy/frameworks/intel/seen/file-names.zeek
|
||||
============================================
|
||||
|
||||
|
||||
:Imports: :doc:`base/frameworks/intel </scripts/base/frameworks/intel/index>`, :doc:`policy/frameworks/intel/seen/where-locations.zeek </scripts/policy/frameworks/intel/seen/where-locations.zeek>`
|
||||
|
||||
Summary
|
||||
~~~~~~~
|
||||
|
||||
Detailed Interface
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
|
||||
|
|
@ -0,0 +1,14 @@
|
|||
:tocdepth: 3
|
||||
|
||||
policy/frameworks/intel/seen/http-headers.zeek
|
||||
==============================================
|
||||
|
||||
|
||||
:Imports: :doc:`base/frameworks/intel </scripts/base/frameworks/intel/index>`, :doc:`base/utils/addrs.zeek </scripts/base/utils/addrs.zeek>`, :doc:`policy/frameworks/intel/seen/where-locations.zeek </scripts/policy/frameworks/intel/seen/where-locations.zeek>`
|
||||
|
||||
Summary
|
||||
~~~~~~~
|
||||
|
||||
Detailed Interface
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
|
||||
14
doc/scripts/policy/frameworks/intel/seen/http-url.zeek.rst
Normal file
14
doc/scripts/policy/frameworks/intel/seen/http-url.zeek.rst
Normal file
|
|
@ -0,0 +1,14 @@
|
|||
:tocdepth: 3
|
||||
|
||||
policy/frameworks/intel/seen/http-url.zeek
|
||||
==========================================
|
||||
|
||||
|
||||
:Imports: :doc:`base/frameworks/intel </scripts/base/frameworks/intel/index>`, :doc:`base/protocols/http/utils.zeek </scripts/base/protocols/http/utils.zeek>`, :doc:`policy/frameworks/intel/seen/where-locations.zeek </scripts/policy/frameworks/intel/seen/where-locations.zeek>`
|
||||
|
||||
Summary
|
||||
~~~~~~~
|
||||
|
||||
Detailed Interface
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
|
||||
52
doc/scripts/policy/frameworks/intel/seen/index.rst
Normal file
52
doc/scripts/policy/frameworks/intel/seen/index.rst
Normal file
|
|
@ -0,0 +1,52 @@
|
|||
:orphan:
|
||||
|
||||
Package: policy/frameworks/intel/seen
|
||||
=====================================
|
||||
|
||||
Scripts that send data to the intelligence framework.
|
||||
|
||||
:doc:`/scripts/policy/frameworks/intel/seen/__load__.zeek`
|
||||
|
||||
|
||||
:doc:`/scripts/policy/frameworks/intel/seen/conn-established.zeek`
|
||||
|
||||
|
||||
:doc:`/scripts/policy/frameworks/intel/seen/where-locations.zeek`
|
||||
|
||||
|
||||
:doc:`/scripts/policy/frameworks/intel/seen/dns.zeek`
|
||||
|
||||
|
||||
:doc:`/scripts/policy/frameworks/intel/seen/file-hashes.zeek`
|
||||
|
||||
|
||||
:doc:`/scripts/policy/frameworks/intel/seen/file-names.zeek`
|
||||
|
||||
|
||||
:doc:`/scripts/policy/frameworks/intel/seen/http-headers.zeek`
|
||||
|
||||
|
||||
:doc:`/scripts/policy/frameworks/intel/seen/http-url.zeek`
|
||||
|
||||
|
||||
:doc:`/scripts/policy/frameworks/intel/seen/pubkey-hashes.zeek`
|
||||
|
||||
|
||||
:doc:`/scripts/policy/frameworks/intel/seen/ssl.zeek`
|
||||
|
||||
|
||||
:doc:`/scripts/policy/frameworks/intel/seen/smb-filenames.zeek`
|
||||
|
||||
|
||||
:doc:`/scripts/policy/frameworks/intel/seen/smtp.zeek`
|
||||
|
||||
|
||||
:doc:`/scripts/policy/frameworks/intel/seen/smtp-url-extraction.zeek`
|
||||
|
||||
|
||||
:doc:`/scripts/policy/frameworks/intel/seen/x509.zeek`
|
||||
|
||||
|
||||
:doc:`/scripts/policy/frameworks/intel/seen/manage-event-groups.zeek`
|
||||
|
||||
|
||||
|
|
@ -0,0 +1,46 @@
|
|||
:tocdepth: 3
|
||||
|
||||
policy/frameworks/intel/seen/manage-event-groups.zeek
|
||||
=====================================================
|
||||
.. zeek:namespace:: Intel
|
||||
|
||||
|
||||
:Namespace: Intel
|
||||
:Imports: :doc:`base/frameworks/reporter </scripts/base/frameworks/reporter/index>`, :doc:`policy/frameworks/intel/seen </scripts/policy/frameworks/intel/seen/index>`
|
||||
|
||||
Summary
|
||||
~~~~~~~
|
||||
Redefinable Options
|
||||
###################
|
||||
================================================================================= ============================================================
|
||||
:zeek:id:`Intel::manage_seen_event_groups`: :zeek:type:`bool` :zeek:attr:`&redef` Whether Intel event groups for the seen scripts are managed.
|
||||
================================================================================= ============================================================
|
||||
|
||||
|
||||
Detailed Interface
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
Redefinable Options
|
||||
###################
|
||||
.. zeek:id:: Intel::manage_seen_event_groups
|
||||
:source-code: policy/frameworks/intel/seen/manage-event-groups.zeek 21 21
|
||||
|
||||
:Type: :zeek:type:`bool`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``T``
|
||||
|
||||
Whether Intel event groups for the seen scripts are managed.
|
||||
|
||||
When loading this script, by default, all :zeek:see:`Intel::Type`
|
||||
event groups are disabled at startup and only enabled when indicators
|
||||
of corresponding types are loaded into the Intel framework's store.
|
||||
This allows to load the ``frameworks/intel/seen`` scripts without
|
||||
incurring event handling overhead when no Intel indicators are loaded.
|
||||
|
||||
One caveat is that the :zeek:see:`Intel::seen_policy` hook will not
|
||||
be invoked for indicator types that are not at all in the Intel
|
||||
framework's store. If you rely on :zeek:see:`Intel::seen_policy` to
|
||||
find unmatched indicators, do not not load this script, set this
|
||||
variable to ``F``, or insert dummy values of the types using
|
||||
:zeek:see:`Intel::insert`.
|
||||
|
||||
|
||||
|
|
@ -0,0 +1,14 @@
|
|||
:tocdepth: 3
|
||||
|
||||
policy/frameworks/intel/seen/pubkey-hashes.zeek
|
||||
===============================================
|
||||
|
||||
|
||||
:Imports: :doc:`base/frameworks/intel </scripts/base/frameworks/intel/index>`, :doc:`base/protocols/ssh </scripts/base/protocols/ssh/index>`, :doc:`policy/frameworks/intel/seen/where-locations.zeek </scripts/policy/frameworks/intel/seen/where-locations.zeek>`
|
||||
|
||||
Summary
|
||||
~~~~~~~
|
||||
|
||||
Detailed Interface
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
|
||||
|
|
@ -0,0 +1,14 @@
|
|||
:tocdepth: 3
|
||||
|
||||
policy/frameworks/intel/seen/smb-filenames.zeek
|
||||
===============================================
|
||||
|
||||
|
||||
:Imports: :doc:`base/frameworks/intel </scripts/base/frameworks/intel/index>`, :doc:`base/protocols/smb </scripts/base/protocols/smb/index>`, :doc:`policy/frameworks/intel/seen/where-locations.zeek </scripts/policy/frameworks/intel/seen/where-locations.zeek>`
|
||||
|
||||
Summary
|
||||
~~~~~~~
|
||||
|
||||
Detailed Interface
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
|
||||
|
|
@ -0,0 +1,14 @@
|
|||
:tocdepth: 3
|
||||
|
||||
policy/frameworks/intel/seen/smtp-url-extraction.zeek
|
||||
=====================================================
|
||||
|
||||
|
||||
:Imports: :doc:`base/frameworks/intel </scripts/base/frameworks/intel/index>`, :doc:`base/protocols/smtp </scripts/base/protocols/smtp/index>`, :doc:`base/utils/urls.zeek </scripts/base/utils/urls.zeek>`, :doc:`policy/frameworks/intel/seen/where-locations.zeek </scripts/policy/frameworks/intel/seen/where-locations.zeek>`
|
||||
|
||||
Summary
|
||||
~~~~~~~
|
||||
|
||||
Detailed Interface
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
|
||||
14
doc/scripts/policy/frameworks/intel/seen/smtp.zeek.rst
Normal file
14
doc/scripts/policy/frameworks/intel/seen/smtp.zeek.rst
Normal file
|
|
@ -0,0 +1,14 @@
|
|||
:tocdepth: 3
|
||||
|
||||
policy/frameworks/intel/seen/smtp.zeek
|
||||
======================================
|
||||
|
||||
|
||||
:Imports: :doc:`base/frameworks/intel </scripts/base/frameworks/intel/index>`, :doc:`base/protocols/smtp </scripts/base/protocols/smtp/index>`, :doc:`base/utils/email.zeek </scripts/base/utils/email.zeek>`, :doc:`policy/frameworks/intel/seen/where-locations.zeek </scripts/policy/frameworks/intel/seen/where-locations.zeek>`
|
||||
|
||||
Summary
|
||||
~~~~~~~
|
||||
|
||||
Detailed Interface
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
|
||||
14
doc/scripts/policy/frameworks/intel/seen/ssl.zeek.rst
Normal file
14
doc/scripts/policy/frameworks/intel/seen/ssl.zeek.rst
Normal file
|
|
@ -0,0 +1,14 @@
|
|||
:tocdepth: 3
|
||||
|
||||
policy/frameworks/intel/seen/ssl.zeek
|
||||
=====================================
|
||||
|
||||
|
||||
:Imports: :doc:`base/frameworks/intel </scripts/base/frameworks/intel/index>`, :doc:`base/protocols/ssl </scripts/base/protocols/ssl/index>`, :doc:`policy/frameworks/intel/seen/where-locations.zeek </scripts/policy/frameworks/intel/seen/where-locations.zeek>`
|
||||
|
||||
Summary
|
||||
~~~~~~~
|
||||
|
||||
Detailed Interface
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
|
||||
|
|
@ -0,0 +1,70 @@
|
|||
:tocdepth: 3
|
||||
|
||||
policy/frameworks/intel/seen/where-locations.zeek
|
||||
=================================================
|
||||
|
||||
|
||||
:Imports: :doc:`base/frameworks/intel </scripts/base/frameworks/intel/index>`
|
||||
|
||||
Summary
|
||||
~~~~~~~
|
||||
Redefinitions
|
||||
#############
|
||||
============================================ ===============================================
|
||||
:zeek:type:`Intel::Where`: :zeek:type:`enum`
|
||||
|
||||
* :zeek:enum:`Conn::IN_ORIG`
|
||||
|
||||
* :zeek:enum:`Conn::IN_RESP`
|
||||
|
||||
* :zeek:enum:`DNS::IN_REQUEST`
|
||||
|
||||
* :zeek:enum:`DNS::IN_RESPONSE`
|
||||
|
||||
* :zeek:enum:`Files::IN_HASH`
|
||||
|
||||
* :zeek:enum:`Files::IN_NAME`
|
||||
|
||||
* :zeek:enum:`HTTP::IN_HOST_HEADER`
|
||||
|
||||
* :zeek:enum:`HTTP::IN_REFERRER_HEADER`
|
||||
|
||||
* :zeek:enum:`HTTP::IN_URL`
|
||||
|
||||
* :zeek:enum:`HTTP::IN_USER_AGENT_HEADER`
|
||||
|
||||
* :zeek:enum:`HTTP::IN_X_FORWARDED_FOR_HEADER`
|
||||
|
||||
* :zeek:enum:`SMB::IN_FILE_NAME`
|
||||
|
||||
* :zeek:enum:`SMTP::IN_CC`
|
||||
|
||||
* :zeek:enum:`SMTP::IN_FROM`
|
||||
|
||||
* :zeek:enum:`SMTP::IN_HEADER`
|
||||
|
||||
* :zeek:enum:`SMTP::IN_MAIL_FROM`
|
||||
|
||||
* :zeek:enum:`SMTP::IN_MESSAGE`
|
||||
|
||||
* :zeek:enum:`SMTP::IN_RCPT_TO`
|
||||
|
||||
* :zeek:enum:`SMTP::IN_RECEIVED_HEADER`
|
||||
|
||||
* :zeek:enum:`SMTP::IN_REPLY_TO`
|
||||
|
||||
* :zeek:enum:`SMTP::IN_TO`
|
||||
|
||||
* :zeek:enum:`SMTP::IN_X_ORIGINATING_IP_HEADER`
|
||||
|
||||
* :zeek:enum:`SSH::IN_SERVER_HOST_KEY`
|
||||
|
||||
* :zeek:enum:`SSL::IN_SERVER_NAME`
|
||||
|
||||
* :zeek:enum:`X509::IN_CERT`
|
||||
============================================ ===============================================
|
||||
|
||||
|
||||
Detailed Interface
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
|
||||
33
doc/scripts/policy/frameworks/intel/seen/x509.zeek.rst
Normal file
33
doc/scripts/policy/frameworks/intel/seen/x509.zeek.rst
Normal file
|
|
@ -0,0 +1,33 @@
|
|||
:tocdepth: 3
|
||||
|
||||
policy/frameworks/intel/seen/x509.zeek
|
||||
======================================
|
||||
.. zeek:namespace:: Intel
|
||||
|
||||
|
||||
:Namespace: Intel
|
||||
:Imports: :doc:`base/files/x509 </scripts/base/files/x509/index>`, :doc:`base/frameworks/intel </scripts/base/frameworks/intel/index>`, :doc:`policy/frameworks/intel/seen/where-locations.zeek </scripts/policy/frameworks/intel/seen/where-locations.zeek>`
|
||||
|
||||
Summary
|
||||
~~~~~~~
|
||||
Runtime Options
|
||||
###############
|
||||
================================================================================================= =============================================================================
|
||||
:zeek:id:`Intel::enable_x509_ext_subject_alternative_name`: :zeek:type:`bool` :zeek:attr:`&redef` Enables the extraction of subject alternate names from the X509 SAN DNS field
|
||||
================================================================================================= =============================================================================
|
||||
|
||||
|
||||
Detailed Interface
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
Runtime Options
|
||||
###############
|
||||
.. zeek:id:: Intel::enable_x509_ext_subject_alternative_name
|
||||
:source-code: policy/frameworks/intel/seen/x509.zeek 9 9
|
||||
|
||||
:Type: :zeek:type:`bool`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``T``
|
||||
|
||||
Enables the extraction of subject alternate names from the X509 SAN DNS field
|
||||
|
||||
|
||||
28
doc/scripts/policy/frameworks/intel/whitelist.zeek.rst
Normal file
28
doc/scripts/policy/frameworks/intel/whitelist.zeek.rst
Normal file
|
|
@ -0,0 +1,28 @@
|
|||
:tocdepth: 3
|
||||
|
||||
policy/frameworks/intel/whitelist.zeek
|
||||
======================================
|
||||
.. zeek:namespace:: Intel
|
||||
|
||||
This script enables whitelisting for intelligence items.
|
||||
|
||||
:Namespace: Intel
|
||||
:Imports: :doc:`base/frameworks/intel </scripts/base/frameworks/intel/index>`
|
||||
|
||||
Summary
|
||||
~~~~~~~
|
||||
Redefinitions
|
||||
#############
|
||||
================================================= ===================================================================================
|
||||
:zeek:type:`Intel::MetaData`: :zeek:type:`record`
|
||||
|
||||
:New Fields: :zeek:type:`Intel::MetaData`
|
||||
|
||||
whitelist: :zeek:type:`bool` :zeek:attr:`&default` = ``F`` :zeek:attr:`&optional`
|
||||
A boolean value to indicate whether the item is whitelisted.
|
||||
================================================= ===================================================================================
|
||||
|
||||
|
||||
Detailed Interface
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
|
||||
19
doc/scripts/policy/frameworks/management/__load__.zeek.rst
Normal file
19
doc/scripts/policy/frameworks/management/__load__.zeek.rst
Normal file
|
|
@ -0,0 +1,19 @@
|
|||
:tocdepth: 3
|
||||
|
||||
policy/frameworks/management/__load__.zeek
|
||||
==========================================
|
||||
|
||||
This loads Management framework functionality needed by both the controller
|
||||
and agents. Note that there's no notion of loading "the Management
|
||||
framework" -- one always loads "management/controller" or
|
||||
"management/agent". This __load__ script exists only to simplify loading all
|
||||
common functionality.
|
||||
|
||||
:Imports: :doc:`policy/frameworks/management/config.zeek </scripts/policy/frameworks/management/config.zeek>`, :doc:`policy/frameworks/management/log.zeek </scripts/policy/frameworks/management/log.zeek>`, :doc:`policy/frameworks/management/persistence.zeek </scripts/policy/frameworks/management/persistence.zeek>`, :doc:`policy/frameworks/management/request.zeek </scripts/policy/frameworks/management/request.zeek>`, :doc:`policy/frameworks/management/types.zeek </scripts/policy/frameworks/management/types.zeek>`, :doc:`policy/frameworks/management/util.zeek </scripts/policy/frameworks/management/util.zeek>`
|
||||
|
||||
Summary
|
||||
~~~~~~~
|
||||
|
||||
Detailed Interface
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
|
||||
|
|
@ -0,0 +1,15 @@
|
|||
:tocdepth: 3
|
||||
|
||||
policy/frameworks/management/agent/__load__.zeek
|
||||
================================================
|
||||
|
||||
The entry point for the Management framework's cluster agent. It runs
|
||||
bootstrap logic for launching an agent process via Zeek's Supervisor.
|
||||
|
||||
|
||||
Summary
|
||||
~~~~~~~
|
||||
|
||||
Detailed Interface
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
|
||||
344
doc/scripts/policy/frameworks/management/agent/api.zeek.rst
Normal file
344
doc/scripts/policy/frameworks/management/agent/api.zeek.rst
Normal file
|
|
@ -0,0 +1,344 @@
|
|||
:tocdepth: 3
|
||||
|
||||
policy/frameworks/management/agent/api.zeek
|
||||
===========================================
|
||||
.. zeek:namespace:: Management::Agent::API
|
||||
|
||||
The event API of cluster agents. Most endpoints consist of event pairs,
|
||||
where the agent answers a request event with a corresponding response
|
||||
event. Such event pairs share the same name prefix and end in "_request" and
|
||||
"_response", respectively.
|
||||
|
||||
:Namespace: Management::Agent::API
|
||||
:Imports: :doc:`base/frameworks/supervisor/control.zeek </scripts/base/frameworks/supervisor/control.zeek>`, :doc:`policy/frameworks/management/types.zeek </scripts/policy/frameworks/management/types.zeek>`
|
||||
|
||||
Summary
|
||||
~~~~~~~
|
||||
Constants
|
||||
#########
|
||||
============================================================== ================================================================
|
||||
:zeek:id:`Management::Agent::API::version`: :zeek:type:`count` A simple versioning scheme, used to track basic compatibility of
|
||||
controller and agent.
|
||||
============================================================== ================================================================
|
||||
|
||||
Events
|
||||
######
|
||||
============================================================================= =====================================================================
|
||||
:zeek:id:`Management::Agent::API::agent_standby_request`: :zeek:type:`event` The controller sends this event to convey that the agent is not
|
||||
currently required.
|
||||
:zeek:id:`Management::Agent::API::agent_standby_response`: :zeek:type:`event` Response to a
|
||||
:zeek:see:`Management::Agent::API::agent_standby_request` event.
|
||||
:zeek:id:`Management::Agent::API::agent_welcome_request`: :zeek:type:`event` The controller sends this event to confirm to the agent that it is
|
||||
part of the current cluster topology.
|
||||
:zeek:id:`Management::Agent::API::agent_welcome_response`: :zeek:type:`event` Response to a
|
||||
:zeek:see:`Management::Agent::API::agent_welcome_request` event.
|
||||
:zeek:id:`Management::Agent::API::deploy_request`: :zeek:type:`event` The controller sends this event to deploy a cluster configuration to
|
||||
this instance.
|
||||
:zeek:id:`Management::Agent::API::deploy_response`: :zeek:type:`event` Response to a :zeek:see:`Management::Agent::API::deploy_request`
|
||||
event.
|
||||
:zeek:id:`Management::Agent::API::get_nodes_request`: :zeek:type:`event` The controller sends this event to request a list of
|
||||
:zeek:see:`Management::NodeStatus` records that capture
|
||||
the status of Supervisor-managed nodes running on this instance.
|
||||
:zeek:id:`Management::Agent::API::get_nodes_response`: :zeek:type:`event` Response to a :zeek:see:`Management::Agent::API::get_nodes_request`
|
||||
event.
|
||||
:zeek:id:`Management::Agent::API::node_dispatch_request`: :zeek:type:`event` The controller sends this to every agent to request a dispatch (the
|
||||
execution of a pre-implemented activity) to all cluster nodes.
|
||||
:zeek:id:`Management::Agent::API::node_dispatch_response`: :zeek:type:`event` Response to a
|
||||
:zeek:see:`Management::Agent::API::node_dispatch_request` event.
|
||||
:zeek:id:`Management::Agent::API::notify_agent_hello`: :zeek:type:`event` The agent sends this event upon peering as a "check-in", informing
|
||||
the controller that an agent of the given name is now available to
|
||||
communicate with.
|
||||
:zeek:id:`Management::Agent::API::notify_change`: :zeek:type:`event`
|
||||
:zeek:id:`Management::Agent::API::notify_error`: :zeek:type:`event`
|
||||
:zeek:id:`Management::Agent::API::notify_log`: :zeek:type:`event`
|
||||
:zeek:id:`Management::Agent::API::restart_request`: :zeek:type:`event` The controller sends this event to ask the agent to restart currently
|
||||
running Zeek cluster nodes.
|
||||
:zeek:id:`Management::Agent::API::restart_response`: :zeek:type:`event` Response to a :zeek:see:`Management::Agent::API::restart_request`
|
||||
event.
|
||||
============================================================================= =====================================================================
|
||||
|
||||
|
||||
Detailed Interface
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
Constants
|
||||
#########
|
||||
.. zeek:id:: Management::Agent::API::version
|
||||
:source-code: policy/frameworks/management/agent/api.zeek 14 14
|
||||
|
||||
:Type: :zeek:type:`count`
|
||||
:Default: ``1``
|
||||
|
||||
A simple versioning scheme, used to track basic compatibility of
|
||||
controller and agent.
|
||||
|
||||
Events
|
||||
######
|
||||
.. zeek:id:: Management::Agent::API::agent_standby_request
|
||||
:source-code: policy/frameworks/management/agent/main.zeek 871 890
|
||||
|
||||
:Type: :zeek:type:`event` (reqid: :zeek:type:`string`)
|
||||
|
||||
The controller sends this event to convey that the agent is not
|
||||
currently required. This status may later change, depending on
|
||||
updates from the client, so the Broker-level peering can remain
|
||||
active. The agent releases any cluster-related resources (including
|
||||
shutdown of existing Zeek cluster nodes) when processing the request,
|
||||
and confirms via the response event. Shutting down an agent at this
|
||||
point has no operational impact on the running cluster.
|
||||
|
||||
|
||||
:param reqid: a request identifier string, echoed in the response event.
|
||||
|
||||
|
||||
.. zeek:id:: Management::Agent::API::agent_standby_response
|
||||
:source-code: policy/frameworks/management/agent/api.zeek 150 150
|
||||
|
||||
:Type: :zeek:type:`event` (reqid: :zeek:type:`string`, result: :zeek:type:`Management::Result`)
|
||||
|
||||
Response to a
|
||||
:zeek:see:`Management::Agent::API::agent_standby_request` event. The
|
||||
agent sends this back to the controller.
|
||||
|
||||
|
||||
:param reqid: the request identifier used in the request event.
|
||||
|
||||
|
||||
:param result: the result record.
|
||||
|
||||
|
||||
.. zeek:id:: Management::Agent::API::agent_welcome_request
|
||||
:source-code: policy/frameworks/management/agent/main.zeek 857 869
|
||||
|
||||
:Type: :zeek:type:`event` (reqid: :zeek:type:`string`)
|
||||
|
||||
The controller sends this event to confirm to the agent that it is
|
||||
part of the current cluster topology. The agent acknowledges with a
|
||||
:zeek:see:`Management::Agent::API::agent_welcome_response` event,
|
||||
upon which the controller may proceed with a cluster deployment to
|
||||
this agent.
|
||||
|
||||
|
||||
:param reqid: a request identifier string, echoed in the response event.
|
||||
|
||||
|
||||
.. zeek:id:: Management::Agent::API::agent_welcome_response
|
||||
:source-code: policy/frameworks/management/controller/main.zeek 900 926
|
||||
|
||||
:Type: :zeek:type:`event` (reqid: :zeek:type:`string`, result: :zeek:type:`Management::Result`)
|
||||
|
||||
Response to a
|
||||
:zeek:see:`Management::Agent::API::agent_welcome_request` event. The
|
||||
agent sends this back to the controller.
|
||||
|
||||
|
||||
:param reqid: the request identifier used in the request event.
|
||||
|
||||
|
||||
:param result: the result record.
|
||||
|
||||
|
||||
.. zeek:id:: Management::Agent::API::deploy_request
|
||||
:source-code: policy/frameworks/management/agent/main.zeek 411 451
|
||||
|
||||
:Type: :zeek:type:`event` (reqid: :zeek:type:`string`, config: :zeek:type:`Management::Configuration`, force: :zeek:type:`bool` :zeek:attr:`&default` = ``F`` :zeek:attr:`&optional`)
|
||||
|
||||
The controller sends this event to deploy a cluster configuration to
|
||||
this instance. Once processed, the agent responds with a
|
||||
:zeek:see:`Management::Agent::API::deploy_response` event. event.
|
||||
|
||||
|
||||
:param reqid: a request identifier string, echoed in the response event.
|
||||
|
||||
|
||||
:param config: a :zeek:see:`Management::Configuration` record describing the
|
||||
cluster topology. This contains the full topology, not just the
|
||||
part pertaining to this instance: the cluster framework requires
|
||||
full cluster visibility to establish needed peerings.
|
||||
|
||||
|
||||
:param force: whether to re-deploy (i.e., restart its Zeek cluster nodes)
|
||||
when the agent already runs this configuration. This relies on
|
||||
the config ID to determine config equality.
|
||||
|
||||
|
||||
.. zeek:id:: Management::Agent::API::deploy_response
|
||||
:source-code: policy/frameworks/management/controller/main.zeek 944 1000
|
||||
|
||||
:Type: :zeek:type:`event` (reqid: :zeek:type:`string`, results: :zeek:type:`Management::ResultVec`)
|
||||
|
||||
Response to a :zeek:see:`Management::Agent::API::deploy_request`
|
||||
event. The agent sends this back to the controller.
|
||||
|
||||
|
||||
:param reqid: the request identifier used in the request event.
|
||||
|
||||
|
||||
:param results: A vector of :zeek:see:`Management::Result` records, each
|
||||
capturing the outcome of a single launched node. For failing
|
||||
nodes, the result's data field is a
|
||||
:zeek:see:`Management::NodeOutputs` record.
|
||||
|
||||
|
||||
.. zeek:id:: Management::Agent::API::get_nodes_request
|
||||
:source-code: policy/frameworks/management/agent/main.zeek 588 597
|
||||
|
||||
:Type: :zeek:type:`event` (reqid: :zeek:type:`string`)
|
||||
|
||||
The controller sends this event to request a list of
|
||||
:zeek:see:`Management::NodeStatus` records that capture
|
||||
the status of Supervisor-managed nodes running on this instance.
|
||||
instances.
|
||||
|
||||
|
||||
:param reqid: a request identifier string, echoed in the response event.
|
||||
|
||||
|
||||
.. zeek:id:: Management::Agent::API::get_nodes_response
|
||||
:source-code: policy/frameworks/management/controller/main.zeek 1153 1197
|
||||
|
||||
:Type: :zeek:type:`event` (reqid: :zeek:type:`string`, result: :zeek:type:`Management::Result`)
|
||||
|
||||
Response to a :zeek:see:`Management::Agent::API::get_nodes_request`
|
||||
event. The agent sends this back to the controller.
|
||||
|
||||
|
||||
:param reqid: the request identifier used in the request event.
|
||||
|
||||
|
||||
:param result: a :zeek:see:`Management::Result` record. Its data
|
||||
member is a vector of :zeek:see:`Management::NodeStatus`
|
||||
records, covering the nodes at this instance. The result may also
|
||||
indicate failure, with error messages indicating what went wrong.
|
||||
|
||||
|
||||
.. zeek:id:: Management::Agent::API::node_dispatch_request
|
||||
:source-code: policy/frameworks/management/agent/main.zeek 761 855
|
||||
|
||||
:Type: :zeek:type:`event` (reqid: :zeek:type:`string`, action: :zeek:type:`vector` of :zeek:type:`string`, nodes: :zeek:type:`set` [:zeek:type:`string`] :zeek:attr:`&default` = ``{ }`` :zeek:attr:`&optional`)
|
||||
|
||||
The controller sends this to every agent to request a dispatch (the
|
||||
execution of a pre-implemented activity) to all cluster nodes. This
|
||||
is the generic controller-agent "back-end" implementation of explicit
|
||||
client-controller "front-end" interactions, including:
|
||||
|
||||
- :zeek:see:`Management::Controller::API::get_id_value_request`: two
|
||||
arguments, the first being "get_id_value" and the second the name
|
||||
of the ID to look up.
|
||||
|
||||
|
||||
:param reqid: a request identifier string, echoed in the response event.
|
||||
|
||||
|
||||
:param action: the requested dispatch command, with any arguments.
|
||||
|
||||
|
||||
:param nodes: a set of cluster node names (e.g. "worker-01") to retrieve
|
||||
the values from. An empty set, supplied by default, means
|
||||
retrieval from all nodes managed by the agent.
|
||||
|
||||
|
||||
.. zeek:id:: Management::Agent::API::node_dispatch_response
|
||||
:source-code: policy/frameworks/management/controller/main.zeek 1230 1295
|
||||
|
||||
:Type: :zeek:type:`event` (reqid: :zeek:type:`string`, results: :zeek:type:`Management::ResultVec`)
|
||||
|
||||
Response to a
|
||||
:zeek:see:`Management::Agent::API::node_dispatch_request` event. Each
|
||||
agent sends this back to the controller to report the dispatch
|
||||
outcomes on all nodes managed by that agent.
|
||||
|
||||
|
||||
:param reqid: the request identifier used in the request event.
|
||||
|
||||
|
||||
:param results: a :zeek:type:`vector` of :zeek:see:`Management::Result`
|
||||
records. Each record covers one Zeek cluster node managed by this
|
||||
agent. Upon success, each :zeek:see:`Management::Result` record's
|
||||
data member contains the dispatches' response in a data type
|
||||
appropriate for the respective dispatch.
|
||||
|
||||
|
||||
.. zeek:id:: Management::Agent::API::notify_agent_hello
|
||||
:source-code: policy/frameworks/management/controller/main.zeek 835 898
|
||||
|
||||
:Type: :zeek:type:`event` (instance: :zeek:type:`string`, id: :zeek:type:`string`, connecting: :zeek:type:`bool`, api_version: :zeek:type:`count`)
|
||||
|
||||
The agent sends this event upon peering as a "check-in", informing
|
||||
the controller that an agent of the given name is now available to
|
||||
communicate with. It is a controller-level equivalent of
|
||||
:zeek:see:`Broker::peer_added` and triggered by it.
|
||||
|
||||
|
||||
:param instance: an instance name, really the agent's name as per
|
||||
:zeek:see:`Management::Agent::get_name`.
|
||||
|
||||
|
||||
:param id: the Broker ID of the agent.
|
||||
|
||||
|
||||
:param connecting: true if this agent connected to the controller,
|
||||
false if the controller connected to the agent.
|
||||
|
||||
|
||||
:param api_version: the API version of this agent.
|
||||
|
||||
|
||||
.. zeek:id:: Management::Agent::API::notify_change
|
||||
:source-code: policy/frameworks/management/controller/main.zeek 929 930
|
||||
|
||||
:Type: :zeek:type:`event` (instance: :zeek:type:`string`, n: :zeek:type:`Management::Node`, old: :zeek:type:`Management::State`, new: :zeek:type:`Management::State`)
|
||||
|
||||
|
||||
.. zeek:id:: Management::Agent::API::notify_error
|
||||
:source-code: policy/frameworks/management/controller/main.zeek 934 935
|
||||
|
||||
:Type: :zeek:type:`event` (instance: :zeek:type:`string`, msg: :zeek:type:`string`, node: :zeek:type:`string` :zeek:attr:`&default` = ``""`` :zeek:attr:`&optional`)
|
||||
|
||||
|
||||
.. zeek:id:: Management::Agent::API::notify_log
|
||||
:source-code: policy/frameworks/management/controller/main.zeek 939 940
|
||||
|
||||
:Type: :zeek:type:`event` (instance: :zeek:type:`string`, msg: :zeek:type:`string`, node: :zeek:type:`string` :zeek:attr:`&default` = ``""`` :zeek:attr:`&optional`)
|
||||
|
||||
|
||||
.. zeek:id:: Management::Agent::API::restart_request
|
||||
:source-code: policy/frameworks/management/agent/main.zeek 934 1008
|
||||
|
||||
:Type: :zeek:type:`event` (reqid: :zeek:type:`string`, nodes: :zeek:type:`set` [:zeek:type:`string`] :zeek:attr:`&default` = ``{ }`` :zeek:attr:`&optional`)
|
||||
|
||||
The controller sends this event to ask the agent to restart currently
|
||||
running Zeek cluster nodes. For nodes currently running, the agent
|
||||
places these nodes into PENDING state and sends restart events to the
|
||||
Supervisor, rendering its responses into a list of
|
||||
:zeek:see:`Management::Result` records summarizing each node restart.
|
||||
When restarted nodes check in with the agent, they switch back to
|
||||
RUNNING state. The agent ignores nodes not currently running.
|
||||
|
||||
|
||||
:param reqid: a request identifier string, echoed in the response event.
|
||||
|
||||
|
||||
:param nodes: a set of cluster node names (e.g. "worker-01") to restart. An
|
||||
empty set, supplied by default, means restart of all of the
|
||||
agent's current cluster nodes.
|
||||
|
||||
|
||||
.. zeek:id:: Management::Agent::API::restart_response
|
||||
:source-code: policy/frameworks/management/controller/main.zeek 1376 1414
|
||||
|
||||
:Type: :zeek:type:`event` (reqid: :zeek:type:`string`, results: :zeek:type:`Management::ResultVec`)
|
||||
|
||||
Response to a :zeek:see:`Management::Agent::API::restart_request`
|
||||
event. The agent sends this back to the controller when the
|
||||
Supervisor has restarted all nodes affected, or a timeout occurs.
|
||||
|
||||
|
||||
:param reqid: the request identifier used in the request event.
|
||||
|
||||
|
||||
:param results: a :zeek:type:`vector` of :zeek:see:`Management::Result`, one
|
||||
for each Supervisor transaction. Each such result identifies both
|
||||
the instance and node.
|
||||
|
||||
|
||||
|
||||
26
doc/scripts/policy/frameworks/management/agent/boot.zeek.rst
Normal file
26
doc/scripts/policy/frameworks/management/agent/boot.zeek.rst
Normal file
|
|
@ -0,0 +1,26 @@
|
|||
:tocdepth: 3
|
||||
|
||||
policy/frameworks/management/agent/boot.zeek
|
||||
============================================
|
||||
|
||||
The cluster agent boot logic runs in Zeek's supervisor and instructs it to
|
||||
launch a Management agent process. The agent's main logic resides in main.zeek,
|
||||
similarly to other frameworks. The new process will execute that script.
|
||||
|
||||
If the current process is not the Zeek supervisor, this does nothing.
|
||||
|
||||
:Imports: :doc:`base/utils/paths.zeek </scripts/base/utils/paths.zeek>`, :doc:`policy/frameworks/management/agent/config.zeek </scripts/policy/frameworks/management/agent/config.zeek>`
|
||||
|
||||
Summary
|
||||
~~~~~~~
|
||||
Redefinitions
|
||||
#############
|
||||
================================================================================== =
|
||||
:zeek:id:`Broker::default_listen_address`: :zeek:type:`string` :zeek:attr:`&redef`
|
||||
:zeek:id:`SupervisorControl::enable_listen`: :zeek:type:`bool` :zeek:attr:`&redef`
|
||||
================================================================================== =
|
||||
|
||||
|
||||
Detailed Interface
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
|
||||
235
doc/scripts/policy/frameworks/management/agent/config.zeek.rst
Normal file
235
doc/scripts/policy/frameworks/management/agent/config.zeek.rst
Normal file
|
|
@ -0,0 +1,235 @@
|
|||
:tocdepth: 3
|
||||
|
||||
policy/frameworks/management/agent/config.zeek
|
||||
==============================================
|
||||
.. zeek:namespace:: Management::Agent
|
||||
|
||||
Configuration settings for a cluster agent.
|
||||
|
||||
:Namespace: Management::Agent
|
||||
:Imports: :doc:`base/misc/installation.zeek </scripts/base/misc/installation.zeek>`, :doc:`policy/frameworks/management </scripts/policy/frameworks/management/index>`, :doc:`policy/frameworks/management/controller/config.zeek </scripts/policy/frameworks/management/controller/config.zeek>`
|
||||
|
||||
Summary
|
||||
~~~~~~~
|
||||
Redefinable Options
|
||||
###################
|
||||
============================================================================================== =====================================================================================
|
||||
:zeek:id:`Management::Agent::archive_cmd`: :zeek:type:`string` :zeek:attr:`&redef` The archival command.
|
||||
:zeek:id:`Management::Agent::archive_dir`: :zeek:type:`string` :zeek:attr:`&redef` The destination interval for archived logs.
|
||||
:zeek:id:`Management::Agent::archive_interval`: :zeek:type:`interval` :zeek:attr:`&redef` The archival interval to use.
|
||||
:zeek:id:`Management::Agent::archive_logs`: :zeek:type:`bool` :zeek:attr:`&redef` Whether the agent should periodically invoke zeek-archiver to
|
||||
finalize logs.
|
||||
:zeek:id:`Management::Agent::controller`: :zeek:type:`Broker::NetworkInfo` :zeek:attr:`&redef` The network coordinates of the controller.
|
||||
:zeek:id:`Management::Agent::default_port`: :zeek:type:`port` :zeek:attr:`&redef` The fallback listen port if :zeek:see:`Management::Agent::listen_port` remains empty.
|
||||
:zeek:id:`Management::Agent::directory`: :zeek:type:`string` :zeek:attr:`&redef` An optional working directory for the agent.
|
||||
:zeek:id:`Management::Agent::listen_address`: :zeek:type:`string` :zeek:attr:`&redef` The network address the agent listens on.
|
||||
:zeek:id:`Management::Agent::listen_port`: :zeek:type:`string` :zeek:attr:`&redef` The network port the agent listens on.
|
||||
:zeek:id:`Management::Agent::name`: :zeek:type:`string` :zeek:attr:`&redef` The name this agent uses to represent the cluster instance it
|
||||
manages.
|
||||
:zeek:id:`Management::Agent::stderr_file`: :zeek:type:`string` :zeek:attr:`&redef` Agent stderr log configuration.
|
||||
:zeek:id:`Management::Agent::stdout_file`: :zeek:type:`string` :zeek:attr:`&redef` Agent stdout log configuration.
|
||||
:zeek:id:`Management::Agent::topic_prefix`: :zeek:type:`string` :zeek:attr:`&redef` The agent's Broker topic prefix.
|
||||
============================================================================================== =====================================================================================
|
||||
|
||||
Functions
|
||||
#########
|
||||
================================================================== =====================================================================
|
||||
:zeek:id:`Management::Agent::endpoint_info`: :zeek:type:`function` Returns a :zeek:see:`Broker::EndpointInfo` record for this instance.
|
||||
:zeek:id:`Management::Agent::get_name`: :zeek:type:`function` Returns the effective name of this agent.
|
||||
:zeek:id:`Management::Agent::instance`: :zeek:type:`function` Returns a :zeek:see:`Management::Instance` describing this
|
||||
instance (its agent name plus listening address/port, as applicable).
|
||||
================================================================== =====================================================================
|
||||
|
||||
|
||||
Detailed Interface
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
Redefinable Options
|
||||
###################
|
||||
.. zeek:id:: Management::Agent::archive_cmd
|
||||
:source-code: policy/frameworks/management/agent/config.zeek 63 63
|
||||
|
||||
:Type: :zeek:type:`string`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``""``
|
||||
|
||||
The archival command. When empty, defaults to the zeek-archiver
|
||||
installed with the Zeek distribution. Whatever the command, the
|
||||
agent will invoke it like zeek-archiver, so take a look at its
|
||||
command-line arguments if you're planning to put in place a
|
||||
substitute. Archival happens from the
|
||||
:zeek:see:`Log::default_rotation_dir` to
|
||||
:zeek:see:`Management::Agent::archive_dir`.
|
||||
|
||||
.. zeek:id:: Management::Agent::archive_dir
|
||||
:source-code: policy/frameworks/management/agent/config.zeek 66 66
|
||||
|
||||
:Type: :zeek:type:`string`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``"/usr/local/zeek/logs"``
|
||||
|
||||
The destination interval for archived logs.
|
||||
|
||||
.. zeek:id:: Management::Agent::archive_interval
|
||||
:source-code: policy/frameworks/management/agent/config.zeek 54 54
|
||||
|
||||
:Type: :zeek:type:`interval`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``0 secs``
|
||||
|
||||
The archival interval to use. When 0, it defaults to the log rotation
|
||||
interval.
|
||||
|
||||
.. zeek:id:: Management::Agent::archive_logs
|
||||
:source-code: policy/frameworks/management/agent/config.zeek 50 50
|
||||
|
||||
:Type: :zeek:type:`bool`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``T``
|
||||
|
||||
Whether the agent should periodically invoke zeek-archiver to
|
||||
finalize logs.
|
||||
|
||||
.. zeek:id:: Management::Agent::controller
|
||||
:source-code: policy/frameworks/management/agent/config.zeek 79 79
|
||||
|
||||
:Type: :zeek:type:`Broker::NetworkInfo`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default:
|
||||
|
||||
::
|
||||
|
||||
{
|
||||
address="127.0.0.1"
|
||||
bound_port=2150/tcp
|
||||
}
|
||||
|
||||
|
||||
The network coordinates of the controller. By default, the agent
|
||||
connects locally to the controller at its default port. Assigning
|
||||
a :zeek:see:`Broker::NetworkInfo` record with IP address "0.0.0.0"
|
||||
means the controller should instead connect to the agent. If you'd
|
||||
like to use that mode, make sure to set
|
||||
:zeek:see:`Management::Agent::listen_address` and
|
||||
:zeek:see:`Management::Agent::listen_port` as needed.
|
||||
|
||||
.. zeek:id:: Management::Agent::default_port
|
||||
:source-code: policy/frameworks/management/agent/config.zeek 46 46
|
||||
|
||||
:Type: :zeek:type:`port`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``2151/tcp``
|
||||
|
||||
The fallback listen port if :zeek:see:`Management::Agent::listen_port` remains empty.
|
||||
|
||||
.. zeek:id:: Management::Agent::directory
|
||||
:source-code: policy/frameworks/management/agent/config.zeek 87 87
|
||||
|
||||
:Type: :zeek:type:`string`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``""``
|
||||
|
||||
An optional working directory for the agent. Agent and controller
|
||||
currently only log locally, not via the Zeek cluster's logger
|
||||
node. This means that if multiple agents and/or controllers work from
|
||||
the same directory, output may get garbled. When not set, defaults to
|
||||
a directory named after the agent (as per its get_name() result).
|
||||
|
||||
.. zeek:id:: Management::Agent::listen_address
|
||||
:source-code: policy/frameworks/management/agent/config.zeek 38 38
|
||||
|
||||
:Type: :zeek:type:`string`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``""``
|
||||
|
||||
The network address the agent listens on. This only takes effect if
|
||||
the agent isn't configured to connect to the controller (see
|
||||
:zeek:see:`Management::Agent::controller`). By default this uses the value of the
|
||||
ZEEK_AGENT_ADDR environment variable, but you may also redef to
|
||||
a specific value. When empty, the implementation falls back to
|
||||
:zeek:see:`Management::default_address`.
|
||||
|
||||
.. zeek:id:: Management::Agent::listen_port
|
||||
:source-code: policy/frameworks/management/agent/config.zeek 43 43
|
||||
|
||||
:Type: :zeek:type:`string`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``""``
|
||||
|
||||
The network port the agent listens on. Counterpart to
|
||||
:zeek:see:`Management::Agent::listen_address`, defaulting to the ZEEK_AGENT_PORT
|
||||
environment variable.
|
||||
|
||||
.. zeek:id:: Management::Agent::name
|
||||
:source-code: policy/frameworks/management/agent/config.zeek 17 17
|
||||
|
||||
:Type: :zeek:type:`string`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``""``
|
||||
|
||||
The name this agent uses to represent the cluster instance it
|
||||
manages. Defaults to the value of the ZEEK_AGENT_NAME environment
|
||||
variable. When that is unset and you don't redef the value,
|
||||
the implementation defaults to "agent-<hostname>".
|
||||
|
||||
.. zeek:id:: Management::Agent::stderr_file
|
||||
:source-code: policy/frameworks/management/agent/config.zeek 30 30
|
||||
|
||||
:Type: :zeek:type:`string`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``"stderr"``
|
||||
|
||||
Agent stderr log configuration. Like :zeek:see:`Management::Agent::stdout_file`,
|
||||
but for the stderr stream.
|
||||
|
||||
.. zeek:id:: Management::Agent::stdout_file
|
||||
:source-code: policy/frameworks/management/agent/config.zeek 26 26
|
||||
|
||||
:Type: :zeek:type:`string`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``"stdout"``
|
||||
|
||||
Agent stdout log configuration. If the string is non-empty, Zeek will
|
||||
produce a free-form log (i.e., not one governed by Zeek's logging
|
||||
framework) in the agent's working directory. If left empty, no such
|
||||
log results.
|
||||
|
||||
Note that the agent also establishes a "proper" Zeek log via the
|
||||
:zeek:see:`Management::Log` module.
|
||||
|
||||
.. zeek:id:: Management::Agent::topic_prefix
|
||||
:source-code: policy/frameworks/management/agent/config.zeek 70 70
|
||||
|
||||
:Type: :zeek:type:`string`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``"zeek/management/agent"``
|
||||
|
||||
The agent's Broker topic prefix. For its own communication, the agent
|
||||
suffixes this with "/<name>", based on :zeek:see:`Management::Agent::get_name`.
|
||||
|
||||
Functions
|
||||
#########
|
||||
.. zeek:id:: Management::Agent::endpoint_info
|
||||
:source-code: policy/frameworks/management/agent/config.zeek 118 140
|
||||
|
||||
:Type: :zeek:type:`function` () : :zeek:type:`Broker::EndpointInfo`
|
||||
|
||||
Returns a :zeek:see:`Broker::EndpointInfo` record for this instance.
|
||||
Similar to :zeek:see:`Management::Agent::instance`, but with slightly different
|
||||
data format.
|
||||
|
||||
.. zeek:id:: Management::Agent::get_name
|
||||
:source-code: policy/frameworks/management/agent/config.zeek 102 108
|
||||
|
||||
:Type: :zeek:type:`function` () : :zeek:type:`string`
|
||||
|
||||
Returns the effective name of this agent.
|
||||
|
||||
.. zeek:id:: Management::Agent::instance
|
||||
:source-code: policy/frameworks/management/agent/config.zeek 110 116
|
||||
|
||||
:Type: :zeek:type:`function` () : :zeek:type:`Management::Instance`
|
||||
|
||||
Returns a :zeek:see:`Management::Instance` describing this
|
||||
instance (its agent name plus listening address/port, as applicable).
|
||||
|
||||
|
||||
37
doc/scripts/policy/frameworks/management/agent/index.rst
Normal file
37
doc/scripts/policy/frameworks/management/agent/index.rst
Normal file
|
|
@ -0,0 +1,37 @@
|
|||
:orphan:
|
||||
|
||||
Package: policy/frameworks/management/agent
|
||||
===========================================
|
||||
|
||||
|
||||
:doc:`/scripts/policy/frameworks/management/agent/__load__.zeek`
|
||||
|
||||
The entry point for the Management framework's cluster agent. It runs
|
||||
bootstrap logic for launching an agent process via Zeek's Supervisor.
|
||||
|
||||
:doc:`/scripts/policy/frameworks/management/agent/api.zeek`
|
||||
|
||||
The event API of cluster agents. Most endpoints consist of event pairs,
|
||||
where the agent answers a request event with a corresponding response
|
||||
event. Such event pairs share the same name prefix and end in "_request" and
|
||||
"_response", respectively.
|
||||
|
||||
:doc:`/scripts/policy/frameworks/management/agent/boot.zeek`
|
||||
|
||||
The cluster agent boot logic runs in Zeek's supervisor and instructs it to
|
||||
launch a Management agent process. The agent's main logic resides in main.zeek,
|
||||
similarly to other frameworks. The new process will execute that script.
|
||||
|
||||
If the current process is not the Zeek supervisor, this does nothing.
|
||||
|
||||
:doc:`/scripts/policy/frameworks/management/agent/config.zeek`
|
||||
|
||||
Configuration settings for a cluster agent.
|
||||
|
||||
:doc:`/scripts/policy/frameworks/management/agent/main.zeek`
|
||||
|
||||
This is the main "runtime" of a cluster agent. Zeek does not load this
|
||||
directly; rather, the agent's bootstrapping module (in ./boot.zeek)
|
||||
specifies it as the script to run in the node newly created via Zeek's
|
||||
supervisor.
|
||||
|
||||
136
doc/scripts/policy/frameworks/management/agent/main.zeek.rst
Normal file
136
doc/scripts/policy/frameworks/management/agent/main.zeek.rst
Normal file
|
|
@ -0,0 +1,136 @@
|
|||
:tocdepth: 3
|
||||
|
||||
policy/frameworks/management/agent/main.zeek
|
||||
============================================
|
||||
.. zeek:namespace:: Management::Agent::Runtime
|
||||
|
||||
This is the main "runtime" of a cluster agent. Zeek does not load this
|
||||
directly; rather, the agent's bootstrapping module (in ./boot.zeek)
|
||||
specifies it as the script to run in the node newly created via Zeek's
|
||||
supervisor.
|
||||
|
||||
:Namespace: Management::Agent::Runtime
|
||||
:Imports: :doc:`base/frameworks/broker </scripts/base/frameworks/broker/index>`, :doc:`base/utils/paths.zeek </scripts/base/utils/paths.zeek>`, :doc:`policy/frameworks/management </scripts/policy/frameworks/management/index>`, :doc:`policy/frameworks/management/agent/api.zeek </scripts/policy/frameworks/management/agent/api.zeek>`, :doc:`policy/frameworks/management/agent/config.zeek </scripts/policy/frameworks/management/agent/config.zeek>`, :doc:`policy/frameworks/management/node/api.zeek </scripts/policy/frameworks/management/node/api.zeek>`, :doc:`policy/frameworks/management/node/config.zeek </scripts/policy/frameworks/management/node/config.zeek>`, :doc:`policy/frameworks/management/supervisor/api.zeek </scripts/policy/frameworks/management/supervisor/api.zeek>`, :doc:`policy/frameworks/management/supervisor/config.zeek </scripts/policy/frameworks/management/supervisor/config.zeek>`
|
||||
|
||||
Summary
|
||||
~~~~~~~
|
||||
Types
|
||||
#####
|
||||
=============================================================================== ================================================================
|
||||
:zeek:type:`Management::Agent::Runtime::DeployState`: :zeek:type:`record` Request state for deploy requests.
|
||||
:zeek:type:`Management::Agent::Runtime::NodeDispatchState`: :zeek:type:`record` Request state for node dispatches, tracking the requested action
|
||||
as well as received responses.
|
||||
:zeek:type:`Management::Agent::Runtime::RestartState`: :zeek:type:`record` Request state for restart requests, tracking received responses.
|
||||
:zeek:type:`Management::Agent::Runtime::SupervisorState`: :zeek:type:`record` Request state specific to the agent's Supervisor interactions.
|
||||
=============================================================================== ================================================================
|
||||
|
||||
Redefinitions
|
||||
#############
|
||||
=========================================================================================== ==============================================================================================================
|
||||
:zeek:type:`Management::Request::Request`: :zeek:type:`record`
|
||||
|
||||
:New Fields: :zeek:type:`Management::Request::Request`
|
||||
|
||||
supervisor_state_agent: :zeek:type:`Management::Agent::Runtime::SupervisorState` :zeek:attr:`&optional`
|
||||
|
||||
deploy_state_agent: :zeek:type:`Management::Agent::Runtime::DeployState` :zeek:attr:`&optional`
|
||||
|
||||
node_dispatch_state_agent: :zeek:type:`Management::Agent::Runtime::NodeDispatchState` :zeek:attr:`&optional`
|
||||
|
||||
restart_state_agent: :zeek:type:`Management::Agent::Runtime::RestartState` :zeek:attr:`&optional`
|
||||
:zeek:id:`Management::Request::timeout_interval`: :zeek:type:`interval` :zeek:attr:`&redef`
|
||||
:zeek:id:`Management::role`: :zeek:type:`Management::Role` :zeek:attr:`&redef`
|
||||
:zeek:id:`table_expire_interval`: :zeek:type:`interval` :zeek:attr:`&redef`
|
||||
=========================================================================================== ==============================================================================================================
|
||||
|
||||
Events
|
||||
######
|
||||
=============================================================================== =
|
||||
:zeek:id:`Management::Agent::Runtime::trigger_log_archival`: :zeek:type:`event`
|
||||
=============================================================================== =
|
||||
|
||||
|
||||
Detailed Interface
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
Types
|
||||
#####
|
||||
.. zeek:type:: Management::Agent::Runtime::DeployState
|
||||
:source-code: policy/frameworks/management/agent/main.zeek 35 39
|
||||
|
||||
:Type: :zeek:type:`record`
|
||||
|
||||
|
||||
.. zeek:field:: nodes_pending :zeek:type:`set` [:zeek:type:`string`]
|
||||
|
||||
Zeek cluster nodes the provided configuration requested
|
||||
and which have not yet checked in with the agent.
|
||||
|
||||
|
||||
Request state for deploy requests.
|
||||
|
||||
.. zeek:type:: Management::Agent::Runtime::NodeDispatchState
|
||||
:source-code: policy/frameworks/management/agent/main.zeek 43 50
|
||||
|
||||
:Type: :zeek:type:`record`
|
||||
|
||||
|
||||
.. zeek:field:: action :zeek:type:`vector` of :zeek:type:`string`
|
||||
|
||||
The dispatched action. The first string is a command,
|
||||
any remaining strings its arguments.
|
||||
|
||||
|
||||
.. zeek:field:: requests :zeek:type:`set` [:zeek:type:`string`] :zeek:attr:`&default` = ``{ }`` :zeek:attr:`&optional`
|
||||
|
||||
Request state for every node managed by this agent.
|
||||
|
||||
|
||||
Request state for node dispatches, tracking the requested action
|
||||
as well as received responses.
|
||||
|
||||
.. zeek:type:: Management::Agent::Runtime::RestartState
|
||||
:source-code: policy/frameworks/management/agent/main.zeek 53 57
|
||||
|
||||
:Type: :zeek:type:`record`
|
||||
|
||||
|
||||
.. zeek:field:: requests :zeek:type:`set` [:zeek:type:`string`] :zeek:attr:`&default` = ``{ }`` :zeek:attr:`&optional`
|
||||
|
||||
Request state for every node the agent asks the Supervisor
|
||||
to restart.
|
||||
|
||||
|
||||
Request state for restart requests, tracking received responses.
|
||||
|
||||
.. zeek:type:: Management::Agent::Runtime::SupervisorState
|
||||
:source-code: policy/frameworks/management/agent/main.zeek 25 32
|
||||
|
||||
:Type: :zeek:type:`record`
|
||||
|
||||
|
||||
.. zeek:field:: node :zeek:type:`string` :zeek:attr:`&default` = ``""`` :zeek:attr:`&optional`
|
||||
|
||||
Name of the node the Supervisor is acting on, if applicable.
|
||||
|
||||
|
||||
.. zeek:field:: status :zeek:type:`Supervisor::Status` :zeek:attr:`&optional`
|
||||
|
||||
The result of a status request.
|
||||
|
||||
|
||||
.. zeek:field:: restart_result :zeek:type:`bool` :zeek:attr:`&optional`
|
||||
|
||||
The result of a restart request.
|
||||
|
||||
|
||||
Request state specific to the agent's Supervisor interactions.
|
||||
|
||||
Events
|
||||
######
|
||||
.. zeek:id:: Management::Agent::Runtime::trigger_log_archival
|
||||
:source-code: policy/frameworks/management/agent/main.zeek 217 261
|
||||
|
||||
:Type: :zeek:type:`event` (run_archival: :zeek:type:`bool` :zeek:attr:`&default` = ``T`` :zeek:attr:`&optional`)
|
||||
|
||||
|
||||
|
||||
139
doc/scripts/policy/frameworks/management/config.zeek.rst
Normal file
139
doc/scripts/policy/frameworks/management/config.zeek.rst
Normal file
|
|
@ -0,0 +1,139 @@
|
|||
:tocdepth: 3
|
||||
|
||||
policy/frameworks/management/config.zeek
|
||||
========================================
|
||||
.. zeek:namespace:: Management
|
||||
|
||||
Management framework configuration settings common to agent and controller.
|
||||
This does not include config settings that exist in both agent and
|
||||
controller but that they set differently, since setting defaults here would
|
||||
be awkward or pointless (since both node types would overwrite them
|
||||
anyway). For role-specific settings, see management/controller/config.zeek
|
||||
and management/agent/config.zeek.
|
||||
|
||||
:Namespace: Management
|
||||
:Imports: :doc:`base/misc/installation.zeek </scripts/base/misc/installation.zeek>`, :doc:`policy/frameworks/management/types.zeek </scripts/policy/frameworks/management/types.zeek>`
|
||||
|
||||
Summary
|
||||
~~~~~~~
|
||||
Redefinable Options
|
||||
###################
|
||||
=============================================================================== ===================================================================
|
||||
:zeek:id:`Management::connect_retry`: :zeek:type:`interval` :zeek:attr:`&redef` The retry interval for Broker connects.
|
||||
:zeek:id:`Management::default_address`: :zeek:type:`string` :zeek:attr:`&redef` The fallback listen address if more specific addresses, such as
|
||||
the controller's :zeek:see:`Management::Controller::listen_address`
|
||||
remains empty.
|
||||
:zeek:id:`Management::role`: :zeek:type:`Management::Role` :zeek:attr:`&redef` The role of this process in cluster management.
|
||||
:zeek:id:`Management::spool_dir`: :zeek:type:`string` :zeek:attr:`&redef` The toplevel directory in which the Management framework creates
|
||||
spool state for any Zeek nodes, including the Zeek cluster, agents,
|
||||
and the controller.
|
||||
:zeek:id:`Management::state_dir`: :zeek:type:`string` :zeek:attr:`&redef` The toplevel directory for variable state, such as Broker data
|
||||
stores.
|
||||
=============================================================================== ===================================================================
|
||||
|
||||
Functions
|
||||
#########
|
||||
=========================================================== ===================================================================
|
||||
:zeek:id:`Management::get_spool_dir`: :zeek:type:`function` Returns the effective spool directory for the management framework.
|
||||
:zeek:id:`Management::get_state_dir`: :zeek:type:`function` Returns the effective state directory for the management framework.
|
||||
=========================================================== ===================================================================
|
||||
|
||||
|
||||
Detailed Interface
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
Redefinable Options
|
||||
###################
|
||||
.. zeek:id:: Management::connect_retry
|
||||
:source-code: policy/frameworks/management/config.zeek 27 27
|
||||
|
||||
:Type: :zeek:type:`interval`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``1.0 sec``
|
||||
|
||||
The retry interval for Broker connects. Defaults to a more
|
||||
aggressive value compared to Broker's 30s.
|
||||
|
||||
.. zeek:id:: Management::default_address
|
||||
:source-code: policy/frameworks/management/config.zeek 23 23
|
||||
|
||||
:Type: :zeek:type:`string`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``"0.0.0.0"``
|
||||
|
||||
The fallback listen address if more specific addresses, such as
|
||||
the controller's :zeek:see:`Management::Controller::listen_address`
|
||||
remains empty. Unless redefined, this listens on all interfaces.
|
||||
|
||||
.. zeek:id:: Management::role
|
||||
:source-code: policy/frameworks/management/config.zeek 18 18
|
||||
|
||||
:Type: :zeek:type:`Management::Role`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``Management::NONE``
|
||||
:Redefinition: from :doc:`/scripts/policy/frameworks/management/agent/main.zeek`
|
||||
|
||||
``=``::
|
||||
|
||||
Management::AGENT
|
||||
|
||||
:Redefinition: from :doc:`/scripts/policy/frameworks/management/controller/main.zeek`
|
||||
|
||||
``=``::
|
||||
|
||||
Management::CONTROLLER
|
||||
|
||||
:Redefinition: from :doc:`/scripts/policy/frameworks/management/node/main.zeek`
|
||||
|
||||
``=``::
|
||||
|
||||
Management::NODE
|
||||
|
||||
|
||||
The role of this process in cluster management. Use this to
|
||||
differentiate code based on the type of node in which it ends up
|
||||
running.
|
||||
|
||||
.. zeek:id:: Management::spool_dir
|
||||
:source-code: policy/frameworks/management/config.zeek 33 33
|
||||
|
||||
:Type: :zeek:type:`string`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``""``
|
||||
|
||||
The toplevel directory in which the Management framework creates
|
||||
spool state for any Zeek nodes, including the Zeek cluster, agents,
|
||||
and the controller. Don't use this directly, use the
|
||||
:zeek:see:`Management::get_spool_dir` function.
|
||||
|
||||
.. zeek:id:: Management::state_dir
|
||||
:source-code: policy/frameworks/management/config.zeek 38 38
|
||||
|
||||
:Type: :zeek:type:`string`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``""``
|
||||
|
||||
The toplevel directory for variable state, such as Broker data
|
||||
stores. Don't use this directly, use the
|
||||
:zeek:see:`Management::get_state_dir` function.
|
||||
|
||||
Functions
|
||||
#########
|
||||
.. zeek:id:: Management::get_spool_dir
|
||||
:source-code: policy/frameworks/management/config.zeek 51 57
|
||||
|
||||
:Type: :zeek:type:`function` () : :zeek:type:`string`
|
||||
|
||||
Returns the effective spool directory for the management framework.
|
||||
That's :zeek:see:`Management::spool_dir` when set, otherwise the
|
||||
installation's spool directory.
|
||||
|
||||
.. zeek:id:: Management::get_state_dir
|
||||
:source-code: policy/frameworks/management/config.zeek 59 65
|
||||
|
||||
:Type: :zeek:type:`function` () : :zeek:type:`string`
|
||||
|
||||
Returns the effective state directory for the management framework.
|
||||
That's :zeek:see:`Management::state_dir` when set, otherwise the
|
||||
installation's state directory.
|
||||
|
||||
|
||||
|
|
@ -0,0 +1,15 @@
|
|||
:tocdepth: 3
|
||||
|
||||
policy/frameworks/management/controller/__load__.zeek
|
||||
=====================================================
|
||||
|
||||
The entry point for the Management framework's cluster controller. It runs
|
||||
bootstrap logic for launching a controller process via Zeek's Supervisor.
|
||||
|
||||
|
||||
Summary
|
||||
~~~~~~~
|
||||
|
||||
Detailed Interface
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
|
||||
402
doc/scripts/policy/frameworks/management/controller/api.zeek.rst
Normal file
402
doc/scripts/policy/frameworks/management/controller/api.zeek.rst
Normal file
|
|
@ -0,0 +1,402 @@
|
|||
:tocdepth: 3
|
||||
|
||||
policy/frameworks/management/controller/api.zeek
|
||||
================================================
|
||||
.. zeek:namespace:: Management::Controller::API
|
||||
|
||||
The event API of cluster controllers. Most endpoints consist of event pairs,
|
||||
where the controller answers the client's request event with a corresponding
|
||||
response event. Such event pairs share the same name prefix and end in
|
||||
"_request" and "_response", respectively.
|
||||
|
||||
:Namespace: Management::Controller::API
|
||||
:Imports: :doc:`policy/frameworks/management/types.zeek </scripts/policy/frameworks/management/types.zeek>`
|
||||
|
||||
Summary
|
||||
~~~~~~~
|
||||
Constants
|
||||
#########
|
||||
=================================================================== ================================================================
|
||||
:zeek:id:`Management::Controller::API::version`: :zeek:type:`count` A simple versioning scheme, used to track basic compatibility of
|
||||
controller, agents, and the client.
|
||||
=================================================================== ================================================================
|
||||
|
||||
Events
|
||||
######
|
||||
======================================================================================== ======================================================================
|
||||
:zeek:id:`Management::Controller::API::deploy_request`: :zeek:type:`event` Trigger deployment of a previously staged configuration.
|
||||
:zeek:id:`Management::Controller::API::deploy_response`: :zeek:type:`event` Response to a :zeek:see:`Management::Controller::API::deploy_request`
|
||||
event.
|
||||
:zeek:id:`Management::Controller::API::get_configuration_request`: :zeek:type:`event` The client sends this event to retrieve the controller's current
|
||||
cluster configuration(s).
|
||||
:zeek:id:`Management::Controller::API::get_configuration_response`: :zeek:type:`event` Response to a
|
||||
:zeek:see:`Management::Controller::API::get_configuration_request`
|
||||
event.
|
||||
:zeek:id:`Management::Controller::API::get_id_value_request`: :zeek:type:`event` The client sends this event to retrieve the current value of a
|
||||
variable in Zeek's global namespace, referenced by the given
|
||||
identifier (i.e., variable name).
|
||||
:zeek:id:`Management::Controller::API::get_id_value_response`: :zeek:type:`event` Response to a
|
||||
:zeek:see:`Management::Controller::API::get_id_value_request`
|
||||
event.
|
||||
:zeek:id:`Management::Controller::API::get_instances_request`: :zeek:type:`event` The client sends this event to request a list of the currently
|
||||
peered agents/instances.
|
||||
:zeek:id:`Management::Controller::API::get_instances_response`: :zeek:type:`event` Response to a
|
||||
:zeek:see:`Management::Controller::API::get_instances_request`
|
||||
event.
|
||||
:zeek:id:`Management::Controller::API::get_nodes_request`: :zeek:type:`event` The client sends this event to request a list of
|
||||
:zeek:see:`Management::NodeStatus` records that capture
|
||||
the status of Supervisor-managed nodes running on the cluster's
|
||||
instances.
|
||||
:zeek:id:`Management::Controller::API::get_nodes_response`: :zeek:type:`event` Response to a
|
||||
:zeek:see:`Management::Controller::API::get_nodes_request` event.
|
||||
:zeek:id:`Management::Controller::API::notify_agents_ready`: :zeek:type:`event` The controller triggers this event when the operational cluster
|
||||
instances align with the ones desired by the cluster
|
||||
configuration.
|
||||
:zeek:id:`Management::Controller::API::restart_request`: :zeek:type:`event` The client sends this event to restart currently running Zeek cluster
|
||||
nodes.
|
||||
:zeek:id:`Management::Controller::API::restart_response`: :zeek:type:`event` Response to a :zeek:see:`Management::Controller::API::restart_request`
|
||||
event.
|
||||
:zeek:id:`Management::Controller::API::stage_configuration_request`: :zeek:type:`event` Upload a configuration to the controller for later deployment.
|
||||
:zeek:id:`Management::Controller::API::stage_configuration_response`: :zeek:type:`event` Response to a
|
||||
:zeek:see:`Management::Controller::API::stage_configuration_request`
|
||||
event.
|
||||
:zeek:id:`Management::Controller::API::test_timeout_request`: :zeek:type:`event` This event causes no further action (other than getting logged) if
|
||||
with_state is F.
|
||||
:zeek:id:`Management::Controller::API::test_timeout_response`: :zeek:type:`event` Response to a
|
||||
:zeek:see:`Management::Controller::API::test_timeout_request`
|
||||
event.
|
||||
======================================================================================== ======================================================================
|
||||
|
||||
|
||||
Detailed Interface
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
Constants
|
||||
#########
|
||||
.. zeek:id:: Management::Controller::API::version
|
||||
:source-code: policy/frameworks/management/controller/api.zeek 13 13
|
||||
|
||||
:Type: :zeek:type:`count`
|
||||
:Default: ``1``
|
||||
|
||||
A simple versioning scheme, used to track basic compatibility of
|
||||
controller, agents, and the client.
|
||||
|
||||
Events
|
||||
######
|
||||
.. zeek:id:: Management::Controller::API::deploy_request
|
||||
:source-code: policy/frameworks/management/controller/main.zeek 1088 1128
|
||||
|
||||
:Type: :zeek:type:`event` (reqid: :zeek:type:`string`)
|
||||
|
||||
Trigger deployment of a previously staged configuration. The client
|
||||
sends this event to the controller, which deploys the configuration
|
||||
to the agents. Agents then terminate any previously running cluster
|
||||
nodes and (re-)launch those defined in the new configuration. Once
|
||||
each agent has responded (or a timeout occurs), the controller sends
|
||||
a response event back to the client, aggregating the results from the
|
||||
agents. The controller keeps the staged configuration available for
|
||||
download, or re-deployment. In addition, the deployed configuration
|
||||
becomes available for download as well, with any augmentations
|
||||
(e.g. node ports filled in by auto-assignment) reflected.
|
||||
|
||||
|
||||
:param reqid: a request identifier string, echoed in the response event.
|
||||
|
||||
|
||||
.. zeek:id:: Management::Controller::API::deploy_response
|
||||
:source-code: policy/frameworks/management/controller/api.zeek 119 119
|
||||
|
||||
:Type: :zeek:type:`event` (reqid: :zeek:type:`string`, results: :zeek:type:`Management::ResultVec`)
|
||||
|
||||
Response to a :zeek:see:`Management::Controller::API::deploy_request`
|
||||
event. The controller sends this back to the client, conveying the
|
||||
outcome of the deployment.
|
||||
|
||||
|
||||
:param reqid: the request identifier used in the request event.
|
||||
|
||||
|
||||
:param results: a vector of :zeek:see:`Management::Result` records.
|
||||
Each member captures the result of launching one cluster
|
||||
node captured in the configuration, or an agent-wide error
|
||||
when the result does not indicate a particular node.
|
||||
|
||||
|
||||
.. zeek:id:: Management::Controller::API::get_configuration_request
|
||||
:source-code: policy/frameworks/management/controller/main.zeek 1063 1086
|
||||
|
||||
:Type: :zeek:type:`event` (reqid: :zeek:type:`string`, deployed: :zeek:type:`bool`)
|
||||
|
||||
The client sends this event to retrieve the controller's current
|
||||
cluster configuration(s).
|
||||
|
||||
|
||||
:param reqid: a request identifier string, echoed in the response event.
|
||||
|
||||
|
||||
:param deployed: when true, returns the deployed configuration (if any),
|
||||
otherwise the staged one (if any).
|
||||
|
||||
|
||||
.. zeek:id:: Management::Controller::API::get_configuration_response
|
||||
:source-code: policy/frameworks/management/controller/api.zeek 89 89
|
||||
|
||||
:Type: :zeek:type:`event` (reqid: :zeek:type:`string`, result: :zeek:type:`Management::Result`)
|
||||
|
||||
Response to a
|
||||
:zeek:see:`Management::Controller::API::get_configuration_request`
|
||||
event. The controller sends this back to the client, with the
|
||||
requested configuration.
|
||||
|
||||
|
||||
:param reqid: the request identifier used in the request event.
|
||||
|
||||
|
||||
:param result: a :zeek:see:`Management::Result` record with a successful
|
||||
:zeek:see:`Management::Configuration` in the data member, if
|
||||
a configuration is currently deployed. Otherwise, a Result
|
||||
record in error state, with no data value assigned.
|
||||
|
||||
|
||||
.. zeek:id:: Management::Controller::API::get_id_value_request
|
||||
:source-code: policy/frameworks/management/controller/main.zeek 1297 1374
|
||||
|
||||
:Type: :zeek:type:`event` (reqid: :zeek:type:`string`, id: :zeek:type:`string`, nodes: :zeek:type:`set` [:zeek:type:`string`] :zeek:attr:`&default` = ``{ }`` :zeek:attr:`&optional`)
|
||||
|
||||
The client sends this event to retrieve the current value of a
|
||||
variable in Zeek's global namespace, referenced by the given
|
||||
identifier (i.e., variable name). The controller asks all agents
|
||||
to retrieve this value from each cluster node, accumulates the
|
||||
returned responses, and responds with a get_id_value_response
|
||||
event back to the client.
|
||||
|
||||
|
||||
:param reqid: a request identifier string, echoed in the response event.
|
||||
|
||||
|
||||
:param id: the name of the variable whose value to retrieve.
|
||||
|
||||
|
||||
:param nodes: a set of cluster node names (e.g. "worker-01") to retrieve
|
||||
the values from. An empty set, supplied by default, means
|
||||
retrieval from all current cluster nodes.
|
||||
|
||||
|
||||
.. zeek:id:: Management::Controller::API::get_id_value_response
|
||||
:source-code: policy/frameworks/management/controller/api.zeek 182 182
|
||||
|
||||
:Type: :zeek:type:`event` (reqid: :zeek:type:`string`, results: :zeek:type:`Management::ResultVec`)
|
||||
|
||||
Response to a
|
||||
:zeek:see:`Management::Controller::API::get_id_value_request`
|
||||
event. The controller sends this back to the client, with a JSON
|
||||
representation of the requested global ID on all relevant instances.
|
||||
|
||||
|
||||
:param reqid: the request identifier used in the request event.
|
||||
|
||||
|
||||
:param results: a :zeek:type:`vector` of :zeek:see:`Management::Result`
|
||||
records. Each record covers one Zeek cluster node. Each record's
|
||||
data field contains a string with the JSON rendering (as produced
|
||||
by :zeek:id:`to_json`, including the error strings it potentially
|
||||
returns).
|
||||
|
||||
|
||||
.. zeek:id:: Management::Controller::API::get_instances_request
|
||||
:source-code: policy/frameworks/management/controller/main.zeek 1130 1151
|
||||
|
||||
:Type: :zeek:type:`event` (reqid: :zeek:type:`string`)
|
||||
|
||||
The client sends this event to request a list of the currently
|
||||
peered agents/instances.
|
||||
|
||||
|
||||
:param reqid: a request identifier string, echoed in the response event.
|
||||
|
||||
|
||||
.. zeek:id:: Management::Controller::API::get_instances_response
|
||||
:source-code: policy/frameworks/management/controller/api.zeek 32 32
|
||||
|
||||
:Type: :zeek:type:`event` (reqid: :zeek:type:`string`, result: :zeek:type:`Management::Result`)
|
||||
|
||||
Response to a
|
||||
:zeek:see:`Management::Controller::API::get_instances_request`
|
||||
event. The controller sends this back to the client.
|
||||
|
||||
|
||||
:param reqid: the request identifier used in the request event.
|
||||
|
||||
|
||||
:param result: a :zeek:see:`Management::Result`. Its data member is a vector
|
||||
of :zeek:see:`Management::Instance` records.
|
||||
|
||||
|
||||
.. zeek:id:: Management::Controller::API::get_nodes_request
|
||||
:source-code: policy/frameworks/management/controller/main.zeek 1199 1228
|
||||
|
||||
:Type: :zeek:type:`event` (reqid: :zeek:type:`string`)
|
||||
|
||||
The client sends this event to request a list of
|
||||
:zeek:see:`Management::NodeStatus` records that capture
|
||||
the status of Supervisor-managed nodes running on the cluster's
|
||||
instances.
|
||||
|
||||
|
||||
:param reqid: a request identifier string, echoed in the response event.
|
||||
|
||||
|
||||
.. zeek:id:: Management::Controller::API::get_nodes_response
|
||||
:source-code: policy/frameworks/management/controller/api.zeek 147 147
|
||||
|
||||
:Type: :zeek:type:`event` (reqid: :zeek:type:`string`, results: :zeek:type:`Management::ResultVec`)
|
||||
|
||||
Response to a
|
||||
:zeek:see:`Management::Controller::API::get_nodes_request` event. The
|
||||
controller sends this back to the client, with a description of the
|
||||
nodes currently managed by the Supervisors on all connected
|
||||
instances. This includes agents and possibly the controller, if it
|
||||
runs jointly with an agent.
|
||||
|
||||
|
||||
:param reqid: the request identifier used in the request event.
|
||||
|
||||
|
||||
:param results: a :zeek:type:`vector` of :zeek:see:`Management::Result`
|
||||
records. Each record covers one cluster instance. Each record's
|
||||
data member is a vector of :zeek:see:`Management::NodeStatus`
|
||||
records, covering the nodes at that instance. Results may also
|
||||
indicate failure, with error messages indicating what went wrong.
|
||||
|
||||
|
||||
.. zeek:id:: Management::Controller::API::notify_agents_ready
|
||||
:source-code: policy/frameworks/management/controller/main.zeek 801 833
|
||||
|
||||
:Type: :zeek:type:`event` (instances: :zeek:type:`set` [:zeek:type:`string`])
|
||||
|
||||
The controller triggers this event when the operational cluster
|
||||
instances align with the ones desired by the cluster
|
||||
configuration. It's essentially a cluster management readiness
|
||||
event. This event is currently only used internally by the controller,
|
||||
and not published to topics.
|
||||
|
||||
|
||||
:param instances: the set of instance names now ready.
|
||||
|
||||
|
||||
.. zeek:id:: Management::Controller::API::restart_request
|
||||
:source-code: policy/frameworks/management/controller/main.zeek 1416 1509
|
||||
|
||||
:Type: :zeek:type:`event` (reqid: :zeek:type:`string`, nodes: :zeek:type:`set` [:zeek:type:`string`] :zeek:attr:`&default` = ``{ }`` :zeek:attr:`&optional`)
|
||||
|
||||
The client sends this event to restart currently running Zeek cluster
|
||||
nodes. The controller relays the request to its agents, which respond
|
||||
with a list of :zeek:see:`Management::Result` records summarizing
|
||||
each node restart. The controller combines these lists, and sends a
|
||||
:zeek:see:`Management::Controller::API::restart_response` event with
|
||||
the result.
|
||||
|
||||
|
||||
:param reqid: a request identifier string, echoed in the response event.
|
||||
|
||||
|
||||
:param nodes: a set of cluster node names (e.g. "worker-01") to restart. An
|
||||
empty set, supplied by default, means restart of all current
|
||||
cluster nodes.
|
||||
|
||||
|
||||
.. zeek:id:: Management::Controller::API::restart_response
|
||||
:source-code: policy/frameworks/management/controller/api.zeek 213 213
|
||||
|
||||
:Type: :zeek:type:`event` (reqid: :zeek:type:`string`, results: :zeek:type:`Management::ResultVec`)
|
||||
|
||||
Response to a :zeek:see:`Management::Controller::API::restart_request`
|
||||
event. The controller sends this back to the client when it has received
|
||||
responses from all agents involved, or a timeout occurs.
|
||||
|
||||
|
||||
:param reqid: the request identifier used in the request event.
|
||||
|
||||
|
||||
:param results: a :zeek:type:`vector` of :zeek:see:`Management::Result`,
|
||||
combining the restart results from all agents. Each such result
|
||||
identifies both the instance and node in question. Results that
|
||||
do not identify an instance are generated by the controller,
|
||||
flagging corner cases, including absence of a deployed cluster
|
||||
or unknown nodes.
|
||||
|
||||
|
||||
.. zeek:id:: Management::Controller::API::stage_configuration_request
|
||||
:source-code: policy/frameworks/management/controller/main.zeek 1002 1061
|
||||
|
||||
:Type: :zeek:type:`event` (reqid: :zeek:type:`string`, config: :zeek:type:`Management::Configuration`)
|
||||
|
||||
Upload a configuration to the controller for later deployment.
|
||||
The client sends this event to the controller, which validates the
|
||||
configuration and indicates the outcome in its response event. No
|
||||
deployment takes place yet, and existing deployed configurations and
|
||||
the running Zeek cluster remain intact. To trigger deployment of an uploaded
|
||||
configuration, use :zeek:see:`Management::Controller::API::deploy_request`.
|
||||
|
||||
|
||||
:param reqid: a request identifier string, echoed in the response event.
|
||||
|
||||
|
||||
:param config: a :zeek:see:`Management::Configuration` record
|
||||
specifying the cluster configuration.
|
||||
|
||||
|
||||
.. zeek:id:: Management::Controller::API::stage_configuration_response
|
||||
:source-code: policy/frameworks/management/controller/api.zeek 63 63
|
||||
|
||||
:Type: :zeek:type:`event` (reqid: :zeek:type:`string`, results: :zeek:type:`Management::ResultVec`)
|
||||
|
||||
Response to a
|
||||
:zeek:see:`Management::Controller::API::stage_configuration_request`
|
||||
event. The controller sends this back to the client, conveying
|
||||
validation results.
|
||||
|
||||
|
||||
:param reqid: the request identifier used in the request event.
|
||||
|
||||
|
||||
:param results: a :zeek:see:`Management::Result` vector, indicating whether
|
||||
the controller accepts the configuration. In case of a success,
|
||||
a single result record indicates so. Otherwise, the sequence is
|
||||
all errors, each indicating a configuration validation error.
|
||||
|
||||
|
||||
.. zeek:id:: Management::Controller::API::test_timeout_request
|
||||
:source-code: policy/frameworks/management/controller/main.zeek 1588 1599
|
||||
|
||||
:Type: :zeek:type:`event` (reqid: :zeek:type:`string`, with_state: :zeek:type:`bool`)
|
||||
|
||||
This event causes no further action (other than getting logged) if
|
||||
with_state is F. When T, the controller establishes request state, and
|
||||
the controller only ever sends the response event when this state times
|
||||
out.
|
||||
|
||||
|
||||
:param reqid: a request identifier string, echoed in the response event when
|
||||
with_state is T.
|
||||
|
||||
|
||||
:param with_state: flag indicating whether the controller should keep (and
|
||||
time out) request state for this request.
|
||||
|
||||
|
||||
.. zeek:id:: Management::Controller::API::test_timeout_response
|
||||
:source-code: policy/frameworks/management/controller/api.zeek 238 238
|
||||
|
||||
:Type: :zeek:type:`event` (reqid: :zeek:type:`string`, result: :zeek:type:`Management::Result`)
|
||||
|
||||
Response to a
|
||||
:zeek:see:`Management::Controller::API::test_timeout_request`
|
||||
event. The controller sends this back to the client if the original
|
||||
request had the with_state flag.
|
||||
|
||||
|
||||
:param reqid: the request identifier used in the request event.
|
||||
|
||||
|
||||
|
||||
|
|
@ -0,0 +1,20 @@
|
|||
:tocdepth: 3
|
||||
|
||||
policy/frameworks/management/controller/boot.zeek
|
||||
=================================================
|
||||
|
||||
The cluster controller's boot logic runs in Zeek's supervisor and instructs
|
||||
it to launch the Management controller process. The controller's main logic
|
||||
resides in main.zeek, similarly to other frameworks. The new process will
|
||||
execute that script.
|
||||
|
||||
If the current process is not the Zeek supervisor, this does nothing.
|
||||
|
||||
:Imports: :doc:`base/utils/paths.zeek </scripts/base/utils/paths.zeek>`, :doc:`policy/frameworks/management/controller/config.zeek </scripts/policy/frameworks/management/controller/config.zeek>`
|
||||
|
||||
Summary
|
||||
~~~~~~~
|
||||
|
||||
Detailed Interface
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
|
||||
|
|
@ -0,0 +1,324 @@
|
|||
:tocdepth: 3
|
||||
|
||||
policy/frameworks/management/controller/config.zeek
|
||||
===================================================
|
||||
.. zeek:namespace:: Management::Controller
|
||||
|
||||
Configuration settings for the cluster controller.
|
||||
|
||||
:Namespace: Management::Controller
|
||||
:Imports: :doc:`policy/frameworks/management </scripts/policy/frameworks/management/index>`
|
||||
|
||||
Summary
|
||||
~~~~~~~
|
||||
Redefinable Options
|
||||
###################
|
||||
======================================================================================================================= =====================================================================================
|
||||
:zeek:id:`Management::Controller::auto_assign_broker_ports`: :zeek:type:`bool` :zeek:attr:`&redef` Whether the controller should auto-assign Broker listening ports to
|
||||
cluster nodes that need them and don't have them explicitly specified
|
||||
in cluster configurations.
|
||||
:zeek:id:`Management::Controller::auto_assign_broker_start_port`: :zeek:type:`port` :zeek:attr:`&redef` The TCP start port to use for auto-assigning cluster node listening
|
||||
ports, if :zeek:see:`Management::Controller::auto_assign_broker_ports` is
|
||||
enabled (the default) and nodes don't come with those ports assigned.
|
||||
:zeek:id:`Management::Controller::auto_assign_metrics_ports`: :zeek:type:`bool` :zeek:attr:`&redef` Whether the controller should auto-assign metrics ports for Prometheus
|
||||
to nodes that need them and don't have them explicitly specified in
|
||||
their cluster configurations.
|
||||
:zeek:id:`Management::Controller::auto_assign_metrics_start_port`: :zeek:type:`port` :zeek:attr:`&redef` The TCP start port to use for auto-assigning metrics exposition ports
|
||||
for Prometheus, if :zeek:see:`Management::Controller::auto_assign_metrics_ports`
|
||||
is enabled (the default).
|
||||
:zeek:id:`Management::Controller::default_port`: :zeek:type:`port` :zeek:attr:`&redef` The fallback listen port if :zeek:see:`Management::Controller::listen_port`
|
||||
remains empty.
|
||||
:zeek:id:`Management::Controller::default_port_websocket`: :zeek:type:`port` :zeek:attr:`&redef` The fallback listen port if :zeek:see:`Management::Controller::listen_port_websocket`
|
||||
remains empty.
|
||||
:zeek:id:`Management::Controller::directory`: :zeek:type:`string` :zeek:attr:`&redef` An optional custom output directory for stdout/stderr.
|
||||
:zeek:id:`Management::Controller::listen_address`: :zeek:type:`string` :zeek:attr:`&redef` The network address the controller listens on for Broker clients.
|
||||
:zeek:id:`Management::Controller::listen_address_websocket`: :zeek:type:`string` :zeek:attr:`&redef` The network address the controller listens on for websocket
|
||||
clients.
|
||||
:zeek:id:`Management::Controller::listen_port`: :zeek:type:`string` :zeek:attr:`&redef` The network port the controller listens on for Broker clients.
|
||||
:zeek:id:`Management::Controller::listen_port_websocket`: :zeek:type:`string` :zeek:attr:`&redef` The network port the controller listens on for websocket clients.
|
||||
:zeek:id:`Management::Controller::name`: :zeek:type:`string` :zeek:attr:`&redef` The name of this controller.
|
||||
:zeek:id:`Management::Controller::stderr_file`: :zeek:type:`string` :zeek:attr:`&redef` The controller's stderr log name.
|
||||
:zeek:id:`Management::Controller::stdout_file`: :zeek:type:`string` :zeek:attr:`&redef` The controller's stdout log name.
|
||||
:zeek:id:`Management::Controller::tls_options_websocket`: :zeek:type:`Cluster::WebSocketTLSOptions` :zeek:attr:`&redef` TLS options for the controller's WebSocket server.
|
||||
:zeek:id:`Management::Controller::topic`: :zeek:type:`string` :zeek:attr:`&redef` The controller's Broker topic.
|
||||
======================================================================================================================= =====================================================================================
|
||||
|
||||
Constants
|
||||
#########
|
||||
================================================================== ====================================================================
|
||||
:zeek:id:`Management::Controller::store_name`: :zeek:type:`string` The name of the Broker store the controller uses to persist internal
|
||||
state to disk.
|
||||
================================================================== ====================================================================
|
||||
|
||||
Functions
|
||||
#########
|
||||
================================================================================= ================================================================
|
||||
:zeek:id:`Management::Controller::endpoint_info`: :zeek:type:`function` Returns a :zeek:see:`Broker::EndpointInfo` record describing the
|
||||
controller's Broker connectivity.
|
||||
:zeek:id:`Management::Controller::endpoint_info_websocket`: :zeek:type:`function` Returns a :zeek:see:`Broker::EndpointInfo` record describing the
|
||||
controller's websocket connectivity.
|
||||
:zeek:id:`Management::Controller::get_name`: :zeek:type:`function` Returns the effective name of the controller.
|
||||
:zeek:id:`Management::Controller::network_info`: :zeek:type:`function` Returns a :zeek:see:`Broker::NetworkInfo` record describing the
|
||||
controller's Broker connectivity.
|
||||
:zeek:id:`Management::Controller::network_info_websocket`: :zeek:type:`function` Returns a :zeek:see:`Broker::NetworkInfo` record describing the
|
||||
controller's websocket connectivity.
|
||||
================================================================================= ================================================================
|
||||
|
||||
|
||||
Detailed Interface
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
Redefinable Options
|
||||
###################
|
||||
.. zeek:id:: Management::Controller::auto_assign_broker_ports
|
||||
:source-code: policy/frameworks/management/controller/config.zeek 75 75
|
||||
|
||||
:Type: :zeek:type:`bool`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``T``
|
||||
|
||||
Whether the controller should auto-assign Broker listening ports to
|
||||
cluster nodes that need them and don't have them explicitly specified
|
||||
in cluster configurations.
|
||||
|
||||
.. zeek:id:: Management::Controller::auto_assign_broker_start_port
|
||||
:source-code: policy/frameworks/management/controller/config.zeek 80 80
|
||||
|
||||
:Type: :zeek:type:`port`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``2200/tcp``
|
||||
|
||||
The TCP start port to use for auto-assigning cluster node listening
|
||||
ports, if :zeek:see:`Management::Controller::auto_assign_broker_ports` is
|
||||
enabled (the default) and nodes don't come with those ports assigned.
|
||||
|
||||
.. zeek:id:: Management::Controller::auto_assign_metrics_ports
|
||||
:source-code: policy/frameworks/management/controller/config.zeek 85 85
|
||||
|
||||
:Type: :zeek:type:`bool`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``T``
|
||||
|
||||
Whether the controller should auto-assign metrics ports for Prometheus
|
||||
to nodes that need them and don't have them explicitly specified in
|
||||
their cluster configurations.
|
||||
|
||||
.. zeek:id:: Management::Controller::auto_assign_metrics_start_port
|
||||
:source-code: policy/frameworks/management/controller/config.zeek 90 90
|
||||
|
||||
:Type: :zeek:type:`port`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``9000/tcp``
|
||||
|
||||
The TCP start port to use for auto-assigning metrics exposition ports
|
||||
for Prometheus, if :zeek:see:`Management::Controller::auto_assign_metrics_ports`
|
||||
is enabled (the default).
|
||||
|
||||
.. zeek:id:: Management::Controller::default_port
|
||||
:source-code: policy/frameworks/management/controller/config.zeek 44 44
|
||||
|
||||
:Type: :zeek:type:`port`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``2150/tcp``
|
||||
|
||||
The fallback listen port if :zeek:see:`Management::Controller::listen_port`
|
||||
remains empty. When set to 0/unknown, the controller won't listen
|
||||
for Broker connections. Don't do this if your management agents
|
||||
connect to the controller (instead of the default other way around),
|
||||
as they require Broker connectivity.
|
||||
|
||||
.. zeek:id:: Management::Controller::default_port_websocket
|
||||
:source-code: policy/frameworks/management/controller/config.zeek 62 62
|
||||
|
||||
:Type: :zeek:type:`port`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``2149/tcp``
|
||||
|
||||
The fallback listen port if :zeek:see:`Management::Controller::listen_port_websocket`
|
||||
remains empty. When set to 0/unknown, the controller won't listen
|
||||
for websocket clients.
|
||||
|
||||
.. zeek:id:: Management::Controller::directory
|
||||
:source-code: policy/frameworks/management/controller/config.zeek 99 99
|
||||
|
||||
:Type: :zeek:type:`string`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``""``
|
||||
|
||||
An optional custom output directory for stdout/stderr. Agent and
|
||||
controller currently only log locally, not via the Zeek cluster's
|
||||
logger node. This means that if both write to the same log file,
|
||||
output gets garbled.
|
||||
|
||||
.. zeek:id:: Management::Controller::listen_address
|
||||
:source-code: policy/frameworks/management/controller/config.zeek 31 31
|
||||
|
||||
:Type: :zeek:type:`string`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``""``
|
||||
|
||||
The network address the controller listens on for Broker clients. By
|
||||
default this uses the ZEEK_CONTROLLER_ADDR environment variable, but
|
||||
you may also redef to a specific value. When empty, the
|
||||
implementation falls back to :zeek:see:`Management::default_address`.
|
||||
|
||||
.. zeek:id:: Management::Controller::listen_address_websocket
|
||||
:source-code: policy/frameworks/management/controller/config.zeek 51 51
|
||||
|
||||
:Type: :zeek:type:`string`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``""``
|
||||
|
||||
The network address the controller listens on for websocket
|
||||
clients. By default this uses the ZEEK_CONTROLLER_WEBSOCKET_ADDR
|
||||
environment variable, but you may also redef to a specific
|
||||
value. When empty, the implementation falls back to
|
||||
:zeek:see:`Management::default_address`.
|
||||
|
||||
.. zeek:id:: Management::Controller::listen_port
|
||||
:source-code: policy/frameworks/management/controller/config.zeek 37 37
|
||||
|
||||
:Type: :zeek:type:`string`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``""``
|
||||
|
||||
The network port the controller listens on for Broker clients.
|
||||
Defaults to the ZEEK_CONTROLLER_PORT environment variable.
|
||||
When that is not set, the implementation falls back to
|
||||
:zeek:see:`Management::Controller::default_port`.
|
||||
|
||||
.. zeek:id:: Management::Controller::listen_port_websocket
|
||||
:source-code: policy/frameworks/management/controller/config.zeek 57 57
|
||||
|
||||
:Type: :zeek:type:`string`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``""``
|
||||
|
||||
The network port the controller listens on for websocket clients.
|
||||
Defaults to the ZEEK_CONTROLLER_WEBSOCKET_PORT environment
|
||||
variable. When that is not set, the implementation falls back to
|
||||
:zeek:see:`Management::Controller::default_port_websocket`.
|
||||
|
||||
.. zeek:id:: Management::Controller::name
|
||||
:source-code: policy/frameworks/management/controller/config.zeek 12 12
|
||||
|
||||
:Type: :zeek:type:`string`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``""``
|
||||
|
||||
The name of this controller. Defaults to the value of the
|
||||
ZEEK_CONTROLLER_NAME environment variable. When that is unset and the
|
||||
user doesn't redef the value, the implementation defaults to
|
||||
"controller-<hostname>".
|
||||
|
||||
.. zeek:id:: Management::Controller::stderr_file
|
||||
:source-code: policy/frameworks/management/controller/config.zeek 25 25
|
||||
|
||||
:Type: :zeek:type:`string`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``"stderr"``
|
||||
|
||||
The controller's stderr log name. Like :zeek:see:`Management::Controller::stdout_file`,
|
||||
but for the stderr stream.
|
||||
|
||||
.. zeek:id:: Management::Controller::stdout_file
|
||||
:source-code: policy/frameworks/management/controller/config.zeek 21 21
|
||||
|
||||
:Type: :zeek:type:`string`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``"stdout"``
|
||||
|
||||
The controller's stdout log name. If the string is non-empty, Zeek
|
||||
will produce a free-form log (i.e., not one governed by Zeek's
|
||||
logging framework) in the controller's working directory. If left
|
||||
empty, no such log results.
|
||||
|
||||
Note that the controller also establishes a "proper" Zeek log via the
|
||||
:zeek:see:`Management::Log` module.
|
||||
|
||||
.. zeek:id:: Management::Controller::tls_options_websocket
|
||||
:source-code: policy/frameworks/management/controller/config.zeek 70 70
|
||||
|
||||
:Type: :zeek:type:`Cluster::WebSocketTLSOptions`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default:
|
||||
|
||||
::
|
||||
|
||||
{
|
||||
cert_file=<uninitialized>
|
||||
key_file=<uninitialized>
|
||||
enable_peer_verification=F
|
||||
ca_file=""
|
||||
ciphers=""
|
||||
}
|
||||
|
||||
|
||||
TLS options for the controller's WebSocket server. The default is
|
||||
to operate unencrypted. To replicate Broker's default encryption
|
||||
without endpoint validation, set the
|
||||
:zeek:field:`Cluster::WebSocketTLSOptions$ca_file` field to
|
||||
"NONE" and :zeek:field:`Cluster::WebSocketTLSOptions$ciphers` to
|
||||
"AECDH-AES256-SHA@SECLEVEL=0:AECDH-AES256-SHA:P-384".
|
||||
|
||||
.. zeek:id:: Management::Controller::topic
|
||||
:source-code: policy/frameworks/management/controller/config.zeek 93 93
|
||||
|
||||
:Type: :zeek:type:`string`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``"zeek/management/controller"``
|
||||
|
||||
The controller's Broker topic. Clients send requests to this topic.
|
||||
|
||||
Constants
|
||||
#########
|
||||
.. zeek:id:: Management::Controller::store_name
|
||||
:source-code: policy/frameworks/management/controller/config.zeek 103 103
|
||||
|
||||
:Type: :zeek:type:`string`
|
||||
:Default: ``"controller"``
|
||||
|
||||
The name of the Broker store the controller uses to persist internal
|
||||
state to disk.
|
||||
|
||||
Functions
|
||||
#########
|
||||
.. zeek:id:: Management::Controller::endpoint_info
|
||||
:source-code: policy/frameworks/management/controller/config.zeek 171 179
|
||||
|
||||
:Type: :zeek:type:`function` () : :zeek:type:`Broker::EndpointInfo`
|
||||
|
||||
Returns a :zeek:see:`Broker::EndpointInfo` record describing the
|
||||
controller's Broker connectivity.
|
||||
|
||||
.. zeek:id:: Management::Controller::endpoint_info_websocket
|
||||
:source-code: policy/frameworks/management/controller/config.zeek 181 189
|
||||
|
||||
:Type: :zeek:type:`function` () : :zeek:type:`Broker::EndpointInfo`
|
||||
|
||||
Returns a :zeek:see:`Broker::EndpointInfo` record describing the
|
||||
controller's websocket connectivity.
|
||||
|
||||
.. zeek:id:: Management::Controller::get_name
|
||||
:source-code: policy/frameworks/management/controller/config.zeek 125 131
|
||||
|
||||
:Type: :zeek:type:`function` () : :zeek:type:`string`
|
||||
|
||||
Returns the effective name of the controller.
|
||||
|
||||
.. zeek:id:: Management::Controller::network_info
|
||||
:source-code: policy/frameworks/management/controller/config.zeek 133 150
|
||||
|
||||
:Type: :zeek:type:`function` () : :zeek:type:`Broker::NetworkInfo`
|
||||
|
||||
Returns a :zeek:see:`Broker::NetworkInfo` record describing the
|
||||
controller's Broker connectivity.
|
||||
|
||||
.. zeek:id:: Management::Controller::network_info_websocket
|
||||
:source-code: policy/frameworks/management/controller/config.zeek 152 169
|
||||
|
||||
:Type: :zeek:type:`function` () : :zeek:type:`Broker::NetworkInfo`
|
||||
|
||||
Returns a :zeek:see:`Broker::NetworkInfo` record describing the
|
||||
controller's websocket connectivity.
|
||||
|
||||
|
||||
|
|
@ -0,0 +1,38 @@
|
|||
:orphan:
|
||||
|
||||
Package: policy/frameworks/management/controller
|
||||
================================================
|
||||
|
||||
|
||||
:doc:`/scripts/policy/frameworks/management/controller/config.zeek`
|
||||
|
||||
Configuration settings for the cluster controller.
|
||||
|
||||
:doc:`/scripts/policy/frameworks/management/controller/__load__.zeek`
|
||||
|
||||
The entry point for the Management framework's cluster controller. It runs
|
||||
bootstrap logic for launching a controller process via Zeek's Supervisor.
|
||||
|
||||
:doc:`/scripts/policy/frameworks/management/controller/api.zeek`
|
||||
|
||||
The event API of cluster controllers. Most endpoints consist of event pairs,
|
||||
where the controller answers the client's request event with a corresponding
|
||||
response event. Such event pairs share the same name prefix and end in
|
||||
"_request" and "_response", respectively.
|
||||
|
||||
:doc:`/scripts/policy/frameworks/management/controller/boot.zeek`
|
||||
|
||||
The cluster controller's boot logic runs in Zeek's supervisor and instructs
|
||||
it to launch the Management controller process. The controller's main logic
|
||||
resides in main.zeek, similarly to other frameworks. The new process will
|
||||
execute that script.
|
||||
|
||||
If the current process is not the Zeek supervisor, this does nothing.
|
||||
|
||||
:doc:`/scripts/policy/frameworks/management/controller/main.zeek`
|
||||
|
||||
This is the main "runtime" of the Management framework's controller. Zeek
|
||||
does not load this directly; rather, the controller's bootstrapping module
|
||||
(in ./boot.zeek) specifies it as the script to run in the node newly created
|
||||
by the supervisor.
|
||||
|
||||
|
|
@ -0,0 +1,176 @@
|
|||
:tocdepth: 3
|
||||
|
||||
policy/frameworks/management/controller/main.zeek
|
||||
=================================================
|
||||
.. zeek:namespace:: Management::Controller::Runtime
|
||||
|
||||
This is the main "runtime" of the Management framework's controller. Zeek
|
||||
does not load this directly; rather, the controller's bootstrapping module
|
||||
(in ./boot.zeek) specifies it as the script to run in the node newly created
|
||||
by the supervisor.
|
||||
|
||||
:Namespace: Management::Controller::Runtime
|
||||
:Imports: :doc:`base/frameworks/broker </scripts/base/frameworks/broker/index>`, :doc:`base/frameworks/cluster </scripts/base/frameworks/cluster/index>`, :doc:`policy/frameworks/management </scripts/policy/frameworks/management/index>`, :doc:`policy/frameworks/management/agent/api.zeek </scripts/policy/frameworks/management/agent/api.zeek>`, :doc:`policy/frameworks/management/agent/config.zeek </scripts/policy/frameworks/management/agent/config.zeek>`, :doc:`policy/frameworks/management/controller/api.zeek </scripts/policy/frameworks/management/controller/api.zeek>`, :doc:`policy/frameworks/management/controller/config.zeek </scripts/policy/frameworks/management/controller/config.zeek>`
|
||||
|
||||
Summary
|
||||
~~~~~~~
|
||||
Types
|
||||
#####
|
||||
==================================================================================== ====================================================================
|
||||
:zeek:type:`Management::Controller::Runtime::ConfigState`: :zeek:type:`enum` A cluster configuration uploaded by the client goes through multiple
|
||||
states on its way to deployment.
|
||||
:zeek:type:`Management::Controller::Runtime::DeployState`: :zeek:type:`record` Request state specific to
|
||||
:zeek:see:`Management::Controller::API::deploy_request` and
|
||||
:zeek:see:`Management::Controller::API::deploy_response`.
|
||||
:zeek:type:`Management::Controller::Runtime::GetNodesState`: :zeek:type:`record` Request state specific to
|
||||
:zeek:see:`Management::Controller::API::get_nodes_request` and
|
||||
:zeek:see:`Management::Controller::API::get_nodes_response`.
|
||||
:zeek:type:`Management::Controller::Runtime::NodeDispatchState`: :zeek:type:`record` Request state for node dispatch requests, to track the requested
|
||||
action and received responses.
|
||||
:zeek:type:`Management::Controller::Runtime::RestartState`: :zeek:type:`record` Request state specific to
|
||||
:zeek:see:`Management::Controller::API::restart_request` and
|
||||
:zeek:see:`Management::Controller::API::restart_response`.
|
||||
:zeek:type:`Management::Controller::Runtime::TestState`: :zeek:type:`record` Dummy state for internal state-keeping test cases.
|
||||
==================================================================================== ====================================================================
|
||||
|
||||
Redefinitions
|
||||
#############
|
||||
============================================================================== =============================================================================================================
|
||||
:zeek:type:`Management::Request::Request`: :zeek:type:`record`
|
||||
|
||||
:New Fields: :zeek:type:`Management::Request::Request`
|
||||
|
||||
deploy_state: :zeek:type:`Management::Controller::Runtime::DeployState` :zeek:attr:`&optional`
|
||||
|
||||
get_nodes_state: :zeek:type:`Management::Controller::Runtime::GetNodesState` :zeek:attr:`&optional`
|
||||
|
||||
node_dispatch_state: :zeek:type:`Management::Controller::Runtime::NodeDispatchState` :zeek:attr:`&optional`
|
||||
|
||||
restart_state: :zeek:type:`Management::Controller::Runtime::RestartState` :zeek:attr:`&optional`
|
||||
|
||||
test_state: :zeek:type:`Management::Controller::Runtime::TestState` :zeek:attr:`&optional`
|
||||
:zeek:id:`Management::role`: :zeek:type:`Management::Role` :zeek:attr:`&redef`
|
||||
:zeek:id:`table_expire_interval`: :zeek:type:`interval` :zeek:attr:`&redef`
|
||||
============================================================================== =============================================================================================================
|
||||
|
||||
|
||||
Detailed Interface
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
Types
|
||||
#####
|
||||
.. zeek:type:: Management::Controller::Runtime::ConfigState
|
||||
:source-code: policy/frameworks/management/controller/main.zeek 24 29
|
||||
|
||||
:Type: :zeek:type:`enum`
|
||||
|
||||
.. zeek:enum:: Management::Controller::Runtime::STAGED Management::Controller::Runtime::ConfigState
|
||||
|
||||
As provided by the client.
|
||||
|
||||
.. zeek:enum:: Management::Controller::Runtime::READY Management::Controller::Runtime::ConfigState
|
||||
|
||||
Necessary updates made, e.g. ports filled in.
|
||||
|
||||
.. zeek:enum:: Management::Controller::Runtime::DEPLOYED Management::Controller::Runtime::ConfigState
|
||||
|
||||
Sent off to the agents for deployment.
|
||||
|
||||
A cluster configuration uploaded by the client goes through multiple
|
||||
states on its way to deployment.
|
||||
|
||||
.. zeek:type:: Management::Controller::Runtime::DeployState
|
||||
:source-code: policy/frameworks/management/controller/main.zeek 33 41
|
||||
|
||||
:Type: :zeek:type:`record`
|
||||
|
||||
|
||||
.. zeek:field:: config :zeek:type:`Management::Configuration`
|
||||
|
||||
The cluster configuration the controller is deploying.
|
||||
|
||||
|
||||
.. zeek:field:: is_internal :zeek:type:`bool` :zeek:attr:`&default` = ``F`` :zeek:attr:`&optional`
|
||||
|
||||
Whether this is a controller-internal deployment, or
|
||||
triggered via a request by a remote peer/client.
|
||||
|
||||
|
||||
.. zeek:field:: requests :zeek:type:`set` [:zeek:type:`string`] :zeek:attr:`&default` = ``{ }`` :zeek:attr:`&optional`
|
||||
|
||||
Request state for every controller/agent transaction.
|
||||
|
||||
|
||||
Request state specific to
|
||||
:zeek:see:`Management::Controller::API::deploy_request` and
|
||||
:zeek:see:`Management::Controller::API::deploy_response`.
|
||||
|
||||
.. zeek:type:: Management::Controller::Runtime::GetNodesState
|
||||
:source-code: policy/frameworks/management/controller/main.zeek 46 49
|
||||
|
||||
:Type: :zeek:type:`record`
|
||||
|
||||
|
||||
.. zeek:field:: requests :zeek:type:`set` [:zeek:type:`string`] :zeek:attr:`&default` = ``{ }`` :zeek:attr:`&optional`
|
||||
|
||||
Request state for every controller/agent transaction.
|
||||
|
||||
|
||||
Request state specific to
|
||||
:zeek:see:`Management::Controller::API::get_nodes_request` and
|
||||
:zeek:see:`Management::Controller::API::get_nodes_response`.
|
||||
|
||||
.. zeek:type:: Management::Controller::Runtime::NodeDispatchState
|
||||
:source-code: policy/frameworks/management/controller/main.zeek 62 72
|
||||
|
||||
:Type: :zeek:type:`record`
|
||||
|
||||
|
||||
.. zeek:field:: action :zeek:type:`vector` of :zeek:type:`string`
|
||||
|
||||
The dispatched action. The first string is a command,
|
||||
any remaining strings its arguments.
|
||||
|
||||
|
||||
.. zeek:field:: requests :zeek:type:`set` [:zeek:type:`string`] :zeek:attr:`&default` = ``{ }`` :zeek:attr:`&optional`
|
||||
|
||||
Request state for every controller/agent transaction.
|
||||
The set of strings tracks the node names from which
|
||||
we still expect responses, before we can respond back
|
||||
to the client.
|
||||
|
||||
|
||||
Request state for node dispatch requests, to track the requested
|
||||
action and received responses. Node dispatches are requests to
|
||||
execute pre-implemented actions on every node in the cluster,
|
||||
and report their outcomes. See
|
||||
:zeek:see:`Management::Agent::API::node_dispatch_request` and
|
||||
:zeek:see:`Management::Agent::API::node_dispatch_response` for the
|
||||
agent/controller interaction, and
|
||||
:zeek:see:`Management::Controller::API::get_id_value_request` and
|
||||
:zeek:see:`Management::Controller::API::get_id_value_response`
|
||||
for an example of a specific API the controller generalizes into
|
||||
a dispatch.
|
||||
|
||||
.. zeek:type:: Management::Controller::Runtime::RestartState
|
||||
:source-code: policy/frameworks/management/controller/main.zeek 77 80
|
||||
|
||||
:Type: :zeek:type:`record`
|
||||
|
||||
|
||||
.. zeek:field:: requests :zeek:type:`set` [:zeek:type:`string`] :zeek:attr:`&default` = ``{ }`` :zeek:attr:`&optional`
|
||||
|
||||
Request state for every controller/agent transaction.
|
||||
|
||||
|
||||
Request state specific to
|
||||
:zeek:see:`Management::Controller::API::restart_request` and
|
||||
:zeek:see:`Management::Controller::API::restart_response`.
|
||||
|
||||
.. zeek:type:: Management::Controller::Runtime::TestState
|
||||
:source-code: policy/frameworks/management/controller/main.zeek 83 84
|
||||
|
||||
:Type: :zeek:type:`record`
|
||||
|
||||
Dummy state for internal state-keeping test cases.
|
||||
|
||||
|
||||
148
doc/scripts/policy/frameworks/management/index.rst
Normal file
148
doc/scripts/policy/frameworks/management/index.rst
Normal file
|
|
@ -0,0 +1,148 @@
|
|||
:orphan:
|
||||
|
||||
Package: policy/frameworks/management
|
||||
=====================================
|
||||
|
||||
|
||||
:doc:`/scripts/policy/frameworks/management/agent/__load__.zeek`
|
||||
|
||||
The entry point for the Management framework's cluster agent. It runs
|
||||
bootstrap logic for launching an agent process via Zeek's Supervisor.
|
||||
|
||||
:doc:`/scripts/policy/frameworks/management/agent/api.zeek`
|
||||
|
||||
The event API of cluster agents. Most endpoints consist of event pairs,
|
||||
where the agent answers a request event with a corresponding response
|
||||
event. Such event pairs share the same name prefix and end in "_request" and
|
||||
"_response", respectively.
|
||||
|
||||
:doc:`/scripts/policy/frameworks/management/types.zeek`
|
||||
|
||||
This module holds the basic types needed for the Management framework. These
|
||||
are used by both cluster agent and controller, and several have corresponding
|
||||
implementations in zeek-client.
|
||||
|
||||
:doc:`/scripts/policy/frameworks/management/agent/boot.zeek`
|
||||
|
||||
The cluster agent boot logic runs in Zeek's supervisor and instructs it to
|
||||
launch a Management agent process. The agent's main logic resides in main.zeek,
|
||||
similarly to other frameworks. The new process will execute that script.
|
||||
|
||||
If the current process is not the Zeek supervisor, this does nothing.
|
||||
|
||||
:doc:`/scripts/policy/frameworks/management/agent/config.zeek`
|
||||
|
||||
Configuration settings for a cluster agent.
|
||||
|
||||
:doc:`/scripts/policy/frameworks/management/__load__.zeek`
|
||||
|
||||
This loads Management framework functionality needed by both the controller
|
||||
and agents. Note that there's no notion of loading "the Management
|
||||
framework" -- one always loads "management/controller" or
|
||||
"management/agent". This __load__ script exists only to simplify loading all
|
||||
common functionality.
|
||||
|
||||
:doc:`/scripts/policy/frameworks/management/config.zeek`
|
||||
|
||||
Management framework configuration settings common to agent and controller.
|
||||
This does not include config settings that exist in both agent and
|
||||
controller but that they set differently, since setting defaults here would
|
||||
be awkward or pointless (since both node types would overwrite them
|
||||
anyway). For role-specific settings, see management/controller/config.zeek
|
||||
and management/agent/config.zeek.
|
||||
|
||||
:doc:`/scripts/policy/frameworks/management/log.zeek`
|
||||
|
||||
This module implements logging abilities for controller and agent. It uses
|
||||
Zeek's logging framework and works only for nodes managed by the
|
||||
supervisor. In this setting Zeek's logging framework operates locally, i.e.,
|
||||
this does not involve logger nodes.
|
||||
|
||||
:doc:`/scripts/policy/frameworks/management/persistence.zeek`
|
||||
|
||||
Common adjustments for any kind of Zeek node when we run the Management
|
||||
framework.
|
||||
|
||||
:doc:`/scripts/policy/frameworks/management/request.zeek`
|
||||
|
||||
This module implements a request state abstraction in the Management
|
||||
framework that both controller and agent use to connect request events to
|
||||
subsequent response ones, and to be able to time out such requests.
|
||||
|
||||
:doc:`/scripts/policy/frameworks/management/util.zeek`
|
||||
|
||||
Utility functions for the Management framework, available to agent
|
||||
and controller.
|
||||
|
||||
:doc:`/scripts/policy/frameworks/management/controller/config.zeek`
|
||||
|
||||
Configuration settings for the cluster controller.
|
||||
|
||||
:doc:`/scripts/policy/frameworks/management/controller/__load__.zeek`
|
||||
|
||||
The entry point for the Management framework's cluster controller. It runs
|
||||
bootstrap logic for launching a controller process via Zeek's Supervisor.
|
||||
|
||||
:doc:`/scripts/policy/frameworks/management/controller/api.zeek`
|
||||
|
||||
The event API of cluster controllers. Most endpoints consist of event pairs,
|
||||
where the controller answers the client's request event with a corresponding
|
||||
response event. Such event pairs share the same name prefix and end in
|
||||
"_request" and "_response", respectively.
|
||||
|
||||
:doc:`/scripts/policy/frameworks/management/controller/boot.zeek`
|
||||
|
||||
The cluster controller's boot logic runs in Zeek's supervisor and instructs
|
||||
it to launch the Management controller process. The controller's main logic
|
||||
resides in main.zeek, similarly to other frameworks. The new process will
|
||||
execute that script.
|
||||
|
||||
If the current process is not the Zeek supervisor, this does nothing.
|
||||
|
||||
:doc:`/scripts/policy/frameworks/management/node/api.zeek`
|
||||
|
||||
The Management event API of cluster nodes. The API consists of request/
|
||||
response event pairs, like elsewhere in the Management, Supervisor, and
|
||||
Control frameworks.
|
||||
|
||||
:doc:`/scripts/policy/frameworks/management/node/config.zeek`
|
||||
|
||||
Configuration settings for nodes controlled by the Management framework.
|
||||
|
||||
:doc:`/scripts/policy/frameworks/management/supervisor/__load__.zeek`
|
||||
|
||||
|
||||
:doc:`/scripts/policy/frameworks/management/supervisor/main.zeek`
|
||||
|
||||
This module provides functionality the Management framework places directly
|
||||
in the Supervisor.
|
||||
|
||||
:doc:`/scripts/policy/frameworks/management/supervisor/api.zeek`
|
||||
|
||||
|
||||
:doc:`/scripts/policy/frameworks/management/supervisor/config.zeek`
|
||||
|
||||
Configuration settings for the Management framework's supervisor extension.
|
||||
|
||||
:doc:`/scripts/policy/frameworks/management/agent/main.zeek`
|
||||
|
||||
This is the main "runtime" of a cluster agent. Zeek does not load this
|
||||
directly; rather, the agent's bootstrapping module (in ./boot.zeek)
|
||||
specifies it as the script to run in the node newly created via Zeek's
|
||||
supervisor.
|
||||
|
||||
:doc:`/scripts/policy/frameworks/management/controller/main.zeek`
|
||||
|
||||
This is the main "runtime" of the Management framework's controller. Zeek
|
||||
does not load this directly; rather, the controller's bootstrapping module
|
||||
(in ./boot.zeek) specifies it as the script to run in the node newly created
|
||||
by the supervisor.
|
||||
|
||||
:doc:`/scripts/policy/frameworks/management/node/__load__.zeek`
|
||||
|
||||
|
||||
:doc:`/scripts/policy/frameworks/management/node/main.zeek`
|
||||
|
||||
This module provides Management framework functionality present in every
|
||||
cluster node, to allowing Management agents to interact with the nodes.
|
||||
|
||||
157
doc/scripts/policy/frameworks/management/log.zeek.rst
Normal file
157
doc/scripts/policy/frameworks/management/log.zeek.rst
Normal file
|
|
@ -0,0 +1,157 @@
|
|||
:tocdepth: 3
|
||||
|
||||
policy/frameworks/management/log.zeek
|
||||
=====================================
|
||||
.. zeek:namespace:: Management::Log
|
||||
|
||||
This module implements logging abilities for controller and agent. It uses
|
||||
Zeek's logging framework and works only for nodes managed by the
|
||||
supervisor. In this setting Zeek's logging framework operates locally, i.e.,
|
||||
this does not involve logger nodes.
|
||||
|
||||
:Namespace: Management::Log
|
||||
:Imports: :doc:`policy/frameworks/management/config.zeek </scripts/policy/frameworks/management/config.zeek>`
|
||||
|
||||
Summary
|
||||
~~~~~~~
|
||||
Types
|
||||
#####
|
||||
========================================================================= =========================================================================
|
||||
:zeek:type:`Management::Log::Info`: :zeek:type:`record` :zeek:attr:`&log` The record type containing the column fields of the agent/controller log.
|
||||
:zeek:type:`Management::Log::Level`: :zeek:type:`enum` The controller/agent log supports four different log levels.
|
||||
========================================================================= =========================================================================
|
||||
|
||||
Redefinitions
|
||||
#############
|
||||
======================================= ======================================
|
||||
:zeek:type:`Log::ID`: :zeek:type:`enum` The cluster logging stream identifier.
|
||||
|
||||
* :zeek:enum:`Management::Log::LOG`
|
||||
======================================= ======================================
|
||||
|
||||
Hooks
|
||||
#####
|
||||
==================================================================== =============================================
|
||||
:zeek:id:`Management::Log::log_policy`: :zeek:type:`Log::PolicyHook` A default logging policy hook for the stream.
|
||||
==================================================================== =============================================
|
||||
|
||||
Functions
|
||||
#########
|
||||
========================================================== ===================================
|
||||
:zeek:id:`Management::Log::debug`: :zeek:type:`function` A debug-level log message writer.
|
||||
:zeek:id:`Management::Log::error`: :zeek:type:`function` An error-level log message writer.
|
||||
:zeek:id:`Management::Log::info`: :zeek:type:`function` An info-level log message writer.
|
||||
:zeek:id:`Management::Log::warning`: :zeek:type:`function` A warning-level log message writer.
|
||||
========================================================== ===================================
|
||||
|
||||
|
||||
Detailed Interface
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
Types
|
||||
#####
|
||||
.. zeek:type:: Management::Log::Info
|
||||
:source-code: policy/frameworks/management/log.zeek 26 37
|
||||
|
||||
:Type: :zeek:type:`record`
|
||||
|
||||
|
||||
.. zeek:field:: ts :zeek:type:`time` :zeek:attr:`&log`
|
||||
|
||||
The time at which a cluster message was generated.
|
||||
|
||||
|
||||
.. zeek:field:: node :zeek:type:`string` :zeek:attr:`&log`
|
||||
|
||||
The name of the node that is creating the log record.
|
||||
|
||||
|
||||
.. zeek:field:: level :zeek:type:`string` :zeek:attr:`&log`
|
||||
|
||||
Log level of this message, converted from the above Level enum
|
||||
|
||||
|
||||
.. zeek:field:: role :zeek:type:`string` :zeek:attr:`&log`
|
||||
|
||||
The role of the node, translated from Management::Role.
|
||||
|
||||
|
||||
.. zeek:field:: message :zeek:type:`string` :zeek:attr:`&log`
|
||||
|
||||
A message indicating information about cluster controller operation.
|
||||
|
||||
:Attributes: :zeek:attr:`&log`
|
||||
|
||||
The record type containing the column fields of the agent/controller log.
|
||||
|
||||
.. zeek:type:: Management::Log::Level
|
||||
:source-code: policy/frameworks/management/log.zeek 18 24
|
||||
|
||||
:Type: :zeek:type:`enum`
|
||||
|
||||
.. zeek:enum:: Management::Log::DEBUG Management::Log::Level
|
||||
|
||||
.. zeek:enum:: Management::Log::INFO Management::Log::Level
|
||||
|
||||
.. zeek:enum:: Management::Log::WARNING Management::Log::Level
|
||||
|
||||
.. zeek:enum:: Management::Log::ERROR Management::Log::Level
|
||||
|
||||
The controller/agent log supports four different log levels.
|
||||
|
||||
Hooks
|
||||
#####
|
||||
.. zeek:id:: Management::Log::log_policy
|
||||
:source-code: policy/frameworks/management/log.zeek 15 15
|
||||
|
||||
:Type: :zeek:type:`Log::PolicyHook`
|
||||
|
||||
A default logging policy hook for the stream.
|
||||
|
||||
Functions
|
||||
#########
|
||||
.. zeek:id:: Management::Log::debug
|
||||
:source-code: policy/frameworks/management/log.zeek 85 93
|
||||
|
||||
:Type: :zeek:type:`function` (message: :zeek:type:`string`) : :zeek:type:`void`
|
||||
|
||||
A debug-level log message writer.
|
||||
|
||||
|
||||
:param message: the message to log.
|
||||
|
||||
|
||||
.. zeek:id:: Management::Log::error
|
||||
:source-code: policy/frameworks/management/log.zeek 115 123
|
||||
|
||||
:Type: :zeek:type:`function` (message: :zeek:type:`string`) : :zeek:type:`void`
|
||||
|
||||
An error-level log message writer. (This only logs a message, it does not
|
||||
terminate Zeek or have other runtime effects.)
|
||||
|
||||
|
||||
:param message: the message to log.
|
||||
|
||||
|
||||
.. zeek:id:: Management::Log::info
|
||||
:source-code: policy/frameworks/management/log.zeek 95 103
|
||||
|
||||
:Type: :zeek:type:`function` (message: :zeek:type:`string`) : :zeek:type:`void`
|
||||
|
||||
An info-level log message writer.
|
||||
|
||||
|
||||
:param message: the message to log.
|
||||
|
||||
|
||||
.. zeek:id:: Management::Log::warning
|
||||
:source-code: policy/frameworks/management/log.zeek 105 113
|
||||
|
||||
:Type: :zeek:type:`function` (message: :zeek:type:`string`) : :zeek:type:`void`
|
||||
|
||||
A warning-level log message writer.
|
||||
|
||||
|
||||
:param message: the message to log.
|
||||
|
||||
|
||||
|
||||
|
|
@ -0,0 +1,14 @@
|
|||
:tocdepth: 3
|
||||
|
||||
policy/frameworks/management/node/__load__.zeek
|
||||
===============================================
|
||||
|
||||
|
||||
:Imports: :doc:`policy/frameworks/management/node/main.zeek </scripts/policy/frameworks/management/node/main.zeek>`
|
||||
|
||||
Summary
|
||||
~~~~~~~
|
||||
|
||||
Detailed Interface
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
|
||||
85
doc/scripts/policy/frameworks/management/node/api.zeek.rst
Normal file
85
doc/scripts/policy/frameworks/management/node/api.zeek.rst
Normal file
|
|
@ -0,0 +1,85 @@
|
|||
:tocdepth: 3
|
||||
|
||||
policy/frameworks/management/node/api.zeek
|
||||
==========================================
|
||||
.. zeek:namespace:: Management::Node::API
|
||||
|
||||
The Management event API of cluster nodes. The API consists of request/
|
||||
response event pairs, like elsewhere in the Management, Supervisor, and
|
||||
Control frameworks.
|
||||
|
||||
:Namespace: Management::Node::API
|
||||
:Imports: :doc:`policy/frameworks/management/types.zeek </scripts/policy/frameworks/management/types.zeek>`
|
||||
|
||||
Summary
|
||||
~~~~~~~
|
||||
Events
|
||||
######
|
||||
============================================================================ =====================================================================
|
||||
:zeek:id:`Management::Node::API::node_dispatch_request`: :zeek:type:`event` Management agents send this event to every Zeek cluster node to run a
|
||||
"dispatch" -- a particular, pre-implemented action.
|
||||
:zeek:id:`Management::Node::API::node_dispatch_response`: :zeek:type:`event` Response to a node_dispatch_request event.
|
||||
:zeek:id:`Management::Node::API::notify_node_hello`: :zeek:type:`event` The cluster nodes send this event upon peering as a "check-in" to
|
||||
the agent, to indicate the node is now available to communicate
|
||||
with.
|
||||
============================================================================ =====================================================================
|
||||
|
||||
|
||||
Detailed Interface
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
Events
|
||||
######
|
||||
.. zeek:id:: Management::Node::API::node_dispatch_request
|
||||
:source-code: policy/frameworks/management/node/main.zeek 58 97
|
||||
|
||||
:Type: :zeek:type:`event` (reqid: :zeek:type:`string`, action: :zeek:type:`vector` of :zeek:type:`string`, nodes: :zeek:type:`set` [:zeek:type:`string`] :zeek:attr:`&default` = ``{ }`` :zeek:attr:`&optional`)
|
||||
|
||||
Management agents send this event to every Zeek cluster node to run a
|
||||
"dispatch" -- a particular, pre-implemented action. This is the agent-node
|
||||
complement to :zeek:see:`Management::Agent::API::node_dispatch_request`.
|
||||
|
||||
|
||||
:param reqid: a request identifier string, echoed in the response event.
|
||||
|
||||
|
||||
:param action: the requested dispatch command, with any arguments.
|
||||
|
||||
|
||||
:param nodes: the cluster node names this dispatch targets. An empty set,
|
||||
supplied by default, means it applies to all nodes. Since nodes
|
||||
receive all dispatch requests, they can use any node names provided
|
||||
here to filter themselves out of responding.
|
||||
|
||||
.. zeek:id:: Management::Node::API::node_dispatch_response
|
||||
:source-code: policy/frameworks/management/agent/main.zeek 690 759
|
||||
|
||||
:Type: :zeek:type:`event` (reqid: :zeek:type:`string`, result: :zeek:type:`Management::Result`)
|
||||
|
||||
Response to a node_dispatch_request event. The nodes send this back
|
||||
to the agent. This is the agent-node equivalent of
|
||||
:zeek:see:`Management::Agent::API::node_dispatch_response`.
|
||||
|
||||
|
||||
:param reqid: the request identifier used in the request event.
|
||||
|
||||
|
||||
:param result: a :zeek:see:`Management::Result` record covering one Zeek
|
||||
cluster node managed by the agent. Upon success, the data field
|
||||
contains a value appropriate for the requested dispatch.
|
||||
|
||||
.. zeek:id:: Management::Node::API::notify_node_hello
|
||||
:source-code: policy/frameworks/management/agent/main.zeek 1010 1033
|
||||
|
||||
:Type: :zeek:type:`event` (node: :zeek:type:`string`)
|
||||
|
||||
The cluster nodes send this event upon peering as a "check-in" to
|
||||
the agent, to indicate the node is now available to communicate
|
||||
with. It is an agent-level equivalent of :zeek:see:`Broker::peer_added`,
|
||||
and similar to :zeek:see:`Management::Agent::API::notify_agent_hello`
|
||||
for agents.
|
||||
|
||||
|
||||
:param node: the name of the node, as given in :zeek:see:`Cluster::node`.
|
||||
|
||||
|
||||
|
||||
|
|
@ -0,0 +1,61 @@
|
|||
:tocdepth: 3
|
||||
|
||||
policy/frameworks/management/node/config.zeek
|
||||
=============================================
|
||||
.. zeek:namespace:: Management::Node
|
||||
|
||||
Configuration settings for nodes controlled by the Management framework.
|
||||
|
||||
:Namespace: Management::Node
|
||||
|
||||
Summary
|
||||
~~~~~~~
|
||||
Redefinable Options
|
||||
###################
|
||||
================================================================================= ======================================
|
||||
:zeek:id:`Management::Node::node_topic`: :zeek:type:`string` :zeek:attr:`&redef` The nodes' Broker topic.
|
||||
:zeek:id:`Management::Node::stderr_file`: :zeek:type:`string` :zeek:attr:`&redef` Cluster node stderr log configuration.
|
||||
:zeek:id:`Management::Node::stdout_file`: :zeek:type:`string` :zeek:attr:`&redef` Cluster node stdout log configuration.
|
||||
================================================================================= ======================================
|
||||
|
||||
|
||||
Detailed Interface
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
Redefinable Options
|
||||
###################
|
||||
.. zeek:id:: Management::Node::node_topic
|
||||
:source-code: policy/frameworks/management/node/config.zeek 8 8
|
||||
|
||||
:Type: :zeek:type:`string`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``"zeek/management/node"``
|
||||
|
||||
The nodes' Broker topic. Cluster nodes automatically subscribe
|
||||
to it, to receive request events from the Management framework.
|
||||
|
||||
.. zeek:id:: Management::Node::stderr_file
|
||||
:source-code: policy/frameworks/management/node/config.zeek 21 21
|
||||
|
||||
:Type: :zeek:type:`string`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``"stderr"``
|
||||
|
||||
Cluster node stderr log configuration. Like
|
||||
:zeek:see:`Management::Node::stdout_file`, but for the stderr stream.
|
||||
|
||||
.. zeek:id:: Management::Node::stdout_file
|
||||
:source-code: policy/frameworks/management/node/config.zeek 17 17
|
||||
|
||||
:Type: :zeek:type:`string`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``"stdout"``
|
||||
|
||||
Cluster node stdout log configuration. If the string is non-empty,
|
||||
Zeek will produce a free-form log (i.e., not one governed by Zeek's
|
||||
logging framework) in the node's working directory. If left empty, no
|
||||
such log results.
|
||||
|
||||
Note that cluster nodes also establish a "proper" management log via
|
||||
the :zeek:see:`Management::Log` module.
|
||||
|
||||
|
||||
24
doc/scripts/policy/frameworks/management/node/index.rst
Normal file
24
doc/scripts/policy/frameworks/management/node/index.rst
Normal file
|
|
@ -0,0 +1,24 @@
|
|||
:orphan:
|
||||
|
||||
Package: policy/frameworks/management/node
|
||||
==========================================
|
||||
|
||||
|
||||
:doc:`/scripts/policy/frameworks/management/node/api.zeek`
|
||||
|
||||
The Management event API of cluster nodes. The API consists of request/
|
||||
response event pairs, like elsewhere in the Management, Supervisor, and
|
||||
Control frameworks.
|
||||
|
||||
:doc:`/scripts/policy/frameworks/management/node/config.zeek`
|
||||
|
||||
Configuration settings for nodes controlled by the Management framework.
|
||||
|
||||
:doc:`/scripts/policy/frameworks/management/node/__load__.zeek`
|
||||
|
||||
|
||||
:doc:`/scripts/policy/frameworks/management/node/main.zeek`
|
||||
|
||||
This module provides Management framework functionality present in every
|
||||
cluster node, to allowing Management agents to interact with the nodes.
|
||||
|
||||
24
doc/scripts/policy/frameworks/management/node/main.zeek.rst
Normal file
24
doc/scripts/policy/frameworks/management/node/main.zeek.rst
Normal file
|
|
@ -0,0 +1,24 @@
|
|||
:tocdepth: 3
|
||||
|
||||
policy/frameworks/management/node/main.zeek
|
||||
===========================================
|
||||
.. zeek:namespace:: Management::Node
|
||||
|
||||
This module provides Management framework functionality present in every
|
||||
cluster node, to allowing Management agents to interact with the nodes.
|
||||
|
||||
:Namespace: Management::Node
|
||||
:Imports: :doc:`base/frameworks/broker/store.zeek </scripts/base/frameworks/broker/store.zeek>`, :doc:`base/frameworks/cluster </scripts/base/frameworks/cluster/index>`, :doc:`base/frameworks/logging/writers/ascii.zeek </scripts/base/frameworks/logging/writers/ascii.zeek>`, :doc:`base/misc/installation.zeek </scripts/base/misc/installation.zeek>`, :doc:`base/utils/paths.zeek </scripts/base/utils/paths.zeek>`, :doc:`policy/frameworks/management </scripts/policy/frameworks/management/index>`, :doc:`policy/frameworks/management/agent/config.zeek </scripts/policy/frameworks/management/agent/config.zeek>`, :doc:`policy/frameworks/management/node/api.zeek </scripts/policy/frameworks/management/node/api.zeek>`, :doc:`policy/frameworks/management/node/config.zeek </scripts/policy/frameworks/management/node/config.zeek>`
|
||||
|
||||
Summary
|
||||
~~~~~~~
|
||||
Redefinitions
|
||||
#############
|
||||
============================================================================== =
|
||||
:zeek:id:`Management::role`: :zeek:type:`Management::Role` :zeek:attr:`&redef`
|
||||
============================================================================== =
|
||||
|
||||
|
||||
Detailed Interface
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
|
||||
|
|
@ -0,0 +1,22 @@
|
|||
:tocdepth: 3
|
||||
|
||||
policy/frameworks/management/persistence.zeek
|
||||
=============================================
|
||||
|
||||
Common adjustments for any kind of Zeek node when we run the Management
|
||||
framework.
|
||||
|
||||
:Imports: :doc:`base/misc/installation.zeek </scripts/base/misc/installation.zeek>`, :doc:`base/utils/paths.zeek </scripts/base/utils/paths.zeek>`, :doc:`policy/frameworks/management/config.zeek </scripts/policy/frameworks/management/config.zeek>`
|
||||
|
||||
Summary
|
||||
~~~~~~~
|
||||
Redefinitions
|
||||
#############
|
||||
============================================================================= =
|
||||
:zeek:id:`Log::default_rotation_dir`: :zeek:type:`string` :zeek:attr:`&redef`
|
||||
============================================================================= =
|
||||
|
||||
|
||||
Detailed Interface
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
|
||||
295
doc/scripts/policy/frameworks/management/request.zeek.rst
Normal file
295
doc/scripts/policy/frameworks/management/request.zeek.rst
Normal file
|
|
@ -0,0 +1,295 @@
|
|||
:tocdepth: 3
|
||||
|
||||
policy/frameworks/management/request.zeek
|
||||
=========================================
|
||||
.. zeek:namespace:: Management::Request
|
||||
|
||||
This module implements a request state abstraction in the Management
|
||||
framework that both controller and agent use to connect request events to
|
||||
subsequent response ones, and to be able to time out such requests.
|
||||
|
||||
:Namespace: Management::Request
|
||||
:Imports: :doc:`policy/frameworks/management/config.zeek </scripts/policy/frameworks/management/config.zeek>`, :doc:`policy/frameworks/management/types.zeek </scripts/policy/frameworks/management/types.zeek>`
|
||||
|
||||
Summary
|
||||
~~~~~~~
|
||||
Redefinable Options
|
||||
###################
|
||||
=========================================================================================== =======================================
|
||||
:zeek:id:`Management::Request::timeout_interval`: :zeek:type:`interval` :zeek:attr:`&redef` The timeout interval for request state.
|
||||
=========================================================================================== =======================================
|
||||
|
||||
State Variables
|
||||
###############
|
||||
=================================================================================== ==========================================================
|
||||
:zeek:id:`Management::Request::null_req`: :zeek:type:`Management::Request::Request` A token request that serves as a null/nonexistent request.
|
||||
=================================================================================== ==========================================================
|
||||
|
||||
Types
|
||||
#####
|
||||
============================================================== ====================================================================
|
||||
:zeek:type:`Management::Request::Request`: :zeek:type:`record` Request records track state associated with a request/response event
|
||||
pair.
|
||||
============================================================== ====================================================================
|
||||
|
||||
Redefinitions
|
||||
#############
|
||||
============================================================== ===========================================================================================================================
|
||||
:zeek:type:`Management::Request::Request`: :zeek:type:`record`
|
||||
|
||||
:New Fields: :zeek:type:`Management::Request::Request`
|
||||
|
||||
finish: :zeek:type:`function` (req: :zeek:type:`Management::Request::Request`) : :zeek:type:`void` :zeek:attr:`&optional`
|
||||
A callback to invoke when this request is finished via
|
||||
:zeek:see:`Management::Request::finish`.
|
||||
============================================================== ===========================================================================================================================
|
||||
|
||||
Events
|
||||
######
|
||||
=================================================================== ======================================================================
|
||||
:zeek:id:`Management::Request::request_expired`: :zeek:type:`event` This event fires when a request times out (as per the
|
||||
Management::Request::timeout_interval) before it has been finished via
|
||||
Management::Request::finish().
|
||||
=================================================================== ======================================================================
|
||||
|
||||
Functions
|
||||
#########
|
||||
================================================================ ========================================================================
|
||||
:zeek:id:`Management::Request::create`: :zeek:type:`function` This function establishes request state.
|
||||
:zeek:id:`Management::Request::finish`: :zeek:type:`function` This function marks a request as complete and causes Zeek to release
|
||||
its internal state.
|
||||
:zeek:id:`Management::Request::is_null`: :zeek:type:`function` This function is a helper predicate to indicate whether a given
|
||||
request is null.
|
||||
:zeek:id:`Management::Request::lookup`: :zeek:type:`function` This function looks up the request for a given request ID and returns
|
||||
it.
|
||||
:zeek:id:`Management::Request::to_string`: :zeek:type:`function` For troubleshooting, this function renders a request record to a string.
|
||||
================================================================ ========================================================================
|
||||
|
||||
|
||||
Detailed Interface
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
Redefinable Options
|
||||
###################
|
||||
.. zeek:id:: Management::Request::timeout_interval
|
||||
:source-code: policy/frameworks/management/request.zeek 52 52
|
||||
|
||||
:Type: :zeek:type:`interval`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``10.0 secs``
|
||||
:Redefinition: from :doc:`/scripts/policy/frameworks/management/agent/main.zeek`
|
||||
|
||||
``=``::
|
||||
|
||||
5.0 secs
|
||||
|
||||
|
||||
The timeout interval for request state. Such state (see the
|
||||
:zeek:see:`Management::Request` module) ties together request and
|
||||
response event pairs. A timeout causes cleanup of request state if
|
||||
regular request/response processing hasn't already done so. It
|
||||
applies both to request state kept in the controller and the agent,
|
||||
though the two use different timeout values: agent-side requests time
|
||||
out more quickly. This allows agents to send more meaningful error
|
||||
messages, while the controller's timeouts serve as a last resort to
|
||||
ensure response to the client.
|
||||
|
||||
State Variables
|
||||
###############
|
||||
.. zeek:id:: Management::Request::null_req
|
||||
:source-code: policy/frameworks/management/request.zeek 55 55
|
||||
|
||||
:Type: :zeek:type:`Management::Request::Request`
|
||||
:Default:
|
||||
|
||||
::
|
||||
|
||||
{
|
||||
id=""
|
||||
parent_id=<uninitialized>
|
||||
results=[]
|
||||
finished=T
|
||||
finish=<uninitialized>
|
||||
supervisor_state_agent=<uninitialized>
|
||||
deploy_state_agent=<uninitialized>
|
||||
node_dispatch_state_agent=<uninitialized>
|
||||
restart_state_agent=<uninitialized>
|
||||
deploy_state=<uninitialized>
|
||||
get_nodes_state=<uninitialized>
|
||||
node_dispatch_state=<uninitialized>
|
||||
restart_state=<uninitialized>
|
||||
test_state=<uninitialized>
|
||||
}
|
||||
|
||||
|
||||
A token request that serves as a null/nonexistent request.
|
||||
|
||||
Types
|
||||
#####
|
||||
.. zeek:type:: Management::Request::Request
|
||||
:source-code: policy/frameworks/management/request.zeek 17 33
|
||||
|
||||
:Type: :zeek:type:`record`
|
||||
|
||||
|
||||
.. zeek:field:: id :zeek:type:`string`
|
||||
|
||||
Each request has a hopefully unique ID provided by the requester.
|
||||
|
||||
|
||||
.. zeek:field:: parent_id :zeek:type:`string` :zeek:attr:`&optional`
|
||||
|
||||
For requests that result based upon another request (such as when
|
||||
the controller sends requests to agents based on a request it
|
||||
received by the client), this specifies that original, "parent"
|
||||
request.
|
||||
|
||||
|
||||
.. zeek:field:: results :zeek:type:`Management::ResultVec` :zeek:attr:`&default` = ``[]`` :zeek:attr:`&optional`
|
||||
|
||||
The results vector builds up the list of results we eventually
|
||||
send to the requestor when we have processed the request.
|
||||
|
||||
|
||||
.. zeek:field:: finished :zeek:type:`bool` :zeek:attr:`&default` = ``F`` :zeek:attr:`&optional`
|
||||
|
||||
An internal flag to track whether a request is complete.
|
||||
|
||||
|
||||
.. zeek:field:: finish :zeek:type:`function` (<recursion>) : :zeek:type:`void` :zeek:attr:`&optional`
|
||||
|
||||
A callback to invoke when this request is finished via
|
||||
:zeek:see:`Management::Request::finish`.
|
||||
|
||||
|
||||
.. zeek:field:: supervisor_state_agent :zeek:type:`Management::Agent::Runtime::SupervisorState` :zeek:attr:`&optional`
|
||||
|
||||
(present if :doc:`/scripts/policy/frameworks/management/agent/main.zeek` is loaded)
|
||||
|
||||
|
||||
.. zeek:field:: deploy_state_agent :zeek:type:`Management::Agent::Runtime::DeployState` :zeek:attr:`&optional`
|
||||
|
||||
(present if :doc:`/scripts/policy/frameworks/management/agent/main.zeek` is loaded)
|
||||
|
||||
|
||||
.. zeek:field:: node_dispatch_state_agent :zeek:type:`Management::Agent::Runtime::NodeDispatchState` :zeek:attr:`&optional`
|
||||
|
||||
(present if :doc:`/scripts/policy/frameworks/management/agent/main.zeek` is loaded)
|
||||
|
||||
|
||||
.. zeek:field:: restart_state_agent :zeek:type:`Management::Agent::Runtime::RestartState` :zeek:attr:`&optional`
|
||||
|
||||
(present if :doc:`/scripts/policy/frameworks/management/agent/main.zeek` is loaded)
|
||||
|
||||
|
||||
.. zeek:field:: deploy_state :zeek:type:`Management::Controller::Runtime::DeployState` :zeek:attr:`&optional`
|
||||
|
||||
(present if :doc:`/scripts/policy/frameworks/management/controller/main.zeek` is loaded)
|
||||
|
||||
|
||||
.. zeek:field:: get_nodes_state :zeek:type:`Management::Controller::Runtime::GetNodesState` :zeek:attr:`&optional`
|
||||
|
||||
(present if :doc:`/scripts/policy/frameworks/management/controller/main.zeek` is loaded)
|
||||
|
||||
|
||||
.. zeek:field:: node_dispatch_state :zeek:type:`Management::Controller::Runtime::NodeDispatchState` :zeek:attr:`&optional`
|
||||
|
||||
(present if :doc:`/scripts/policy/frameworks/management/controller/main.zeek` is loaded)
|
||||
|
||||
|
||||
.. zeek:field:: restart_state :zeek:type:`Management::Controller::Runtime::RestartState` :zeek:attr:`&optional`
|
||||
|
||||
(present if :doc:`/scripts/policy/frameworks/management/controller/main.zeek` is loaded)
|
||||
|
||||
|
||||
.. zeek:field:: test_state :zeek:type:`Management::Controller::Runtime::TestState` :zeek:attr:`&optional`
|
||||
|
||||
(present if :doc:`/scripts/policy/frameworks/management/controller/main.zeek` is loaded)
|
||||
|
||||
|
||||
Request records track state associated with a request/response event
|
||||
pair. Calls to
|
||||
:zeek:see:`Management::Request::create` establish such state
|
||||
when an entity sends off a request event, while
|
||||
:zeek:see:`Management::Request::finish` clears the state when
|
||||
a corresponding response event comes in, or the state times out.
|
||||
|
||||
Events
|
||||
######
|
||||
.. zeek:id:: Management::Request::request_expired
|
||||
:source-code: policy/frameworks/management/request.zeek 84 84
|
||||
|
||||
:Type: :zeek:type:`event` (req: :zeek:type:`Management::Request::Request`)
|
||||
|
||||
This event fires when a request times out (as per the
|
||||
Management::Request::timeout_interval) before it has been finished via
|
||||
Management::Request::finish().
|
||||
|
||||
|
||||
:param req: the request state that is expiring.
|
||||
|
||||
|
||||
Functions
|
||||
#########
|
||||
.. zeek:id:: Management::Request::create
|
||||
:source-code: policy/frameworks/management/request.zeek 119 124
|
||||
|
||||
:Type: :zeek:type:`function` (reqid: :zeek:type:`string` :zeek:attr:`&default` = ``9Ye7pQPhuMe`` :zeek:attr:`&optional`) : :zeek:type:`Management::Request::Request`
|
||||
|
||||
This function establishes request state.
|
||||
|
||||
|
||||
:param reqid: the identifier to use for the request.
|
||||
|
||||
|
||||
.. zeek:id:: Management::Request::finish
|
||||
:source-code: policy/frameworks/management/request.zeek 134 148
|
||||
|
||||
:Type: :zeek:type:`function` (reqid: :zeek:type:`string`) : :zeek:type:`bool`
|
||||
|
||||
This function marks a request as complete and causes Zeek to release
|
||||
its internal state. When the request does not exist, this does
|
||||
nothing.
|
||||
|
||||
|
||||
:param reqid: the ID of the request state to release.
|
||||
|
||||
|
||||
.. zeek:id:: Management::Request::is_null
|
||||
:source-code: policy/frameworks/management/request.zeek 150 156
|
||||
|
||||
:Type: :zeek:type:`function` (request: :zeek:type:`Management::Request::Request`) : :zeek:type:`bool`
|
||||
|
||||
This function is a helper predicate to indicate whether a given
|
||||
request is null.
|
||||
|
||||
|
||||
:param request: a Request record to check.
|
||||
|
||||
|
||||
:returns: T if the given request matches the null_req instance, F otherwise.
|
||||
|
||||
|
||||
.. zeek:id:: Management::Request::lookup
|
||||
:source-code: policy/frameworks/management/request.zeek 126 132
|
||||
|
||||
:Type: :zeek:type:`function` (reqid: :zeek:type:`string`) : :zeek:type:`Management::Request::Request`
|
||||
|
||||
This function looks up the request for a given request ID and returns
|
||||
it. When no such request exists, returns Management::Request::null_req.
|
||||
|
||||
|
||||
:param reqid: the ID of the request state to retrieve.
|
||||
|
||||
|
||||
.. zeek:id:: Management::Request::to_string
|
||||
:source-code: policy/frameworks/management/request.zeek 158 168
|
||||
|
||||
:Type: :zeek:type:`function` (request: :zeek:type:`Management::Request::Request`) : :zeek:type:`string`
|
||||
|
||||
For troubleshooting, this function renders a request record to a string.
|
||||
|
||||
|
||||
:param request: the request to render.
|
||||
|
||||
|
||||
|
||||
|
|
@ -0,0 +1,14 @@
|
|||
:tocdepth: 3
|
||||
|
||||
policy/frameworks/management/supervisor/__load__.zeek
|
||||
=====================================================
|
||||
|
||||
|
||||
:Imports: :doc:`policy/frameworks/management/supervisor/main.zeek </scripts/policy/frameworks/management/supervisor/main.zeek>`
|
||||
|
||||
Summary
|
||||
~~~~~~~
|
||||
|
||||
Detailed Interface
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
|
||||
|
|
@ -0,0 +1,44 @@
|
|||
:tocdepth: 3
|
||||
|
||||
policy/frameworks/management/supervisor/api.zeek
|
||||
================================================
|
||||
.. zeek:namespace:: Management::Supervisor::API
|
||||
|
||||
|
||||
:Namespace: Management::Supervisor::API
|
||||
:Imports: :doc:`policy/frameworks/management/types.zeek </scripts/policy/frameworks/management/types.zeek>`
|
||||
|
||||
Summary
|
||||
~~~~~~~
|
||||
Events
|
||||
######
|
||||
============================================================================ =====================================================================
|
||||
:zeek:id:`Management::Supervisor::API::notify_node_exit`: :zeek:type:`event` The Supervisor generates this event whenever it has received a status
|
||||
update from the stem, indicating that a node exited.
|
||||
============================================================================ =====================================================================
|
||||
|
||||
|
||||
Detailed Interface
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
Events
|
||||
######
|
||||
.. zeek:id:: Management::Supervisor::API::notify_node_exit
|
||||
:source-code: policy/frameworks/management/agent/main.zeek 263 269
|
||||
|
||||
:Type: :zeek:type:`event` (node: :zeek:type:`string`, outputs: :zeek:type:`Management::NodeOutputs`)
|
||||
|
||||
The Supervisor generates this event whenever it has received a status
|
||||
update from the stem, indicating that a node exited.
|
||||
|
||||
|
||||
:param node: the name of a node previously created via
|
||||
:zeek:see:`Supervisor::create`.
|
||||
|
||||
|
||||
:param outputs: stdout/stderr context for the node. The contained strings
|
||||
span up to the 100 most recent lines in the corresponding
|
||||
stream. See :zeek:see:`Management::Supervisor::output_max_lines`
|
||||
to adjust the line limit.
|
||||
|
||||
|
||||
|
||||
|
|
@ -0,0 +1,75 @@
|
|||
:tocdepth: 3
|
||||
|
||||
policy/frameworks/management/supervisor/config.zeek
|
||||
===================================================
|
||||
.. zeek:namespace:: Management::Supervisor
|
||||
|
||||
Configuration settings for the Management framework's supervisor extension.
|
||||
|
||||
:Namespace: Management::Supervisor
|
||||
|
||||
Summary
|
||||
~~~~~~~
|
||||
Redefinable Options
|
||||
###################
|
||||
=========================================================================================== =================================================================
|
||||
:zeek:id:`Management::Supervisor::output_max_lines`: :zeek:type:`count` :zeek:attr:`&redef` The maximum number of stdout/stderr output lines to convey in
|
||||
:zeek:see:`Management::Supervisor::API::notify_node_exit` events.
|
||||
:zeek:id:`Management::Supervisor::print_stderr`: :zeek:type:`bool` :zeek:attr:`&redef` Whether to print the stderr sent up to the Supervisor by created
|
||||
nodes to the terminal.
|
||||
:zeek:id:`Management::Supervisor::print_stdout`: :zeek:type:`bool` :zeek:attr:`&redef` Whether to print the stdout sent up to the Supervisor by created
|
||||
nodes to the terminal.
|
||||
:zeek:id:`Management::Supervisor::topic_prefix`: :zeek:type:`string` :zeek:attr:`&redef` The Broker topic for Management framework communication with the
|
||||
Supervisor.
|
||||
=========================================================================================== =================================================================
|
||||
|
||||
|
||||
Detailed Interface
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
Redefinable Options
|
||||
###################
|
||||
.. zeek:id:: Management::Supervisor::output_max_lines
|
||||
:source-code: policy/frameworks/management/supervisor/config.zeek 24 24
|
||||
|
||||
:Type: :zeek:type:`count`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``100``
|
||||
|
||||
The maximum number of stdout/stderr output lines to convey in
|
||||
:zeek:see:`Management::Supervisor::API::notify_node_exit` events.
|
||||
|
||||
.. zeek:id:: Management::Supervisor::print_stderr
|
||||
:source-code: policy/frameworks/management/supervisor/config.zeek 20 20
|
||||
|
||||
:Type: :zeek:type:`bool`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``F``
|
||||
|
||||
Whether to print the stderr sent up to the Supervisor by created
|
||||
nodes to the terminal. By default, this is disabled since this output
|
||||
already ends up in a node-specific stderr file, per
|
||||
:zeek:see:`Management::Node::stderr_file`.
|
||||
|
||||
.. zeek:id:: Management::Supervisor::print_stdout
|
||||
:source-code: policy/frameworks/management/supervisor/config.zeek 14 14
|
||||
|
||||
:Type: :zeek:type:`bool`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``F``
|
||||
|
||||
Whether to print the stdout sent up to the Supervisor by created
|
||||
nodes to the terminal. By default, this is disabled since this output
|
||||
already ends up in a node-specific stdout file, per
|
||||
:zeek:see:`Management::Node::stdout_file`.
|
||||
|
||||
.. zeek:id:: Management::Supervisor::topic_prefix
|
||||
:source-code: policy/frameworks/management/supervisor/config.zeek 8 8
|
||||
|
||||
:Type: :zeek:type:`string`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``"zeek/management/supervisor"``
|
||||
|
||||
The Broker topic for Management framework communication with the
|
||||
Supervisor. The agent subscribes to this.
|
||||
|
||||
|
||||
|
|
@ -0,0 +1,21 @@
|
|||
:orphan:
|
||||
|
||||
Package: policy/frameworks/management/supervisor
|
||||
================================================
|
||||
|
||||
|
||||
:doc:`/scripts/policy/frameworks/management/supervisor/__load__.zeek`
|
||||
|
||||
|
||||
:doc:`/scripts/policy/frameworks/management/supervisor/main.zeek`
|
||||
|
||||
This module provides functionality the Management framework places directly
|
||||
in the Supervisor.
|
||||
|
||||
:doc:`/scripts/policy/frameworks/management/supervisor/api.zeek`
|
||||
|
||||
|
||||
:doc:`/scripts/policy/frameworks/management/supervisor/config.zeek`
|
||||
|
||||
Configuration settings for the Management framework's supervisor extension.
|
||||
|
||||
|
|
@ -0,0 +1,18 @@
|
|||
:tocdepth: 3
|
||||
|
||||
policy/frameworks/management/supervisor/main.zeek
|
||||
=================================================
|
||||
.. zeek:namespace:: Management::Supervisor
|
||||
|
||||
This module provides functionality the Management framework places directly
|
||||
in the Supervisor.
|
||||
|
||||
:Namespace: Management::Supervisor
|
||||
:Imports: :doc:`base/utils/paths.zeek </scripts/base/utils/paths.zeek>`, :doc:`base/utils/queue.zeek </scripts/base/utils/queue.zeek>`, :doc:`policy/frameworks/management/node/config.zeek </scripts/policy/frameworks/management/node/config.zeek>`, :doc:`policy/frameworks/management/supervisor/api.zeek </scripts/policy/frameworks/management/supervisor/api.zeek>`, :doc:`policy/frameworks/management/supervisor/config.zeek </scripts/policy/frameworks/management/supervisor/config.zeek>`, :doc:`policy/frameworks/management/types.zeek </scripts/policy/frameworks/management/types.zeek>`
|
||||
|
||||
Summary
|
||||
~~~~~~~
|
||||
|
||||
Detailed Interface
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
|
||||
386
doc/scripts/policy/frameworks/management/types.zeek.rst
Normal file
386
doc/scripts/policy/frameworks/management/types.zeek.rst
Normal file
|
|
@ -0,0 +1,386 @@
|
|||
:tocdepth: 3
|
||||
|
||||
policy/frameworks/management/types.zeek
|
||||
=======================================
|
||||
.. zeek:namespace:: Management
|
||||
|
||||
This module holds the basic types needed for the Management framework. These
|
||||
are used by both cluster agent and controller, and several have corresponding
|
||||
implementations in zeek-client.
|
||||
|
||||
:Namespace: Management
|
||||
|
||||
Summary
|
||||
~~~~~~~
|
||||
Types
|
||||
#####
|
||||
=========================================================== =====================================================================
|
||||
:zeek:type:`Management::Configuration`: :zeek:type:`record` Data structure capturing a cluster's complete configuration.
|
||||
:zeek:type:`Management::Instance`: :zeek:type:`record` Configuration describing a Zeek instance running a Cluster
|
||||
Agent.
|
||||
:zeek:type:`Management::InstanceVec`: :zeek:type:`vector`
|
||||
:zeek:type:`Management::Node`: :zeek:type:`record` Configuration describing a Cluster Node process.
|
||||
:zeek:type:`Management::NodeOutputs`: :zeek:type:`record` In :zeek:see:`Management::Controller::API::deploy_response` events,
|
||||
each :zeek:see:`Management::Result` indicates the outcome of a
|
||||
launched cluster node.
|
||||
:zeek:type:`Management::NodeStatus`: :zeek:type:`record` The status of a Supervisor-managed node, as reported to the client in
|
||||
a get_nodes_request/get_nodes_response transaction.
|
||||
:zeek:type:`Management::NodeStatusVec`: :zeek:type:`vector`
|
||||
:zeek:type:`Management::Option`: :zeek:type:`record` A Zeek-side option with value.
|
||||
:zeek:type:`Management::Result`: :zeek:type:`record` Return value for request-response API event pairs.
|
||||
:zeek:type:`Management::ResultVec`: :zeek:type:`vector`
|
||||
:zeek:type:`Management::Role`: :zeek:type:`enum` Management infrastructure node type.
|
||||
:zeek:type:`Management::State`: :zeek:type:`enum` State that a Cluster Node can be in.
|
||||
=========================================================== =====================================================================
|
||||
|
||||
Functions
|
||||
#########
|
||||
================================================================== =========================================================
|
||||
:zeek:id:`Management::result_to_string`: :zeek:type:`function` Given a :zeek:see:`Management::Result` record,
|
||||
this function returns a string summarizing it.
|
||||
:zeek:id:`Management::result_vec_to_string`: :zeek:type:`function` Given a vector of :zeek:see:`Management::Result` records,
|
||||
this function returns a string summarizing them.
|
||||
================================================================== =========================================================
|
||||
|
||||
|
||||
Detailed Interface
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
Types
|
||||
#####
|
||||
.. zeek:type:: Management::Configuration
|
||||
:source-code: policy/frameworks/management/types.zeek 67 74
|
||||
|
||||
:Type: :zeek:type:`record`
|
||||
|
||||
|
||||
.. zeek:field:: id :zeek:type:`string` :zeek:attr:`&default` = ``fD0qxAnfwOe`` :zeek:attr:`&optional`
|
||||
|
||||
Unique identifier for a particular configuration
|
||||
|
||||
|
||||
.. zeek:field:: instances :zeek:type:`set` [:zeek:type:`Management::Instance`] :zeek:attr:`&default` = ``{ }`` :zeek:attr:`&optional`
|
||||
|
||||
The instances in the cluster.
|
||||
|
||||
|
||||
.. zeek:field:: nodes :zeek:type:`set` [:zeek:type:`Management::Node`] :zeek:attr:`&default` = ``{ }`` :zeek:attr:`&optional`
|
||||
|
||||
The set of nodes in the cluster, as distributed over the instances.
|
||||
|
||||
|
||||
Data structure capturing a cluster's complete configuration.
|
||||
|
||||
.. zeek:type:: Management::Instance
|
||||
:source-code: policy/frameworks/management/types.zeek 27 34
|
||||
|
||||
:Type: :zeek:type:`record`
|
||||
|
||||
|
||||
.. zeek:field:: name :zeek:type:`string`
|
||||
|
||||
Unique, human-readable instance name
|
||||
|
||||
|
||||
.. zeek:field:: host :zeek:type:`addr`
|
||||
|
||||
IP address of system
|
||||
|
||||
|
||||
.. zeek:field:: listen_port :zeek:type:`port` :zeek:attr:`&optional`
|
||||
|
||||
Agent listening port. Not needed if agents connect to controller.
|
||||
|
||||
|
||||
Configuration describing a Zeek instance running a Cluster
|
||||
Agent. Normally, there'll be one instance per cluster
|
||||
system: a single physical system.
|
||||
|
||||
.. zeek:type:: Management::InstanceVec
|
||||
:source-code: policy/frameworks/management/types.zeek 36 36
|
||||
|
||||
:Type: :zeek:type:`vector` of :zeek:type:`Management::Instance`
|
||||
|
||||
|
||||
.. zeek:type:: Management::Node
|
||||
:source-code: policy/frameworks/management/types.zeek 52 64
|
||||
|
||||
:Type: :zeek:type:`record`
|
||||
|
||||
|
||||
.. zeek:field:: name :zeek:type:`string`
|
||||
|
||||
Cluster-unique, human-readable node name
|
||||
|
||||
|
||||
.. zeek:field:: instance :zeek:type:`string`
|
||||
|
||||
Name of instance where node is to run
|
||||
|
||||
|
||||
.. zeek:field:: role :zeek:type:`Supervisor::ClusterRole`
|
||||
|
||||
Role of the node.
|
||||
|
||||
|
||||
.. zeek:field:: state :zeek:type:`Management::State`
|
||||
|
||||
Desired, or current, run state.
|
||||
|
||||
|
||||
.. zeek:field:: p :zeek:type:`port` :zeek:attr:`&optional`
|
||||
|
||||
Port on which this node will listen
|
||||
|
||||
|
||||
.. zeek:field:: scripts :zeek:type:`vector` of :zeek:type:`string` :zeek:attr:`&optional`
|
||||
|
||||
Additional Zeek scripts for node
|
||||
|
||||
|
||||
.. zeek:field:: options :zeek:type:`set` [:zeek:type:`Management::Option`] :zeek:attr:`&optional`
|
||||
|
||||
Zeek options for node
|
||||
|
||||
|
||||
.. zeek:field:: interface :zeek:type:`string` :zeek:attr:`&optional`
|
||||
|
||||
Interface to sniff
|
||||
|
||||
|
||||
.. zeek:field:: cpu_affinity :zeek:type:`int` :zeek:attr:`&optional`
|
||||
|
||||
CPU/core number to pin to
|
||||
|
||||
|
||||
.. zeek:field:: env :zeek:type:`table` [:zeek:type:`string`] of :zeek:type:`string` :zeek:attr:`&default` = ``{ }`` :zeek:attr:`&optional`
|
||||
|
||||
Custom environment vars
|
||||
|
||||
|
||||
.. zeek:field:: metrics_port :zeek:type:`port` :zeek:attr:`&optional`
|
||||
|
||||
Metrics exposure port, for Prometheus
|
||||
|
||||
|
||||
Configuration describing a Cluster Node process.
|
||||
|
||||
.. zeek:type:: Management::NodeOutputs
|
||||
:source-code: policy/frameworks/management/types.zeek 122 125
|
||||
|
||||
:Type: :zeek:type:`record`
|
||||
|
||||
|
||||
.. zeek:field:: stdout :zeek:type:`string`
|
||||
|
||||
The stdout stream of a Zeek process
|
||||
|
||||
|
||||
.. zeek:field:: stderr :zeek:type:`string`
|
||||
|
||||
The stderr stream of a Zeek process
|
||||
|
||||
|
||||
In :zeek:see:`Management::Controller::API::deploy_response` events,
|
||||
each :zeek:see:`Management::Result` indicates the outcome of a
|
||||
launched cluster node. If a node does not launch properly (meaning
|
||||
it doesn't check in with the agent on the machine it's running on),
|
||||
the result will indicate failure, and its data field will be an
|
||||
instance of this record, capturing the stdout and stderr output of
|
||||
the failing node.
|
||||
|
||||
.. zeek:type:: Management::NodeStatus
|
||||
:source-code: policy/frameworks/management/types.zeek 78 94
|
||||
|
||||
:Type: :zeek:type:`record`
|
||||
|
||||
|
||||
.. zeek:field:: node :zeek:type:`string`
|
||||
|
||||
Cluster-unique, human-readable node name
|
||||
|
||||
|
||||
.. zeek:field:: state :zeek:type:`Management::State`
|
||||
|
||||
Current run state of the node.
|
||||
|
||||
|
||||
.. zeek:field:: mgmt_role :zeek:type:`Management::Role` :zeek:attr:`&default` = ``Management::NONE`` :zeek:attr:`&optional`
|
||||
|
||||
Role the node plays in cluster management.
|
||||
|
||||
|
||||
.. zeek:field:: cluster_role :zeek:type:`Supervisor::ClusterRole` :zeek:attr:`&default` = ``Supervisor::NONE`` :zeek:attr:`&optional`
|
||||
|
||||
Role the node plays in the Zeek cluster.
|
||||
|
||||
|
||||
.. zeek:field:: pid :zeek:type:`int` :zeek:attr:`&optional`
|
||||
|
||||
Process ID of the node. This is optional because the Supervisor may not have
|
||||
a PID when a node is still bootstrapping.
|
||||
|
||||
|
||||
.. zeek:field:: p :zeek:type:`port` :zeek:attr:`&optional`
|
||||
|
||||
The node's Broker peering listening port, if any.
|
||||
|
||||
|
||||
.. zeek:field:: metrics_port :zeek:type:`port` :zeek:attr:`&optional`
|
||||
|
||||
The node's metrics port for Prometheus, if any.
|
||||
|
||||
|
||||
The status of a Supervisor-managed node, as reported to the client in
|
||||
a get_nodes_request/get_nodes_response transaction.
|
||||
|
||||
.. zeek:type:: Management::NodeStatusVec
|
||||
:source-code: policy/frameworks/management/types.zeek 96 96
|
||||
|
||||
:Type: :zeek:type:`vector` of :zeek:type:`Management::NodeStatus`
|
||||
|
||||
|
||||
.. zeek:type:: Management::Option
|
||||
:source-code: policy/frameworks/management/types.zeek 19 22
|
||||
|
||||
:Type: :zeek:type:`record`
|
||||
|
||||
|
||||
.. zeek:field:: name :zeek:type:`string`
|
||||
|
||||
Name of option
|
||||
|
||||
|
||||
.. zeek:field:: value :zeek:type:`string`
|
||||
|
||||
Value of option
|
||||
|
||||
|
||||
A Zeek-side option with value.
|
||||
|
||||
.. zeek:type:: Management::Result
|
||||
:source-code: policy/frameworks/management/types.zeek 104 111
|
||||
|
||||
:Type: :zeek:type:`record`
|
||||
|
||||
|
||||
.. zeek:field:: reqid :zeek:type:`string`
|
||||
|
||||
Request ID of operation this result refers to
|
||||
|
||||
|
||||
.. zeek:field:: success :zeek:type:`bool` :zeek:attr:`&default` = ``T`` :zeek:attr:`&optional`
|
||||
|
||||
True if successful
|
||||
|
||||
|
||||
.. zeek:field:: instance :zeek:type:`string` :zeek:attr:`&optional`
|
||||
|
||||
Name of associated instance (for context)
|
||||
|
||||
|
||||
.. zeek:field:: data :zeek:type:`any` :zeek:attr:`&optional`
|
||||
|
||||
Addl data returned for successful operation
|
||||
|
||||
|
||||
.. zeek:field:: error :zeek:type:`string` :zeek:attr:`&optional`
|
||||
|
||||
Descriptive error on failure
|
||||
|
||||
|
||||
.. zeek:field:: node :zeek:type:`string` :zeek:attr:`&optional`
|
||||
|
||||
Name of associated node (for context)
|
||||
|
||||
|
||||
Return value for request-response API event pairs. Some responses
|
||||
contain one, others multiple of these. The request ID allows clients
|
||||
to string requests and responses together. Agents and the controller
|
||||
fill in the instance and node fields whenever there's sufficient
|
||||
context to define them. Any result produced by an agent will carry an
|
||||
instance value, for example.
|
||||
|
||||
.. zeek:type:: Management::ResultVec
|
||||
:source-code: policy/frameworks/management/types.zeek 113 113
|
||||
|
||||
:Type: :zeek:type:`vector` of :zeek:type:`Management::Result`
|
||||
|
||||
|
||||
.. zeek:type:: Management::Role
|
||||
:source-code: policy/frameworks/management/types.zeek 11 17
|
||||
|
||||
:Type: :zeek:type:`enum`
|
||||
|
||||
.. zeek:enum:: Management::NONE Management::Role
|
||||
|
||||
No active role in cluster management
|
||||
|
||||
.. zeek:enum:: Management::AGENT Management::Role
|
||||
|
||||
A cluster management agent.
|
||||
|
||||
.. zeek:enum:: Management::CONTROLLER Management::Role
|
||||
|
||||
The cluster's controller.
|
||||
|
||||
.. zeek:enum:: Management::NODE Management::Role
|
||||
|
||||
A managed cluster node (worker, manager, etc).
|
||||
|
||||
Management infrastructure node type. This intentionally does not
|
||||
include the managed cluster node types (worker, logger, etc) -- those
|
||||
continue to be managed by the cluster framework.
|
||||
|
||||
.. zeek:type:: Management::State
|
||||
:source-code: policy/frameworks/management/types.zeek 42 50
|
||||
|
||||
:Type: :zeek:type:`enum`
|
||||
|
||||
.. zeek:enum:: Management::PENDING Management::State
|
||||
|
||||
Not yet running
|
||||
|
||||
.. zeek:enum:: Management::RUNNING Management::State
|
||||
|
||||
Running and operating normally
|
||||
|
||||
.. zeek:enum:: Management::STOPPED Management::State
|
||||
|
||||
Explicitly stopped
|
||||
|
||||
.. zeek:enum:: Management::FAILED Management::State
|
||||
|
||||
Failed to start; and permanently halted
|
||||
|
||||
.. zeek:enum:: Management::CRASHED Management::State
|
||||
|
||||
Crashed, will be restarted,
|
||||
|
||||
.. zeek:enum:: Management::UNKNOWN Management::State
|
||||
|
||||
State not known currently (e.g., because of lost connectivity)
|
||||
|
||||
State that a Cluster Node can be in. State changes trigger an
|
||||
API notification (see notify_change()). The Pending state corresponds
|
||||
to the Supervisor not yet reporting a PID for a node when it has not
|
||||
yet fully launched.
|
||||
|
||||
Functions
|
||||
#########
|
||||
.. zeek:id:: Management::result_to_string
|
||||
:source-code: policy/frameworks/management/types.zeek 136 160
|
||||
|
||||
:Type: :zeek:type:`function` (res: :zeek:type:`Management::Result`) : :zeek:type:`string`
|
||||
|
||||
Given a :zeek:see:`Management::Result` record,
|
||||
this function returns a string summarizing it.
|
||||
|
||||
.. zeek:id:: Management::result_vec_to_string
|
||||
:source-code: policy/frameworks/management/types.zeek 162 170
|
||||
|
||||
:Type: :zeek:type:`function` (res: :zeek:type:`Management::ResultVec`) : :zeek:type:`string`
|
||||
|
||||
Given a vector of :zeek:see:`Management::Result` records,
|
||||
this function returns a string summarizing them.
|
||||
|
||||
|
||||
38
doc/scripts/policy/frameworks/management/util.zeek.rst
Normal file
38
doc/scripts/policy/frameworks/management/util.zeek.rst
Normal file
|
|
@ -0,0 +1,38 @@
|
|||
:tocdepth: 3
|
||||
|
||||
policy/frameworks/management/util.zeek
|
||||
======================================
|
||||
.. zeek:namespace:: Management::Util
|
||||
|
||||
Utility functions for the Management framework, available to agent
|
||||
and controller.
|
||||
|
||||
:Namespace: Management::Util
|
||||
|
||||
Summary
|
||||
~~~~~~~
|
||||
Functions
|
||||
#########
|
||||
================================================================= ============================================================
|
||||
:zeek:id:`Management::Util::set_to_vector`: :zeek:type:`function` Renders a set of strings to an alphabetically sorted vector.
|
||||
================================================================= ============================================================
|
||||
|
||||
|
||||
Detailed Interface
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
Functions
|
||||
#########
|
||||
.. zeek:id:: Management::Util::set_to_vector
|
||||
:source-code: policy/frameworks/management/util.zeek 15 25
|
||||
|
||||
:Type: :zeek:type:`function` (ss: :zeek:type:`set` [:zeek:type:`string`]) : :zeek:type:`vector` of :zeek:type:`string`
|
||||
|
||||
Renders a set of strings to an alphabetically sorted vector.
|
||||
|
||||
|
||||
:param ss: the string set to convert.
|
||||
|
||||
|
||||
:returns: the vector of all strings in ss.
|
||||
|
||||
|
||||
|
|
@ -0,0 +1,412 @@
|
|||
:tocdepth: 3
|
||||
|
||||
policy/frameworks/netcontrol/catch-and-release.zeek
|
||||
===================================================
|
||||
.. zeek:namespace:: NetControl
|
||||
|
||||
Implementation of catch-and-release functionality for NetControl.
|
||||
|
||||
:Namespace: NetControl
|
||||
:Imports: :doc:`base/frameworks/cluster </scripts/base/frameworks/cluster/index>`, :doc:`base/frameworks/netcontrol </scripts/base/frameworks/netcontrol/index>`
|
||||
|
||||
Summary
|
||||
~~~~~~~
|
||||
Runtime Options
|
||||
###############
|
||||
======================================================================================================= ====================================================================================
|
||||
:zeek:id:`NetControl::catch_release_warn_blocked_ip_encountered`: :zeek:type:`bool` :zeek:attr:`&redef` If true, catch and release warns if packets of an IP address are still seen after it
|
||||
should have been blocked.
|
||||
======================================================================================================= ====================================================================================
|
||||
|
||||
Redefinable Options
|
||||
###################
|
||||
======================================================================================= =====================================================================================
|
||||
:zeek:id:`NetControl::catch_release_intervals`: :zeek:type:`vector` :zeek:attr:`&redef` Time intervals for which subsequent drops of the same IP take
|
||||
effect.
|
||||
:zeek:id:`NetControl::watch_connections`: :zeek:type:`bool` :zeek:attr:`&redef` If true, catch_release_seen is called on the connection originator in new_connection,
|
||||
connection_established, partial_connection, connection_attempt, connection_rejected,
|
||||
connection_reset and connection_pending
|
||||
======================================================================================= =====================================================================================
|
||||
|
||||
Types
|
||||
#####
|
||||
=============================================================== =========================================================================
|
||||
:zeek:type:`NetControl::BlockInfo`: :zeek:type:`record` This record is used for storing information about current blocks that are
|
||||
part of catch and release.
|
||||
:zeek:type:`NetControl::CatchReleaseActions`: :zeek:type:`enum` The enum that contains the different kinds of messages that are logged by
|
||||
catch and release.
|
||||
:zeek:type:`NetControl::CatchReleaseInfo`: :zeek:type:`record` The record type that is used for representing and logging
|
||||
=============================================================== =========================================================================
|
||||
|
||||
Redefinitions
|
||||
#############
|
||||
======================================= ========================================
|
||||
:zeek:type:`Log::ID`: :zeek:type:`enum`
|
||||
|
||||
* :zeek:enum:`NetControl::CATCH_RELEASE`
|
||||
======================================= ========================================
|
||||
|
||||
Events
|
||||
######
|
||||
======================================================================= ===================================================================================
|
||||
:zeek:id:`NetControl::catch_release_add`: :zeek:type:`event`
|
||||
:zeek:id:`NetControl::catch_release_block_delete`: :zeek:type:`event`
|
||||
:zeek:id:`NetControl::catch_release_block_new`: :zeek:type:`event`
|
||||
:zeek:id:`NetControl::catch_release_delete`: :zeek:type:`event`
|
||||
:zeek:id:`NetControl::catch_release_encountered`: :zeek:type:`event`
|
||||
:zeek:id:`NetControl::catch_release_forgotten`: :zeek:type:`event` Event is raised when catch and release cases management of an IP address because no
|
||||
activity was seen within the watch_until period.
|
||||
:zeek:id:`NetControl::log_netcontrol_catch_release`: :zeek:type:`event` Event that can be handled to access the :zeek:type:`NetControl::CatchReleaseInfo`
|
||||
record as it is sent on to the logging framework.
|
||||
======================================================================= ===================================================================================
|
||||
|
||||
Hooks
|
||||
#####
|
||||
============================================================================= =
|
||||
:zeek:id:`NetControl::log_policy_catch_release`: :zeek:type:`Log::PolicyHook`
|
||||
============================================================================= =
|
||||
|
||||
Functions
|
||||
#########
|
||||
=========================================================================== =======================================================================================================
|
||||
:zeek:id:`NetControl::catch_release_seen`: :zeek:type:`function` This function can be called to notify the catch and release script that activity by
|
||||
an IP address was seen.
|
||||
:zeek:id:`NetControl::drop_address_catch_release`: :zeek:type:`function` Stops all packets involving an IP address from being forwarded.
|
||||
:zeek:id:`NetControl::get_catch_release_info`: :zeek:type:`function` Get the :zeek:see:`NetControl::BlockInfo` record for an address currently blocked by catch and release.
|
||||
:zeek:id:`NetControl::unblock_address_catch_release`: :zeek:type:`function` Removes an address from being watched with catch and release.
|
||||
=========================================================================== =======================================================================================================
|
||||
|
||||
|
||||
Detailed Interface
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
Runtime Options
|
||||
###############
|
||||
.. zeek:id:: NetControl::catch_release_warn_blocked_ip_encountered
|
||||
:source-code: policy/frameworks/netcontrol/catch-and-release.zeek 148 148
|
||||
|
||||
:Type: :zeek:type:`bool`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``F``
|
||||
|
||||
If true, catch and release warns if packets of an IP address are still seen after it
|
||||
should have been blocked.
|
||||
|
||||
Redefinable Options
|
||||
###################
|
||||
.. zeek:id:: NetControl::catch_release_intervals
|
||||
:source-code: policy/frameworks/netcontrol/catch-and-release.zeek 152 152
|
||||
|
||||
:Type: :zeek:type:`vector` of :zeek:type:`interval`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default:
|
||||
|
||||
::
|
||||
|
||||
[10.0 mins, 1.0 hr, 1.0 day, 7.0 days]
|
||||
|
||||
|
||||
Time intervals for which subsequent drops of the same IP take
|
||||
effect.
|
||||
|
||||
.. zeek:id:: NetControl::watch_connections
|
||||
:source-code: policy/frameworks/netcontrol/catch-and-release.zeek 144 144
|
||||
|
||||
:Type: :zeek:type:`bool`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``T``
|
||||
|
||||
If true, catch_release_seen is called on the connection originator in new_connection,
|
||||
connection_established, partial_connection, connection_attempt, connection_rejected,
|
||||
connection_reset and connection_pending
|
||||
|
||||
Types
|
||||
#####
|
||||
.. zeek:type:: NetControl::BlockInfo
|
||||
:source-code: policy/frameworks/netcontrol/catch-and-release.zeek 16 29
|
||||
|
||||
:Type: :zeek:type:`record`
|
||||
|
||||
|
||||
.. zeek:field:: block_until :zeek:type:`time` :zeek:attr:`&optional`
|
||||
|
||||
Absolute time indicating until when a block is inserted using NetControl.
|
||||
|
||||
|
||||
.. zeek:field:: watch_until :zeek:type:`time`
|
||||
|
||||
Absolute time indicating until when an IP address is watched to reblock it.
|
||||
|
||||
|
||||
.. zeek:field:: num_reblocked :zeek:type:`count` :zeek:attr:`&default` = ``0`` :zeek:attr:`&optional`
|
||||
|
||||
Number of times an IP address was reblocked.
|
||||
|
||||
|
||||
.. zeek:field:: current_interval :zeek:type:`count`
|
||||
|
||||
Number indicating at which catch and release interval we currently are.
|
||||
|
||||
|
||||
.. zeek:field:: current_block_id :zeek:type:`string`
|
||||
|
||||
ID of the inserted block, if any.
|
||||
|
||||
|
||||
.. zeek:field:: location :zeek:type:`string` :zeek:attr:`&optional`
|
||||
|
||||
User specified string.
|
||||
|
||||
|
||||
This record is used for storing information about current blocks that are
|
||||
part of catch and release.
|
||||
|
||||
.. zeek:type:: NetControl::CatchReleaseActions
|
||||
:source-code: policy/frameworks/netcontrol/catch-and-release.zeek 33 51
|
||||
|
||||
:Type: :zeek:type:`enum`
|
||||
|
||||
.. zeek:enum:: NetControl::INFO NetControl::CatchReleaseActions
|
||||
|
||||
Log lines marked with info are purely informational; no action was taken.
|
||||
|
||||
.. zeek:enum:: NetControl::ADDED NetControl::CatchReleaseActions
|
||||
|
||||
A rule for the specified IP address already existed in NetControl (outside
|
||||
of catch-and-release). Catch and release did not add a new rule, but is now
|
||||
watching the IP address and will add a new rule after the current rule expires.
|
||||
|
||||
.. zeek:enum:: NetControl::DROP_REQUESTED NetControl::CatchReleaseActions
|
||||
|
||||
A drop was requested by catch and release.
|
||||
|
||||
.. zeek:enum:: NetControl::DROPPED NetControl::CatchReleaseActions
|
||||
|
||||
An address was successfully blocked by catch and release.
|
||||
|
||||
.. zeek:enum:: NetControl::UNBLOCK NetControl::CatchReleaseActions
|
||||
|
||||
An address was unblocked after the timeout expired.
|
||||
|
||||
.. zeek:enum:: NetControl::FORGOTTEN NetControl::CatchReleaseActions
|
||||
|
||||
An address was forgotten because it did not reappear within the `watch_until` interval.
|
||||
|
||||
.. zeek:enum:: NetControl::SEEN_AGAIN NetControl::CatchReleaseActions
|
||||
|
||||
A watched IP address was seen again; catch and release will re-block it.
|
||||
|
||||
The enum that contains the different kinds of messages that are logged by
|
||||
catch and release.
|
||||
|
||||
.. zeek:type:: NetControl::CatchReleaseInfo
|
||||
:source-code: policy/frameworks/netcontrol/catch-and-release.zeek 53 78
|
||||
|
||||
:Type: :zeek:type:`record`
|
||||
|
||||
|
||||
.. zeek:field:: ts :zeek:type:`time` :zeek:attr:`&log`
|
||||
|
||||
The absolute time indicating when the action for this log-line occurred.
|
||||
|
||||
|
||||
.. zeek:field:: rule_id :zeek:type:`string` :zeek:attr:`&log` :zeek:attr:`&optional`
|
||||
|
||||
The rule id that this log line refers to.
|
||||
|
||||
|
||||
.. zeek:field:: ip :zeek:type:`addr` :zeek:attr:`&log`
|
||||
|
||||
The IP address that this line refers to.
|
||||
|
||||
|
||||
.. zeek:field:: action :zeek:type:`NetControl::CatchReleaseActions` :zeek:attr:`&log`
|
||||
|
||||
The action that was taken in this log-line.
|
||||
|
||||
|
||||
.. zeek:field:: block_interval :zeek:type:`interval` :zeek:attr:`&log` :zeek:attr:`&optional`
|
||||
|
||||
The current block_interval (for how long the address is blocked).
|
||||
|
||||
|
||||
.. zeek:field:: watch_interval :zeek:type:`interval` :zeek:attr:`&log` :zeek:attr:`&optional`
|
||||
|
||||
The current watch_interval (for how long the address will be watched and re-block if it reappears).
|
||||
|
||||
|
||||
.. zeek:field:: blocked_until :zeek:type:`time` :zeek:attr:`&log` :zeek:attr:`&optional`
|
||||
|
||||
The absolute time until which the address is blocked.
|
||||
|
||||
|
||||
.. zeek:field:: watched_until :zeek:type:`time` :zeek:attr:`&log` :zeek:attr:`&optional`
|
||||
|
||||
The absolute time until which the address will be monitored.
|
||||
|
||||
|
||||
.. zeek:field:: num_blocked :zeek:type:`count` :zeek:attr:`&log` :zeek:attr:`&optional`
|
||||
|
||||
Number of times that this address was blocked in the current cycle.
|
||||
|
||||
|
||||
.. zeek:field:: location :zeek:type:`string` :zeek:attr:`&log` :zeek:attr:`&optional`
|
||||
|
||||
The user specified location string.
|
||||
|
||||
|
||||
.. zeek:field:: message :zeek:type:`string` :zeek:attr:`&log` :zeek:attr:`&optional`
|
||||
|
||||
Additional informational string by the catch and release framework about this log-line.
|
||||
|
||||
|
||||
.. zeek:field:: plugin :zeek:type:`string` :zeek:attr:`&log` :zeek:attr:`&optional`
|
||||
|
||||
Plugin triggering the log entry.
|
||||
|
||||
|
||||
The record type that is used for representing and logging
|
||||
|
||||
Events
|
||||
######
|
||||
.. zeek:id:: NetControl::catch_release_add
|
||||
:source-code: policy/frameworks/netcontrol/catch-and-release.zeek 161 161
|
||||
|
||||
:Type: :zeek:type:`event` (a: :zeek:type:`addr`, location: :zeek:type:`string`)
|
||||
|
||||
|
||||
.. zeek:id:: NetControl::catch_release_block_delete
|
||||
:source-code: policy/frameworks/netcontrol/catch-and-release.zeek 160 160
|
||||
|
||||
:Type: :zeek:type:`event` (a: :zeek:type:`addr`)
|
||||
|
||||
|
||||
.. zeek:id:: NetControl::catch_release_block_new
|
||||
:source-code: policy/frameworks/netcontrol/catch-and-release.zeek 159 159
|
||||
|
||||
:Type: :zeek:type:`event` (a: :zeek:type:`addr`, b: :zeek:type:`NetControl::BlockInfo`)
|
||||
|
||||
|
||||
.. zeek:id:: NetControl::catch_release_delete
|
||||
:source-code: policy/frameworks/netcontrol/catch-and-release.zeek 162 162
|
||||
|
||||
:Type: :zeek:type:`event` (a: :zeek:type:`addr`, reason: :zeek:type:`string`)
|
||||
|
||||
|
||||
.. zeek:id:: NetControl::catch_release_encountered
|
||||
:source-code: policy/frameworks/netcontrol/catch-and-release.zeek 163 163
|
||||
|
||||
:Type: :zeek:type:`event` (a: :zeek:type:`addr`)
|
||||
|
||||
|
||||
.. zeek:id:: NetControl::catch_release_forgotten
|
||||
:source-code: policy/frameworks/netcontrol/catch-and-release.zeek 139 139
|
||||
|
||||
:Type: :zeek:type:`event` (a: :zeek:type:`addr`, bi: :zeek:type:`NetControl::BlockInfo`)
|
||||
|
||||
Event is raised when catch and release cases management of an IP address because no
|
||||
activity was seen within the watch_until period.
|
||||
|
||||
|
||||
:param a: The address that is no longer being managed.
|
||||
|
||||
|
||||
:param bi: The :zeek:see:`NetControl::BlockInfo` record containing information about the block.
|
||||
|
||||
.. zeek:id:: NetControl::log_netcontrol_catch_release
|
||||
:source-code: policy/frameworks/netcontrol/catch-and-release.zeek 156 156
|
||||
|
||||
:Type: :zeek:type:`event` (rec: :zeek:type:`NetControl::CatchReleaseInfo`)
|
||||
|
||||
Event that can be handled to access the :zeek:type:`NetControl::CatchReleaseInfo`
|
||||
record as it is sent on to the logging framework.
|
||||
|
||||
Hooks
|
||||
#####
|
||||
.. zeek:id:: NetControl::log_policy_catch_release
|
||||
:source-code: policy/frameworks/netcontrol/catch-and-release.zeek 12 12
|
||||
|
||||
:Type: :zeek:type:`Log::PolicyHook`
|
||||
|
||||
|
||||
Functions
|
||||
#########
|
||||
.. zeek:id:: NetControl::catch_release_seen
|
||||
:source-code: policy/frameworks/netcontrol/catch-and-release.zeek 447 511
|
||||
|
||||
:Type: :zeek:type:`function` (a: :zeek:type:`addr`) : :zeek:type:`void`
|
||||
|
||||
This function can be called to notify the catch and release script that activity by
|
||||
an IP address was seen. If the respective IP address is currently monitored by catch and
|
||||
release and not blocked, the block will be reinstated. See the documentation of watch_new_connection
|
||||
which events the catch and release functionality usually monitors for activity.
|
||||
|
||||
|
||||
:param a: The address that was seen and should be re-dropped if it is being watched.
|
||||
|
||||
.. zeek:id:: NetControl::drop_address_catch_release
|
||||
:source-code: policy/frameworks/netcontrol/catch-and-release.zeek 353 419
|
||||
|
||||
:Type: :zeek:type:`function` (a: :zeek:type:`addr`, location: :zeek:type:`string` :zeek:attr:`&default` = ``""`` :zeek:attr:`&optional`) : :zeek:type:`NetControl::BlockInfo`
|
||||
|
||||
Stops all packets involving an IP address from being forwarded. This function
|
||||
uses catch-and-release functionality, where the IP address is only dropped for
|
||||
a short amount of time that is incremented steadily when the IP is encountered
|
||||
again.
|
||||
|
||||
In cluster mode, this function works on workers as well as the manager. On managers,
|
||||
the returned :zeek:see:`NetControl::BlockInfo` record will not contain the block ID,
|
||||
which will be assigned on the manager.
|
||||
|
||||
|
||||
:param a: The address to be dropped.
|
||||
|
||||
|
||||
:param t: How long to drop it, with 0 being indefinitely.
|
||||
|
||||
|
||||
:param location: An optional string describing where the drop was triggered.
|
||||
|
||||
|
||||
:returns: The :zeek:see:`NetControl::BlockInfo` record containing information about
|
||||
the inserted block.
|
||||
|
||||
.. zeek:id:: NetControl::get_catch_release_info
|
||||
:source-code: policy/frameworks/netcontrol/catch-and-release.zeek 345 351
|
||||
|
||||
:Type: :zeek:type:`function` (a: :zeek:type:`addr`) : :zeek:type:`NetControl::BlockInfo`
|
||||
|
||||
Get the :zeek:see:`NetControl::BlockInfo` record for an address currently blocked by catch and release.
|
||||
If the address is unknown to catch and release, the watch_until time will be set to 0.
|
||||
|
||||
In cluster mode, this function works on the manager and workers. On workers, the data will
|
||||
lag slightly behind the manager; if you add a block, it will not be instantly available via
|
||||
this function.
|
||||
|
||||
|
||||
:param a: The address to get information about.
|
||||
|
||||
|
||||
:returns: The :zeek:see:`NetControl::BlockInfo` record containing information about
|
||||
the inserted block.
|
||||
|
||||
.. zeek:id:: NetControl::unblock_address_catch_release
|
||||
:source-code: policy/frameworks/netcontrol/catch-and-release.zeek 422 445
|
||||
|
||||
:Type: :zeek:type:`function` (a: :zeek:type:`addr`, reason: :zeek:type:`string` :zeek:attr:`&default` = ``""`` :zeek:attr:`&optional`) : :zeek:type:`bool`
|
||||
|
||||
Removes an address from being watched with catch and release. Returns true if the
|
||||
address was found and removed; returns false if it was unknown to catch and release.
|
||||
|
||||
If the address is currently blocked, and the block was inserted by catch and release,
|
||||
the block is removed.
|
||||
|
||||
|
||||
:param a: The address to be unblocked.
|
||||
|
||||
|
||||
:param reason: A reason for the unblock.
|
||||
|
||||
|
||||
:returns: True if the address was unblocked.
|
||||
|
||||
|
||||
14
doc/scripts/policy/frameworks/notice/__load__.zeek.rst
Normal file
14
doc/scripts/policy/frameworks/notice/__load__.zeek.rst
Normal file
|
|
@ -0,0 +1,14 @@
|
|||
:tocdepth: 3
|
||||
|
||||
policy/frameworks/notice/__load__.zeek
|
||||
======================================
|
||||
|
||||
|
||||
:Imports: :doc:`policy/frameworks/notice/extend-email/hostnames.zeek </scripts/policy/frameworks/notice/extend-email/hostnames.zeek>`
|
||||
|
||||
Summary
|
||||
~~~~~~~
|
||||
|
||||
Detailed Interface
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
|
||||
30
doc/scripts/policy/frameworks/notice/actions/drop.zeek.rst
Normal file
30
doc/scripts/policy/frameworks/notice/actions/drop.zeek.rst
Normal file
|
|
@ -0,0 +1,30 @@
|
|||
:tocdepth: 3
|
||||
|
||||
policy/frameworks/notice/actions/drop.zeek
|
||||
==========================================
|
||||
.. zeek:namespace:: Notice
|
||||
|
||||
This script extends the built in notice code to implement the IP address
|
||||
dropping functionality.
|
||||
|
||||
:Namespace: Notice
|
||||
:Imports: :doc:`base/frameworks/netcontrol </scripts/base/frameworks/netcontrol/index>`, :doc:`base/frameworks/notice/main.zeek </scripts/base/frameworks/notice/main.zeek>`, :doc:`policy/frameworks/netcontrol/catch-and-release.zeek </scripts/policy/frameworks/netcontrol/catch-and-release.zeek>`
|
||||
|
||||
Summary
|
||||
~~~~~~~
|
||||
Redefinitions
|
||||
#############
|
||||
============================================== ===================================================================================================
|
||||
:zeek:type:`Notice::Info`: :zeek:type:`record`
|
||||
|
||||
:New Fields: :zeek:type:`Notice::Info`
|
||||
|
||||
dropped: :zeek:type:`bool` :zeek:attr:`&log` :zeek:attr:`&default` = ``F`` :zeek:attr:`&optional`
|
||||
Indicate if the $src IP address was dropped and denied
|
||||
network access.
|
||||
============================================== ===================================================================================================
|
||||
|
||||
|
||||
Detailed Interface
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
|
||||
42
doc/scripts/policy/frameworks/notice/community-id.zeek.rst
Normal file
42
doc/scripts/policy/frameworks/notice/community-id.zeek.rst
Normal file
|
|
@ -0,0 +1,42 @@
|
|||
:tocdepth: 3
|
||||
|
||||
policy/frameworks/notice/community-id.zeek
|
||||
==========================================
|
||||
.. zeek:namespace:: CommunityID::Notice
|
||||
|
||||
|
||||
:Namespace: CommunityID::Notice
|
||||
:Imports: :doc:`base/frameworks/notice </scripts/base/frameworks/notice/index>`, :doc:`base/protocols/conn </scripts/base/protocols/conn/index>`, :doc:`policy/protocols/conn/community-id-logging.zeek </scripts/policy/protocols/conn/community-id-logging.zeek>`
|
||||
|
||||
Summary
|
||||
~~~~~~~
|
||||
Runtime Options
|
||||
###############
|
||||
============================================================================== =
|
||||
:zeek:id:`CommunityID::Notice::enabled`: :zeek:type:`bool` :zeek:attr:`&redef`
|
||||
============================================================================== =
|
||||
|
||||
Redefinitions
|
||||
#############
|
||||
============================================== ============================================================================
|
||||
:zeek:type:`Notice::Info`: :zeek:type:`record`
|
||||
|
||||
:New Fields: :zeek:type:`Notice::Info`
|
||||
|
||||
community_id: :zeek:type:`string` :zeek:attr:`&optional` :zeek:attr:`&log`
|
||||
============================================== ============================================================================
|
||||
|
||||
|
||||
Detailed Interface
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
Runtime Options
|
||||
###############
|
||||
.. zeek:id:: CommunityID::Notice::enabled
|
||||
:source-code: policy/frameworks/notice/community-id.zeek 14 14
|
||||
|
||||
:Type: :zeek:type:`bool`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``T``
|
||||
|
||||
|
||||
|
||||
|
|
@ -0,0 +1,20 @@
|
|||
:tocdepth: 3
|
||||
|
||||
policy/frameworks/notice/extend-email/hostnames.zeek
|
||||
====================================================
|
||||
.. zeek:namespace:: Notice
|
||||
|
||||
Loading this script extends the :zeek:enum:`Notice::ACTION_EMAIL` action
|
||||
by appending to the email the hostnames associated with
|
||||
:zeek:type:`Notice::Info`'s *src* and *dst* fields as determined by a
|
||||
DNS lookup.
|
||||
|
||||
:Namespace: Notice
|
||||
:Imports: :doc:`base/frameworks/notice/main.zeek </scripts/base/frameworks/notice/main.zeek>`
|
||||
|
||||
Summary
|
||||
~~~~~~~
|
||||
|
||||
Detailed Interface
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
|
||||
24
doc/scripts/policy/frameworks/notice/index.rst
Normal file
24
doc/scripts/policy/frameworks/notice/index.rst
Normal file
|
|
@ -0,0 +1,24 @@
|
|||
:orphan:
|
||||
|
||||
Package: policy/frameworks/notice
|
||||
=================================
|
||||
|
||||
|
||||
:doc:`/scripts/policy/frameworks/notice/__load__.zeek`
|
||||
|
||||
|
||||
:doc:`/scripts/policy/frameworks/notice/extend-email/hostnames.zeek`
|
||||
|
||||
Loading this script extends the :zeek:enum:`Notice::ACTION_EMAIL` action
|
||||
by appending to the email the hostnames associated with
|
||||
:zeek:type:`Notice::Info`'s *src* and *dst* fields as determined by a
|
||||
DNS lookup.
|
||||
|
||||
:doc:`/scripts/policy/frameworks/notice/actions/drop.zeek`
|
||||
|
||||
This script extends the built in notice code to implement the IP address
|
||||
dropping functionality.
|
||||
|
||||
:doc:`/scripts/policy/frameworks/notice/community-id.zeek`
|
||||
|
||||
|
||||
115
doc/scripts/policy/frameworks/packet-filter/shunt.zeek.rst
Normal file
115
doc/scripts/policy/frameworks/packet-filter/shunt.zeek.rst
Normal file
|
|
@ -0,0 +1,115 @@
|
|||
:tocdepth: 3
|
||||
|
||||
policy/frameworks/packet-filter/shunt.zeek
|
||||
==========================================
|
||||
.. zeek:namespace:: PacketFilter
|
||||
|
||||
|
||||
:Namespace: PacketFilter
|
||||
:Imports: :doc:`base/frameworks/notice </scripts/base/frameworks/notice/index>`, :doc:`base/frameworks/packet-filter </scripts/base/frameworks/packet-filter/index>`
|
||||
|
||||
Summary
|
||||
~~~~~~~
|
||||
Redefinable Options
|
||||
###################
|
||||
=============================================================================== =======================================================================
|
||||
:zeek:id:`PacketFilter::max_bpf_shunts`: :zeek:type:`count` :zeek:attr:`&redef` The maximum number of BPF based shunts that Zeek is allowed to perform.
|
||||
=============================================================================== =======================================================================
|
||||
|
||||
Redefinitions
|
||||
#############
|
||||
============================================ ============================================================
|
||||
:zeek:type:`Notice::Type`: :zeek:type:`enum`
|
||||
|
||||
* :zeek:enum:`PacketFilter::Cannot_BPF_Shunt_Conn`:
|
||||
Limitations in BPF make shunting some connections with BPF
|
||||
impossible.
|
||||
|
||||
* :zeek:enum:`PacketFilter::No_More_Conn_Shunts_Available`:
|
||||
Indicative that :zeek:id:`PacketFilter::max_bpf_shunts`
|
||||
connections are already being shunted with BPF filters and
|
||||
no more are allowed.
|
||||
============================================ ============================================================
|
||||
|
||||
Functions
|
||||
#########
|
||||
========================================================================== ============================================================================
|
||||
:zeek:id:`PacketFilter::current_shunted_conns`: :zeek:type:`function` Retrieve the currently shunted connections.
|
||||
:zeek:id:`PacketFilter::current_shunted_host_pairs`: :zeek:type:`function` Retrieve the currently shunted host pairs.
|
||||
:zeek:id:`PacketFilter::force_unshunt_host_pair`: :zeek:type:`function` Performs the same function as the :zeek:id:`PacketFilter::unshunt_host_pair`
|
||||
function, but it forces an immediate filter update.
|
||||
:zeek:id:`PacketFilter::shunt_conn`: :zeek:type:`function` Call this function to use BPF to shunt a connection (to prevent the
|
||||
data packets from reaching Zeek).
|
||||
:zeek:id:`PacketFilter::shunt_host_pair`: :zeek:type:`function` This function will use a BPF expression to shunt traffic between
|
||||
the two hosts given in the ``conn_id`` so that the traffic is never
|
||||
exposed to Zeek's traffic processing.
|
||||
:zeek:id:`PacketFilter::unshunt_host_pair`: :zeek:type:`function` Remove shunting for a host pair given as a ``conn_id``.
|
||||
========================================================================== ============================================================================
|
||||
|
||||
|
||||
Detailed Interface
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
Redefinable Options
|
||||
###################
|
||||
.. zeek:id:: PacketFilter::max_bpf_shunts
|
||||
:source-code: policy/frameworks/packet-filter/shunt.zeek 8 8
|
||||
|
||||
:Type: :zeek:type:`count`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``100``
|
||||
|
||||
The maximum number of BPF based shunts that Zeek is allowed to perform.
|
||||
|
||||
Functions
|
||||
#########
|
||||
.. zeek:id:: PacketFilter::current_shunted_conns
|
||||
:source-code: policy/frameworks/packet-filter/shunt.zeek 86 89
|
||||
|
||||
:Type: :zeek:type:`function` () : :zeek:type:`set` [:zeek:type:`conn_id`]
|
||||
|
||||
Retrieve the currently shunted connections.
|
||||
|
||||
.. zeek:id:: PacketFilter::current_shunted_host_pairs
|
||||
:source-code: policy/frameworks/packet-filter/shunt.zeek 91 94
|
||||
|
||||
:Type: :zeek:type:`function` () : :zeek:type:`set` [:zeek:type:`conn_id`]
|
||||
|
||||
Retrieve the currently shunted host pairs.
|
||||
|
||||
.. zeek:id:: PacketFilter::force_unshunt_host_pair
|
||||
:source-code: policy/frameworks/packet-filter/shunt.zeek 133 142
|
||||
|
||||
:Type: :zeek:type:`function` (id: :zeek:type:`conn_id`) : :zeek:type:`bool`
|
||||
|
||||
Performs the same function as the :zeek:id:`PacketFilter::unshunt_host_pair`
|
||||
function, but it forces an immediate filter update.
|
||||
|
||||
.. zeek:id:: PacketFilter::shunt_conn
|
||||
:source-code: policy/frameworks/packet-filter/shunt.zeek 144 162
|
||||
|
||||
:Type: :zeek:type:`function` (id: :zeek:type:`conn_id`) : :zeek:type:`bool`
|
||||
|
||||
Call this function to use BPF to shunt a connection (to prevent the
|
||||
data packets from reaching Zeek). For TCP connections, control
|
||||
packets are still allowed through so that Zeek can continue logging
|
||||
the connection and it can stop shunting once the connection ends.
|
||||
|
||||
.. zeek:id:: PacketFilter::shunt_host_pair
|
||||
:source-code: policy/frameworks/packet-filter/shunt.zeek 108 118
|
||||
|
||||
:Type: :zeek:type:`function` (id: :zeek:type:`conn_id`) : :zeek:type:`bool`
|
||||
|
||||
This function will use a BPF expression to shunt traffic between
|
||||
the two hosts given in the ``conn_id`` so that the traffic is never
|
||||
exposed to Zeek's traffic processing.
|
||||
|
||||
.. zeek:id:: PacketFilter::unshunt_host_pair
|
||||
:source-code: policy/frameworks/packet-filter/shunt.zeek 120 131
|
||||
|
||||
:Type: :zeek:type:`function` (id: :zeek:type:`conn_id`) : :zeek:type:`bool`
|
||||
|
||||
Remove shunting for a host pair given as a ``conn_id``. The filter
|
||||
is not immediately removed. It waits for the occasional filter
|
||||
update done by the ``PacketFilter`` framework.
|
||||
|
||||
|
||||
21
doc/scripts/policy/frameworks/signatures/iso-9660.zeek.rst
Normal file
21
doc/scripts/policy/frameworks/signatures/iso-9660.zeek.rst
Normal file
|
|
@ -0,0 +1,21 @@
|
|||
:tocdepth: 3
|
||||
|
||||
policy/frameworks/signatures/iso-9660.zeek
|
||||
==========================================
|
||||
|
||||
Load signature for ISO 9660 disk image and increase
|
||||
default_file_bof_buffer_size to make it functional.
|
||||
|
||||
|
||||
Summary
|
||||
~~~~~~~
|
||||
Redefinitions
|
||||
#############
|
||||
=============================================================================== =
|
||||
:zeek:id:`default_file_bof_buffer_size`: :zeek:type:`count` :zeek:attr:`&redef`
|
||||
=============================================================================== =
|
||||
|
||||
|
||||
Detailed Interface
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
|
||||
|
|
@ -0,0 +1,49 @@
|
|||
:tocdepth: 3
|
||||
|
||||
policy/frameworks/software/version-changes.zeek
|
||||
===============================================
|
||||
.. zeek:namespace:: Software
|
||||
|
||||
Provides the possibility to define software names that are interesting to
|
||||
watch for changes. A notice is generated if software versions change on a
|
||||
host.
|
||||
|
||||
:Namespace: Software
|
||||
:Imports: :doc:`base/frameworks/notice </scripts/base/frameworks/notice/index>`, :doc:`base/frameworks/software </scripts/base/frameworks/software/index>`
|
||||
|
||||
Summary
|
||||
~~~~~~~
|
||||
Runtime Options
|
||||
###############
|
||||
====================================================================================== ====================================================================
|
||||
:zeek:id:`Software::interesting_version_changes`: :zeek:type:`set` :zeek:attr:`&redef` Some software is more interesting when the version changes and this
|
||||
is a set of all software that should raise a notice when a different
|
||||
version is seen on a host.
|
||||
====================================================================================== ====================================================================
|
||||
|
||||
Redefinitions
|
||||
#############
|
||||
============================================ ======================================================
|
||||
:zeek:type:`Notice::Type`: :zeek:type:`enum`
|
||||
|
||||
* :zeek:enum:`Software::Software_Version_Change`:
|
||||
For certain software, a version changing may matter.
|
||||
============================================ ======================================================
|
||||
|
||||
|
||||
Detailed Interface
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
Runtime Options
|
||||
###############
|
||||
.. zeek:id:: Software::interesting_version_changes
|
||||
:source-code: policy/frameworks/software/version-changes.zeek 22 22
|
||||
|
||||
:Type: :zeek:type:`set` [:zeek:type:`string`]
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``{}``
|
||||
|
||||
Some software is more interesting when the version changes and this
|
||||
is a set of all software that should raise a notice when a different
|
||||
version is seen on a host.
|
||||
|
||||
|
||||
111
doc/scripts/policy/frameworks/software/vulnerable.zeek.rst
Normal file
111
doc/scripts/policy/frameworks/software/vulnerable.zeek.rst
Normal file
|
|
@ -0,0 +1,111 @@
|
|||
:tocdepth: 3
|
||||
|
||||
policy/frameworks/software/vulnerable.zeek
|
||||
==========================================
|
||||
.. zeek:namespace:: Software
|
||||
|
||||
Provides a variable to define vulnerable versions of software and if
|
||||
a version of that software is as old or older than the defined version a
|
||||
notice will be generated.
|
||||
|
||||
:Namespace: Software
|
||||
:Imports: :doc:`base/frameworks/control </scripts/base/frameworks/control/index>`, :doc:`base/frameworks/notice </scripts/base/frameworks/notice/index>`, :doc:`base/frameworks/software </scripts/base/frameworks/software/index>`
|
||||
|
||||
Summary
|
||||
~~~~~~~
|
||||
Runtime Options
|
||||
###############
|
||||
=================================================================================================== =============================================================
|
||||
:zeek:id:`Software::vulnerable_versions_update_endpoint`: :zeek:type:`string` :zeek:attr:`&redef` The DNS zone where runtime vulnerable software updates will
|
||||
be loaded from.
|
||||
:zeek:id:`Software::vulnerable_versions_update_interval`: :zeek:type:`interval` :zeek:attr:`&redef` The interval at which vulnerable versions should grab updates
|
||||
over DNS.
|
||||
=================================================================================================== =============================================================
|
||||
|
||||
Redefinable Options
|
||||
###################
|
||||
================================================================================ ===============================================================
|
||||
:zeek:id:`Software::vulnerable_versions`: :zeek:type:`table` :zeek:attr:`&redef` This is a table of software versions indexed by the name of the
|
||||
software and a set of version ranges that are declared to be
|
||||
vulnerable for that software.
|
||||
================================================================================ ===============================================================
|
||||
|
||||
Types
|
||||
#####
|
||||
================================================================== =
|
||||
:zeek:type:`Software::VulnerableVersionRange`: :zeek:type:`record`
|
||||
================================================================== =
|
||||
|
||||
Redefinitions
|
||||
#############
|
||||
============================================ ===============================================================
|
||||
:zeek:type:`Notice::Type`: :zeek:type:`enum`
|
||||
|
||||
* :zeek:enum:`Software::Vulnerable_Version`:
|
||||
Indicates that a vulnerable version of software was detected.
|
||||
============================================ ===============================================================
|
||||
|
||||
|
||||
Detailed Interface
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
Runtime Options
|
||||
###############
|
||||
.. zeek:id:: Software::vulnerable_versions_update_endpoint
|
||||
:source-code: policy/frameworks/software/vulnerable.zeek 32 32
|
||||
|
||||
:Type: :zeek:type:`string`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``""``
|
||||
|
||||
The DNS zone where runtime vulnerable software updates will
|
||||
be loaded from.
|
||||
|
||||
.. zeek:id:: Software::vulnerable_versions_update_interval
|
||||
:source-code: policy/frameworks/software/vulnerable.zeek 36 36
|
||||
|
||||
:Type: :zeek:type:`interval`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``1.0 hr``
|
||||
|
||||
The interval at which vulnerable versions should grab updates
|
||||
over DNS.
|
||||
|
||||
Redefinable Options
|
||||
###################
|
||||
.. zeek:id:: Software::vulnerable_versions
|
||||
:source-code: policy/frameworks/software/vulnerable.zeek 41 41
|
||||
|
||||
:Type: :zeek:type:`table` [:zeek:type:`string`] of :zeek:type:`set` [:zeek:type:`Software::VulnerableVersionRange`]
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``{}``
|
||||
|
||||
This is a table of software versions indexed by the name of the
|
||||
software and a set of version ranges that are declared to be
|
||||
vulnerable for that software.
|
||||
|
||||
Types
|
||||
#####
|
||||
.. zeek:type:: Software::VulnerableVersionRange
|
||||
:source-code: policy/frameworks/software/vulnerable.zeek 17 28
|
||||
|
||||
:Type: :zeek:type:`record`
|
||||
|
||||
|
||||
.. zeek:field:: min :zeek:type:`Software::Version` :zeek:attr:`&optional`
|
||||
|
||||
The minimal version of a vulnerable version range. This
|
||||
field can be undefined if all previous versions of a piece
|
||||
of software are vulnerable.
|
||||
|
||||
|
||||
.. zeek:field:: max :zeek:type:`Software::Version`
|
||||
|
||||
The maximum vulnerable version. This field is deliberately
|
||||
not optional because a maximum vulnerable version must
|
||||
always be defined. This assumption may become incorrect
|
||||
if all future versions of some software are to be considered
|
||||
vulnerable. :)
|
||||
|
||||
|
||||
|
||||
|
||||
|
|
@ -0,0 +1,98 @@
|
|||
:tocdepth: 3
|
||||
|
||||
policy/frameworks/software/windows-version-detection.zeek
|
||||
=========================================================
|
||||
.. zeek:namespace:: OS
|
||||
|
||||
Windows systems access a Microsoft Certificate Revocation List (CRL) periodically. The
|
||||
user agent for these requests reveals which version of Crypt32.dll installed on the system,
|
||||
which can uniquely identify the version of Windows that's running.
|
||||
|
||||
This script will log the version of Windows that was identified to the Software framework.
|
||||
|
||||
:Namespace: OS
|
||||
:Imports: :doc:`base/frameworks/software </scripts/base/frameworks/software/index>`, :doc:`base/protocols/http </scripts/base/protocols/http/index>`
|
||||
|
||||
Summary
|
||||
~~~~~~~
|
||||
Redefinable Options
|
||||
###################
|
||||
========================================================================= =
|
||||
:zeek:id:`OS::crypto_api_mapping`: :zeek:type:`table` :zeek:attr:`&redef`
|
||||
========================================================================= =
|
||||
|
||||
Types
|
||||
#####
|
||||
============================================================ =
|
||||
:zeek:type:`Software::name_and_version`: :zeek:type:`record`
|
||||
============================================================ =
|
||||
|
||||
Redefinitions
|
||||
#############
|
||||
============================================== ==================================================
|
||||
:zeek:type:`Software::Type`: :zeek:type:`enum`
|
||||
|
||||
* :zeek:enum:`OS::WINDOWS`:
|
||||
Identifier for Windows operating system versions
|
||||
============================================== ==================================================
|
||||
|
||||
|
||||
Detailed Interface
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
Redefinable Options
|
||||
###################
|
||||
.. zeek:id:: OS::crypto_api_mapping
|
||||
:source-code: policy/frameworks/software/windows-version-detection.zeek 23 23
|
||||
|
||||
:Type: :zeek:type:`table` [:zeek:type:`string`] of :zeek:type:`Software::name_and_version`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default:
|
||||
|
||||
::
|
||||
|
||||
{
|
||||
["Microsoft-CryptoAPI/5.131.3790.1830"] = [name="Windows", version=[major=5, minor=131, minor2=3790, minor3=1830, addl="XP x64 or Server 2003 SP1"]],
|
||||
["Microsoft-CryptoAPI/5.131.2600.3249"] = [name="Windows", version=[major=5, minor=131, minor2=2600, minor3=3249, addl="XP SP3 RC Beta"]],
|
||||
["Microsoft-CryptoAPI/5.131.2600.5508"] = [name="Windows", version=[major=5, minor=131, minor2=2600, minor3=5508, addl="XP SP3 RC2 Update 2"]],
|
||||
["Microsoft-CryptoAPI/5.131.2600.3180"] = [name="Windows", version=[major=5, minor=131, minor2=2600, minor3=3180, addl="XP SP3 Beta 1"]],
|
||||
["Microsoft-CryptoAPI/5.131.2600.3264"] = [name="Windows", version=[major=5, minor=131, minor2=2600, minor3=3264, addl="XP SP3 RC1"]],
|
||||
["Microsoft-CryptoAPI/6.2"] = [name="Windows", version=[major=6, minor=2, minor2=<uninitialized>, minor3=<uninitialized>, addl="8 or Server 2012"]],
|
||||
["Microsoft-CryptoAPI/5.131.2600.3282"] = [name="Windows", version=[major=5, minor=131, minor2=2600, minor3=3282, addl="XP SP3 RC1 Update"]],
|
||||
["Microsoft-CryptoAPI/5.131.3790.5235"] = [name="Windows", version=[major=5, minor=131, minor2=3790, minor3=5235, addl="XP x64 or Server 2003 with MS13-095"]],
|
||||
["Microsoft-CryptoAPI/6.4"] = [name="Windows", version=[major=6, minor=4, minor2=<uninitialized>, minor3=<uninitialized>, addl="10 Technical Preview"]],
|
||||
["Microsoft-CryptoAPI/5.131.2600.0"] = [name="Windows", version=[major=5, minor=131, minor2=2600, minor3=0, addl="XP SP0"]],
|
||||
["Microsoft-CryptoAPI/5.131.2600.5512"] = [name="Windows", version=[major=5, minor=131, minor2=2600, minor3=5512, addl="XP SP3"]],
|
||||
["Microsoft-CryptoAPI/5.131.2195.6661"] = [name="Windows", version=[major=5, minor=131, minor2=2195, minor3=6661, addl="2000 SP4"]],
|
||||
["Microsoft-CryptoAPI/5.131.2195.6926"] = [name="Windows", version=[major=5, minor=131, minor2=2195, minor3=6926, addl="2000 with Hotfix 98830"]],
|
||||
["Microsoft-CryptoAPI/6.0"] = [name="Windows", version=[major=6, minor=0, minor2=<uninitialized>, minor3=<uninitialized>, addl="Vista or Server 2008"]],
|
||||
["Microsoft-CryptoAPI/5.131.3790.0"] = [name="Windows", version=[major=5, minor=131, minor2=3790, minor3=0, addl="XP x64 or Server 2003 SP0"]],
|
||||
["Microsoft-CryptoAPI/6.3"] = [name="Windows", version=[major=6, minor=3, minor2=<uninitialized>, minor3=<uninitialized>, addl="8.1 or Server 2012 R2"]],
|
||||
["Microsoft-CryptoAPI/5.131.2600.3205"] = [name="Windows", version=[major=5, minor=131, minor2=2600, minor3=3205, addl="XP SP3 Beta 2"]],
|
||||
["Microsoft-CryptoAPI/5.131.2600.1106"] = [name="Windows", version=[major=5, minor=131, minor2=2600, minor3=1106, addl="XP SP1"]],
|
||||
["Microsoft-CryptoAPI/5.131.2600.2180"] = [name="Windows", version=[major=5, minor=131, minor2=2600, minor3=2180, addl="XP SP2"]],
|
||||
["Microsoft-CryptoAPI/5.131.2195.6824"] = [name="Windows", version=[major=5, minor=131, minor2=2195, minor3=6824, addl="2000 with MS04-11"]],
|
||||
["Microsoft-CryptoAPI/5.131.2600.3300"] = [name="Windows", version=[major=5, minor=131, minor2=2600, minor3=3300, addl="XP SP3 RC2"]],
|
||||
["Microsoft-CryptoAPI/6.1"] = [name="Windows", version=[major=6, minor=1, minor2=<uninitialized>, minor3=<uninitialized>, addl="7 or Server 2008 R2"]],
|
||||
["Microsoft-CryptoAPI/10.0"] = [name="Windows", version=[major=10, minor=0, minor2=<uninitialized>, minor3=<uninitialized>, addl=<uninitialized>]],
|
||||
["Microsoft-CryptoAPI/5.131.2600.3311"] = [name="Windows", version=[major=5, minor=131, minor2=2600, minor3=3311, addl="XP SP3 RC2 Update"]],
|
||||
["Microsoft-CryptoAPI/5.131.3790.3959"] = [name="Windows", version=[major=5, minor=131, minor2=3790, minor3=3959, addl="XP x64 or Server 2003 SP2"]]
|
||||
}
|
||||
|
||||
|
||||
|
||||
Types
|
||||
#####
|
||||
.. zeek:type:: Software::name_and_version
|
||||
:source-code: policy/frameworks/software/windows-version-detection.zeek 18 21
|
||||
|
||||
:Type: :zeek:type:`record`
|
||||
|
||||
|
||||
.. zeek:field:: name :zeek:type:`string`
|
||||
|
||||
|
||||
.. zeek:field:: version :zeek:type:`Software::Version`
|
||||
|
||||
|
||||
|
||||
|
||||
16
doc/scripts/policy/frameworks/spicy/resource-usage.zeek.rst
Normal file
16
doc/scripts/policy/frameworks/spicy/resource-usage.zeek.rst
Normal file
|
|
@ -0,0 +1,16 @@
|
|||
:tocdepth: 3
|
||||
|
||||
policy/frameworks/spicy/resource-usage.zeek
|
||||
===========================================
|
||||
.. zeek:namespace:: Spicy
|
||||
|
||||
Logs Spicy-related resource usage continuously for debugging purposes.
|
||||
|
||||
:Namespace: Spicy
|
||||
|
||||
Summary
|
||||
~~~~~~~
|
||||
|
||||
Detailed Interface
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
|
||||
|
|
@ -0,0 +1,14 @@
|
|||
:tocdepth: 3
|
||||
|
||||
policy/frameworks/storage/backend/redis/__load__.zeek
|
||||
=====================================================
|
||||
|
||||
|
||||
:Imports: :doc:`policy/frameworks/storage/backend/redis/main.zeek </scripts/policy/frameworks/storage/backend/redis/main.zeek>`
|
||||
|
||||
Summary
|
||||
~~~~~~~
|
||||
|
||||
Detailed Interface
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
|
||||
|
|
@ -0,0 +1,13 @@
|
|||
:orphan:
|
||||
|
||||
Package: policy/frameworks/storage/backend/redis
|
||||
================================================
|
||||
|
||||
|
||||
:doc:`/scripts/policy/frameworks/storage/backend/redis/__load__.zeek`
|
||||
|
||||
|
||||
:doc:`/scripts/policy/frameworks/storage/backend/redis/main.zeek`
|
||||
|
||||
Redis storage backend support
|
||||
|
||||
|
|
@ -0,0 +1,108 @@
|
|||
:tocdepth: 3
|
||||
|
||||
policy/frameworks/storage/backend/redis/main.zeek
|
||||
=================================================
|
||||
.. zeek:namespace:: Storage::Backend::Redis
|
||||
|
||||
Redis storage backend support
|
||||
|
||||
:Namespace: Storage::Backend::Redis
|
||||
:Imports: :doc:`base/frameworks/storage/main.zeek </scripts/base/frameworks/storage/main.zeek>`
|
||||
|
||||
Summary
|
||||
~~~~~~~
|
||||
Redefinable Options
|
||||
###################
|
||||
======================================================================================================== ==============================================
|
||||
:zeek:id:`Storage::Backend::Redis::default_connect_timeout`: :zeek:type:`interval` :zeek:attr:`&redef` Default value for connection attempt timeouts.
|
||||
:zeek:id:`Storage::Backend::Redis::default_operation_timeout`: :zeek:type:`interval` :zeek:attr:`&redef` Default value for operation timeouts.
|
||||
======================================================================================================== ==============================================
|
||||
|
||||
Types
|
||||
#####
|
||||
================================================================== ==============================================
|
||||
:zeek:type:`Storage::Backend::Redis::Options`: :zeek:type:`record` Options record for the built-in Redis backend.
|
||||
================================================================== ==============================================
|
||||
|
||||
Redefinitions
|
||||
#############
|
||||
========================================================= =============================================================================
|
||||
:zeek:type:`Storage::BackendOptions`: :zeek:type:`record`
|
||||
|
||||
:New Fields: :zeek:type:`Storage::BackendOptions`
|
||||
|
||||
redis: :zeek:type:`Storage::Backend::Redis::Options` :zeek:attr:`&optional`
|
||||
========================================================= =============================================================================
|
||||
|
||||
|
||||
Detailed Interface
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
Redefinable Options
|
||||
###################
|
||||
.. zeek:id:: Storage::Backend::Redis::default_connect_timeout
|
||||
:source-code: policy/frameworks/storage/backend/redis/main.zeek 10 10
|
||||
|
||||
:Type: :zeek:type:`interval`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``5.0 secs``
|
||||
|
||||
Default value for connection attempt timeouts. This can be overridden
|
||||
per-connection with the ``connect_timeout`` backend option.
|
||||
|
||||
.. zeek:id:: Storage::Backend::Redis::default_operation_timeout
|
||||
:source-code: policy/frameworks/storage/backend/redis/main.zeek 14 14
|
||||
|
||||
:Type: :zeek:type:`interval`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``5.0 secs``
|
||||
|
||||
Default value for operation timeouts. This can be overridden per-connection
|
||||
with the ``operation_timeout`` backend option.
|
||||
|
||||
Types
|
||||
#####
|
||||
.. zeek:type:: Storage::Backend::Redis::Options
|
||||
:source-code: policy/frameworks/storage/backend/redis/main.zeek 17 49
|
||||
|
||||
:Type: :zeek:type:`record`
|
||||
|
||||
|
||||
.. zeek:field:: server_host :zeek:type:`string` :zeek:attr:`&optional`
|
||||
|
||||
|
||||
.. zeek:field:: server_port :zeek:type:`port` :zeek:attr:`&default` = ``6379/tcp`` :zeek:attr:`&optional`
|
||||
|
||||
|
||||
.. zeek:field:: server_unix_socket :zeek:type:`string` :zeek:attr:`&optional`
|
||||
|
||||
|
||||
.. zeek:field:: key_prefix :zeek:type:`string` :zeek:attr:`&default` = ``""`` :zeek:attr:`&optional`
|
||||
|
||||
|
||||
.. zeek:field:: connect_timeout :zeek:type:`interval` :zeek:attr:`&default` = :zeek:see:`Storage::Backend::Redis::default_connect_timeout` :zeek:attr:`&optional`
|
||||
|
||||
Timeout for connection attempts to the backend. Connection attempts
|
||||
that exceed this time will return
|
||||
:zeek:see:`Storage::CONNECTION_FAILED`.
|
||||
|
||||
|
||||
.. zeek:field:: operation_timeout :zeek:type:`interval` :zeek:attr:`&default` = :zeek:see:`Storage::Backend::Redis::default_operation_timeout` :zeek:attr:`&optional`
|
||||
|
||||
Timeout for operation requests sent to the backend. Operations that
|
||||
exceed this time will return :zeek:see:`Storage::TIMEOUT`.
|
||||
|
||||
|
||||
.. zeek:field:: username :zeek:type:`string` :zeek:attr:`&optional`
|
||||
|
||||
A username to use for authentication the server is protected by an ACL.
|
||||
|
||||
|
||||
.. zeek:field:: password :zeek:type:`string` :zeek:attr:`&optional`
|
||||
|
||||
A username to use for authentication the server is protected by an ACL
|
||||
or by a simple password.
|
||||
|
||||
|
||||
Options record for the built-in Redis backend.
|
||||
|
||||
|
||||
|
|
@ -0,0 +1,14 @@
|
|||
:tocdepth: 3
|
||||
|
||||
policy/frameworks/storage/backend/sqlite/__load__.zeek
|
||||
======================================================
|
||||
|
||||
|
||||
:Imports: :doc:`policy/frameworks/storage/backend/sqlite/main.zeek </scripts/policy/frameworks/storage/backend/sqlite/main.zeek>`
|
||||
|
||||
Summary
|
||||
~~~~~~~
|
||||
|
||||
Detailed Interface
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
|
||||
|
|
@ -0,0 +1,13 @@
|
|||
:orphan:
|
||||
|
||||
Package: policy/frameworks/storage/backend/sqlite
|
||||
=================================================
|
||||
|
||||
|
||||
:doc:`/scripts/policy/frameworks/storage/backend/sqlite/__load__.zeek`
|
||||
|
||||
|
||||
:doc:`/scripts/policy/frameworks/storage/backend/sqlite/main.zeek`
|
||||
|
||||
SQLite storage backend support
|
||||
|
||||
|
|
@ -0,0 +1,91 @@
|
|||
:tocdepth: 3
|
||||
|
||||
policy/frameworks/storage/backend/sqlite/main.zeek
|
||||
==================================================
|
||||
.. zeek:namespace:: Storage::Backend::SQLite
|
||||
|
||||
SQLite storage backend support
|
||||
|
||||
:Namespace: Storage::Backend::SQLite
|
||||
:Imports: :doc:`base/frameworks/storage/main.zeek </scripts/base/frameworks/storage/main.zeek>`
|
||||
|
||||
Summary
|
||||
~~~~~~~
|
||||
Types
|
||||
#####
|
||||
=================================================================== ===============================================
|
||||
:zeek:type:`Storage::Backend::SQLite::Options`: :zeek:type:`record` Options record for the built-in SQLite backend.
|
||||
=================================================================== ===============================================
|
||||
|
||||
Redefinitions
|
||||
#############
|
||||
========================================================= ===============================================================================
|
||||
:zeek:type:`Storage::BackendOptions`: :zeek:type:`record`
|
||||
|
||||
:New Fields: :zeek:type:`Storage::BackendOptions`
|
||||
|
||||
sqlite: :zeek:type:`Storage::Backend::SQLite::Options` :zeek:attr:`&optional`
|
||||
========================================================= ===============================================================================
|
||||
|
||||
|
||||
Detailed Interface
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
Types
|
||||
#####
|
||||
.. zeek:type:: Storage::Backend::SQLite::Options
|
||||
:source-code: policy/frameworks/storage/backend/sqlite/main.zeek 9 50
|
||||
|
||||
:Type: :zeek:type:`record`
|
||||
|
||||
|
||||
.. zeek:field:: database_path :zeek:type:`string`
|
||||
|
||||
Path to the database file on disk. Setting this to ":memory:" will tell
|
||||
SQLite to use an in-memory database. Relative paths will be opened
|
||||
relative to the directory where Zeek was started from. Zeek will not
|
||||
create intermediate directories if they do not already exist. See
|
||||
https://www.sqlite.org/c3ref/open.html for more rules on paths that can
|
||||
be passed here.
|
||||
|
||||
|
||||
.. zeek:field:: table_name :zeek:type:`string`
|
||||
|
||||
Name of the table used for storing data. It is possible to use the same
|
||||
database file for two separate tables, as long as the this value is
|
||||
different between the two.
|
||||
|
||||
|
||||
.. zeek:field:: busy_timeout :zeek:type:`interval` :zeek:attr:`&default` = ``5.0 secs`` :zeek:attr:`&optional`
|
||||
|
||||
The timeout for the connection to the database. This is set
|
||||
per-connection. It is equivalent to setting a ``busy_timeout`` pragma
|
||||
value, but that value will be ignored in favor of this field.
|
||||
|
||||
|
||||
.. zeek:field:: pragma_commands :zeek:type:`table` [:zeek:type:`string`] of :zeek:type:`string` :zeek:attr:`&ordered` :zeek:attr:`&default` = *...* :zeek:attr:`&optional`
|
||||
|
||||
Key/value table for passing pragma commands when opening the database.
|
||||
These must be pairs that can be passed to the ``pragma`` command in
|
||||
sqlite. The ``integrity_check`` pragma is run automatically and does
|
||||
not need to be included here. For pragmas without a second argument,
|
||||
set the value to an empty string. Setting the ``busy_timeout`` pragma
|
||||
here will be ignored.
|
||||
|
||||
|
||||
.. zeek:field:: pragma_timeout :zeek:type:`interval` :zeek:attr:`&default` = ``500.0 msecs`` :zeek:attr:`&optional`
|
||||
|
||||
The total amount of time that an SQLite backend will spend attempting
|
||||
to run an individual pragma command before giving up and returning an
|
||||
initialization error. Setting this to zero will result in the backend
|
||||
attempting forever until success.
|
||||
|
||||
|
||||
.. zeek:field:: pragma_wait_on_busy :zeek:type:`interval` :zeek:attr:`&default` = ``5.0 msecs`` :zeek:attr:`&optional`
|
||||
|
||||
The amount of time that at SQLite backend will wait between failures
|
||||
to run an individual pragma command.
|
||||
|
||||
|
||||
Options record for the built-in SQLite backend.
|
||||
|
||||
|
||||
225
doc/scripts/policy/frameworks/telemetry/log.zeek.rst
Normal file
225
doc/scripts/policy/frameworks/telemetry/log.zeek.rst
Normal file
|
|
@ -0,0 +1,225 @@
|
|||
:tocdepth: 3
|
||||
|
||||
policy/frameworks/telemetry/log.zeek
|
||||
====================================
|
||||
.. zeek:namespace:: Telemetry
|
||||
|
||||
Implementation of a :file:`telemetry.log` and :file:`telemetry_histogram.log`
|
||||
file using metrics accessible via the Telemetry module.
|
||||
|
||||
:Namespace: Telemetry
|
||||
:Imports: :doc:`base/frameworks/telemetry </scripts/base/frameworks/telemetry/index>`
|
||||
|
||||
Summary
|
||||
~~~~~~~
|
||||
Runtime Options
|
||||
###############
|
||||
============================================================================= ===============================================================
|
||||
:zeek:id:`Telemetry::log_interval`: :zeek:type:`interval` :zeek:attr:`&redef` How often metrics are reported.
|
||||
:zeek:id:`Telemetry::log_prefixes`: :zeek:type:`set` :zeek:attr:`&redef` Only metrics with prefixes in this set will be included in the
|
||||
:file:`telemetry.log` and :file:`telemetry_histogram.log` files
|
||||
by default.
|
||||
============================================================================= ===============================================================
|
||||
|
||||
Types
|
||||
#####
|
||||
========================================================== =======================================================
|
||||
:zeek:type:`Telemetry::HistogramInfo`: :zeek:type:`record` Record type used for logging histogram metrics.
|
||||
:zeek:type:`Telemetry::Info`: :zeek:type:`record` Record type used for logging counter and gauge metrics.
|
||||
========================================================== =======================================================
|
||||
|
||||
Redefinitions
|
||||
#############
|
||||
======================================= =======================================
|
||||
:zeek:type:`Log::ID`: :zeek:type:`enum`
|
||||
|
||||
* :zeek:enum:`Telemetry::LOG`
|
||||
|
||||
* :zeek:enum:`Telemetry::LOG_HISTOGRAM`
|
||||
======================================= =======================================
|
||||
|
||||
Events
|
||||
######
|
||||
================================================================= =========================================================
|
||||
:zeek:id:`Telemetry::log_telemetry`: :zeek:type:`event` Event triggered for every record in the stream.
|
||||
:zeek:id:`Telemetry::log_telemetry_histogram`: :zeek:type:`event` Event triggered for every record in the histogram stream.
|
||||
================================================================= =========================================================
|
||||
|
||||
Hooks
|
||||
#####
|
||||
======================================================================== =======================================================
|
||||
:zeek:id:`Telemetry::log_policy`: :zeek:type:`Log::PolicyHook` A default logging policy hook for the stream.
|
||||
:zeek:id:`Telemetry::log_policy_histogram`: :zeek:type:`Log::PolicyHook` A default logging policy hook for the histogram stream.
|
||||
======================================================================== =======================================================
|
||||
|
||||
|
||||
Detailed Interface
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
Runtime Options
|
||||
###############
|
||||
.. zeek:id:: Telemetry::log_interval
|
||||
:source-code: policy/frameworks/telemetry/log.zeek 12 12
|
||||
|
||||
:Type: :zeek:type:`interval`
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default: ``1.0 min``
|
||||
|
||||
How often metrics are reported.
|
||||
|
||||
.. zeek:id:: Telemetry::log_prefixes
|
||||
:source-code: policy/frameworks/telemetry/log.zeek 23 23
|
||||
|
||||
:Type: :zeek:type:`set` [:zeek:type:`string`]
|
||||
:Attributes: :zeek:attr:`&redef`
|
||||
:Default:
|
||||
|
||||
::
|
||||
|
||||
{
|
||||
"process",
|
||||
"zeek"
|
||||
}
|
||||
|
||||
|
||||
Only metrics with prefixes in this set will be included in the
|
||||
:file:`telemetry.log` and :file:`telemetry_histogram.log` files
|
||||
by default. Setting this option to an empty set includes all
|
||||
prefixes.
|
||||
|
||||
For more fine-grained customization, setting this option to an
|
||||
empty set and implementing the :zeek:see:`Telemetry::log_policy`
|
||||
and :zeek:see:`Telemetry::log_policy_histogram` hooks to filter
|
||||
individual records is recommended.
|
||||
|
||||
Types
|
||||
#####
|
||||
.. zeek:type:: Telemetry::HistogramInfo
|
||||
:source-code: policy/frameworks/telemetry/log.zeek 51 78
|
||||
|
||||
:Type: :zeek:type:`record`
|
||||
|
||||
|
||||
.. zeek:field:: ts :zeek:type:`time` :zeek:attr:`&log`
|
||||
|
||||
Timestamp of reporting.
|
||||
|
||||
|
||||
.. zeek:field:: peer :zeek:type:`string` :zeek:attr:`&log`
|
||||
|
||||
Peer that generated this log.
|
||||
|
||||
|
||||
.. zeek:field:: name :zeek:type:`string` :zeek:attr:`&log`
|
||||
|
||||
The name of the metric.
|
||||
|
||||
|
||||
.. zeek:field:: labels :zeek:type:`vector` of :zeek:type:`string` :zeek:attr:`&log`
|
||||
|
||||
The names of the individual labels.
|
||||
|
||||
|
||||
.. zeek:field:: label_values :zeek:type:`vector` of :zeek:type:`string` :zeek:attr:`&log`
|
||||
|
||||
The values of the labels as listed in ``labels``.
|
||||
|
||||
|
||||
.. zeek:field:: bounds :zeek:type:`vector` of :zeek:type:`double` :zeek:attr:`&log`
|
||||
|
||||
The bounds of the individual buckets
|
||||
|
||||
|
||||
.. zeek:field:: values :zeek:type:`vector` of :zeek:type:`double` :zeek:attr:`&log`
|
||||
|
||||
The number of observations within each individual bucket.
|
||||
|
||||
|
||||
.. zeek:field:: sum :zeek:type:`double` :zeek:attr:`&log`
|
||||
|
||||
The sum over all observations
|
||||
|
||||
|
||||
.. zeek:field:: observations :zeek:type:`double` :zeek:attr:`&log`
|
||||
|
||||
The total number of observations.
|
||||
|
||||
|
||||
Record type used for logging histogram metrics.
|
||||
|
||||
.. zeek:type:: Telemetry::Info
|
||||
:source-code: policy/frameworks/telemetry/log.zeek 26 48
|
||||
|
||||
:Type: :zeek:type:`record`
|
||||
|
||||
|
||||
.. zeek:field:: ts :zeek:type:`time` :zeek:attr:`&log`
|
||||
|
||||
Timestamp of reporting.
|
||||
|
||||
|
||||
.. zeek:field:: peer :zeek:type:`string` :zeek:attr:`&log`
|
||||
|
||||
Peer that generated this log.
|
||||
|
||||
|
||||
.. zeek:field:: metric_type :zeek:type:`string` :zeek:attr:`&log`
|
||||
|
||||
Contains the value "counter" or "gauge" depending on
|
||||
the underlying metric type.
|
||||
|
||||
|
||||
.. zeek:field:: name :zeek:type:`string` :zeek:attr:`&log`
|
||||
|
||||
The name of the metric.
|
||||
|
||||
|
||||
.. zeek:field:: labels :zeek:type:`vector` of :zeek:type:`string` :zeek:attr:`&log`
|
||||
|
||||
The names of the individual labels.
|
||||
|
||||
|
||||
.. zeek:field:: label_values :zeek:type:`vector` of :zeek:type:`string` :zeek:attr:`&log`
|
||||
|
||||
The values of the labels as listed in ``labels``.
|
||||
|
||||
|
||||
.. zeek:field:: value :zeek:type:`double` :zeek:attr:`&log`
|
||||
|
||||
The value of this metric.
|
||||
|
||||
|
||||
Record type used for logging counter and gauge metrics.
|
||||
|
||||
Events
|
||||
######
|
||||
.. zeek:id:: Telemetry::log_telemetry
|
||||
:source-code: policy/frameworks/telemetry/log.zeek 87 87
|
||||
|
||||
:Type: :zeek:type:`event` (rec: :zeek:type:`Telemetry::Info`)
|
||||
|
||||
Event triggered for every record in the stream.
|
||||
|
||||
.. zeek:id:: Telemetry::log_telemetry_histogram
|
||||
:source-code: policy/frameworks/telemetry/log.zeek 90 90
|
||||
|
||||
:Type: :zeek:type:`event` (rec: :zeek:type:`Telemetry::HistogramInfo`)
|
||||
|
||||
Event triggered for every record in the histogram stream.
|
||||
|
||||
Hooks
|
||||
#####
|
||||
.. zeek:id:: Telemetry::log_policy
|
||||
:source-code: policy/frameworks/telemetry/log.zeek 81 81
|
||||
|
||||
:Type: :zeek:type:`Log::PolicyHook`
|
||||
|
||||
A default logging policy hook for the stream.
|
||||
|
||||
.. zeek:id:: Telemetry::log_policy_histogram
|
||||
:source-code: policy/frameworks/telemetry/log.zeek 84 84
|
||||
|
||||
:Type: :zeek:type:`Log::PolicyHook`
|
||||
|
||||
A default logging policy hook for the histogram stream.
|
||||
|
||||
|
||||
Loading…
Add table
Add a link
Reference in a new issue