Mirror/zeek
1
0
Fork 0
mirror of https://github.com/zeek/zeek.git synced 2025-10-08 09:38:19 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 3

Copy docs into Zeek repo directly

Browse source 
This is based on commit 2731def9159247e6da8a3191783c89683363689c from the
zeek-docs repo.
This commit is contained in:
Tim Wojtulewicz 2025-09-15 15:52:18 -07:00
parent 83f1e74643
commit ded98cd373
1074 changed files with 169319 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

14
doc/scripts/base/frameworks/analyzer/__load__.zeek.rst Normal file
View file

@ -0,0 +1,14 @@
:tocdepth: 3
base/frameworks/analyzer/__load__.zeek
======================================
:Imports: :doc:`base/frameworks/analyzer/dpd.zeek </scripts/base/frameworks/analyzer/dpd.zeek>`, :doc:`base/frameworks/analyzer/logging.zeek </scripts/base/frameworks/analyzer/logging.zeek>`, :doc:`base/frameworks/analyzer/main.zeek </scripts/base/frameworks/analyzer/main.zeek>`
Summary
~~~~~~~
Detailed Interface
~~~~~~~~~~~~~~~~~~

85
doc/scripts/base/frameworks/analyzer/dpd.zeek.rst Normal file
View file

@ -0,0 +1,85 @@
:tocdepth: 3
base/frameworks/analyzer/dpd.zeek
=================================
.. zeek:namespace:: DPD
Disables analyzers if protocol violations occur, and adds service information
to connection log.
:Namespace: DPD
:Imports: :doc:`base/frameworks/analyzer/main.zeek </scripts/base/frameworks/analyzer/main.zeek>`
Summary
~~~~~~~
Runtime Options
###############
=========================================================================================== ===============================================================
:zeek:id:`DPD::ignore_violations`: :zeek:type:`set` :zeek:attr:`&redef` Analyzers which you don't want to remove on violations.
:zeek:id:`DPD::ignore_violations_after`: :zeek:type:`count` :zeek:attr:`&redef` Ignore violations which go this many bytes into the connection.
:zeek:id:`DPD::track_removed_services_in_connection`: :zeek:type:`bool` :zeek:attr:`&redef` Change behavior of service field in conn.log:
Failed services are no longer removed.
=========================================================================================== ===============================================================
Redefinitions
#############
============================================ ==================================================================================================================
:zeek:type:`connection`: :zeek:type:`record`
:New Fields: :zeek:type:`connection`
failed_analyzers: :zeek:type:`set` [:zeek:type:`string`] :zeek:attr:`&default` = ``{ }`` :zeek:attr:`&optional`
The set of prototol analyzers that were removed due to a protocol
violation after the same analyzer had previously been confirmed.
============================================ ==================================================================================================================
Detailed Interface
~~~~~~~~~~~~~~~~~~
Runtime Options
###############
.. zeek:id:: DPD::ignore_violations
:source-code: base/frameworks/analyzer/dpd.zeek 10 10
:Type: :zeek:type:`set` [:zeek:type:`Analyzer::Tag`]
:Attributes: :zeek:attr:`&redef`
:Default: ``{}``
:Redefinition: from :doc:`/scripts/base/protocols/dce-rpc/main.zeek`
``+=``::
Analyzer::ANALYZER_DCE_RPC
:Redefinition: from :doc:`/scripts/base/protocols/ntlm/main.zeek`
``+=``::
Analyzer::ANALYZER_NTLM
Analyzers which you don't want to remove on violations.
.. zeek:id:: DPD::ignore_violations_after
:source-code: base/frameworks/analyzer/dpd.zeek 14 14
:Type: :zeek:type:`count`
:Attributes: :zeek:attr:`&redef`
:Default: ``10240``
Ignore violations which go this many bytes into the connection.
Set to 0 to never ignore protocol violations.
.. zeek:id:: DPD::track_removed_services_in_connection
:source-code: base/frameworks/analyzer/dpd.zeek 21 21
:Type: :zeek:type:`bool`
:Attributes: :zeek:attr:`&redef`
:Default: ``F``
Change behavior of service field in conn.log:
Failed services are no longer removed. Instead, for a failed
service, a second entry with a "-" in front of it is added.
E.g. a http connection with a violation would be logged as
"http,-http".

42
doc/scripts/base/frameworks/analyzer/index.rst Normal file
View file

@ -0,0 +1,42 @@
:orphan:
Package: base/frameworks/analyzer
=================================
The analyzer framework allows to dynamically enable or disable Zeek's
protocol analyzers, as well as to manage the well-known ports which
automatically activate a particular analyzer for new connections.
:doc:`/scripts/base/frameworks/analyzer/main.zeek`
Framework for managing Zeek's protocol analyzers.
The analyzer framework allows to dynamically enable or disable analyzers, as
well as to manage the well-known ports which automatically activate a
particular analyzer for new connections.
Protocol analyzers are identified by unique tags of type
:zeek:type:`Analyzer::Tag`, such as :zeek:enum:`Analyzer::ANALYZER_HTTP`.
These tags are defined internally by
the analyzers themselves, and documented in their analyzer-specific
description along with the events that they generate.
Analyzer tags are also inserted into a global :zeek:type:`AllAnalyzers::Tag` enum
type. This type contains duplicates of all of the :zeek:type:`Analyzer::Tag`,
:zeek:type:`PacketAnalyzer::Tag` and :zeek:type:`Files::Tag` enum values
and can be used for arguments to function/hook/event definitions where they
need to handle any analyzer type. See :zeek:id:`Analyzer::register_for_ports`
for an example.
:doc:`/scripts/base/frameworks/analyzer/__load__.zeek`
:doc:`/scripts/base/frameworks/analyzer/dpd.zeek`
Disables analyzers if protocol violations occur, and adds service information
to connection log.
:doc:`/scripts/base/frameworks/analyzer/logging.zeek`
Logging analyzer violations into analyzer.log

148
doc/scripts/base/frameworks/analyzer/logging.zeek.rst Normal file
View file

@ -0,0 +1,148 @@
:tocdepth: 3
base/frameworks/analyzer/logging.zeek
=====================================
.. zeek:namespace:: Analyzer::Logging
Logging analyzer violations into analyzer.log
:Namespace: Analyzer::Logging
:Imports: :doc:`base/frameworks/analyzer/main.zeek </scripts/base/frameworks/analyzer/main.zeek>`, :doc:`base/frameworks/logging </scripts/base/frameworks/logging/index>`
Summary
~~~~~~~
Runtime Options
###############
=========================================================================================== ==============================================================
:zeek:id:`Analyzer::Logging::failure_data_max_size`: :zeek:type:`count` :zeek:attr:`&redef` If a violation contains information about the data causing it,
include at most this many bytes of it in the log.
=========================================================================================== ==============================================================
Types
#####
========================================================= ===========================================================================
:zeek:type:`Analyzer::Logging::Info`: :zeek:type:`record` The record type defining the columns to log in the analyzer logging stream.
========================================================= ===========================================================================
Redefinitions
#############
======================================= ===========================================
:zeek:type:`Log::ID`: :zeek:type:`enum` Add the analyzer logging stream identifier.
* :zeek:enum:`Analyzer::Logging::LOG`
======================================= ===========================================
Events
######
============================================================== ===============================================================================
:zeek:id:`Analyzer::Logging::log_analyzer`: :zeek:type:`event` An event that can be handled to access the :zeek:type:`Analyzer::Logging::Info`
record as it is sent on to the logging framework.
============================================================== ===============================================================================
Hooks
#####
====================================================================== =============================================
:zeek:id:`Analyzer::Logging::log_policy`: :zeek:type:`Log::PolicyHook` A default logging policy hook for the stream.
====================================================================== =============================================
Detailed Interface
~~~~~~~~~~~~~~~~~~
Runtime Options
###############
.. zeek:id:: Analyzer::Logging::failure_data_max_size
:source-code: base/frameworks/analyzer/logging.zeek 39 39
:Type: :zeek:type:`count`
:Attributes: :zeek:attr:`&redef`
:Default: ``40``
If a violation contains information about the data causing it,
include at most this many bytes of it in the log.
Types
#####
.. zeek:type:: Analyzer::Logging::Info
:source-code: base/frameworks/analyzer/logging.zeek 13 35
:Type: :zeek:type:`record`
.. zeek:field:: ts :zeek:type:`time` :zeek:attr:`&log`
Timestamp of the violation.
.. zeek:field:: analyzer_kind :zeek:type:`string` :zeek:attr:`&log`
The kind of analyzer involved. Currently "packet", "file"
or "protocol".
.. zeek:field:: analyzer_name :zeek:type:`string` :zeek:attr:`&log`
The name of the analyzer as produced by :zeek:see:`Analyzer::name`
for the analyzer's tag.
.. zeek:field:: uid :zeek:type:`string` :zeek:attr:`&log` :zeek:attr:`&optional`
Connection UID if available.
.. zeek:field:: fuid :zeek:type:`string` :zeek:attr:`&log` :zeek:attr:`&optional`
File UID if available.
.. zeek:field:: id :zeek:type:`conn_id` :zeek:attr:`&log` :zeek:attr:`&optional`
Connection identifier if available.
.. zeek:field:: proto :zeek:type:`transport_proto` :zeek:attr:`&log` :zeek:attr:`&optional`
Transport protocol for the violation, if available.
.. zeek:field:: failure_reason :zeek:type:`string` :zeek:attr:`&log`
Failure or violation reason, if available.
.. zeek:field:: failure_data :zeek:type:`string` :zeek:attr:`&log` :zeek:attr:`&optional`
Data causing failure or violation if available. Truncated
to :zeek:see:`Analyzer::Logging::failure_data_max_size`.
.. zeek:field:: packet_segment :zeek:type:`string` :zeek:attr:`&optional` :zeek:attr:`&log`
(present if :doc:`/scripts/policy/frameworks/analyzer/packet-segment-logging.zeek` is loaded)
A chunk of the payload that most likely resulted in the
analyzer violation.
The record type defining the columns to log in the analyzer logging stream.
Events
######
.. zeek:id:: Analyzer::Logging::log_analyzer
:source-code: base/frameworks/analyzer/logging.zeek 43 43
:Type: :zeek:type:`event` (rec: :zeek:type:`Analyzer::Logging::Info`)
An event that can be handled to access the :zeek:type:`Analyzer::Logging::Info`
record as it is sent on to the logging framework.
Hooks
#####
.. zeek:id:: Analyzer::Logging::log_policy
:source-code: policy/frameworks/analyzer/packet-segment-logging.zeek 38 50
:Type: :zeek:type:`Log::PolicyHook`
A default logging policy hook for the stream.

364
doc/scripts/base/frameworks/analyzer/main.zeek.rst Normal file
View file

@ -0,0 +1,364 @@
:tocdepth: 3
base/frameworks/analyzer/main.zeek
==================================
.. zeek:namespace:: Analyzer
Framework for managing Zeek's protocol analyzers.
The analyzer framework allows to dynamically enable or disable analyzers, as
well as to manage the well-known ports which automatically activate a
particular analyzer for new connections.
Protocol analyzers are identified by unique tags of type
:zeek:type:`Analyzer::Tag`, such as :zeek:enum:`Analyzer::ANALYZER_HTTP`.
These tags are defined internally by
the analyzers themselves, and documented in their analyzer-specific
description along with the events that they generate.
Analyzer tags are also inserted into a global :zeek:type:`AllAnalyzers::Tag` enum
type. This type contains duplicates of all of the :zeek:type:`Analyzer::Tag`,
:zeek:type:`PacketAnalyzer::Tag` and :zeek:type:`Files::Tag` enum values
and can be used for arguments to function/hook/event definitions where they
need to handle any analyzer type. See :zeek:id:`Analyzer::register_for_ports`
for an example.
:Namespace: Analyzer
:Imports: :doc:`base/bif/analyzer.bif.zeek </scripts/base/bif/analyzer.bif.zeek>`, :doc:`base/bif/file_analysis.bif.zeek </scripts/base/bif/file_analysis.bif.zeek>`, :doc:`base/bif/packet_analysis.bif.zeek </scripts/base/bif/packet_analysis.bif.zeek>`, :doc:`base/frameworks/packet-filter/utils.zeek </scripts/base/frameworks/packet-filter/utils.zeek>`
Summary
~~~~~~~
State Variables
###############
============================================================================== ===================================================================
:zeek:id:`Analyzer::disable_all`: :zeek:type:`bool` :zeek:attr:`&redef` If true, all available analyzers are initially disabled at startup.
:zeek:id:`Analyzer::disabled_analyzers`: :zeek:type:`set` :zeek:attr:`&redef` A set of analyzers to disable by default at startup.
:zeek:id:`Analyzer::ports`: :zeek:type:`table` A table of ports mapped to analyzers that handle those ports.
:zeek:id:`Analyzer::requested_analyzers`: :zeek:type:`set` :zeek:attr:`&redef` A set of protocol, packet or file analyzer tags requested to
be enabled during startup.
============================================================================== ===================================================================
Events
######
======================================================== ========================================================================
:zeek:id:`Analyzer::analyzer_failed`: :zeek:type:`event` Event that is raised when an analyzer raised a service violation and was
removed.
======================================================== ========================================================================
Functions
#########
================================================================ =======================================================================
:zeek:id:`Analyzer::all_registered_ports`: :zeek:type:`function` Returns a table of all ports-to-analyzer mappings currently registered.
:zeek:id:`Analyzer::analyzer_to_bpf`: :zeek:type:`function` Automatically creates a BPF filter for the specified protocol based
on the data supplied for the protocol through the
:zeek:see:`Analyzer::register_for_ports` function.
:zeek:id:`Analyzer::disable_analyzer`: :zeek:type:`function` Disables an analyzer.
:zeek:id:`Analyzer::enable_analyzer`: :zeek:type:`function` Enables an analyzer.
:zeek:id:`Analyzer::get_bpf`: :zeek:type:`function` Create a BPF filter which matches all of the ports defined
by the various protocol analysis scripts as "registered ports"
for the protocol.
:zeek:id:`Analyzer::get_tag`: :zeek:type:`function` Translates an analyzer's name to a tag enum value.
:zeek:id:`Analyzer::has_tag`: :zeek:type:`function` Check whether the given analyzer name exists.
:zeek:id:`Analyzer::kind`: :zeek:type:`function` Translates an analyzer type to a string with the analyzer's type.
:zeek:id:`Analyzer::name`: :zeek:type:`function` Translates an analyzer type to a string with the analyzer's name.
:zeek:id:`Analyzer::register_for_port`: :zeek:type:`function` Registers an individual well-known port for an analyzer.
:zeek:id:`Analyzer::register_for_ports`: :zeek:type:`function` Registers a set of well-known ports for an analyzer.
:zeek:id:`Analyzer::registered_ports`: :zeek:type:`function` Returns a set of all well-known ports currently registered for a
specific analyzer.
:zeek:id:`Analyzer::schedule_analyzer`: :zeek:type:`function` Schedules an analyzer for a future connection originating from a
given IP address and port.
================================================================ =======================================================================
Detailed Interface
~~~~~~~~~~~~~~~~~~
State Variables
###############
.. zeek:id:: Analyzer::disable_all
:source-code: base/frameworks/analyzer/main.zeek 28 28
:Type: :zeek:type:`bool`
:Attributes: :zeek:attr:`&redef`
:Default: ``F``
If true, all available analyzers are initially disabled at startup.
One can then selectively enable them with
:zeek:id:`Analyzer::enable_analyzer`.
.. zeek:id:: Analyzer::disabled_analyzers
:source-code: base/frameworks/analyzer/main.zeek 156 156
:Type: :zeek:type:`set` [:zeek:type:`AllAnalyzers::Tag`]
:Attributes: :zeek:attr:`&redef`
:Default:
::
{
AllAnalyzers::ANALYZER_ANALYZER_TCPSTATS
}
A set of analyzers to disable by default at startup. The default set
contains legacy analyzers that are no longer supported.
.. zeek:id:: Analyzer::ports
:source-code: base/frameworks/analyzer/main.zeek 164 164
:Type: :zeek:type:`table` [:zeek:type:`AllAnalyzers::Tag`] of :zeek:type:`set` [:zeek:type:`port`]
:Default: ``{}``
A table of ports mapped to analyzers that handle those ports. This is
used by BPF filtering and DPD. Session analyzers can add to this using
Analyzer::register_for_port(s) and packet analyzers can add to this
using PacketAnalyzer::register_for_port(s).
.. zeek:id:: Analyzer::requested_analyzers
:source-code: base/frameworks/analyzer/main.zeek 174 174
:Type: :zeek:type:`set` [:zeek:type:`AllAnalyzers::Tag`]
:Attributes: :zeek:attr:`&redef`
:Default: ``{}``
A set of protocol, packet or file analyzer tags requested to
be enabled during startup.
By default, all analyzers in Zeek are enabled. When all analyzers
are disabled through :zeek:see:`Analyzer::disable_all`, this set
set allows to record analyzers to be enabled during Zeek startup.
This set can be added to via :zeek:see:`redef`.
Events
######
.. zeek:id:: Analyzer::analyzer_failed
:source-code: base/frameworks/analyzer/main.zeek 191 191
:Type: :zeek:type:`event` (ts: :zeek:type:`time`, atype: :zeek:type:`AllAnalyzers::Tag`, info: :zeek:type:`AnalyzerViolationInfo`)
Event that is raised when an analyzer raised a service violation and was
removed.
The event is also raised if the analyzer already was no longer active by
the time that the violation was handled - so if it happens at the very
end of a connection.
Currently this event is only raised for protocol analyzers, as packet
and file analyzers are never actively removed/disabled.
:param ts: time at which the violation occurred
:param atype: atype: The analyzer tag, such as ``Analyzer::ANALYZER_HTTP``.
:param info: Details about the violation. This record should include a :zeek:type:`connection`
Functions
#########
.. zeek:id:: Analyzer::all_registered_ports
:source-code: base/frameworks/analyzer/main.zeek 265 268
:Type: :zeek:type:`function` () : :zeek:type:`table` [:zeek:type:`AllAnalyzers::Tag`] of :zeek:type:`set` [:zeek:type:`port`]
Returns a table of all ports-to-analyzer mappings currently registered.
:returns: A table mapping each analyzer to the set of ports
registered for it.
.. zeek:id:: Analyzer::analyzer_to_bpf
:source-code: base/frameworks/analyzer/main.zeek 304 314
:Type: :zeek:type:`function` (tag: :zeek:type:`Analyzer::Tag`) : :zeek:type:`string`
Automatically creates a BPF filter for the specified protocol based
on the data supplied for the protocol through the
:zeek:see:`Analyzer::register_for_ports` function.
:param tag: The analyzer tag.
:returns: BPF filter string.
.. zeek:id:: Analyzer::disable_analyzer
:source-code: base/frameworks/analyzer/main.zeek 224 233
:Type: :zeek:type:`function` (tag: :zeek:type:`AllAnalyzers::Tag`) : :zeek:type:`bool`
Disables an analyzer. Once disabled, the analyzer will not be used
further for analysis of future connections.
:param tag: The tag of the analyzer to disable.
:returns: True if the analyzer was successfully disabled.
.. zeek:id:: Analyzer::enable_analyzer
:source-code: base/frameworks/analyzer/main.zeek 213 222
:Type: :zeek:type:`function` (tag: :zeek:type:`AllAnalyzers::Tag`) : :zeek:type:`bool`
Enables an analyzer. Once enabled, the analyzer may be used for analysis
of future connections as decided by Zeek's dynamic protocol detection.
:param tag: The tag of the analyzer to enable.
:returns: True if the analyzer was successfully enabled.
.. zeek:id:: Analyzer::get_bpf
:source-code: base/frameworks/analyzer/main.zeek 316 324
:Type: :zeek:type:`function` () : :zeek:type:`string`
Create a BPF filter which matches all of the ports defined
by the various protocol analysis scripts as "registered ports"
for the protocol.
.. zeek:id:: Analyzer::get_tag
:source-code: base/frameworks/analyzer/main.zeek 293 296
:Type: :zeek:type:`function` (name: :zeek:type:`string`) : :zeek:type:`AllAnalyzers::Tag`
Translates an analyzer's name to a tag enum value.
The analyzer is assumed to exist; call
:zeek:see:`Analyzer::has_tag` first to verify that name is a
valid analyzer name.
:param name: The analyzer name.
:returns: The analyzer tag corresponding to the name.
.. zeek:id:: Analyzer::has_tag
:source-code: base/frameworks/analyzer/main.zeek 288 291
:Type: :zeek:type:`function` (name: :zeek:type:`string`) : :zeek:type:`bool`
Check whether the given analyzer name exists.
This can be used before calling :zeek:see:`Analyzer::get_tag` to
verify that the given name as string is a valid analyzer name.
:param name: The analyzer name.
:returns: True if the given name is a valid analyzer, else false.
.. zeek:id:: Analyzer::kind
:source-code: base/frameworks/analyzer/main.zeek 275 286
:Type: :zeek:type:`function` (atype: :zeek:type:`AllAnalyzers::Tag`) : :zeek:type:`string`
Translates an analyzer type to a string with the analyzer's type.
Possible values are "protocol", "packet", "file", or "unknown".
:param tag: The analyzer tag.
:returns: The analyzer kind corresponding to the tag.
.. zeek:id:: Analyzer::name
:source-code: base/frameworks/analyzer/main.zeek 270 273
:Type: :zeek:type:`function` (atype: :zeek:type:`AllAnalyzers::Tag`) : :zeek:type:`string`
Translates an analyzer type to a string with the analyzer's name.
:param tag: The analyzer tag.
:returns: The analyzer name corresponding to the tag.
.. zeek:id:: Analyzer::register_for_port
:source-code: base/frameworks/analyzer/main.zeek 248 258
:Type: :zeek:type:`function` (tag: :zeek:type:`Analyzer::Tag`, p: :zeek:type:`port`) : :zeek:type:`bool`
Registers an individual well-known port for an analyzer. If a future
connection on this port is seen, the analyzer will be automatically
assigned to parsing it. The function *adds* to all ports already
registered, it doesn't replace them.
:param tag: The tag of the analyzer.
:param p: The well-known port to associate with the analyzer.
:returns: True if the port was successfully registered.
.. zeek:id:: Analyzer::register_for_ports
:source-code: base/frameworks/analyzer/main.zeek 235 246
:Type: :zeek:type:`function` (tag: :zeek:type:`Analyzer::Tag`, ports: :zeek:type:`set` [:zeek:type:`port`]) : :zeek:type:`bool`
Registers a set of well-known ports for an analyzer. If a future
connection on one of these ports is seen, the analyzer will be
automatically assigned to parsing it. The function *adds* to all ports
already registered, it doesn't replace them.
:param tag: The tag of the analyzer.
:param ports: The set of well-known ports to associate with the analyzer.
:returns: True if the ports were successfully registered.
.. zeek:id:: Analyzer::registered_ports
:source-code: base/frameworks/analyzer/main.zeek 260 263
:Type: :zeek:type:`function` (tag: :zeek:type:`AllAnalyzers::Tag`) : :zeek:type:`set` [:zeek:type:`port`]
Returns a set of all well-known ports currently registered for a
specific analyzer.
:param tag: The tag of the analyzer.
:returns: The set of ports.
.. zeek:id:: Analyzer::schedule_analyzer
:source-code: base/frameworks/analyzer/main.zeek 299 302
:Type: :zeek:type:`function` (orig: :zeek:type:`addr`, resp: :zeek:type:`addr`, resp_p: :zeek:type:`port`, analyzer: :zeek:type:`Analyzer::Tag`, tout: :zeek:type:`interval`) : :zeek:type:`bool`
Schedules an analyzer for a future connection originating from a
given IP address and port.
:param orig: The IP address originating a connection in the future.
0.0.0.0 can be used as a wildcard to match any originator address.
:param resp: The IP address responding to a connection from *orig*.
:param resp_p: The destination port at *resp*.
:param analyzer: The analyzer ID.
:param tout: A timeout interval after which the scheduling request will be
discarded if the connection has not yet been seen.
:returns: True if successful.

14
doc/scripts/base/frameworks/broker/__load__.zeek.rst Normal file
View file

@ -0,0 +1,14 @@
:tocdepth: 3
base/frameworks/broker/__load__.zeek
====================================
:Imports: :doc:`base/frameworks/broker/backpressure.zeek </scripts/base/frameworks/broker/backpressure.zeek>`, :doc:`base/frameworks/broker/log.zeek </scripts/base/frameworks/broker/log.zeek>`, :doc:`base/frameworks/broker/main.zeek </scripts/base/frameworks/broker/main.zeek>`, :doc:`base/frameworks/broker/store.zeek </scripts/base/frameworks/broker/store.zeek>`
Summary
~~~~~~~
Detailed Interface
~~~~~~~~~~~~~~~~~~

24
doc/scripts/base/frameworks/broker/backpressure.zeek.rst Normal file
View file

@ -0,0 +1,24 @@
:tocdepth: 3
base/frameworks/broker/backpressure.zeek
========================================
This handles Broker peers that fall so far behind in handling messages that
this node sends it that the local Broker endpoint decides to unpeer them.
Zeek captures this as follows:
- In broker.log, with a regular "peer-removed" entry indicating CAF's reason.
- Via eventing through :zeek:see:`Broker::peer_removed` as done in this script.
The cluster framework additionally captures the unpeering as follows:
- In cluster.log, with a higher-level message indicating the node names involved.
- Via telemetry, using a labeled counter.
Summary
~~~~~~~
Detailed Interface
~~~~~~~~~~~~~~~~~~

36
doc/scripts/base/frameworks/broker/index.rst Normal file
View file

@ -0,0 +1,36 @@
:orphan:
Package: base/frameworks/broker
===============================
The Broker communication framework facilitates connecting to remote Zeek
instances to share state and transfer events.
:doc:`/scripts/base/frameworks/broker/__load__.zeek`
:doc:`/scripts/base/frameworks/broker/main.zeek`
The Broker-based communication API and its various options.
:doc:`/scripts/base/frameworks/broker/store.zeek`
The Broker-based data store API and its various options.
:doc:`/scripts/base/frameworks/broker/log.zeek`
:doc:`/scripts/base/frameworks/broker/backpressure.zeek`
This handles Broker peers that fall so far behind in handling messages that
this node sends it that the local Broker endpoint decides to unpeer them.
Zeek captures this as follows:
- In broker.log, with a regular "peer-removed" entry indicating CAF's reason.
- Via eventing through :zeek:see:`Broker::peer_removed` as done in this script.
The cluster framework additionally captures the unpeering as follows:
- In cluster.log, with a higher-level message indicating the node names involved.
- Via telemetry, using a labeled counter.

121
doc/scripts/base/frameworks/broker/log.zeek.rst Normal file
View file

@ -0,0 +1,121 @@
:tocdepth: 3
base/frameworks/broker/log.zeek
===============================
.. zeek:namespace:: Broker
:Namespace: Broker
:Imports: :doc:`base/frameworks/broker/main.zeek </scripts/base/frameworks/broker/main.zeek>`
Summary
~~~~~~~
Types
#####
============================================== =============================================================
:zeek:type:`Broker::Info`: :zeek:type:`record` A record type containing the column fields of the Broker log.
:zeek:type:`Broker::Type`: :zeek:type:`enum` The type of a Broker activity being logged.
============================================== =============================================================
Redefinitions
#############
======================================= =====================================
:zeek:type:`Log::ID`: :zeek:type:`enum` The Broker logging stream identifier.
* :zeek:enum:`Broker::LOG`
======================================= =====================================
Hooks
#####
=========================================================== =============================================
:zeek:id:`Broker::log_policy`: :zeek:type:`Log::PolicyHook` A default logging policy hook for the stream.
=========================================================== =============================================
Detailed Interface
~~~~~~~~~~~~~~~~~~
Types
#####
.. zeek:type:: Broker::Info
:source-code: base/frameworks/broker/log.zeek 33 45
:Type: :zeek:type:`record`
.. zeek:field:: ts :zeek:type:`time` :zeek:attr:`&log`
The network time at which a Broker event occurred.
.. zeek:field:: ty :zeek:type:`Broker::Type` :zeek:attr:`&log`
The type of the Broker event.
.. zeek:field:: ev :zeek:type:`string` :zeek:attr:`&log`
The event being logged.
.. zeek:field:: peer :zeek:type:`Broker::NetworkInfo` :zeek:attr:`&log` :zeek:attr:`&optional`
The peer (if any) with which a Broker event is
concerned.
.. zeek:field:: message :zeek:type:`string` :zeek:attr:`&log` :zeek:attr:`&optional`
An optional message describing the Broker event in more detail
A record type containing the column fields of the Broker log.
.. zeek:type:: Broker::Type
:source-code: base/frameworks/broker/log.zeek 13 31
:Type: :zeek:type:`enum`
.. zeek:enum:: Broker::STATUS Broker::Type
An informational status update.
.. zeek:enum:: Broker::ERROR Broker::Type
An error situation.
.. zeek:enum:: Broker::CRITICAL_EVENT Broker::Type
Fatal event, normal operation has most likely broken down.
.. zeek:enum:: Broker::ERROR_EVENT Broker::Type
Unrecoverable event that imparts at least part of the system.
.. zeek:enum:: Broker::WARNING_EVENT Broker::Type
Unexpected or conspicuous event that may still be recoverable.
.. zeek:enum:: Broker::INFO_EVENT Broker::Type
Noteworthy event during normal operation.
.. zeek:enum:: Broker::VERBOSE_EVENT Broker::Type
Information that might be relevant for a user to understand system behavior.
.. zeek:enum:: Broker::DEBUG_EVENT Broker::Type
An event that is relevant only for troubleshooting and debugging.
The type of a Broker activity being logged.
Hooks
#####
.. zeek:id:: Broker::log_policy
:source-code: base/frameworks/broker/log.zeek 10 10
:Type: :zeek:type:`Log::PolicyHook`
A default logging policy hook for the stream.

995
doc/scripts/base/frameworks/broker/main.zeek.rst Normal file
View file

@ -0,0 +1,995 @@
:tocdepth: 3
base/frameworks/broker/main.zeek
================================
.. zeek:namespace:: Broker
The Broker-based communication API and its various options.
:Namespace: Broker
:Imports: :doc:`base/bif/comm.bif.zeek </scripts/base/bif/comm.bif.zeek>`, :doc:`base/bif/messaging.bif.zeek </scripts/base/bif/messaging.bif.zeek>`
Summary
~~~~~~~
Runtime Options
###############
================================================================================= =================================================================
:zeek:id:`Broker::peer_counts_as_iosource`: :zeek:type:`bool` :zeek:attr:`&redef` Whether calling :zeek:see:`Broker::peer` will register the Broker
system as an I/O source that will block the process from shutting
down.
================================================================================= =================================================================
Redefinable Options
###################
======================================================================================================= ===========================================================================
:zeek:id:`Broker::aggressive_interval`: :zeek:type:`count` :zeek:attr:`&redef` Frequency of work-stealing polling attempts for Broker/CAF threads
in "aggressive" mode.
:zeek:id:`Broker::aggressive_polls`: :zeek:type:`count` :zeek:attr:`&redef` Number of work-stealing polling attempts for Broker/CAF threads
in "aggressive" mode.
:zeek:id:`Broker::buffer_stats_reset_interval`: :zeek:type:`interval` :zeek:attr:`&redef` How frequently Zeek resets some peering/client buffer statistics,
such as ``max_queued_recently`` in :zeek:see:`BrokerPeeringStats`.
:zeek:id:`Broker::default_connect_retry`: :zeek:type:`interval` :zeek:attr:`&redef` Default interval to retry connecting to a peer if it cannot be made to
work initially, or if it ever becomes disconnected.
:zeek:id:`Broker::default_listen_address`: :zeek:type:`string` :zeek:attr:`&redef` Default address on which to listen.
:zeek:id:`Broker::default_listen_address_websocket`: :zeek:type:`string` :zeek:attr:`&redef` Default address on which to listen for WebSocket connections.
:zeek:id:`Broker::default_listen_retry`: :zeek:type:`interval` :zeek:attr:`&redef` Default interval to retry listening on a port if it's currently in
use already.
:zeek:id:`Broker::default_log_topic_prefix`: :zeek:type:`string` :zeek:attr:`&redef` The default topic prefix where logs will be published.
:zeek:id:`Broker::default_port`: :zeek:type:`port` :zeek:attr:`&redef` Default port for native Broker communication.
:zeek:id:`Broker::default_port_websocket`: :zeek:type:`port` :zeek:attr:`&redef` Default port for Broker WebSocket communication.
:zeek:id:`Broker::disable_ssl`: :zeek:type:`bool` :zeek:attr:`&redef` If true, do not use SSL for network connections.
:zeek:id:`Broker::forward_messages`: :zeek:type:`bool` :zeek:attr:`&redef` Forward all received messages to subscribing peers.
:zeek:id:`Broker::log_batch_interval`: :zeek:type:`interval` :zeek:attr:`&redef` Max time to buffer log messages before sending the current set out as a
batch.
:zeek:id:`Broker::log_batch_size`: :zeek:type:`count` :zeek:attr:`&redef` The max number of log entries per log stream to batch together when
sending log messages to a remote logger.
:zeek:id:`Broker::log_severity_level`: :zeek:type:`Broker::LogSeverityLevel` :zeek:attr:`&redef` The log event severity level for the Broker log output.
:zeek:id:`Broker::log_stderr_severity_level`: :zeek:type:`Broker::LogSeverityLevel` :zeek:attr:`&redef` Event severity level for also printing the Broker log output to stderr.
:zeek:id:`Broker::max_threads`: :zeek:type:`count` :zeek:attr:`&redef` Max number of threads to use for Broker/CAF functionality.
:zeek:id:`Broker::moderate_interval`: :zeek:type:`count` :zeek:attr:`&redef` Frequency of work-stealing polling attempts for Broker/CAF threads
in "moderate" mode.
:zeek:id:`Broker::moderate_polls`: :zeek:type:`count` :zeek:attr:`&redef` Number of work-stealing polling attempts for Broker/CAF threads
in "moderate" mode.
:zeek:id:`Broker::moderate_sleep`: :zeek:type:`interval` :zeek:attr:`&redef` Interval of time for under-utilized Broker/CAF threads to sleep
when in "moderate" mode.
:zeek:id:`Broker::peer_buffer_size`: :zeek:type:`count` :zeek:attr:`&redef` Max number of items we buffer at most per peer.
:zeek:id:`Broker::peer_overflow_policy`: :zeek:type:`string` :zeek:attr:`&redef` Configures how Broker responds to peers that cannot keep up with the
incoming message rate.
:zeek:id:`Broker::relaxed_interval`: :zeek:type:`count` :zeek:attr:`&redef` Frequency of work-stealing polling attempts for Broker/CAF threads
in "relaxed" mode.
:zeek:id:`Broker::relaxed_sleep`: :zeek:type:`interval` :zeek:attr:`&redef` Interval of time for under-utilized Broker/CAF threads to sleep
when in "relaxed" mode.
:zeek:id:`Broker::scheduler_policy`: :zeek:type:`string` :zeek:attr:`&redef` The CAF scheduling policy to use.
:zeek:id:`Broker::ssl_cafile`: :zeek:type:`string` :zeek:attr:`&redef` Path to a file containing concatenated trusted certificates
in PEM format.
:zeek:id:`Broker::ssl_capath`: :zeek:type:`string` :zeek:attr:`&redef` Path to an OpenSSL-style directory of trusted certificates.
:zeek:id:`Broker::ssl_certificate`: :zeek:type:`string` :zeek:attr:`&redef` Path to a file containing a X.509 certificate for this
node in PEM format.
:zeek:id:`Broker::ssl_keyfile`: :zeek:type:`string` :zeek:attr:`&redef` Path to the file containing the private key for this node's
certificate.
:zeek:id:`Broker::ssl_passphrase`: :zeek:type:`string` :zeek:attr:`&redef` Passphrase to decrypt the private key specified by
:zeek:see:`Broker::ssl_keyfile`.
:zeek:id:`Broker::web_socket_buffer_size`: :zeek:type:`count` :zeek:attr:`&redef` Same as :zeek:see:`Broker::peer_buffer_size` but for WebSocket clients.
:zeek:id:`Broker::web_socket_overflow_policy`: :zeek:type:`string` :zeek:attr:`&redef` Same as :zeek:see:`Broker::peer_overflow_policy` but for WebSocket clients.
======================================================================================================= ===========================================================================
Types
#####
======================================================== ====================================================================
:zeek:type:`Broker::Data`: :zeek:type:`record` Opaque communication data.
:zeek:type:`Broker::DataVector`: :zeek:type:`vector` Opaque communication data sequence.
:zeek:type:`Broker::EndpointInfo`: :zeek:type:`record`
:zeek:type:`Broker::ErrorCode`: :zeek:type:`enum` Enumerates the possible error types.
:zeek:type:`Broker::Event`: :zeek:type:`record` Opaque event communication data.
:zeek:type:`Broker::LogSeverityLevel`: :zeek:type:`enum` The possible log event severity levels for Broker.
:zeek:type:`Broker::NetworkInfo`: :zeek:type:`record`
:zeek:type:`Broker::PeerInfo`: :zeek:type:`record`
:zeek:type:`Broker::PeerInfos`: :zeek:type:`vector`
:zeek:type:`Broker::PeerStatus`: :zeek:type:`enum` The possible states of a peer endpoint.
:zeek:type:`Broker::TableItem`: :zeek:type:`record` Opaque communication data used as a convenient way to wrap key-value
pairs that comprise table entries.
======================================================== ====================================================================
Functions
#########
======================================================================= =======================================================================
:zeek:id:`Broker::default_log_topic`: :zeek:type:`function` The default implementation for :zeek:see:`Broker::log_topic`.
:zeek:id:`Broker::flush_logs`: :zeek:type:`function` Sends all pending log messages to remote peers.
:zeek:id:`Broker::forward`: :zeek:type:`function` Register a topic prefix subscription for events that should only be
forwarded to any subscribing peers and not raise any event handlers
on the receiving/forwarding node.
:zeek:id:`Broker::is_outbound_peering`: :zeek:type:`function` Whether the local node originally initiated the peering with the
given endpoint.
:zeek:id:`Broker::listen`: :zeek:type:`function` Listen for remote connections using the native Broker protocol.
:zeek:id:`Broker::log_topic`: :zeek:type:`function` :zeek:attr:`&redef` A function that will be called for each log entry to determine what
broker topic string will be used for sending it to peers.
:zeek:id:`Broker::node_id`: :zeek:type:`function` Get a unique identifier for the local broker endpoint.
:zeek:id:`Broker::peer`: :zeek:type:`function` Initiate a remote connection.
:zeek:id:`Broker::peering_stats`: :zeek:type:`function` Obtain each peering's send-buffer statistics.
:zeek:id:`Broker::peers`: :zeek:type:`function` Get a list of all peer connections.
:zeek:id:`Broker::publish_id`: :zeek:type:`function` Publishes the value of an identifier to a given topic.
:zeek:id:`Broker::subscribe`: :zeek:type:`function` Register interest in all peer event messages that use a certain topic
prefix.
:zeek:id:`Broker::unpeer`: :zeek:type:`function` Remove a remote connection.
:zeek:id:`Broker::unsubscribe`: :zeek:type:`function` Unregister interest in all peer event messages that use a topic prefix.
======================================================================= =======================================================================
Detailed Interface
~~~~~~~~~~~~~~~~~~
Runtime Options
###############
.. zeek:id:: Broker::peer_counts_as_iosource
:source-code: base/frameworks/broker/main.zeek 153 153
:Type: :zeek:type:`bool`
:Attributes: :zeek:attr:`&redef`
:Default: ``T``
Whether calling :zeek:see:`Broker::peer` will register the Broker
system as an I/O source that will block the process from shutting
down. For example, set this to false when you are reading pcaps,
but also want to initiate a Broker peering and still shutdown after
done reading the pcap.
Redefinable Options
###################
.. zeek:id:: Broker::aggressive_interval
:source-code: base/frameworks/broker/main.zeek 135 135
:Type: :zeek:type:`count`
:Attributes: :zeek:attr:`&redef`
:Default: ``4``
Frequency of work-stealing polling attempts for Broker/CAF threads
in "aggressive" mode. Only used for the "stealing" scheduler policy.
.. zeek:id:: Broker::aggressive_polls
:source-code: base/frameworks/broker/main.zeek 127 127
:Type: :zeek:type:`count`
:Attributes: :zeek:attr:`&redef`
:Default: ``5``
Number of work-stealing polling attempts for Broker/CAF threads
in "aggressive" mode. Only used for the "stealing" scheduler policy.
.. zeek:id:: Broker::buffer_stats_reset_interval
:source-code: base/frameworks/broker/main.zeek 104 104
:Type: :zeek:type:`interval`
:Attributes: :zeek:attr:`&redef`
:Default: ``1.0 min``
How frequently Zeek resets some peering/client buffer statistics,
such as ``max_queued_recently`` in :zeek:see:`BrokerPeeringStats`.
.. zeek:id:: Broker::default_connect_retry
:source-code: base/frameworks/broker/main.zeek 39 39
:Type: :zeek:type:`interval`
:Attributes: :zeek:attr:`&redef`
:Default: ``1.0 sec``
Default interval to retry connecting to a peer if it cannot be made to
work initially, or if it ever becomes disconnected. Use of the
ZEEK_DEFAULT_CONNECT_RETRY environment variable (set as number of
seconds) will override this option and also any values given to
:zeek:see:`Broker::peer`.
.. zeek:id:: Broker::default_listen_address
:source-code: base/frameworks/broker/main.zeek 27 27
:Type: :zeek:type:`string`
:Attributes: :zeek:attr:`&redef`
:Default: ``""``
:Redefinition: from :doc:`/scripts/policy/frameworks/management/agent/boot.zeek`
``=``::
127.0.0.1
Default address on which to listen.
.. zeek:see:: Broker::listen
.. zeek:id:: Broker::default_listen_address_websocket
:source-code: base/frameworks/broker/main.zeek 32 32
:Type: :zeek:type:`string`
:Attributes: :zeek:attr:`&redef`
:Default: ``""``
Default address on which to listen for WebSocket connections.
.. zeek:see:: Cluster::listen_websocket
.. zeek:id:: Broker::default_listen_retry
:source-code: base/frameworks/broker/main.zeek 22 22
:Type: :zeek:type:`interval`
:Attributes: :zeek:attr:`&redef`
:Default: ``1.0 sec``
Default interval to retry listening on a port if it's currently in
use already. Use of the ZEEK_DEFAULT_LISTEN_RETRY environment variable
(set as a number of seconds) will override this option and also
any values given to :zeek:see:`Broker::listen`.
.. zeek:id:: Broker::default_log_topic_prefix
:source-code: base/frameworks/broker/main.zeek 157 157
:Type: :zeek:type:`string`
:Attributes: :zeek:attr:`&redef`
:Default: ``"zeek/logs/"``
The default topic prefix where logs will be published. The log's stream
id is appended when writing to a particular stream.
.. zeek:id:: Broker::default_port
:source-code: base/frameworks/broker/main.zeek 8 8
:Type: :zeek:type:`port`
:Attributes: :zeek:attr:`&redef`
:Default: ``9999/tcp``
Default port for native Broker communication. Where not specified
otherwise, this is the port to connect to and listen on.
.. zeek:id:: Broker::default_port_websocket
:source-code: base/frameworks/broker/main.zeek 16 16
:Type: :zeek:type:`port`
:Attributes: :zeek:attr:`&redef`
:Default: ``9997/tcp``
Default port for Broker WebSocket communication. Where not specified
otherwise, this is the port to connect to and listen on for
WebSocket connections.
See the Broker documentation for a specification of the message
format over WebSocket connections.
.. zeek:id:: Broker::disable_ssl
:source-code: base/frameworks/broker/main.zeek 45 45
:Type: :zeek:type:`bool`
:Attributes: :zeek:attr:`&redef`
:Default: ``F``
If true, do not use SSL for network connections. By default, SSL will
even be used if no certificates / CAs have been configured. In that case
(which is the default) the communication will be encrypted, but not
authenticated.
.. zeek:id:: Broker::forward_messages
:source-code: base/frameworks/broker/main.zeek 146 146
:Type: :zeek:type:`bool`
:Attributes: :zeek:attr:`&redef`
:Default: ``F``
Forward all received messages to subscribing peers.
.. zeek:id:: Broker::log_batch_interval
:source-code: base/frameworks/broker/main.zeek 78 78
:Type: :zeek:type:`interval`
:Attributes: :zeek:attr:`&redef`
:Default: ``1.0 sec``
Max time to buffer log messages before sending the current set out as a
batch.
.. zeek:id:: Broker::log_batch_size
:source-code: base/frameworks/broker/main.zeek 74 74
:Type: :zeek:type:`count`
:Attributes: :zeek:attr:`&redef`
:Default: ``400``
The max number of log entries per log stream to batch together when
sending log messages to a remote logger.
.. zeek:id:: Broker::log_severity_level
:source-code: base/frameworks/broker/main.zeek 195 195
:Type: :zeek:type:`Broker::LogSeverityLevel`
:Attributes: :zeek:attr:`&redef`
:Default: ``Broker::LOG_WARNING``
The log event severity level for the Broker log output.
.. zeek:id:: Broker::log_stderr_severity_level
:source-code: base/frameworks/broker/main.zeek 198 198
:Type: :zeek:type:`Broker::LogSeverityLevel`
:Attributes: :zeek:attr:`&redef`
:Default: ``Broker::LOG_CRITICAL``
Event severity level for also printing the Broker log output to stderr.
.. zeek:id:: Broker::max_threads
:source-code: base/frameworks/broker/main.zeek 82 82
:Type: :zeek:type:`count`
:Attributes: :zeek:attr:`&redef`
:Default: ``1``
Max number of threads to use for Broker/CAF functionality. The
``ZEEK_BROKER_MAX_THREADS`` environment variable overrides this setting.
.. zeek:id:: Broker::moderate_interval
:source-code: base/frameworks/broker/main.zeek 139 139
:Type: :zeek:type:`count`
:Attributes: :zeek:attr:`&redef`
:Default: ``2``
Frequency of work-stealing polling attempts for Broker/CAF threads
in "moderate" mode. Only used for the "stealing" scheduler policy.
.. zeek:id:: Broker::moderate_polls
:source-code: base/frameworks/broker/main.zeek 131 131
:Type: :zeek:type:`count`
:Attributes: :zeek:attr:`&redef`
:Default: ``5``
Number of work-stealing polling attempts for Broker/CAF threads
in "moderate" mode. Only used for the "stealing" scheduler policy.
.. zeek:id:: Broker::moderate_sleep
:source-code: base/frameworks/broker/main.zeek 119 119
:Type: :zeek:type:`interval`
:Attributes: :zeek:attr:`&redef`
:Default: ``16.0 msecs``
Interval of time for under-utilized Broker/CAF threads to sleep
when in "moderate" mode. Only used for the "stealing" scheduler policy.
.. zeek:id:: Broker::peer_buffer_size
:source-code: base/frameworks/broker/main.zeek 87 87
:Type: :zeek:type:`count`
:Attributes: :zeek:attr:`&redef`
:Default: ``8192``
Max number of items we buffer at most per peer. What action to take when
the buffer reaches its maximum size is determined by
:zeek:see:`Broker::peer_overflow_policy`.
.. zeek:id:: Broker::peer_overflow_policy
:source-code: base/frameworks/broker/main.zeek 94 94
:Type: :zeek:type:`string`
:Attributes: :zeek:attr:`&redef`
:Default: ``"drop_oldest"``
Configures how Broker responds to peers that cannot keep up with the
incoming message rate. Available strategies:
- disconnect: drop the connection to the unresponsive peer
- drop_newest: replace the newest message in the buffer
- drop_oldest: removed the olsted message from the buffer, then append
.. zeek:id:: Broker::relaxed_interval
:source-code: base/frameworks/broker/main.zeek 143 143
:Type: :zeek:type:`count`
:Attributes: :zeek:attr:`&redef`
:Default: ``1``
Frequency of work-stealing polling attempts for Broker/CAF threads
in "relaxed" mode. Only used for the "stealing" scheduler policy.
.. zeek:id:: Broker::relaxed_sleep
:source-code: base/frameworks/broker/main.zeek 123 123
:Type: :zeek:type:`interval`
:Attributes: :zeek:attr:`&redef`
:Default: ``64.0 msecs``
Interval of time for under-utilized Broker/CAF threads to sleep
when in "relaxed" mode. Only used for the "stealing" scheduler policy.
.. zeek:id:: Broker::scheduler_policy
:source-code: base/frameworks/broker/main.zeek 115 115
:Type: :zeek:type:`string`
:Attributes: :zeek:attr:`&redef`
:Default: ``"sharing"``
The CAF scheduling policy to use. Available options are "sharing" and
"stealing". The "sharing" policy uses a single, global work queue along
with mutex and condition variable used for accessing it, which may be
better for cases that don't require much concurrency or need lower power
consumption. The "stealing" policy uses multiple work queues protected
by spinlocks, which may be better for use-cases that have more
concurrency needs. E.g. may be worth testing the "stealing" policy
along with dedicating more threads if a lot of data store processing is
required.
.. zeek:id:: Broker::ssl_cafile
:source-code: base/frameworks/broker/main.zeek 50 50
:Type: :zeek:type:`string`
:Attributes: :zeek:attr:`&redef`
:Default: ``""``
Path to a file containing concatenated trusted certificates
in PEM format. If set, Zeek will require valid certificates for
all peers.
.. zeek:id:: Broker::ssl_capath
:source-code: base/frameworks/broker/main.zeek 55 55
:Type: :zeek:type:`string`
:Attributes: :zeek:attr:`&redef`
:Default: ``""``
Path to an OpenSSL-style directory of trusted certificates.
If set, Zeek will require valid certificates for
all peers.
.. zeek:id:: Broker::ssl_certificate
:source-code: base/frameworks/broker/main.zeek 60 60
:Type: :zeek:type:`string`
:Attributes: :zeek:attr:`&redef`
:Default: ``""``
Path to a file containing a X.509 certificate for this
node in PEM format. If set, Zeek will require valid certificates for
all peers.
.. zeek:id:: Broker::ssl_keyfile
:source-code: base/frameworks/broker/main.zeek 70 70
:Type: :zeek:type:`string`
:Attributes: :zeek:attr:`&redef`
:Default: ``""``
Path to the file containing the private key for this node's
certificate. If set, Zeek will require valid certificates for
all peers.
.. zeek:id:: Broker::ssl_passphrase
:source-code: base/frameworks/broker/main.zeek 65 65
:Type: :zeek:type:`string`
:Attributes: :zeek:attr:`&redef`
:Default: ``""``
Passphrase to decrypt the private key specified by
:zeek:see:`Broker::ssl_keyfile`. If set, Zeek will require valid
certificates for all peers.
.. zeek:id:: Broker::web_socket_buffer_size
:source-code: base/frameworks/broker/main.zeek 97 97
:Type: :zeek:type:`count`
:Attributes: :zeek:attr:`&redef`
:Default: ``8192``
Same as :zeek:see:`Broker::peer_buffer_size` but for WebSocket clients.
.. zeek:id:: Broker::web_socket_overflow_policy
:source-code: base/frameworks/broker/main.zeek 100 100
:Type: :zeek:type:`string`
:Attributes: :zeek:attr:`&redef`
:Default: ``"drop_oldest"``
Same as :zeek:see:`Broker::peer_overflow_policy` but for WebSocket clients.
Types
#####
.. zeek:type:: Broker::Data
:source-code: base/frameworks/broker/main.zeek 275 277
:Type: :zeek:type:`record`
.. zeek:field:: data :zeek:type:`opaque` of Broker::Data :zeek:attr:`&optional`
Opaque communication data.
.. zeek:type:: Broker::DataVector
:source-code: base/frameworks/broker/main.zeek 280 280
:Type: :zeek:type:`vector` of :zeek:type:`Broker::Data`
Opaque communication data sequence.
.. zeek:type:: Broker::EndpointInfo
:source-code: base/frameworks/broker/main.zeek 256 261
:Type: :zeek:type:`record`
.. zeek:field:: id :zeek:type:`string`
A unique identifier of the node.
.. zeek:field:: network :zeek:type:`Broker::NetworkInfo` :zeek:attr:`&optional`
Network-level information.
.. zeek:type:: Broker::ErrorCode
:source-code: base/frameworks/broker/main.zeek 200 200
:Type: :zeek:type:`enum`
.. zeek:enum:: Broker::NO_ERROR Broker::ErrorCode
(present if :doc:`/scripts/base/bif/comm.bif.zeek` is loaded)
.. zeek:enum:: Broker::UNSPECIFIED Broker::ErrorCode
The unspecified default error code.
.. zeek:enum:: Broker::PEER_INCOMPATIBLE Broker::ErrorCode
Version incompatibility.
.. zeek:enum:: Broker::PEER_INVALID Broker::ErrorCode
Referenced peer does not exist.
.. zeek:enum:: Broker::PEER_UNAVAILABLE Broker::ErrorCode
Remote peer not listening.
.. zeek:enum:: Broker::PEER_DISCONNECT_DURING_HANDSHAKE Broker::ErrorCode
Remote peer disconnected during the handshake.
.. zeek:enum:: Broker::PEER_TIMEOUT Broker::ErrorCode
A peering request timed out.
.. zeek:enum:: Broker::MASTER_EXISTS Broker::ErrorCode
Master with given name already exists.
.. zeek:enum:: Broker::NO_SUCH_MASTER Broker::ErrorCode
Master with given name does not exist.
.. zeek:enum:: Broker::NO_SUCH_KEY Broker::ErrorCode
The given data store key does not exist.
.. zeek:enum:: Broker::REQUEST_TIMEOUT Broker::ErrorCode
The store operation timed out.
.. zeek:enum:: Broker::TYPE_CLASH Broker::ErrorCode
The operation expected a different type than provided.
.. zeek:enum:: Broker::INVALID_DATA Broker::ErrorCode
The data value cannot be used to carry out the desired operation.
.. zeek:enum:: Broker::BACKEND_FAILURE Broker::ErrorCode
The storage backend failed to execute the operation.
.. zeek:enum:: Broker::STALE_DATA Broker::ErrorCode
The storage backend failed to execute the operation.
.. zeek:enum:: Broker::CANNOT_OPEN_FILE Broker::ErrorCode
(present if :doc:`/scripts/base/bif/comm.bif.zeek` is loaded)
.. zeek:enum:: Broker::CANNOT_WRITE_FILE Broker::ErrorCode
(present if :doc:`/scripts/base/bif/comm.bif.zeek` is loaded)
.. zeek:enum:: Broker::INVALID_TOPIC_KEY Broker::ErrorCode
(present if :doc:`/scripts/base/bif/comm.bif.zeek` is loaded)
.. zeek:enum:: Broker::END_OF_FILE Broker::ErrorCode
(present if :doc:`/scripts/base/bif/comm.bif.zeek` is loaded)
.. zeek:enum:: Broker::INVALID_TAG Broker::ErrorCode
(present if :doc:`/scripts/base/bif/comm.bif.zeek` is loaded)
.. zeek:enum:: Broker::INVALID_STATUS Broker::ErrorCode
(present if :doc:`/scripts/base/bif/comm.bif.zeek` is loaded)
.. zeek:enum:: Broker::CAF_ERROR Broker::ErrorCode
Catch-all for a CAF-level problem.
Enumerates the possible error types.
.. zeek:type:: Broker::Event
:source-code: base/frameworks/broker/main.zeek 283 288
:Type: :zeek:type:`record`
.. zeek:field:: name :zeek:type:`string` :zeek:attr:`&optional`
The name of the event. Not set if invalid event or arguments.
.. zeek:field:: args :zeek:type:`Broker::DataVector`
The arguments to the event.
Opaque event communication data.
.. zeek:type:: Broker::LogSeverityLevel
:source-code: base/frameworks/broker/main.zeek 179 193
:Type: :zeek:type:`enum`
.. zeek:enum:: Broker::LOG_CRITICAL Broker::LogSeverityLevel
Fatal event, normal operation has most likely broken down.
.. zeek:enum:: Broker::LOG_ERROR Broker::LogSeverityLevel
Unrecoverable event that imparts at least part of the system.
.. zeek:enum:: Broker::LOG_WARNING Broker::LogSeverityLevel
Unexpected or conspicuous event that may still be recoverable.
.. zeek:enum:: Broker::LOG_INFO Broker::LogSeverityLevel
Noteworthy event during normal operation.
.. zeek:enum:: Broker::LOG_VERBOSE Broker::LogSeverityLevel
Information that might be relevant for a user to understand system behavior.
.. zeek:enum:: Broker::LOG_DEBUG Broker::LogSeverityLevel
An event that is relevant only for troubleshooting and debugging.
The possible log event severity levels for Broker.
.. zeek:type:: Broker::NetworkInfo
:source-code: base/frameworks/broker/main.zeek 249 254
:Type: :zeek:type:`record`
.. zeek:field:: address :zeek:type:`string` :zeek:attr:`&log`
The IP address or hostname where the endpoint listens.
.. zeek:field:: bound_port :zeek:type:`port` :zeek:attr:`&log`
The port where the endpoint is bound to.
.. zeek:type:: Broker::PeerInfo
:source-code: base/frameworks/broker/main.zeek 263 270
:Type: :zeek:type:`record`
.. zeek:field:: peer :zeek:type:`Broker::EndpointInfo`
.. zeek:field:: status :zeek:type:`Broker::PeerStatus`
.. zeek:field:: is_outbound :zeek:type:`bool`
Whether the local node created the peering, as opposed to a
remote establishing it by connecting to us.
.. zeek:type:: Broker::PeerInfos
:source-code: base/frameworks/broker/main.zeek 272 272
:Type: :zeek:type:`vector` of :zeek:type:`Broker::PeerInfo`
.. zeek:type:: Broker::PeerStatus
:source-code: base/frameworks/broker/main.zeek 234 234
:Type: :zeek:type:`enum`
.. zeek:enum:: Broker::INITIALIZING Broker::PeerStatus
The peering process is initiated.
.. zeek:enum:: Broker::CONNECTING Broker::PeerStatus
Connection establishment in process.
.. zeek:enum:: Broker::CONNECTED Broker::PeerStatus
Connection established, peering pending.
.. zeek:enum:: Broker::PEERED Broker::PeerStatus
Successfully peered.
.. zeek:enum:: Broker::DISCONNECTED Broker::PeerStatus
Connection to remote peer lost.
.. zeek:enum:: Broker::RECONNECTING Broker::PeerStatus
Reconnecting to peer after a lost connection.
The possible states of a peer endpoint.
.. zeek:type:: Broker::TableItem
:source-code: base/frameworks/broker/main.zeek 292 295
:Type: :zeek:type:`record`
.. zeek:field:: key :zeek:type:`Broker::Data`
.. zeek:field:: val :zeek:type:`Broker::Data`
Opaque communication data used as a convenient way to wrap key-value
pairs that comprise table entries.
Functions
#########
.. zeek:id:: Broker::default_log_topic
:source-code: base/frameworks/broker/main.zeek 160 163
:Type: :zeek:type:`function` (id: :zeek:type:`Log::ID`, path: :zeek:type:`string`) : :zeek:type:`string`
The default implementation for :zeek:see:`Broker::log_topic`.
.. zeek:id:: Broker::flush_logs
:source-code: base/frameworks/broker/main.zeek 498 501
:Type: :zeek:type:`function` () : :zeek:type:`count`
Sends all pending log messages to remote peers. This normally
doesn't need to be used except for test cases that are time-sensitive.
.. zeek:id:: Broker::forward
:source-code: base/frameworks/broker/main.zeek 513 516
:Type: :zeek:type:`function` (topic_prefix: :zeek:type:`string`) : :zeek:type:`bool`
Register a topic prefix subscription for events that should only be
forwarded to any subscribing peers and not raise any event handlers
on the receiving/forwarding node. i.e. it's the same as
:zeek:see:`Broker::subscribe` except matching events are not raised
on the receiver, just forwarded. Use :zeek:see:`Broker::unsubscribe`
with the same argument to undo this operation.
:param topic_prefix: a prefix to match against remote message topics.
e.g. an empty prefix matches everything and "a" matches
"alice" and "amy" but not "bob".
:returns: true if a new event forwarding/subscription is now registered.
.. zeek:id:: Broker::is_outbound_peering
:source-code: base/frameworks/broker/main.zeek 478 481
:Type: :zeek:type:`function` (a: :zeek:type:`string`, p: :zeek:type:`port`) : :zeek:type:`bool`
Whether the local node originally initiated the peering with the
given endpoint.
:param a: the address used in previous successful call to :zeek:see:`Broker::peer`.
:param p: the port used in previous successful call to :zeek:see:`Broker::peer`.
Returns:: True if this node initiated the peering.
.. zeek:id:: Broker::listen
:source-code: base/frameworks/broker/main.zeek 450 466
:Type: :zeek:type:`function` (a: :zeek:type:`string` :zeek:attr:`&default` = :zeek:see:`Broker::default_listen_address` :zeek:attr:`&optional`, p: :zeek:type:`port` :zeek:attr:`&default` = :zeek:see:`Broker::default_port` :zeek:attr:`&optional`, retry: :zeek:type:`interval` :zeek:attr:`&default` = :zeek:see:`Broker::default_listen_retry` :zeek:attr:`&optional`) : :zeek:type:`port`
Listen for remote connections using the native Broker protocol.
:param a: an address string on which to accept connections, e.g.
"127.0.0.1". An empty string refers to INADDR_ANY.
:param p: the TCP port to listen on. The value 0 means that the OS should choose
the next available free port.
:param retry: If non-zero, retries listening in regular intervals if the port cannot be
acquired immediately. 0 disables retries. If the
ZEEK_DEFAULT_LISTEN_RETRY environment variable is set (as number
of seconds), it overrides any value given here.
:returns: the bound port or 0/? on failure.
.. zeek:see:: Broker::status
.. zeek:id:: Broker::log_topic
:source-code: base/frameworks/broker/main.zeek 160 163
:Type: :zeek:type:`function` (id: :zeek:type:`Log::ID`, path: :zeek:type:`string`) : :zeek:type:`string`
:Attributes: :zeek:attr:`&redef`
A function that will be called for each log entry to determine what
broker topic string will be used for sending it to peers. The
default implementation will return a value based on
:zeek:see:`Broker::default_log_topic_prefix`.
:param id: the ID associated with the log stream entry that will be sent.
:param path: the path to which the log stream entry will be output.
:returns: a string representing the broker topic to which the log
will be sent.
.. zeek:id:: Broker::node_id
:source-code: base/frameworks/broker/main.zeek 488 491
:Type: :zeek:type:`function` () : :zeek:type:`string`
Get a unique identifier for the local broker endpoint.
:returns: a unique identifier for the local broker endpoint.
.. zeek:id:: Broker::peer
:source-code: base/frameworks/broker/main.zeek 468 471
:Type: :zeek:type:`function` (a: :zeek:type:`string`, p: :zeek:type:`port` :zeek:attr:`&default` = :zeek:see:`Broker::default_port` :zeek:attr:`&optional`, retry: :zeek:type:`interval` :zeek:attr:`&default` = :zeek:see:`Broker::default_connect_retry` :zeek:attr:`&optional`) : :zeek:type:`bool`
Initiate a remote connection.
:param a: an address to connect to, e.g. "localhost" or "127.0.0.1".
:param p: the TCP port on which the remote side is listening.
:param retry: an interval at which to retry establishing the
connection with the remote peer if it cannot be made initially, or
if it ever becomes disconnected. If the
ZEEK_DEFAULT_CONNECT_RETRY environment variable is set (as number
of seconds), it overrides any value given here.
:returns: true if it's possible to try connecting with the peer and
it's a new peer. The actual connection may not be established
until a later point in time.
.. zeek:see:: Broker::status
.. zeek:id:: Broker::peering_stats
:source-code: base/frameworks/broker/main.zeek 493 496
:Type: :zeek:type:`function` () : :zeek:type:`table` [:zeek:type:`string`] of :zeek:type:`BrokerPeeringStats`
Obtain each peering's send-buffer statistics. The keys are Broker
endpoint IDs.
:returns: per-peering statistics.
.. zeek:id:: Broker::peers
:source-code: base/frameworks/broker/main.zeek 483 486
:Type: :zeek:type:`function` () : :zeek:type:`vector` of :zeek:type:`Broker::PeerInfo`
Get a list of all peer connections.
:returns: a list of all peer connections.
.. zeek:id:: Broker::publish_id
:source-code: base/frameworks/broker/main.zeek 503 506
:Type: :zeek:type:`function` (topic: :zeek:type:`string`, id: :zeek:type:`string`) : :zeek:type:`bool`
Publishes the value of an identifier to a given topic. The subscribers
will update their local value for that identifier on receipt.
:param topic: a topic associated with the message.
:param id: the identifier to publish.
:returns: true if the message is sent.
.. zeek:id:: Broker::subscribe
:source-code: base/frameworks/broker/main.zeek 508 511
:Type: :zeek:type:`function` (topic_prefix: :zeek:type:`string`) : :zeek:type:`bool`
Register interest in all peer event messages that use a certain topic
prefix. Note that subscriptions may not be altered immediately after
calling (except during :zeek:see:`zeek_init`).
:param topic_prefix: a prefix to match against remote message topics.
e.g. an empty prefix matches everything and "a" matches
"alice" and "amy" but not "bob".
:returns: true if it's a new event subscription and it is now registered.
.. zeek:id:: Broker::unpeer
:source-code: base/frameworks/broker/main.zeek 473 476
:Type: :zeek:type:`function` (a: :zeek:type:`string`, p: :zeek:type:`port`) : :zeek:type:`bool`
Remove a remote connection.
Note that this does not terminate the connection to the peer, it
just means that we won't exchange any further information with it
unless peering resumes later.
:param a: the address used in previous successful call to :zeek:see:`Broker::peer`.
:param p: the port used in previous successful call to :zeek:see:`Broker::peer`.
:returns: true if the arguments match a previously successful call to
:zeek:see:`Broker::peer`.
:param TODO: We do not have a function yet to terminate a connection.
.. zeek:id:: Broker::unsubscribe
:source-code: base/frameworks/broker/main.zeek 518 521
:Type: :zeek:type:`function` (topic_prefix: :zeek:type:`string`) : :zeek:type:`bool`
Unregister interest in all peer event messages that use a topic prefix.
Note that subscriptions may not be altered immediately after calling
(except during :zeek:see:`zeek_init`).
:param topic_prefix: a prefix previously supplied to a successful call to
:zeek:see:`Broker::subscribe` or :zeek:see:`Broker::forward`.
:returns: true if interest in the topic prefix is no longer advertised.

1377
doc/scripts/base/frameworks/broker/store.zeek.rst Normal file
View file

File diff suppressed because it is too large Load diff

14
doc/scripts/base/frameworks/cluster/__load__.zeek.rst Normal file
View file

@ -0,0 +1,14 @@
:tocdepth: 3
base/frameworks/cluster/__load__.zeek
=====================================
:Imports: :doc:`base/frameworks/cluster/main.zeek </scripts/base/frameworks/cluster/main.zeek>`, :doc:`base/frameworks/cluster/pools.zeek </scripts/base/frameworks/cluster/pools.zeek>`, :doc:`base/frameworks/cluster/telemetry.zeek </scripts/base/frameworks/cluster/telemetry.zeek>`
Summary
~~~~~~~
Detailed Interface
~~~~~~~~~~~~~~~~~~

38
doc/scripts/base/frameworks/cluster/index.rst Normal file
View file

@ -0,0 +1,38 @@
:orphan:
Package: base/frameworks/cluster
================================
The cluster framework provides for establishing and controlling a cluster
of Zeek instances.
:doc:`/scripts/base/frameworks/cluster/__load__.zeek`
:doc:`/scripts/base/frameworks/cluster/main.zeek`
A framework for establishing and controlling a cluster of Zeek instances.
In order to use the cluster framework, a script named
``cluster-layout.zeek`` must exist somewhere in Zeek's script search path
which has a cluster definition of the :zeek:id:`Cluster::nodes` variable.
The ``CLUSTER_NODE`` environment variable or :zeek:id:`Cluster::node`
must also be sent and the cluster framework loaded as a package like
``@load base/frameworks/cluster``.
.. warning::
The file ``cluster-layout.zeek`` should only contain the definition
of :zeek:id:`Cluster::nodes`. Specifically, avoid loading other Zeek
scripts or using :zeek:see:`redef` for anything but :zeek:id:`Cluster::nodes`.
Due to ``cluster-layout.zeek`` being loaded very early, it is easy to
introduce circular loading issues.
:doc:`/scripts/base/frameworks/cluster/pools.zeek`
Defines an interface for managing pools of cluster nodes. Pools are
a useful way to distribute work or data among nodes within a cluster.
:doc:`/scripts/base/frameworks/cluster/telemetry.zeek`

999
doc/scripts/base/frameworks/cluster/main.zeek.rst Normal file
View file

@ -0,0 +1,999 @@
:tocdepth: 3
base/frameworks/cluster/main.zeek
=================================
.. zeek:namespace:: Cluster
A framework for establishing and controlling a cluster of Zeek instances.
In order to use the cluster framework, a script named
``cluster-layout.zeek`` must exist somewhere in Zeek's script search path
which has a cluster definition of the :zeek:id:`Cluster::nodes` variable.
The ``CLUSTER_NODE`` environment variable or :zeek:id:`Cluster::node`
must also be sent and the cluster framework loaded as a package like
``@load base/frameworks/cluster``.
.. warning::
The file ``cluster-layout.zeek`` should only contain the definition
of :zeek:id:`Cluster::nodes`. Specifically, avoid loading other Zeek
scripts or using :zeek:see:`redef` for anything but :zeek:id:`Cluster::nodes`.
Due to ``cluster-layout.zeek`` being loaded very early, it is easy to
introduce circular loading issues.
:Namespace: Cluster
:Imports: :doc:`base/bif/cluster.bif.zeek </scripts/base/bif/cluster.bif.zeek>`, :doc:`base/bif/plugins/Zeek_Cluster_WebSocket.events.bif.zeek </scripts/base/bif/plugins/Zeek_Cluster_WebSocket.events.bif.zeek>`, :doc:`base/frameworks/broker </scripts/base/frameworks/broker/index>`, :doc:`base/frameworks/control </scripts/base/frameworks/control/index>`
Summary
~~~~~~~
Redefinable Options
###################
==================================================================================================== ===============================================================================
:zeek:id:`Cluster::default_backend`: :zeek:type:`Broker::BackendType` :zeek:attr:`&redef` The type of data store backend that will be used for all data stores if
no other has already been specified by the user in :zeek:see:`Cluster::stores`.
:zeek:id:`Cluster::default_master_node`: :zeek:type:`string` :zeek:attr:`&redef` Name of the node on which master data stores will be created if no other
has already been specified by the user in :zeek:see:`Cluster::stores`.
:zeek:id:`Cluster::default_persistent_backend`: :zeek:type:`Broker::BackendType` :zeek:attr:`&redef` The type of persistent data store backend that will be used for all data
stores if no other has already been specified by the user in
:zeek:see:`Cluster::stores`.
:zeek:id:`Cluster::default_store_dir`: :zeek:type:`string` :zeek:attr:`&redef` Setting a default dir will, for persistent backends that have not
been given an explicit file path via :zeek:see:`Cluster::stores`,
automatically create a path within this dir that is based on the name of
the data store.
:zeek:id:`Cluster::default_websocket_max_event_queue_size`: :zeek:type:`count` :zeek:attr:`&redef` The default maximum queue size for WebSocket event dispatcher instances.
:zeek:id:`Cluster::default_websocket_ping_interval`: :zeek:type:`interval` :zeek:attr:`&redef` The default ping interval for WebSocket clients.
:zeek:id:`Cluster::enable_round_robin_logging`: :zeek:type:`bool` :zeek:attr:`&redef` Whether to distribute log messages among available logging nodes.
:zeek:id:`Cluster::logger_topic`: :zeek:type:`string` :zeek:attr:`&redef` The topic name used for exchanging messages that are relevant to
logger nodes in a cluster.
:zeek:id:`Cluster::manager_is_logger`: :zeek:type:`bool` :zeek:attr:`&redef` Indicates whether or not the manager will act as the logger and receive
logs.
:zeek:id:`Cluster::manager_topic`: :zeek:type:`string` :zeek:attr:`&redef` The topic name used for exchanging messages that are relevant to
manager nodes in a cluster.
:zeek:id:`Cluster::node`: :zeek:type:`string` :zeek:attr:`&redef` This is usually supplied on the command line for each instance
of the cluster that is started up.
:zeek:id:`Cluster::node_topic_prefix`: :zeek:type:`string` :zeek:attr:`&redef` The topic prefix used for exchanging messages that are relevant to
a named node in a cluster.
:zeek:id:`Cluster::nodeid_topic_prefix`: :zeek:type:`string` :zeek:attr:`&redef` The topic prefix used for exchanging messages that are relevant to
a unique node in a cluster.
:zeek:id:`Cluster::nodes`: :zeek:type:`table` :zeek:attr:`&redef` The cluster layout definition.
:zeek:id:`Cluster::proxy_topic`: :zeek:type:`string` :zeek:attr:`&redef` The topic name used for exchanging messages that are relevant to
proxy nodes in a cluster.
:zeek:id:`Cluster::retry_interval`: :zeek:type:`interval` :zeek:attr:`&redef` Interval for retrying failed connections between cluster nodes.
:zeek:id:`Cluster::worker_topic`: :zeek:type:`string` :zeek:attr:`&redef` The topic name used for exchanging messages that are relevant to
worker nodes in a cluster.
==================================================================================================== ===============================================================================
Constants
#########
====================================================== ==================================================================
:zeek:id:`Cluster::broadcast_topics`: :zeek:type:`set` A set of topic names to be used for broadcasting messages that are
relevant to all nodes in a cluster.
====================================================== ==================================================================
State Variables
###############
================================================================================================ ======================================================================
:zeek:id:`Cluster::stores`: :zeek:type:`table` :zeek:attr:`&default` = *...* :zeek:attr:`&redef` A table of cluster-enabled data stores that have been created, indexed
by their name.
================================================================================================ ======================================================================
Types
#####
================================================================= ==========================================================================
:zeek:type:`Cluster::EndpointInfo`: :zeek:type:`record` Information about a WebSocket endpoint.
:zeek:type:`Cluster::Event`: :zeek:type:`record` An event instance for cluster pub/sub.
:zeek:type:`Cluster::Info`: :zeek:type:`record` :zeek:attr:`&log` The record type which contains the column fields of the cluster log.
:zeek:type:`Cluster::NamedNode`: :zeek:type:`record` Record to represent a cluster node including its name.
:zeek:type:`Cluster::NetworkInfo`: :zeek:type:`record` Network information of an endpoint.
:zeek:type:`Cluster::Node`: :zeek:type:`record` Record type to indicate a node in a cluster.
:zeek:type:`Cluster::NodeType`: :zeek:type:`enum` Types of nodes that are allowed to participate in the cluster
configuration.
:zeek:type:`Cluster::StoreInfo`: :zeek:type:`record` Information regarding a cluster-enabled data store.
:zeek:type:`Cluster::WebSocketServerOptions`: :zeek:type:`record` WebSocket server options to pass to :zeek:see:`Cluster::listen_websocket`.
:zeek:type:`Cluster::WebSocketTLSOptions`: :zeek:type:`record` The TLS options for a WebSocket server.
:zeek:type:`Cluster::BackendTag`: :zeek:type:`enum`
:zeek:type:`Cluster::EventSerializerTag`: :zeek:type:`enum`
:zeek:type:`Cluster::LogSerializerTag`: :zeek:type:`enum`
================================================================= ==========================================================================
Redefinitions
#############
======================================= ======================================
:zeek:type:`Log::ID`: :zeek:type:`enum` The cluster logging stream identifier.
* :zeek:enum:`Cluster::LOG`
======================================= ======================================
Events
######
================================================= =======================================================================
:zeek:id:`Cluster::hello`: :zeek:type:`event` When using broker-enabled cluster framework, nodes broadcast this event
to exchange their user-defined name along with a string that uniquely
identifies it for the duration of its lifetime.
:zeek:id:`Cluster::node_down`: :zeek:type:`event` When using broker-enabled cluster framework, this event will be emitted
locally whenever a connected cluster node becomes disconnected.
:zeek:id:`Cluster::node_up`: :zeek:type:`event` When using broker-enabled cluster framework, this event will be emitted
locally whenever a cluster node connects or reconnects.
================================================= =======================================================================
Hooks
#####
============================================================ =============================================================
:zeek:id:`Cluster::log_policy`: :zeek:type:`Log::PolicyHook` A default logging policy hook for the stream.
:zeek:id:`Cluster::on_subscribe`: :zeek:type:`hook` A hook invoked for every :zeek:see:`Cluster::subscribe` call.
:zeek:id:`Cluster::on_unsubscribe`: :zeek:type:`hook` A hook invoked for every :zeek:see:`Cluster::subscribe` call.
============================================================ =============================================================
Functions
#########
=========================================================================== =====================================================================
:zeek:id:`Cluster::create_store`: :zeek:type:`function` Sets up a cluster-enabled data store.
:zeek:id:`Cluster::get_active_node_count`: :zeek:type:`function` Returns the number of nodes per type, the calling node is currently
connected to.
:zeek:id:`Cluster::get_node_count`: :zeek:type:`function` Returns the number of nodes defined in the cluster layout for a given
node type.
:zeek:id:`Cluster::init`: :zeek:type:`function` Initialize the cluster backend.
:zeek:id:`Cluster::is_enabled`: :zeek:type:`function` This function can be called at any time to determine if the cluster
framework is being enabled for this run.
:zeek:id:`Cluster::listen_websocket`: :zeek:type:`function` Start listening on a WebSocket address.
:zeek:id:`Cluster::local_node_metrics_port`: :zeek:type:`function` This function can be called at any time to determine the configured
metrics port for Prometheus being used by current Zeek instance.
:zeek:id:`Cluster::local_node_type`: :zeek:type:`function` This function can be called at any time to determine what type of
cluster node the current Zeek instance is going to be acting as.
:zeek:id:`Cluster::log`: :zeek:type:`function` Write a message to the cluster logging stream.
:zeek:id:`Cluster::node_id`: :zeek:type:`function` :zeek:attr:`&redef` Function returning this node's identifier.
:zeek:id:`Cluster::node_topic`: :zeek:type:`function` :zeek:attr:`&redef` Retrieve the topic associated with a specific node in the cluster.
:zeek:id:`Cluster::nodeid_to_node`: :zeek:type:`function` Retrieve the cluster-level naming of a node based on its node ID,
a backend-specific identifier.
:zeek:id:`Cluster::nodeid_topic`: :zeek:type:`function` :zeek:attr:`&redef` Retrieve the topic associated with a specific node in the cluster.
:zeek:id:`Cluster::subscribe`: :zeek:type:`function` Subscribe to the given topic.
:zeek:id:`Cluster::unsubscribe`: :zeek:type:`function` Unsubscribe from the given topic.
=========================================================================== =====================================================================
Detailed Interface
~~~~~~~~~~~~~~~~~~
Redefinable Options
###################
.. zeek:id:: Cluster::default_backend
:source-code: base/frameworks/cluster/main.zeek 70 70
:Type: :zeek:type:`Broker::BackendType`
:Attributes: :zeek:attr:`&redef`
:Default: ``Broker::MEMORY``
The type of data store backend that will be used for all data stores if
no other has already been specified by the user in :zeek:see:`Cluster::stores`.
.. zeek:id:: Cluster::default_master_node
:source-code: base/frameworks/cluster/main.zeek 66 66
:Type: :zeek:type:`string`
:Attributes: :zeek:attr:`&redef`
:Default: ``""``
Name of the node on which master data stores will be created if no other
has already been specified by the user in :zeek:see:`Cluster::stores`.
An empty value means "use whatever name corresponds to the manager
node".
.. zeek:id:: Cluster::default_persistent_backend
:source-code: base/frameworks/cluster/main.zeek 76 76
:Type: :zeek:type:`Broker::BackendType`
:Attributes: :zeek:attr:`&redef`
:Default: ``Broker::SQLITE``
The type of persistent data store backend that will be used for all data
stores if no other has already been specified by the user in
:zeek:see:`Cluster::stores`. This will be used when script authors call
:zeek:see:`Cluster::create_store` with the *persistent* argument set true.
.. zeek:id:: Cluster::default_store_dir
:source-code: base/frameworks/cluster/main.zeek 95 95
:Type: :zeek:type:`string`
:Attributes: :zeek:attr:`&redef`
:Default: ``""``
Setting a default dir will, for persistent backends that have not
been given an explicit file path via :zeek:see:`Cluster::stores`,
automatically create a path within this dir that is based on the name of
the data store.
.. zeek:id:: Cluster::default_websocket_max_event_queue_size
:source-code: base/frameworks/cluster/main.zeek 86 86
:Type: :zeek:type:`count`
:Attributes: :zeek:attr:`&redef`
:Default: ``32``
The default maximum queue size for WebSocket event dispatcher instances.
If the maximum queue size is reached, events from external WebSocket
clients will be stalled and processed once the queue has been drained.
An internal metric named ``cluster_onloop_queue_stalls`` and
labeled with a ``WebSocketEventDispatcher:<host>:<port>`` tag
is incremented when the maximum queue size is reached.
.. zeek:id:: Cluster::default_websocket_ping_interval
:source-code: base/frameworks/cluster/main.zeek 89 89
:Type: :zeek:type:`interval`
:Attributes: :zeek:attr:`&redef`
:Default: ``5.0 secs``
The default ping interval for WebSocket clients.
.. zeek:id:: Cluster::enable_round_robin_logging
:source-code: base/frameworks/cluster/main.zeek 25 25
:Type: :zeek:type:`bool`
:Attributes: :zeek:attr:`&redef`
:Default: ``T``
Whether to distribute log messages among available logging nodes.
.. zeek:id:: Cluster::logger_topic
:source-code: base/frameworks/cluster/main.zeek 29 29
:Type: :zeek:type:`string`
:Attributes: :zeek:attr:`&redef`
:Default: ``"zeek/cluster/logger"``
:Redefinition: from :doc:`/scripts/policy/frameworks/cluster/backend/zeromq/main.zeek`
``=``::
zeek.cluster.logger
The topic name used for exchanging messages that are relevant to
logger nodes in a cluster. Used with broker-enabled cluster communication.
.. zeek:id:: Cluster::manager_is_logger
:source-code: base/frameworks/cluster/main.zeek 252 252
:Type: :zeek:type:`bool`
:Attributes: :zeek:attr:`&redef`
:Default: ``T``
Indicates whether or not the manager will act as the logger and receive
logs. This value should be set in the cluster-layout.zeek script (the
value should be true only if no logger is specified in Cluster::nodes).
Note that ZeekControl handles this automatically.
.. zeek:id:: Cluster::manager_topic
:source-code: base/frameworks/cluster/main.zeek 33 33
:Type: :zeek:type:`string`
:Attributes: :zeek:attr:`&redef`
:Default: ``"zeek/cluster/manager"``
:Redefinition: from :doc:`/scripts/policy/frameworks/cluster/backend/zeromq/main.zeek`
``=``::
zeek.cluster.manager
The topic name used for exchanging messages that are relevant to
manager nodes in a cluster. Used with broker-enabled cluster communication.
.. zeek:id:: Cluster::node
:source-code: base/frameworks/cluster/main.zeek 256 256
:Type: :zeek:type:`string`
:Attributes: :zeek:attr:`&redef`
:Default: ``""``
This is usually supplied on the command line for each instance
of the cluster that is started up.
.. zeek:id:: Cluster::node_topic_prefix
:source-code: base/frameworks/cluster/main.zeek 56 56
:Type: :zeek:type:`string`
:Attributes: :zeek:attr:`&redef`
:Default: ``"zeek/cluster/node/"``
The topic prefix used for exchanging messages that are relevant to
a named node in a cluster. Used with broker-enabled cluster communication.
.. zeek:id:: Cluster::nodeid_topic_prefix
:source-code: base/frameworks/cluster/main.zeek 60 60
:Type: :zeek:type:`string`
:Attributes: :zeek:attr:`&redef`
:Default: ``"zeek/cluster/nodeid/"``
The topic prefix used for exchanging messages that are relevant to
a unique node in a cluster. Used with broker-enabled cluster communication.
.. zeek:id:: Cluster::nodes
:source-code: base/frameworks/cluster/main.zeek 237 237
:Type: :zeek:type:`table` [:zeek:type:`string`] of :zeek:type:`Cluster::Node`
:Attributes: :zeek:attr:`&redef`
:Default: ``{}``
The cluster layout definition. This should be placed into a filter
named cluster-layout.zeek somewhere in the ZEEKPATH. It will be
automatically loaded if the CLUSTER_NODE environment variable is set.
Note that ZeekControl handles all of this automatically.
The table is typically indexed by node names/labels (e.g. "manager"
or "worker-1").
.. zeek:id:: Cluster::proxy_topic
:source-code: base/frameworks/cluster/main.zeek 37 37
:Type: :zeek:type:`string`
:Attributes: :zeek:attr:`&redef`
:Default: ``"zeek/cluster/proxy"``
:Redefinition: from :doc:`/scripts/policy/frameworks/cluster/backend/zeromq/main.zeek`
``=``::
zeek.cluster.proxy
The topic name used for exchanging messages that are relevant to
proxy nodes in a cluster. Used with broker-enabled cluster communication.
.. zeek:id:: Cluster::retry_interval
:source-code: base/frameworks/cluster/main.zeek 268 268
:Type: :zeek:type:`interval`
:Attributes: :zeek:attr:`&redef`
:Default: ``1.0 sec``
Interval for retrying failed connections between cluster nodes.
If set, the ZEEK_DEFAULT_CONNECT_RETRY (given in number of seconds)
environment variable overrides this option.
.. zeek:id:: Cluster::worker_topic
:source-code: base/frameworks/cluster/main.zeek 41 41
:Type: :zeek:type:`string`
:Attributes: :zeek:attr:`&redef`
:Default: ``"zeek/cluster/worker"``
:Redefinition: from :doc:`/scripts/policy/frameworks/cluster/backend/zeromq/main.zeek`
``=``::
zeek.cluster.worker
The topic name used for exchanging messages that are relevant to
worker nodes in a cluster. Used with broker-enabled cluster communication.
Constants
#########
.. zeek:id:: Cluster::broadcast_topics
:source-code: base/frameworks/cluster/main.zeek 47 47
:Type: :zeek:type:`set` [:zeek:type:`string`]
:Default:
::
{
"zeek/cluster/manager",
"zeek/cluster/logger",
"zeek/cluster/proxy",
"zeek/cluster/worker"
}
A set of topic names to be used for broadcasting messages that are
relevant to all nodes in a cluster. Currently, there is not a common
topic to broadcast to, because enabling implicit Broker forwarding would
cause a routing loop for this topic.
State Variables
###############
.. zeek:id:: Cluster::stores
:source-code: base/frameworks/cluster/main.zeek 130 130
:Type: :zeek:type:`table` [:zeek:type:`string`] of :zeek:type:`Cluster::StoreInfo`
:Attributes: :zeek:attr:`&default` = *[name=<uninitialized>, store=<uninitialized>, master_node=, master=F, backend=Broker::MEMORY, options=[sqlite=[path=, synchronous=<uninitialized>, journal_mode=<uninitialized>, failure_mode=Broker::SQLITE_FAILURE_MODE_FAIL, integrity_check=F]], clone_resync_interval=10.0 secs, clone_stale_interval=5.0 mins, clone_mutation_buffer_interval=2.0 mins]* :zeek:attr:`&redef`
:Default: ``{}``
A table of cluster-enabled data stores that have been created, indexed
by their name. This table will be populated automatically by
:zeek:see:`Cluster::create_store`, but if you need to customize
the options related to a particular data store, you may redef this
table. Calls to :zeek:see:`Cluster::create_store` will first check
the table for an entry of the same name and, if found, will use the
predefined options there when setting up the store.
Types
#####
.. zeek:type:: Cluster::EndpointInfo
:source-code: base/frameworks/cluster/main.zeek 396 401
:Type: :zeek:type:`record`
.. zeek:field:: id :zeek:type:`string`
.. zeek:field:: network :zeek:type:`Cluster::NetworkInfo`
.. zeek:field:: application_name :zeek:type:`string` :zeek:attr:`&optional`
The value of the X-Application-Name HTTP header, if any.
Information about a WebSocket endpoint.
.. zeek:type:: Cluster::Event
:source-code: base/frameworks/cluster/main.zeek 337 342
:Type: :zeek:type:`record`
.. zeek:field:: ev :zeek:type:`any`
The event handler to be invoked on the remote node.
.. zeek:field:: args :zeek:type:`vector` of :zeek:type:`any`
The arguments for the event.
An event instance for cluster pub/sub.
See :zeek:see:`Cluster::publish` and :zeek:see:`Cluster::make_event`.
.. zeek:type:: Cluster::Info
:source-code: base/frameworks/cluster/main.zeek 151 158
:Type: :zeek:type:`record`
.. zeek:field:: ts :zeek:type:`time` :zeek:attr:`&log`
The time at which a cluster message was generated.
.. zeek:field:: node :zeek:type:`string` :zeek:attr:`&log`
The name of the node that is creating the log record.
.. zeek:field:: message :zeek:type:`string` :zeek:attr:`&log`
A message indicating information about the cluster's operation.
:Attributes: :zeek:attr:`&log`
The record type which contains the column fields of the cluster log.
.. zeek:type:: Cluster::NamedNode
:source-code: base/frameworks/cluster/main.zeek 204 207
:Type: :zeek:type:`record`
.. zeek:field:: name :zeek:type:`string`
.. zeek:field:: node :zeek:type:`Cluster::Node`
Record to represent a cluster node including its name.
.. zeek:type:: Cluster::NetworkInfo
:source-code: base/frameworks/cluster/main.zeek 388 393
:Type: :zeek:type:`record`
.. zeek:field:: address :zeek:type:`string`
The IP address or hostname where the endpoint listens.
.. zeek:field:: bound_port :zeek:type:`port`
The port where the endpoint is bound to.
Network information of an endpoint.
.. zeek:type:: Cluster::Node
:source-code: base/frameworks/cluster/main.zeek 181 201
:Type: :zeek:type:`record`
.. zeek:field:: node_type :zeek:type:`Cluster::NodeType`
Identifies the type of cluster node in this node's configuration.
.. zeek:field:: ip :zeek:type:`addr`
The IP address of the cluster node.
.. zeek:field:: zone_id :zeek:type:`string` :zeek:attr:`&default` = ``""`` :zeek:attr:`&optional`
If the *ip* field is a non-global IPv6 address, this field
can specify a particular :rfc:`4007` ``zone_id``.
.. zeek:field:: p :zeek:type:`port` :zeek:attr:`&default` = ``0/unknown`` :zeek:attr:`&optional`
The port that this node will listen on for peer connections.
A value of ``0/unknown`` means the node is not pre-configured to listen.
.. zeek:field:: manager :zeek:type:`string` :zeek:attr:`&optional`
Name of the manager node this node uses. For workers and proxies.
.. zeek:field:: id :zeek:type:`string` :zeek:attr:`&optional`
A unique identifier assigned to the node by the broker framework.
This field is only set while a node is connected.
.. zeek:field:: metrics_port :zeek:type:`port` :zeek:attr:`&optional`
The port used to expose metrics to Prometheus. Setting this in a cluster
configuration will override the setting for Telemetry::metrics_port for
the node.
Record type to indicate a node in a cluster.
.. zeek:type:: Cluster::NodeType
:source-code: base/frameworks/cluster/main.zeek 162 179
:Type: :zeek:type:`enum`
.. zeek:enum:: Cluster::NONE Cluster::NodeType
A dummy node type indicating the local node is not operating
within a cluster.
.. zeek:enum:: Cluster::CONTROL Cluster::NodeType
A node type which is allowed to view/manipulate the configuration
of other nodes in the cluster.
.. zeek:enum:: Cluster::LOGGER Cluster::NodeType
A node type responsible for log management.
.. zeek:enum:: Cluster::MANAGER Cluster::NodeType
A node type responsible for policy management.
.. zeek:enum:: Cluster::PROXY Cluster::NodeType
A node type for relaying worker node communication and synchronizing
worker node state.
.. zeek:enum:: Cluster::WORKER Cluster::NodeType
The node type doing all the actual traffic analysis.
Types of nodes that are allowed to participate in the cluster
configuration.
.. zeek:type:: Cluster::StoreInfo
:source-code: base/frameworks/cluster/main.zeek 98 121
:Type: :zeek:type:`record`
.. zeek:field:: name :zeek:type:`string` :zeek:attr:`&optional`
The name of the data store.
.. zeek:field:: store :zeek:type:`opaque` of Broker::Store :zeek:attr:`&optional`
The store handle.
.. zeek:field:: master_node :zeek:type:`string` :zeek:attr:`&default` = :zeek:see:`Cluster::default_master_node` :zeek:attr:`&optional`
The name of the cluster node on which the master version of the data
store resides.
.. zeek:field:: master :zeek:type:`bool` :zeek:attr:`&default` = ``F`` :zeek:attr:`&optional`
Whether the data store is the master version or a clone.
.. zeek:field:: backend :zeek:type:`Broker::BackendType` :zeek:attr:`&default` = :zeek:see:`Cluster::default_backend` :zeek:attr:`&optional`
The type of backend used for storing data.
.. zeek:field:: options :zeek:type:`Broker::BackendOptions` :zeek:attr:`&default` = *...* :zeek:attr:`&optional`
Parameters used for configuring the backend.
.. zeek:field:: clone_resync_interval :zeek:type:`interval` :zeek:attr:`&default` = :zeek:see:`Broker::default_clone_resync_interval` :zeek:attr:`&optional`
A resync/reconnect interval to pass through to
:zeek:see:`Broker::create_clone`.
.. zeek:field:: clone_stale_interval :zeek:type:`interval` :zeek:attr:`&default` = :zeek:see:`Broker::default_clone_stale_interval` :zeek:attr:`&optional`
A staleness duration to pass through to
:zeek:see:`Broker::create_clone`.
.. zeek:field:: clone_mutation_buffer_interval :zeek:type:`interval` :zeek:attr:`&default` = :zeek:see:`Broker::default_clone_mutation_buffer_interval` :zeek:attr:`&optional`
A mutation buffer interval to pass through to
:zeek:see:`Broker::create_clone`.
Information regarding a cluster-enabled data store.
.. zeek:type:: Cluster::WebSocketServerOptions
:source-code: base/frameworks/cluster/main.zeek 364 378
:Type: :zeek:type:`record`
.. zeek:field:: listen_addr :zeek:type:`addr` :zeek:attr:`&optional`
The address to listen on, cannot be used together with ``listen_host``.
.. zeek:field:: listen_port :zeek:type:`port`
The port the WebSocket server is supposed to listen on.
.. zeek:field:: max_event_queue_size :zeek:type:`count` :zeek:attr:`&default` = :zeek:see:`Cluster::default_websocket_max_event_queue_size` :zeek:attr:`&optional`
The maximum event queue size for this server.
.. zeek:field:: ping_interval :zeek:type:`interval` :zeek:attr:`&default` = :zeek:see:`Cluster::default_websocket_ping_interval` :zeek:attr:`&optional`
Ping interval to use. A WebSocket client not responding to
the pings will be disconnected. Set to a negative value to
disable pings. Subsecond intervals are currently not supported.
.. zeek:field:: tls_options :zeek:type:`Cluster::WebSocketTLSOptions` :zeek:attr:`&default` = *...* :zeek:attr:`&optional`
The TLS options used for this WebSocket server. By default,
TLS is disabled. See also :zeek:see:`Cluster::WebSocketTLSOptions`.
WebSocket server options to pass to :zeek:see:`Cluster::listen_websocket`.
.. zeek:type:: Cluster::WebSocketTLSOptions
:source-code: base/frameworks/cluster/main.zeek 348 361
:Type: :zeek:type:`record`
.. zeek:field:: cert_file :zeek:type:`string` :zeek:attr:`&optional`
The cert file to use.
.. zeek:field:: key_file :zeek:type:`string` :zeek:attr:`&optional`
The key file to use.
.. zeek:field:: enable_peer_verification :zeek:type:`bool` :zeek:attr:`&default` = ``F`` :zeek:attr:`&optional`
Expect peers to send client certificates.
.. zeek:field:: ca_file :zeek:type:`string` :zeek:attr:`&default` = ``""`` :zeek:attr:`&optional`
The CA certificate or CA bundle used for peer verification.
Empty will use the implementations's default when
``enable_peer_verification`` is T.
.. zeek:field:: ciphers :zeek:type:`string` :zeek:attr:`&default` = ``""`` :zeek:attr:`&optional`
The ciphers to use. Empty will use the implementation's defaults.
The TLS options for a WebSocket server.
If cert_file and key_file are set, TLS is enabled. If both
are unset, TLS is disabled. Any other combination is an error.
.. zeek:type:: Cluster::BackendTag
:Type: :zeek:type:`enum`
.. zeek:enum:: Cluster::CLUSTER_BACKEND_BROKER Cluster::BackendTag
.. zeek:enum:: Cluster::CLUSTER_BACKEND_BROKER_WEBSOCKET_SHIM Cluster::BackendTag
.. zeek:enum:: Cluster::CLUSTER_BACKEND_ZEROMQ Cluster::BackendTag
.. zeek:type:: Cluster::EventSerializerTag
:Type: :zeek:type:`enum`
.. zeek:enum:: Cluster::EVENT_SERIALIZER_BROKER_BIN_V1 Cluster::EventSerializerTag
.. zeek:enum:: Cluster::EVENT_SERIALIZER_BROKER_JSON_V1 Cluster::EventSerializerTag
.. zeek:type:: Cluster::LogSerializerTag
:Type: :zeek:type:`enum`
.. zeek:enum:: Cluster::LOG_SERIALIZER_ZEEK_BIN_V1 Cluster::LogSerializerTag
Events
######
.. zeek:id:: Cluster::hello
:source-code: base/frameworks/cluster/main.zeek 510 535
:Type: :zeek:type:`event` (name: :zeek:type:`string`, id: :zeek:type:`string`)
When using broker-enabled cluster framework, nodes broadcast this event
to exchange their user-defined name along with a string that uniquely
identifies it for the duration of its lifetime. This string may change
if the node dies and has to reconnect later.
.. zeek:id:: Cluster::node_down
:source-code: base/frameworks/cluster/main.zeek 282 282
:Type: :zeek:type:`event` (name: :zeek:type:`string`, id: :zeek:type:`string`)
When using broker-enabled cluster framework, this event will be emitted
locally whenever a connected cluster node becomes disconnected.
.. zeek:id:: Cluster::node_up
:source-code: base/frameworks/cluster/main.zeek 278 278
:Type: :zeek:type:`event` (name: :zeek:type:`string`, id: :zeek:type:`string`)
When using broker-enabled cluster framework, this event will be emitted
locally whenever a cluster node connects or reconnects.
Hooks
#####
.. zeek:id:: Cluster::log_policy
:source-code: base/frameworks/cluster/main.zeek 148 148
:Type: :zeek:type:`Log::PolicyHook`
A default logging policy hook for the stream.
.. zeek:id:: Cluster::on_subscribe
:source-code: base/frameworks/cluster/main.zeek 408 408
:Type: :zeek:type:`hook` (topic: :zeek:type:`string`) : :zeek:type:`bool`
A hook invoked for every :zeek:see:`Cluster::subscribe` call.
Breaking from this hook has no effect.
:param topic: The topic string as given to :zeek:see:`Cluster::subscribe`.
.. zeek:id:: Cluster::on_unsubscribe
:source-code: base/frameworks/cluster/main.zeek 415 415
:Type: :zeek:type:`hook` (topic: :zeek:type:`string`) : :zeek:type:`bool`
A hook invoked for every :zeek:see:`Cluster::subscribe` call.
Breaking from this hook has no effect.
:param topic: The topic string as given to :zeek:see:`Cluster::subscribe`.
Functions
#########
.. zeek:id:: Cluster::create_store
:source-code: base/frameworks/cluster/main.zeek 590 665
:Type: :zeek:type:`function` (name: :zeek:type:`string`, persistent: :zeek:type:`bool` :zeek:attr:`&default` = ``F`` :zeek:attr:`&optional`) : :zeek:type:`Cluster::StoreInfo`
Sets up a cluster-enabled data store. They will also still properly
function for uses that are not operating a cluster.
:param name: the name of the data store to create.
:param persistent: whether the data store must be persistent.
:returns: the store's information. For master stores, the store will be
ready to use immediately. For clones, the store field will not
be set until the node containing the master store has connected.
.. zeek:id:: Cluster::get_active_node_count
:source-code: base/frameworks/cluster/main.zeek 454 457
:Type: :zeek:type:`function` (node_type: :zeek:type:`Cluster::NodeType`) : :zeek:type:`count`
Returns the number of nodes per type, the calling node is currently
connected to. This is primarily intended for use by the manager to find
out how many nodes should be responding to requests.
.. zeek:id:: Cluster::get_node_count
:source-code: base/frameworks/cluster/main.zeek 441 452
:Type: :zeek:type:`function` (node_type: :zeek:type:`Cluster::NodeType`) : :zeek:type:`count`
Returns the number of nodes defined in the cluster layout for a given
node type.
.. zeek:id:: Cluster::init
:source-code: base/frameworks/cluster/main.zeek 672 675
:Type: :zeek:type:`function` () : :zeek:type:`bool`
Initialize the cluster backend.
Cluster backends usually invoke this from a :zeek:see:`zeek_init` handler.
:returns: T on success, else F.
.. zeek:id:: Cluster::is_enabled
:source-code: base/frameworks/cluster/main.zeek 459 462
:Type: :zeek:type:`function` () : :zeek:type:`bool`
This function can be called at any time to determine if the cluster
framework is being enabled for this run.
:returns: True if :zeek:id:`Cluster::node` has been set.
.. zeek:id:: Cluster::listen_websocket
:source-code: base/frameworks/cluster/main.zeek 687 690
:Type: :zeek:type:`function` (options: :zeek:type:`Cluster::WebSocketServerOptions`) : :zeek:type:`bool`
Start listening on a WebSocket address.
:param options: The server :zeek:see:`Cluster::WebSocketServerOptions` to use.
:returns: T on success, else F.
.. zeek:id:: Cluster::local_node_metrics_port
:source-code: base/frameworks/cluster/main.zeek 475 487
:Type: :zeek:type:`function` () : :zeek:type:`port`
This function can be called at any time to determine the configured
metrics port for Prometheus being used by current Zeek instance. If
:zeek:id:`Cluster::is_enabled` returns false or the node isn't found,
``0/unknown`` is returned.
:returns: The metrics port used by the calling node.
.. zeek:id:: Cluster::local_node_type
:source-code: base/frameworks/cluster/main.zeek 464 473
:Type: :zeek:type:`function` () : :zeek:type:`Cluster::NodeType`
This function can be called at any time to determine what type of
cluster node the current Zeek instance is going to be acting as.
If :zeek:id:`Cluster::is_enabled` returns false, then
:zeek:enum:`Cluster::NONE` is returned.
:returns: The :zeek:type:`Cluster::NodeType` the calling node acts as.
.. zeek:id:: Cluster::log
:source-code: base/frameworks/cluster/main.zeek 667 670
:Type: :zeek:type:`function` (msg: :zeek:type:`string`) : :zeek:type:`void`
Write a message to the cluster logging stream.
.. zeek:id:: Cluster::node_id
:source-code: policy/frameworks/cluster/backend/zeromq/main.zeek 355 357
:Type: :zeek:type:`function` () : :zeek:type:`string`
:Attributes: :zeek:attr:`&redef`
Function returning this node's identifier.
By default this is :zeek:see:`Broker::node_id`, but can be
redefined by other cluster backends. This identifier should be
a short lived identifier that resets when a node is restarted.
.. zeek:id:: Cluster::node_topic
:source-code: policy/frameworks/cluster/backend/zeromq/main.zeek 340 342
:Type: :zeek:type:`function` (name: :zeek:type:`string`) : :zeek:type:`string`
:Attributes: :zeek:attr:`&redef`
Retrieve the topic associated with a specific node in the cluster.
:param name: the name of the cluster node (e.g. "manager").
:returns: a topic string that may used to send a message exclusively to
a given cluster node.
.. zeek:id:: Cluster::nodeid_to_node
:source-code: base/frameworks/cluster/main.zeek 499 508
:Type: :zeek:type:`function` (id: :zeek:type:`string`) : :zeek:type:`Cluster::NamedNode`
Retrieve the cluster-level naming of a node based on its node ID,
a backend-specific identifier.
:param id: the node ID of a peer.
:returns: the :zeek:see:`Cluster::NamedNode` for the requested node, if
known, otherwise a "null" instance with an empty name field.
.. zeek:id:: Cluster::nodeid_topic
:source-code: policy/frameworks/cluster/backend/zeromq/main.zeek 344 346
:Type: :zeek:type:`function` (id: :zeek:type:`string`) : :zeek:type:`string`
:Attributes: :zeek:attr:`&redef`
Retrieve the topic associated with a specific node in the cluster.
:param id: the id of the cluster node (from :zeek:see:`Broker::EndpointInfo`
or :zeek:see:`Broker::node_id`.
:returns: a topic string that may used to send a message exclusively to
a given cluster node.
.. zeek:id:: Cluster::subscribe
:source-code: base/frameworks/cluster/main.zeek 677 680
:Type: :zeek:type:`function` (topic: :zeek:type:`string`) : :zeek:type:`bool`
Subscribe to the given topic.
:param topic: The topic to subscribe to.
:returns: T on success, else F.
.. zeek:id:: Cluster::unsubscribe
:source-code: base/frameworks/cluster/main.zeek 682 685
:Type: :zeek:type:`function` (topic: :zeek:type:`string`) : :zeek:type:`bool`
Unsubscribe from the given topic.
:param topic: The topic to unsubscribe from.
:returns: T on success, else F.

352
doc/scripts/base/frameworks/cluster/pools.zeek.rst Normal file
View file

@ -0,0 +1,352 @@
:tocdepth: 3
base/frameworks/cluster/pools.zeek
==================================
.. zeek:namespace:: Cluster
Defines an interface for managing pools of cluster nodes. Pools are
a useful way to distribute work or data among nodes within a cluster.
:Namespace: Cluster
:Imports: :doc:`base/frameworks/cluster/main.zeek </scripts/base/frameworks/cluster/main.zeek>`, :doc:`base/utils/hash_hrw.zeek </scripts/base/utils/hash_hrw.zeek>`
Summary
~~~~~~~
State Variables
###############
======================================================================================== =======================================================
:zeek:id:`Cluster::logger_pool`: :zeek:type:`Cluster::Pool` A pool containing all the logger nodes of a cluster.
:zeek:id:`Cluster::logger_pool_spec`: :zeek:type:`Cluster::PoolSpec` :zeek:attr:`&redef` The specification for :zeek:see:`Cluster::logger_pool`.
:zeek:id:`Cluster::proxy_pool`: :zeek:type:`Cluster::Pool` A pool containing all the proxy nodes of a cluster.
:zeek:id:`Cluster::proxy_pool_spec`: :zeek:type:`Cluster::PoolSpec` :zeek:attr:`&redef` The specification for :zeek:see:`Cluster::proxy_pool`.
:zeek:id:`Cluster::worker_pool`: :zeek:type:`Cluster::Pool` A pool containing all the worker nodes of a cluster.
:zeek:id:`Cluster::worker_pool_spec`: :zeek:type:`Cluster::PoolSpec` :zeek:attr:`&redef` The specification for :zeek:see:`Cluster::worker_pool`.
======================================================================================== =======================================================
Types
#####
========================================================= ===========================================================
:zeek:type:`Cluster::PoolNode`: :zeek:type:`record` Store state of a cluster within the context of a work pool.
:zeek:type:`Cluster::PoolNodeTable`: :zeek:type:`table`
:zeek:type:`Cluster::PoolSpec`: :zeek:type:`record` A pool specification.
:zeek:type:`Cluster::RoundRobinTable`: :zeek:type:`table`
========================================================= ===========================================================
Functions
#########
======================================================== ======================================================================
:zeek:id:`Cluster::hrw_topic`: :zeek:type:`function` Retrieve the topic associated with the node mapped via Rendezvous hash
of an arbitrary key.
:zeek:id:`Cluster::register_pool`: :zeek:type:`function` Registers and initializes a pool.
:zeek:id:`Cluster::rr_log_topic`: :zeek:type:`function` Distributes log message topics among logger nodes via round-robin.
:zeek:id:`Cluster::rr_topic`: :zeek:type:`function` Retrieve the topic associated with the node in a round-robin fashion.
======================================================== ======================================================================
Detailed Interface
~~~~~~~~~~~~~~~~~~
State Variables
###############
.. zeek:id:: Cluster::logger_pool
:source-code: base/frameworks/cluster/pools.zeek 91 91
:Type: :zeek:type:`Cluster::Pool`
:Default:
::
{
spec=[topic=<uninitialized>, node_type=<uninitialized>, max_nodes=<uninitialized>, exclusive=F]
nodes={
}
node_list=[]
hrw_pool=[sites={
}]
rr_key_seq={
}
alive_count=0
}
A pool containing all the logger nodes of a cluster.
The pool's node membership/availability is automatically
maintained by the cluster framework.
.. zeek:id:: Cluster::logger_pool_spec
:source-code: base/frameworks/cluster/pools.zeek 74 74
:Type: :zeek:type:`Cluster::PoolSpec`
:Attributes: :zeek:attr:`&redef`
:Default:
::
{
topic="zeek/cluster/pool/logger"
node_type=Cluster::LOGGER
max_nodes=<uninitialized>
exclusive=F
}
:Redefinition: from :doc:`/scripts/policy/frameworks/cluster/backend/zeromq/main.zeek`
``=``::
Cluster::PoolSpec($topic=zeek.cluster.pool.logger, $node_type=Cluster::LOGGER)
The specification for :zeek:see:`Cluster::logger_pool`.
.. zeek:id:: Cluster::proxy_pool
:source-code: base/frameworks/cluster/pools.zeek 81 81
:Type: :zeek:type:`Cluster::Pool`
:Default:
::
{
spec=[topic=<uninitialized>, node_type=<uninitialized>, max_nodes=<uninitialized>, exclusive=F]
nodes={
}
node_list=[]
hrw_pool=[sites={
}]
rr_key_seq={
}
alive_count=0
}
A pool containing all the proxy nodes of a cluster.
The pool's node membership/availability is automatically
maintained by the cluster framework.
.. zeek:id:: Cluster::proxy_pool_spec
:source-code: base/frameworks/cluster/pools.zeek 64 64
:Type: :zeek:type:`Cluster::PoolSpec`
:Attributes: :zeek:attr:`&redef`
:Default:
::
{
topic="zeek/cluster/pool/proxy"
node_type=Cluster::PROXY
max_nodes=<uninitialized>
exclusive=F
}
:Redefinition: from :doc:`/scripts/policy/frameworks/cluster/backend/zeromq/main.zeek`
``=``::
Cluster::PoolSpec($topic=zeek.cluster.pool.proxy, $node_type=Cluster::PROXY)
The specification for :zeek:see:`Cluster::proxy_pool`.
.. zeek:id:: Cluster::worker_pool
:source-code: base/frameworks/cluster/pools.zeek 86 86
:Type: :zeek:type:`Cluster::Pool`
:Default:
::
{
spec=[topic=<uninitialized>, node_type=<uninitialized>, max_nodes=<uninitialized>, exclusive=F]
nodes={
}
node_list=[]
hrw_pool=[sites={
}]
rr_key_seq={
}
alive_count=0
}
A pool containing all the worker nodes of a cluster.
The pool's node membership/availability is automatically
maintained by the cluster framework.
.. zeek:id:: Cluster::worker_pool_spec
:source-code: base/frameworks/cluster/pools.zeek 69 69
:Type: :zeek:type:`Cluster::PoolSpec`
:Attributes: :zeek:attr:`&redef`
:Default:
::
{
topic="zeek/cluster/pool/worker"
node_type=Cluster::WORKER
max_nodes=<uninitialized>
exclusive=F
}
:Redefinition: from :doc:`/scripts/policy/frameworks/cluster/backend/zeromq/main.zeek`
``=``::
Cluster::PoolSpec($topic=zeek.cluster.pool.worker, $node_type=Cluster::WORKER)
The specification for :zeek:see:`Cluster::worker_pool`.
Types
#####
.. zeek:type:: Cluster::PoolNode
:source-code: base/frameworks/cluster/pools.zeek 11 23
:Type: :zeek:type:`record`
.. zeek:field:: name :zeek:type:`string`
The node name (e.g. "manager").
.. zeek:field:: alias :zeek:type:`string`
An alias of *name* used to prevent hashing collisions when creating
*site_id*.
.. zeek:field:: site_id :zeek:type:`count`
A 32-bit unique identifier for the pool node, derived from name/alias.
.. zeek:field:: alive :zeek:type:`bool` :zeek:attr:`&default` = ``F`` :zeek:attr:`&optional`
Whether the node is currently alive and can receive work.
.. zeek:field:: topic :zeek:type:`string`
The pre-computed result from Cluster::node_topic
Store state of a cluster within the context of a work pool.
.. zeek:type:: Cluster::PoolNodeTable
:source-code: base/frameworks/cluster/pools.zeek 42 42
:Type: :zeek:type:`table` [:zeek:type:`string`] of :zeek:type:`Cluster::PoolNode`
.. zeek:type:: Cluster::PoolSpec
:source-code: base/frameworks/cluster/pools.zeek 26 40
:Type: :zeek:type:`record`
.. zeek:field:: topic :zeek:type:`string`
A topic string that can be used to reach all nodes within a pool.
.. zeek:field:: node_type :zeek:type:`Cluster::NodeType`
The type of nodes that are contained within the pool.
.. zeek:field:: max_nodes :zeek:type:`count` :zeek:attr:`&optional`
The maximum number of nodes that may belong to the pool.
If not set, then all available nodes will be added to the pool,
else the cluster framework will automatically limit the pool
membership according to the threshold.
.. zeek:field:: exclusive :zeek:type:`bool` :zeek:attr:`&default` = ``F`` :zeek:attr:`&optional`
Whether the pool requires exclusive access to nodes. If true,
then *max_nodes* nodes will not be assigned to any other pool.
When using this flag, *max_nodes* must also be set.
A pool specification.
.. zeek:type:: Cluster::RoundRobinTable
:source-code: base/frameworks/cluster/pools.zeek 43 43
:Type: :zeek:type:`table` [:zeek:type:`string`] of :zeek:type:`int`
Functions
#########
.. zeek:id:: Cluster::hrw_topic
:source-code: base/frameworks/cluster/pools.zeek 170 178
:Type: :zeek:type:`function` (pool: :zeek:type:`Cluster::Pool`, key: :zeek:type:`any`) : :zeek:type:`string`
Retrieve the topic associated with the node mapped via Rendezvous hash
of an arbitrary key.
:param pool: the pool of nodes to consider.
:param key: data used for input to the hashing function that will uniformly
distribute keys among available nodes.
:returns: a topic string associated with a cluster node that is alive
or an empty string if nothing is alive.
.. zeek:id:: Cluster::register_pool
:source-code: base/frameworks/cluster/pools.zeek 163 168
:Type: :zeek:type:`function` (spec: :zeek:type:`Cluster::PoolSpec`) : :zeek:type:`Cluster::Pool`
Registers and initializes a pool.
.. zeek:id:: Cluster::rr_log_topic
:source-code: base/frameworks/cluster/pools.zeek 216 225
:Type: :zeek:type:`function` (id: :zeek:type:`Log::ID`, path: :zeek:type:`string`) : :zeek:type:`string`
Distributes log message topics among logger nodes via round-robin.
This will be automatically assigned to :zeek:see:`Broker::log_topic`
if :zeek:see:`Cluster::enable_round_robin_logging` is enabled.
If no logger nodes are active, then this will return the value
of :zeek:see:`Broker::default_log_topic`.
.. zeek:id:: Cluster::rr_topic
:source-code: base/frameworks/cluster/pools.zeek 180 214
:Type: :zeek:type:`function` (pool: :zeek:type:`Cluster::Pool`, key: :zeek:type:`string` :zeek:attr:`&default` = ``""`` :zeek:attr:`&optional`) : :zeek:type:`string`
Retrieve the topic associated with the node in a round-robin fashion.
:param pool: the pool of nodes to consider.
:param key: an arbitrary string to identify the purpose for which you're
requesting the topic. e.g. consider using a name-spaced key
like "Intel::cluster_rr_key" if you need to guarantee that
a group of messages get distributed in a well-defined pattern
without other messages being interleaved within the round-robin.
Usually sharing the default key is fine for load-balancing
purposes.
:returns: a topic string associated with a cluster node that is alive,
or an empty string if nothing is alive.

127
doc/scripts/base/frameworks/cluster/telemetry.zeek.rst Normal file
View file

@ -0,0 +1,127 @@
:tocdepth: 3
base/frameworks/cluster/telemetry.zeek
======================================
.. zeek:namespace:: Cluster::Telemetry
:Namespace: Cluster::Telemetry
Summary
~~~~~~~
Redefinable Options
###################
================================================================================================================= =================================================================
:zeek:id:`Cluster::Telemetry::core_metrics`: :zeek:type:`set` :zeek:attr:`&redef` The telemetry types to enable for the core backend.
:zeek:id:`Cluster::Telemetry::message_size_bounds`: :zeek:type:`vector` :zeek:attr:`&redef` For the DEBUG metrics, the histogram buckets to use.
:zeek:id:`Cluster::Telemetry::topic_normalizations`: :zeek:type:`table` :zeek:attr:`&ordered` :zeek:attr:`&redef` Table used for normalizing topic names that contain random parts.
:zeek:id:`Cluster::Telemetry::websocket_metrics`: :zeek:type:`set` :zeek:attr:`&redef` The telemetry types to enable for WebSocket backends.
================================================================================================================= =================================================================
Types
#####
======================================================== =============================
:zeek:type:`Cluster::Telemetry::Type`: :zeek:type:`enum` Module for cluster telemetry.
======================================================== =============================
Detailed Interface
~~~~~~~~~~~~~~~~~~
Redefinable Options
###################
.. zeek:id:: Cluster::Telemetry::core_metrics
:source-code: base/frameworks/cluster/telemetry.zeek 19 19
:Type: :zeek:type:`set` [:zeek:type:`Cluster::Telemetry::Type`]
:Attributes: :zeek:attr:`&redef`
:Default:
::
{
Cluster::Telemetry::INFO
}
The telemetry types to enable for the core backend.
.. zeek:id:: Cluster::Telemetry::message_size_bounds
:source-code: base/frameworks/cluster/telemetry.zeek 36 36
:Type: :zeek:type:`vector` of :zeek:type:`double`
:Attributes: :zeek:attr:`&redef`
:Default:
::
[10.0, 50.0, 100.0, 500.0, 1000.0, 5000.0, 10000.0, 50000.0]
For the DEBUG metrics, the histogram buckets to use.
.. zeek:id:: Cluster::Telemetry::topic_normalizations
:source-code: base/frameworks/cluster/telemetry.zeek 31 31
:Type: :zeek:type:`table` [:zeek:type:`pattern`] of :zeek:type:`string`
:Attributes: :zeek:attr:`&ordered` :zeek:attr:`&redef`
:Default:
::
{
[/^?(^zeek\/cluster\/nodeid\/.*)$?/] = "zeek/cluster/nodeid/__normalized__"
}
:Redefinition: from :doc:`/scripts/policy/frameworks/cluster/backend/zeromq/main.zeek`
``+=``::
/^?(^zeek\.cluster\.nodeid\..*)$?/ = zeek.cluster.nodeid.__normalized__
Table used for normalizing topic names that contain random parts.
Map to an empty string to skip recording a specific metric
completely.
.. zeek:id:: Cluster::Telemetry::websocket_metrics
:source-code: base/frameworks/cluster/telemetry.zeek 24 24
:Type: :zeek:type:`set` [:zeek:type:`Cluster::Telemetry::Type`]
:Attributes: :zeek:attr:`&redef`
:Default:
::
{
Cluster::Telemetry::INFO
}
The telemetry types to enable for WebSocket backends.
Types
#####
.. zeek:type:: Cluster::Telemetry::Type
:source-code: base/frameworks/cluster/telemetry.zeek 5 17
:Type: :zeek:type:`enum`
.. zeek:enum:: Cluster::Telemetry::INFO Cluster::Telemetry::Type
Creates counter metrics for incoming and for outgoing
events without labels.
.. zeek:enum:: Cluster::Telemetry::VERBOSE Cluster::Telemetry::Type
Creates counter metrics for incoming and outgoing events
labeled with handler and normalized topic names.
.. zeek:enum:: Cluster::Telemetry::DEBUG Cluster::Telemetry::Type
Creates histogram metrics using the serialized message size
for events, labeled by topic, handler and script location
(outgoing only).
Module for cluster telemetry.

14
doc/scripts/base/frameworks/config/__load__.zeek.rst Normal file
View file

@ -0,0 +1,14 @@
:tocdepth: 3
base/frameworks/config/__load__.zeek
====================================
:Imports: :doc:`base/frameworks/config/input.zeek </scripts/base/frameworks/config/input.zeek>`, :doc:`base/frameworks/config/main.zeek </scripts/base/frameworks/config/main.zeek>`, :doc:`base/frameworks/config/weird.zeek </scripts/base/frameworks/config/weird.zeek>`
Summary
~~~~~~~
Detailed Interface
~~~~~~~~~~~~~~~~~~

25
doc/scripts/base/frameworks/config/index.rst Normal file
View file

@ -0,0 +1,25 @@
:orphan:
Package: base/frameworks/config
===============================
The configuration framework provides a way to change the Zeek configuration
in "option" values at run-time.
:doc:`/scripts/base/frameworks/config/__load__.zeek`
:doc:`/scripts/base/frameworks/config/main.zeek`
The configuration framework provides a way to change Zeek options
(as specified by the "option" keyword) at runtime. It also logs runtime
changes to options to config.log.
:doc:`/scripts/base/frameworks/config/input.zeek`
File input for the configuration framework using the input framework.
:doc:`/scripts/base/frameworks/config/weird.zeek`
This script sets up the config framework change handlers for weirds.

56
doc/scripts/base/frameworks/config/input.zeek.rst Normal file
View file

@ -0,0 +1,56 @@
:tocdepth: 3
base/frameworks/config/input.zeek
=================================
.. zeek:namespace:: Config
File input for the configuration framework using the input framework.
:Namespace: Config
:Imports: :doc:`base/frameworks/cluster </scripts/base/frameworks/cluster/index>`, :doc:`base/frameworks/config/main.zeek </scripts/base/frameworks/config/main.zeek>`
Summary
~~~~~~~
Redefinable Options
###################
===================================================================== ===============================================
:zeek:id:`Config::config_files`: :zeek:type:`set` :zeek:attr:`&redef` Configuration files that will be read off disk.
===================================================================== ===============================================
Functions
#########
===================================================== ===================================================================
:zeek:id:`Config::read_config`: :zeek:type:`function` Read specified configuration file and apply values; updates to file
are not tracked.
===================================================== ===================================================================
Detailed Interface
~~~~~~~~~~~~~~~~~~
Redefinable Options
###################
.. zeek:id:: Config::config_files
:source-code: base/frameworks/config/input.zeek 15 15
:Type: :zeek:type:`set` [:zeek:type:`string`]
:Attributes: :zeek:attr:`&redef`
:Default: ``{}``
Configuration files that will be read off disk. Files are reread
every time they are updated so updates should be atomic with "mv"
instead of writing the file in place.
If the same configuration option is defined in several files with
different values, behavior is unspecified.
Functions
#########
.. zeek:id:: Config::read_config
:source-code: base/frameworks/config/input.zeek 61 77
:Type: :zeek:type:`function` (filename: :zeek:type:`string`) : :zeek:type:`void`
Read specified configuration file and apply values; updates to file
are not tracked.

132
doc/scripts/base/frameworks/config/main.zeek.rst Normal file
View file

@ -0,0 +1,132 @@
:tocdepth: 3
base/frameworks/config/main.zeek
================================
.. zeek:namespace:: Config
The configuration framework provides a way to change Zeek options
(as specified by the "option" keyword) at runtime. It also logs runtime
changes to options to config.log.
:Namespace: Config
:Imports: :doc:`base/frameworks/cluster </scripts/base/frameworks/cluster/index>`
Summary
~~~~~~~
Types
#####
============================================== ==================================
:zeek:type:`Config::Info`: :zeek:type:`record` Represents the data in config.log.
============================================== ==================================
Redefinitions
#############
======================================= =====================================
:zeek:type:`Log::ID`: :zeek:type:`enum` The config logging stream identifier.
* :zeek:enum:`Config::LOG`
======================================= =====================================
Events
######
================================================= =================================================================
:zeek:id:`Config::log_config`: :zeek:type:`event` Event that can be handled to access the :zeek:type:`Config::Info`
record as it is sent on to the logging framework.
================================================= =================================================================
Hooks
#####
=========================================================== =============================================
:zeek:id:`Config::log_policy`: :zeek:type:`Log::PolicyHook` A default logging policy hook for the stream.
=========================================================== =============================================
Functions
#########
=================================================== ==================================================================
:zeek:id:`Config::set_value`: :zeek:type:`function` This function is the config framework layer around the lower-level
:zeek:see:`Option::set` call.
=================================================== ==================================================================
Detailed Interface
~~~~~~~~~~~~~~~~~~
Types
#####
.. zeek:type:: Config::Info
:source-code: base/frameworks/config/main.zeek 17 28
:Type: :zeek:type:`record`
.. zeek:field:: ts :zeek:type:`time` :zeek:attr:`&log`
Timestamp at which the configuration change occurred.
.. zeek:field:: id :zeek:type:`string` :zeek:attr:`&log`
ID of the value that was changed.
.. zeek:field:: old_value :zeek:type:`string` :zeek:attr:`&log`
Value before the change.
.. zeek:field:: new_value :zeek:type:`string` :zeek:attr:`&log`
Value after the change.
.. zeek:field:: location :zeek:type:`string` :zeek:attr:`&optional` :zeek:attr:`&log`
Optional location that triggered the change.
Represents the data in config.log.
Events
######
.. zeek:id:: Config::log_config
:source-code: base/frameworks/config/main.zeek 32 32
:Type: :zeek:type:`event` (rec: :zeek:type:`Config::Info`)
Event that can be handled to access the :zeek:type:`Config::Info`
record as it is sent on to the logging framework.
Hooks
#####
.. zeek:id:: Config::log_policy
:source-code: base/frameworks/config/main.zeek 14 14
:Type: :zeek:type:`Log::PolicyHook`
A default logging policy hook for the stream.
Functions
#########
.. zeek:id:: Config::set_value
:source-code: base/frameworks/config/main.zeek 99 102
:Type: :zeek:type:`function` (ID: :zeek:type:`string`, val: :zeek:type:`any`, location: :zeek:type:`string` :zeek:attr:`&default` = ``""`` :zeek:attr:`&optional`) : :zeek:type:`bool`
This function is the config framework layer around the lower-level
:zeek:see:`Option::set` call. Config::set_value will set the configuration
value for all nodes in the cluster, no matter where it was called. Note
that :zeek:see:`Option::set` does not distribute configuration changes
to other nodes.
:param ID: The ID of the option to update.
:param val: The new value of the option.
:param location: Optional parameter detailing where this change originated from.
:returns: true on success, false when an error occurs.

17
doc/scripts/base/frameworks/config/weird.zeek.rst Normal file
View file

@ -0,0 +1,17 @@
:tocdepth: 3
base/frameworks/config/weird.zeek
=================================
.. zeek:namespace:: Config
This script sets up the config framework change handlers for weirds.
:Namespace: Config
:Imports: :doc:`base/frameworks/config/main.zeek </scripts/base/frameworks/config/main.zeek>`
Summary
~~~~~~~
Detailed Interface
~~~~~~~~~~~~~~~~~~

14
doc/scripts/base/frameworks/control/__load__.zeek.rst Normal file
View file

@ -0,0 +1,14 @@
:tocdepth: 3
base/frameworks/control/__load__.zeek
=====================================
:Imports: :doc:`base/frameworks/control/main.zeek </scripts/base/frameworks/control/main.zeek>`
Summary
~~~~~~~
Detailed Interface
~~~~~~~~~~~~~~~~~~

18
doc/scripts/base/frameworks/control/index.rst Normal file
View file

@ -0,0 +1,18 @@
:orphan:
Package: base/frameworks/control
================================
The control framework provides the foundation for providing "commands"
that can be taken remotely at runtime to modify a running Zeek instance
or collect information from the running instance.
:doc:`/scripts/base/frameworks/control/__load__.zeek`
:doc:`/scripts/base/frameworks/control/main.zeek`
The control framework provides the foundation for providing "commands"
that can be taken remotely at runtime to modify a running Zeek instance
or collect information from the running instance.

239
doc/scripts/base/frameworks/control/main.zeek.rst Normal file
View file

@ -0,0 +1,239 @@
:tocdepth: 3
base/frameworks/control/main.zeek
=================================
.. zeek:namespace:: Control
The control framework provides the foundation for providing "commands"
that can be taken remotely at runtime to modify a running Zeek instance
or collect information from the running instance.
:Namespace: Control
Summary
~~~~~~~
Redefinable Options
###################
============================================================================ ================================================================
:zeek:id:`Control::arg`: :zeek:type:`string` :zeek:attr:`&redef` This can be used by commands that take an argument.
:zeek:id:`Control::cmd`: :zeek:type:`string` :zeek:attr:`&redef` The command that is being done.
:zeek:id:`Control::commands`: :zeek:type:`set` :zeek:attr:`&redef` The commands that can currently be given on the command line for
remote control.
:zeek:id:`Control::controllee_listen`: :zeek:type:`bool` :zeek:attr:`&redef` Whether the controllee should call :zeek:see:`Broker::listen`.
:zeek:id:`Control::host`: :zeek:type:`addr` :zeek:attr:`&redef` The address of the host that will be controlled.
:zeek:id:`Control::host_port`: :zeek:type:`port` :zeek:attr:`&redef` The port of the host that will be controlled.
:zeek:id:`Control::zone_id`: :zeek:type:`string` :zeek:attr:`&redef` If :zeek:id:`Control::host` is a non-global IPv6 address and
requires a specific :rfc:`4007` ``zone_id``, it can be set here.
============================================================================ ================================================================
Constants
#########
===================================================== =================================================================
:zeek:id:`Control::ignore_ids`: :zeek:type:`set` Variable IDs that are to be ignored by the update process.
:zeek:id:`Control::topic_prefix`: :zeek:type:`string` The topic prefix used for exchanging control messages via Broker.
===================================================== =================================================================
Events
######
===================================================================== =====================================================================
:zeek:id:`Control::configuration_update`: :zeek:type:`event` This event is a wrapper and alias for the
:zeek:id:`Control::configuration_update_request` event.
:zeek:id:`Control::configuration_update_request`: :zeek:type:`event` Inform the remote Zeek instance that it's configuration may have been
updated.
:zeek:id:`Control::configuration_update_response`: :zeek:type:`event` Message in response to a configuration update request.
:zeek:id:`Control::id_value_request`: :zeek:type:`event` Event for requesting the value of an ID (a variable).
:zeek:id:`Control::id_value_response`: :zeek:type:`event` Event for returning the value of an ID after an
:zeek:id:`Control::id_value_request` event.
:zeek:id:`Control::net_stats_request`: :zeek:type:`event` Requests the current net_stats.
:zeek:id:`Control::net_stats_response`: :zeek:type:`event` Returns the current net_stats.
:zeek:id:`Control::peer_status_request`: :zeek:type:`event` Requests the current communication status.
:zeek:id:`Control::peer_status_response`: :zeek:type:`event` Returns the current communication status.
:zeek:id:`Control::shutdown_request`: :zeek:type:`event` Requests that the Zeek instance begins shutting down.
:zeek:id:`Control::shutdown_response`: :zeek:type:`event` Message in response to a shutdown request.
===================================================================== =====================================================================
Detailed Interface
~~~~~~~~~~~~~~~~~~
Redefinable Options
###################
.. zeek:id:: Control::arg
:source-code: base/frameworks/control/main.zeek 30 30
:Type: :zeek:type:`string`
:Attributes: :zeek:attr:`&redef`
:Default: ``""``
This can be used by commands that take an argument.
.. zeek:id:: Control::cmd
:source-code: base/frameworks/control/main.zeek 27 27
:Type: :zeek:type:`string`
:Attributes: :zeek:attr:`&redef`
:Default: ``""``
The command that is being done. It's typically set on the
command line.
.. zeek:id:: Control::commands
:source-code: base/frameworks/control/main.zeek 34 34
:Type: :zeek:type:`set` [:zeek:type:`string`]
:Attributes: :zeek:attr:`&redef`
:Default:
::
{
"peer_status",
"id_value",
"net_stats",
"configuration_update",
"shutdown"
}
The commands that can currently be given on the command line for
remote control.
.. zeek:id:: Control::controllee_listen
:source-code: base/frameworks/control/main.zeek 13 13
:Type: :zeek:type:`bool`
:Attributes: :zeek:attr:`&redef`
:Default: ``T``
Whether the controllee should call :zeek:see:`Broker::listen`.
In a cluster, this isn't needed since the setup process calls it.
.. zeek:id:: Control::host
:source-code: base/frameworks/control/main.zeek 16 16
:Type: :zeek:type:`addr`
:Attributes: :zeek:attr:`&redef`
:Default: ``0.0.0.0``
The address of the host that will be controlled.
.. zeek:id:: Control::host_port
:source-code: base/frameworks/control/main.zeek 19 19
:Type: :zeek:type:`port`
:Attributes: :zeek:attr:`&redef`
:Default: ``0/tcp``
The port of the host that will be controlled.
.. zeek:id:: Control::zone_id
:source-code: base/frameworks/control/main.zeek 23 23
:Type: :zeek:type:`string`
:Attributes: :zeek:attr:`&redef`
:Default: ``""``
If :zeek:id:`Control::host` is a non-global IPv6 address and
requires a specific :rfc:`4007` ``zone_id``, it can be set here.
Constants
#########
.. zeek:id:: Control::ignore_ids
:source-code: base/frameworks/control/main.zeek 43 43
:Type: :zeek:type:`set` [:zeek:type:`string`]
:Default: ``{}``
Variable IDs that are to be ignored by the update process.
.. zeek:id:: Control::topic_prefix
:source-code: base/frameworks/control/main.zeek 9 9
:Type: :zeek:type:`string`
:Default: ``"zeek/control"``
The topic prefix used for exchanging control messages via Broker.
Events
######
.. zeek:id:: Control::configuration_update
:source-code: policy/frameworks/software/vulnerable.zeek 125 128
:Type: :zeek:type:`event` ()
This event is a wrapper and alias for the
:zeek:id:`Control::configuration_update_request` event.
This event is also a primary hooking point for the control framework.
.. zeek:id:: Control::configuration_update_request
:source-code: policy/frameworks/control/controllee.zeek 63 74
:Type: :zeek:type:`event` ()
Inform the remote Zeek instance that it's configuration may have been
updated.
.. zeek:id:: Control::configuration_update_response
:source-code: policy/frameworks/control/controller.zeek 45 48
:Type: :zeek:type:`event` ()
Message in response to a configuration update request.
.. zeek:id:: Control::id_value_request
:source-code: policy/frameworks/control/controllee.zeek 26 31
:Type: :zeek:type:`event` (id: :zeek:type:`string`)
Event for requesting the value of an ID (a variable).
.. zeek:id:: Control::id_value_response
:source-code: policy/frameworks/control/controller.zeek 30 33
:Type: :zeek:type:`event` (id: :zeek:type:`string`, val: :zeek:type:`string`)
Event for returning the value of an ID after an
:zeek:id:`Control::id_value_request` event.
.. zeek:id:: Control::net_stats_request
:source-code: policy/frameworks/control/controllee.zeek 54 61
:Type: :zeek:type:`event` ()
Requests the current net_stats.
.. zeek:id:: Control::net_stats_response
:source-code: policy/frameworks/control/controller.zeek 40 43
:Type: :zeek:type:`event` (s: :zeek:type:`string`)
Returns the current net_stats.
.. zeek:id:: Control::peer_status_request
:source-code: policy/frameworks/control/controllee.zeek 33 52
:Type: :zeek:type:`event` ()
Requests the current communication status.
.. zeek:id:: Control::peer_status_response
:source-code: policy/frameworks/control/controller.zeek 35 38
:Type: :zeek:type:`event` (s: :zeek:type:`string`)
Returns the current communication status.
.. zeek:id:: Control::shutdown_request
:source-code: policy/frameworks/control/controllee.zeek 76 83
:Type: :zeek:type:`event` ()
Requests that the Zeek instance begins shutting down.
.. zeek:id:: Control::shutdown_response
:source-code: policy/frameworks/control/controller.zeek 50 53
:Type: :zeek:type:`event` ()
Message in response to a shutdown request.

14
doc/scripts/base/frameworks/files/__load__.zeek.rst Normal file
View file

@ -0,0 +1,14 @@
:tocdepth: 3
base/frameworks/files/__load__.zeek
===================================
:Imports: :doc:`base/frameworks/files/magic </scripts/base/frameworks/files/magic/index>`, :doc:`base/frameworks/files/main.zeek </scripts/base/frameworks/files/main.zeek>`
Summary
~~~~~~~
Detailed Interface
~~~~~~~~~~~~~~~~~~

20
doc/scripts/base/frameworks/files/index.rst Normal file
View file

@ -0,0 +1,20 @@
:orphan:
Package: base/frameworks/files
==============================
The file analysis framework provides an interface for driving the analysis
of files, possibly independent of any network protocol over which they're
transported.
:doc:`/scripts/base/frameworks/files/__load__.zeek`
:doc:`/scripts/base/frameworks/files/main.zeek`
An interface for driving the analysis of files, possibly independent of
any network protocol over which they're transported.
:doc:`/scripts/base/frameworks/files/magic/__load__.zeek`

13
doc/scripts/base/frameworks/files/magic/__load__.zeek.rst Normal file
View file

@ -0,0 +1,13 @@
:tocdepth: 3
base/frameworks/files/magic/__load__.zeek
=========================================
Summary
~~~~~~~
Detailed Interface
~~~~~~~~~~~~~~~~~~

9
doc/scripts/base/frameworks/files/magic/index.rst Normal file
View file

@ -0,0 +1,9 @@
:orphan:
Package: base/frameworks/files/magic
====================================
:doc:`/scripts/base/frameworks/files/magic/__load__.zeek`

732
doc/scripts/base/frameworks/files/main.zeek.rst Normal file
View file

@ -0,0 +1,732 @@
:tocdepth: 3
base/frameworks/files/main.zeek
===============================
.. zeek:namespace:: Files
An interface for driving the analysis of files, possibly independent of
any network protocol over which they're transported.
:Namespace: Files
:Imports: :doc:`base/bif/file_analysis.bif.zeek </scripts/base/bif/file_analysis.bif.zeek>`, :doc:`base/frameworks/analyzer </scripts/base/frameworks/analyzer/index>`, :doc:`base/frameworks/logging </scripts/base/frameworks/logging/index>`, :doc:`base/utils/site.zeek </scripts/base/utils/site.zeek>`
Summary
~~~~~~~
Runtime Options
###############
=========================================================================== ========================================
:zeek:id:`Files::enable_reassembler`: :zeek:type:`bool` :zeek:attr:`&redef` The default setting for file reassembly.
=========================================================================== ========================================
Redefinable Options
###################
=========================================================================================== ================================================================
:zeek:id:`Files::analyze_by_mime_type_automatically`: :zeek:type:`bool` :zeek:attr:`&redef` Decide if you want to automatically attached analyzers to
files based on the detected mime type of the file.
:zeek:id:`Files::disable`: :zeek:type:`table` :zeek:attr:`&redef` A table that can be used to disable file analysis completely for
any files transferred over given network protocol analyzers.
:zeek:id:`Files::reassembly_buffer_size`: :zeek:type:`count` :zeek:attr:`&redef` The default per-file reassembly buffer size.
=========================================================================================== ================================================================
Types
#####
========================================================================= ==============================================================
:zeek:type:`Files::AnalyzerArgs`: :zeek:type:`record` :zeek:attr:`&redef` A structure which parameterizes a type of file analysis.
:zeek:type:`Files::Info`: :zeek:type:`record` :zeek:attr:`&redef` Contains all metadata related to the analysis of a given file.
:zeek:type:`Files::ProtoRegistration`: :zeek:type:`record`
========================================================================= ==============================================================
Redefinitions
#############
============================================================= =======================================================
:zeek:type:`Log::ID`: :zeek:type:`enum`
* :zeek:enum:`Files::LOG`:
Logging stream for file analysis.
:zeek:type:`fa_file`: :zeek:type:`record` :zeek:attr:`&redef`
:New Fields: :zeek:type:`fa_file`
info: :zeek:type:`Files::Info` :zeek:attr:`&optional`
============================================================= =======================================================
Events
######
=============================================== ====================================================================
:zeek:id:`Files::log_files`: :zeek:type:`event` Event that can be handled to access the Info record as it is sent on
to the logging framework.
=============================================== ====================================================================
Hooks
#####
========================================================== =============================================
:zeek:id:`Files::log_policy`: :zeek:type:`Log::PolicyHook` A default logging policy hook for the stream.
========================================================== =============================================
Functions
#########
======================================================================= =============================================================================
:zeek:id:`Files::add_analyzer`: :zeek:type:`function` Adds an analyzer to the analysis of a given file.
:zeek:id:`Files::all_registered_mime_types`: :zeek:type:`function` Returns a table of all MIME-type-to-analyzer mappings currently registered.
:zeek:id:`Files::analyzer_enabled`: :zeek:type:`function` Checks whether a file analyzer is generally enabled.
:zeek:id:`Files::analyzer_name`: :zeek:type:`function` Translates a file analyzer enum value to a string with the
analyzer's name.
:zeek:id:`Files::describe`: :zeek:type:`function` Provides a text description regarding metadata of the file.
:zeek:id:`Files::disable_analyzer`: :zeek:type:`function` Disables a file analyzer.
:zeek:id:`Files::disable_reassembly`: :zeek:type:`function` Disables the file reassembler on this file.
:zeek:id:`Files::enable_analyzer`: :zeek:type:`function` Enables a file analyzer.
:zeek:id:`Files::enable_reassembly`: :zeek:type:`function` Allows the file reassembler to be used if it's necessary because the
file is transferred out of order.
:zeek:id:`Files::file_exists`: :zeek:type:`function` Lookup to see if a particular file id exists and is still valid.
:zeek:id:`Files::lookup_file`: :zeek:type:`function` Lookup an :zeek:see:`fa_file` record with the file id.
:zeek:id:`Files::register_analyzer_add_callback`: :zeek:type:`function` Register a callback for file analyzers to use if they need to do some
manipulation when they are being added to a file before the core code
takes over.
:zeek:id:`Files::register_for_mime_type`: :zeek:type:`function` Registers a MIME type for an analyzer.
:zeek:id:`Files::register_for_mime_types`: :zeek:type:`function` Registers a set of MIME types for an analyzer.
:zeek:id:`Files::register_protocol`: :zeek:type:`function` Register callbacks for protocols that work with the Files framework.
:zeek:id:`Files::registered_mime_types`: :zeek:type:`function` Returns a set of all MIME types currently registered for a specific analyzer.
:zeek:id:`Files::remove_analyzer`: :zeek:type:`function` Removes an analyzer from the analysis of a given file.
:zeek:id:`Files::set_reassembly_buffer_size`: :zeek:type:`function` Set the maximum size the reassembly buffer is allowed to grow
for the given file.
:zeek:id:`Files::set_timeout_interval`: :zeek:type:`function` Sets the *timeout_interval* field of :zeek:see:`fa_file`, which is
used to determine the length of inactivity that is allowed for a file
before internal state related to it is cleaned up.
:zeek:id:`Files::stop`: :zeek:type:`function` Stops/ignores any further analysis of a given file.
======================================================================= =============================================================================
Detailed Interface
~~~~~~~~~~~~~~~~~~
Runtime Options
###############
.. zeek:id:: Files::enable_reassembler
:source-code: base/frameworks/files/main.zeek 127 127
:Type: :zeek:type:`bool`
:Attributes: :zeek:attr:`&redef`
:Default: ``T``
The default setting for file reassembly.
Redefinable Options
###################
.. zeek:id:: Files::analyze_by_mime_type_automatically
:source-code: base/frameworks/files/main.zeek 124 124
:Type: :zeek:type:`bool`
:Attributes: :zeek:attr:`&redef`
:Default: ``T``
Decide if you want to automatically attached analyzers to
files based on the detected mime type of the file.
.. zeek:id:: Files::disable
:source-code: base/frameworks/files/main.zeek 120 120
:Type: :zeek:type:`table` [:zeek:type:`Files::Tag`] of :zeek:type:`bool`
:Attributes: :zeek:attr:`&redef`
:Default: ``{}``
A table that can be used to disable file analysis completely for
any files transferred over given network protocol analyzers.
.. zeek:id:: Files::reassembly_buffer_size
:source-code: base/frameworks/files/main.zeek 130 130
:Type: :zeek:type:`count`
:Attributes: :zeek:attr:`&redef`
:Default: ``524288``
The default per-file reassembly buffer size.
Types
#####
.. zeek:type:: Files::AnalyzerArgs
:source-code: base/frameworks/files/main.zeek 21 32
:Type: :zeek:type:`record`
.. zeek:field:: chunk_event :zeek:type:`event` (f: :zeek:type:`fa_file`, data: :zeek:type:`string`, off: :zeek:type:`count`) :zeek:attr:`&optional`
An event which will be generated for all new file contents,
chunk-wise. Used when *tag* (in the
:zeek:see:`Files::add_analyzer` function) is
:zeek:see:`Files::ANALYZER_DATA_EVENT`.
.. zeek:field:: stream_event :zeek:type:`event` (f: :zeek:type:`fa_file`, data: :zeek:type:`string`) :zeek:attr:`&optional`
An event which will be generated for all new file contents,
stream-wise. Used when *tag* is
:zeek:see:`Files::ANALYZER_DATA_EVENT`.
.. zeek:field:: extract_filename :zeek:type:`string` :zeek:attr:`&optional`
(present if :doc:`/scripts/base/files/extract/main.zeek` is loaded)
The local filename to which to write an extracted file.
This field is used in the core by the extraction plugin
to know where to write the file to. If not specified, then
a filename in the format "extract-<source>-<id>" is
automatically assigned (using the *source* and *id*
fields of :zeek:see:`fa_file`).
.. zeek:field:: extract_limit :zeek:type:`count` :zeek:attr:`&default` = :zeek:see:`FileExtract::default_limit` :zeek:attr:`&optional`
(present if :doc:`/scripts/base/files/extract/main.zeek` is loaded)
The maximum allowed file size in bytes of *extract_filename*.
Once reached, a :zeek:see:`file_extraction_limit` event is
raised and the analyzer will be removed unless
:zeek:see:`FileExtract::set_limit` is called to increase the
limit. A value of zero means "no limit".
.. zeek:field:: extract_limit_includes_missing :zeek:type:`bool` :zeek:attr:`&default` = :zeek:see:`FileExtract::default_limit_includes_missing` :zeek:attr:`&optional`
(present if :doc:`/scripts/base/files/extract/main.zeek` is loaded)
By default, missing bytes in files count towards the extract file size.
Missing bytes can, e.g., occur due to missed traffic, or offsets
used when downloading files.
Setting this option to false changes this behavior so that holes
in files do no longer count towards these limits. Files with
holes are created as sparse files on disk. Their apparent size
can exceed this file size limit.
:Attributes: :zeek:attr:`&redef`
A structure which parameterizes a type of file analysis.
.. zeek:type:: Files::Info
:source-code: base/frameworks/files/main.zeek 37 116
:Type: :zeek:type:`record`
.. zeek:field:: ts :zeek:type:`time` :zeek:attr:`&log`
The time when the file was first seen.
.. zeek:field:: fuid :zeek:type:`string` :zeek:attr:`&log`
An identifier associated with a single file.
.. zeek:field:: uid :zeek:type:`string` :zeek:attr:`&log` :zeek:attr:`&optional`
If this file, or parts of it, were transferred over a
network connection, this is the uid for the connection.
.. zeek:field:: id :zeek:type:`conn_id` :zeek:attr:`&log` :zeek:attr:`&optional`
If this file, or parts of it, were transferred over a
network connection, this shows the connection.
.. zeek:field:: source :zeek:type:`string` :zeek:attr:`&log` :zeek:attr:`&optional`
An identification of the source of the file data. E.g. it
may be a network protocol over which it was transferred, or a
local file path which was read, or some other input source.
.. zeek:field:: depth :zeek:type:`count` :zeek:attr:`&default` = ``0`` :zeek:attr:`&optional` :zeek:attr:`&log`
A value to represent the depth of this file in relation
to its source. In SMTP, it is the depth of the MIME
attachment on the message. In HTTP, it is the depth of the
request within the TCP connection.
.. zeek:field:: analyzers :zeek:type:`set` [:zeek:type:`string`] :zeek:attr:`&default` = ``{ }`` :zeek:attr:`&optional` :zeek:attr:`&log`
A set of analysis types done during the file analysis.
.. zeek:field:: mime_type :zeek:type:`string` :zeek:attr:`&log` :zeek:attr:`&optional`
A mime type provided by the strongest file magic signature
match against the *bof_buffer* field of :zeek:see:`fa_file`,
or in the cases where no buffering of the beginning of file
occurs, an initial guess of the mime type based on the first
data seen.
.. zeek:field:: filename :zeek:type:`string` :zeek:attr:`&log` :zeek:attr:`&optional`
A filename for the file if one is available from the source
for the file. These will frequently come from
"Content-Disposition" headers in network protocols.
.. zeek:field:: duration :zeek:type:`interval` :zeek:attr:`&log` :zeek:attr:`&default` = ``0 secs`` :zeek:attr:`&optional`
The duration the file was analyzed for.
.. zeek:field:: local_orig :zeek:type:`bool` :zeek:attr:`&log` :zeek:attr:`&optional`
If the source of this file is a network connection, this field
indicates if the data originated from the local network or not as
determined by the configured :zeek:see:`Site::local_nets`.
.. zeek:field:: is_orig :zeek:type:`bool` :zeek:attr:`&log` :zeek:attr:`&optional`
If the source of this file is a network connection, this field
indicates if the file is being sent by the originator of the
connection or the responder.
.. zeek:field:: seen_bytes :zeek:type:`count` :zeek:attr:`&log` :zeek:attr:`&default` = ``0`` :zeek:attr:`&optional`
Number of bytes provided to the file analysis engine for the file.
The value refers to the total number of bytes processed for this
file across all connections seen by the current Zeek instance.
.. zeek:field:: total_bytes :zeek:type:`count` :zeek:attr:`&log` :zeek:attr:`&optional`
Total number of bytes that are supposed to comprise the full file.
.. zeek:field:: missing_bytes :zeek:type:`count` :zeek:attr:`&log` :zeek:attr:`&default` = ``0`` :zeek:attr:`&optional`
The number of bytes in the file stream that were completely missed
during the process of analysis e.g. due to dropped packets.
The value refers to number of bytes missed for this file
across all connections seen by the current Zeek instance.
.. zeek:field:: overflow_bytes :zeek:type:`count` :zeek:attr:`&log` :zeek:attr:`&default` = ``0`` :zeek:attr:`&optional`
The number of bytes in the file stream that were not delivered to
stream file analyzers. This could be overlapping bytes or
bytes that couldn't be reassembled.
.. zeek:field:: timedout :zeek:type:`bool` :zeek:attr:`&log` :zeek:attr:`&default` = ``F`` :zeek:attr:`&optional`
Whether the file analysis timed out at least once for the file.
.. zeek:field:: parent_fuid :zeek:type:`string` :zeek:attr:`&log` :zeek:attr:`&optional`
Identifier associated with a container file from which this one was
extracted as part of the file analysis.
.. zeek:field:: md5 :zeek:type:`string` :zeek:attr:`&log` :zeek:attr:`&optional`
(present if :doc:`/scripts/base/files/hash/main.zeek` is loaded)
An MD5 digest of the file contents.
.. zeek:field:: sha1 :zeek:type:`string` :zeek:attr:`&log` :zeek:attr:`&optional`
(present if :doc:`/scripts/base/files/hash/main.zeek` is loaded)
A SHA1 digest of the file contents.
.. zeek:field:: sha256 :zeek:type:`string` :zeek:attr:`&log` :zeek:attr:`&optional`
(present if :doc:`/scripts/base/files/hash/main.zeek` is loaded)
A SHA256 digest of the file contents.
.. zeek:field:: x509 :zeek:type:`X509::Info` :zeek:attr:`&optional`
(present if :doc:`/scripts/base/files/x509/main.zeek` is loaded)
Information about X509 certificates. This is used to keep
certificate information until all events have been received.
.. zeek:field:: extracted :zeek:type:`string` :zeek:attr:`&optional` :zeek:attr:`&log`
(present if :doc:`/scripts/base/files/extract/main.zeek` is loaded)
Local filename of extracted file.
.. zeek:field:: extracted_cutoff :zeek:type:`bool` :zeek:attr:`&optional` :zeek:attr:`&log`
(present if :doc:`/scripts/base/files/extract/main.zeek` is loaded)
Set to true if the file being extracted was cut off
so the whole file was not logged.
.. zeek:field:: extracted_size :zeek:type:`count` :zeek:attr:`&optional` :zeek:attr:`&log`
(present if :doc:`/scripts/base/files/extract/main.zeek` is loaded)
The number of bytes extracted to disk.
.. zeek:field:: entropy :zeek:type:`double` :zeek:attr:`&log` :zeek:attr:`&optional`
(present if :doc:`/scripts/policy/frameworks/files/entropy-test-all-files.zeek` is loaded)
The information density of the contents of the file,
expressed as a number of bits per character.
:Attributes: :zeek:attr:`&redef`
Contains all metadata related to the analysis of a given file.
For the most part, fields here are derived from ones of the same name
in :zeek:see:`fa_file`.
.. zeek:type:: Files::ProtoRegistration
:source-code: base/frameworks/files/main.zeek 255 265
:Type: :zeek:type:`record`
.. zeek:field:: get_file_handle :zeek:type:`function` (c: :zeek:type:`connection`, is_orig: :zeek:type:`bool`) : :zeek:type:`string`
A callback to generate a file handle on demand when
one is needed by the core.
.. zeek:field:: describe :zeek:type:`function` (f: :zeek:type:`fa_file`) : :zeek:type:`string` :zeek:attr:`&default` = :zeek:type:`function` :zeek:attr:`&optional`
A callback to "describe" a file. In the case of an HTTP
transfer the most obvious description would be the URL.
It's like an extremely compressed version of the normal log.
Events
######
.. zeek:id:: Files::log_files
:source-code: base/frameworks/files/main.zeek 326 326
:Type: :zeek:type:`event` (rec: :zeek:type:`Files::Info`)
Event that can be handled to access the Info record as it is sent on
to the logging framework.
Hooks
#####
.. zeek:id:: Files::log_policy
:source-code: base/files/x509/main.zeek 180 184
:Type: :zeek:type:`Log::PolicyHook`
A default logging policy hook for the stream.
Functions
#########
.. zeek:id:: Files::add_analyzer
:source-code: base/frameworks/files/main.zeek 415 431
:Type: :zeek:type:`function` (f: :zeek:type:`fa_file`, tag: :zeek:type:`Files::Tag`, args: :zeek:type:`Files::AnalyzerArgs` :zeek:attr:`&default` = *[chunk_event=<uninitialized>, stream_event=<uninitialized>, extract_filename=<uninitialized>, extract_limit=104857600, extract_limit_includes_missing=T]* :zeek:attr:`&optional`) : :zeek:type:`bool`
Adds an analyzer to the analysis of a given file.
:param f: the file.
:param tag: the analyzer type.
:param args: any parameters the analyzer takes.
:returns: true if the analyzer will be added, or false if analysis
for the file isn't currently active or the *args*
were invalid for the analyzer type.
.. zeek:id:: Files::all_registered_mime_types
:source-code: base/frameworks/files/main.zeek 495 498
:Type: :zeek:type:`function` () : :zeek:type:`table` [:zeek:type:`Files::Tag`] of :zeek:type:`set` [:zeek:type:`string`]
Returns a table of all MIME-type-to-analyzer mappings currently registered.
:returns: A table mapping each analyzer to the set of MIME types
registered for it.
.. zeek:id:: Files::analyzer_enabled
:source-code: base/frameworks/files/main.zeek 410 413
:Type: :zeek:type:`function` (tag: :zeek:type:`Files::Tag`) : :zeek:type:`bool`
Checks whether a file analyzer is generally enabled.
:param tag: the analyzer type to check.
:returns: true if the analyzer is generally enabled, else false.
.. zeek:id:: Files::analyzer_name
:source-code: base/frameworks/files/main.zeek 448 451
:Type: :zeek:type:`function` (tag: :zeek:type:`Files::Tag`) : :zeek:type:`string`
Translates a file analyzer enum value to a string with the
analyzer's name.
:param tag: The analyzer tag.
:returns: The analyzer name corresponding to the tag.
.. zeek:id:: Files::describe
:source-code: base/frameworks/files/main.zeek 500 511
:Type: :zeek:type:`function` (f: :zeek:type:`fa_file`) : :zeek:type:`string`
Provides a text description regarding metadata of the file.
For example, with HTTP it would return a URL.
:param f: The file to be described.
:returns: a text description regarding metadata of the file.
.. zeek:id:: Files::disable_analyzer
:source-code: base/frameworks/files/main.zeek 405 408
:Type: :zeek:type:`function` (tag: :zeek:type:`Files::Tag`) : :zeek:type:`bool`
Disables a file analyzer.
:param tag: the analyzer type to disable.
:returns: false if the analyzer tag could not be found, else true.
.. zeek:id:: Files::disable_reassembly
:source-code: base/frameworks/files/main.zeek 390 393
:Type: :zeek:type:`function` (f: :zeek:type:`fa_file`) : :zeek:type:`void`
Disables the file reassembler on this file. If the file is not
transferred out of order this will have no effect.
:param f: the file.
.. zeek:id:: Files::enable_analyzer
:source-code: base/frameworks/files/main.zeek 400 403
:Type: :zeek:type:`function` (tag: :zeek:type:`Files::Tag`) : :zeek:type:`bool`
Enables a file analyzer.
:param tag: the analyzer type to enable.
:returns: false if the analyzer tag could not be found, else true.
.. zeek:id:: Files::enable_reassembly
:source-code: base/frameworks/files/main.zeek 385 388
:Type: :zeek:type:`function` (f: :zeek:type:`fa_file`) : :zeek:type:`void`
Allows the file reassembler to be used if it's necessary because the
file is transferred out of order.
:param f: the file.
.. zeek:id:: Files::file_exists
:source-code: base/frameworks/files/main.zeek 370 373
:Type: :zeek:type:`function` (fuid: :zeek:type:`string`) : :zeek:type:`bool`
Lookup to see if a particular file id exists and is still valid.
:param fuid: the file id.
:returns: T if the file uid is known.
.. zeek:id:: Files::lookup_file
:source-code: base/frameworks/files/main.zeek 375 378
:Type: :zeek:type:`function` (fuid: :zeek:type:`string`) : :zeek:type:`fa_file`
Lookup an :zeek:see:`fa_file` record with the file id.
:param fuid: the file id.
:returns: the associated :zeek:see:`fa_file` record.
.. zeek:id:: Files::register_analyzer_add_callback
:source-code: base/frameworks/files/main.zeek 433 436
:Type: :zeek:type:`function` (tag: :zeek:type:`Files::Tag`, callback: :zeek:type:`function` (f: :zeek:type:`fa_file`, args: :zeek:type:`Files::AnalyzerArgs`) : :zeek:type:`void`) : :zeek:type:`void`
Register a callback for file analyzers to use if they need to do some
manipulation when they are being added to a file before the core code
takes over. This is unlikely to be interesting for users and should
only be called by file analyzer authors but is *not required*.
:param tag: Tag for the file analyzer.
:param callback: Function to execute when the given file analyzer is being added.
.. zeek:id:: Files::register_for_mime_type
:source-code: base/frameworks/files/main.zeek 473 488
:Type: :zeek:type:`function` (tag: :zeek:type:`Files::Tag`, mt: :zeek:type:`string`) : :zeek:type:`bool`
Registers a MIME type for an analyzer. If a future file with this type is seen,
the analyzer will be automatically assigned to parsing it. The function *adds*
to all MIME types already registered, it doesn't replace them.
:param tag: The tag of the analyzer.
:param mt: The MIME type in the form "foo/bar" (case-insensitive).
:returns: True if the MIME type was successfully registered.
.. zeek:id:: Files::register_for_mime_types
:source-code: base/frameworks/files/main.zeek 460 471
:Type: :zeek:type:`function` (tag: :zeek:type:`Files::Tag`, mime_types: :zeek:type:`set` [:zeek:type:`string`]) : :zeek:type:`bool`
Registers a set of MIME types for an analyzer. If a future connection on one of
these types is seen, the analyzer will be automatically assigned to parsing it.
The function *adds* to all MIME types already registered, it doesn't replace
them.
:param tag: The tag of the analyzer.
:param mts: The set of MIME types, each in the form "foo/bar" (case-insensitive).
:returns: True if the MIME types were successfully registered.
.. zeek:id:: Files::register_protocol
:source-code: base/frameworks/files/main.zeek 453 458
:Type: :zeek:type:`function` (tag: :zeek:type:`Analyzer::Tag`, reg: :zeek:type:`Files::ProtoRegistration`) : :zeek:type:`bool`
Register callbacks for protocols that work with the Files framework.
The callbacks must uniquely identify a file and each protocol can
only have a single callback registered for it.
:param tag: Tag for the protocol analyzer having a callback being registered.
:param reg: A :zeek:see:`Files::ProtoRegistration` record.
:returns: true if the protocol being registered was not previously registered.
.. zeek:id:: Files::registered_mime_types
:source-code: base/frameworks/files/main.zeek 490 493
:Type: :zeek:type:`function` (tag: :zeek:type:`Files::Tag`) : :zeek:type:`set` [:zeek:type:`string`]
Returns a set of all MIME types currently registered for a specific analyzer.
:param tag: The tag of the analyzer.
:returns: The set of MIME types.
.. zeek:id:: Files::remove_analyzer
:source-code: base/frameworks/files/main.zeek 438 441
:Type: :zeek:type:`function` (f: :zeek:type:`fa_file`, tag: :zeek:type:`Files::Tag`, args: :zeek:type:`Files::AnalyzerArgs` :zeek:attr:`&default` = *[chunk_event=<uninitialized>, stream_event=<uninitialized>, extract_filename=<uninitialized>, extract_limit=104857600, extract_limit_includes_missing=T]* :zeek:attr:`&optional`) : :zeek:type:`bool`
Removes an analyzer from the analysis of a given file.
:param f: the file.
:param tag: the analyzer type.
:param args: the analyzer (type and args) to remove.
:returns: true if the analyzer will be removed, or false if analysis
for the file isn't currently active.
.. zeek:id:: Files::set_reassembly_buffer_size
:source-code: base/frameworks/files/main.zeek 395 398
:Type: :zeek:type:`function` (f: :zeek:type:`fa_file`, max: :zeek:type:`count`) : :zeek:type:`void`
Set the maximum size the reassembly buffer is allowed to grow
for the given file.
:param f: the file.
:param max: Maximum allowed size of the reassembly buffer.
.. zeek:id:: Files::set_timeout_interval
:source-code: base/frameworks/files/main.zeek 380 383
:Type: :zeek:type:`function` (f: :zeek:type:`fa_file`, t: :zeek:type:`interval`) : :zeek:type:`bool`
Sets the *timeout_interval* field of :zeek:see:`fa_file`, which is
used to determine the length of inactivity that is allowed for a file
before internal state related to it is cleaned up. When used within
a :zeek:see:`file_timeout` handler, the analysis will delay timing out
again for the period specified by *t*.
:param f: the file.
:param t: the amount of time the file can remain inactive before discarding.
:returns: true if the timeout interval was set, or false if analysis
for the file isn't currently active.
.. zeek:id:: Files::stop
:source-code: base/frameworks/files/main.zeek 443 446
:Type: :zeek:type:`function` (f: :zeek:type:`fa_file`) : :zeek:type:`bool`
Stops/ignores any further analysis of a given file.
:param f: the file.
:returns: true if analysis for the given file will be ignored for the
rest of its contents, or false if analysis for the file
isn't currently active.

14
doc/scripts/base/frameworks/input/__load__.zeek.rst Normal file
View file

@ -0,0 +1,14 @@
:tocdepth: 3
base/frameworks/input/__load__.zeek
===================================
:Imports: :doc:`base/frameworks/input/main.zeek </scripts/base/frameworks/input/main.zeek>`, :doc:`base/frameworks/input/readers/ascii.zeek </scripts/base/frameworks/input/readers/ascii.zeek>`, :doc:`base/frameworks/input/readers/benchmark.zeek </scripts/base/frameworks/input/readers/benchmark.zeek>`, :doc:`base/frameworks/input/readers/binary.zeek </scripts/base/frameworks/input/readers/binary.zeek>`, :doc:`base/frameworks/input/readers/config.zeek </scripts/base/frameworks/input/readers/config.zeek>`, :doc:`base/frameworks/input/readers/raw.zeek </scripts/base/frameworks/input/readers/raw.zeek>`, :doc:`base/frameworks/input/readers/sqlite.zeek </scripts/base/frameworks/input/readers/sqlite.zeek>`
Summary
~~~~~~~
Detailed Interface
~~~~~~~~~~~~~~~~~~

50
doc/scripts/base/frameworks/input/index.rst Normal file
View file

@ -0,0 +1,50 @@
:orphan:
Package: base/frameworks/input
==============================
The input framework provides a way to read previously stored data either as
an event stream or into a Zeek table.
:doc:`/scripts/base/frameworks/input/__load__.zeek`
:doc:`/scripts/base/frameworks/input/main.zeek`
The input framework provides a way to read previously stored data either
as an event stream or into a Zeek table.
:doc:`/scripts/base/frameworks/input/readers/ascii.zeek`
Interface for the ascii input reader.
The defaults are set to match Zeek's ASCII output.
:doc:`/scripts/base/frameworks/input/readers/raw.zeek`
Interface for the raw input reader.
:doc:`/scripts/base/frameworks/input/readers/benchmark.zeek`
Interface for the benchmark input reader.
:doc:`/scripts/base/frameworks/input/readers/binary.zeek`
Interface for the binary input reader.
:doc:`/scripts/base/frameworks/input/readers/config.zeek`
Interface for the config input reader.
:doc:`/scripts/base/frameworks/input/readers/sqlite.zeek`
Interface for the SQLite input reader. Redefinable options are available
to tweak the input format of the SQLite reader.
See :doc:`/frameworks/logging-input-sqlite` for an introduction on how to
use the SQLite reader.
When using the SQLite reader, you have to specify the SQL query that returns
the desired data by setting ``query`` in the ``config`` table. See the
introduction mentioned above for an example.

492
doc/scripts/base/frameworks/input/main.zeek.rst Normal file
View file

@ -0,0 +1,492 @@
:tocdepth: 3
base/frameworks/input/main.zeek
===============================
.. zeek:namespace:: Input
The input framework provides a way to read previously stored data either
as an event stream or into a Zeek table.
:Namespace: Input
:Imports: :doc:`base/bif/input.bif.zeek </scripts/base/bif/input.bif.zeek>`
Summary
~~~~~~~
Runtime Options
###############
================================================================================ ==============================
:zeek:id:`Input::default_mode`: :zeek:type:`Input::Mode` :zeek:attr:`&redef` The default reader mode used.
:zeek:id:`Input::default_reader`: :zeek:type:`Input::Reader` :zeek:attr:`&redef` The default input reader used.
================================================================================ ==============================
Redefinable Options
###################
================================================================================= =========================================================
:zeek:id:`Input::accept_unsupported_types`: :zeek:type:`bool` :zeek:attr:`&redef` Flag that controls if the input framework accepts records
that contain types that are not supported (at the moment
file and function).
:zeek:id:`Input::empty_field`: :zeek:type:`string` :zeek:attr:`&redef` String to use for empty fields.
:zeek:id:`Input::separator`: :zeek:type:`string` :zeek:attr:`&redef` Separator between fields.
:zeek:id:`Input::set_separator`: :zeek:type:`string` :zeek:attr:`&redef` Separator between set elements.
:zeek:id:`Input::unset_field`: :zeek:type:`string` :zeek:attr:`&redef` String to use for an unset &optional field.
================================================================================= =========================================================
Types
#####
============================================================ ===================================================================
:zeek:type:`Input::AnalysisDescription`: :zeek:type:`record` A file analysis input stream type used to forward input data to the
file analysis framework.
:zeek:type:`Input::Event`: :zeek:type:`enum` Type that describes what kind of change occurred.
:zeek:type:`Input::EventDescription`: :zeek:type:`record` An event input stream type used to send input data to a Zeek event.
:zeek:type:`Input::Mode`: :zeek:type:`enum` Type that defines the input stream read mode.
:zeek:type:`Input::TableDescription`: :zeek:type:`record` A table input stream type used to send data to a Zeek table.
:zeek:type:`Input::Reader`: :zeek:type:`enum`
============================================================ ===================================================================
Events
######
================================================= ====================================================================
:zeek:id:`Input::end_of_data`: :zeek:type:`event` Event that is called when the end of a data source has been reached,
including after an update.
================================================= ====================================================================
Functions
#########
===================================================== ============================================================
:zeek:id:`Input::add_analysis`: :zeek:type:`function` Create a new file analysis input stream from a given source.
:zeek:id:`Input::add_event`: :zeek:type:`function` Create a new event input stream from a given source.
:zeek:id:`Input::add_table`: :zeek:type:`function` Create a new table input stream from a given source.
:zeek:id:`Input::force_update`: :zeek:type:`function` Forces the current input to be checked for changes.
:zeek:id:`Input::remove`: :zeek:type:`function` Remove an input stream.
===================================================== ============================================================
Detailed Interface
~~~~~~~~~~~~~~~~~~
Runtime Options
###############
.. zeek:id:: Input::default_mode
:source-code: base/frameworks/input/main.zeek 31 31
:Type: :zeek:type:`Input::Mode`
:Attributes: :zeek:attr:`&redef`
:Default: ``Input::MANUAL``
The default reader mode used. Defaults to :zeek:see:`Input::MANUAL`.
.. zeek:id:: Input::default_reader
:source-code: base/frameworks/input/main.zeek 28 28
:Type: :zeek:type:`Input::Reader`
:Attributes: :zeek:attr:`&redef`
:Default: ``Input::READER_ASCII``
The default input reader used. Defaults to :zeek:see:`Input::READER_ASCII`.
Redefinable Options
###################
.. zeek:id:: Input::accept_unsupported_types
:source-code: base/frameworks/input/main.zeek 56 56
:Type: :zeek:type:`bool`
:Attributes: :zeek:attr:`&redef`
:Default: ``F``
Flag that controls if the input framework accepts records
that contain types that are not supported (at the moment
file and function). If true, the input framework will
warn in these cases, but continue. If false, it will
abort. Defaults to false (abort).
.. zeek:id:: Input::empty_field
:source-code: base/frameworks/input/main.zeek 45 45
:Type: :zeek:type:`string`
:Attributes: :zeek:attr:`&redef`
:Default: ``"(empty)"``
String to use for empty fields.
Individual readers can use a different value.
.. zeek:id:: Input::separator
:source-code: base/frameworks/input/main.zeek 36 36
:Type: :zeek:type:`string`
:Attributes: :zeek:attr:`&redef`
:Default: ``"\x09"``
Separator between fields.
Please note that the separator has to be exactly one character long.
Individual readers can use a different value.
.. zeek:id:: Input::set_separator
:source-code: base/frameworks/input/main.zeek 41 41
:Type: :zeek:type:`string`
:Attributes: :zeek:attr:`&redef`
:Default: ``","``
Separator between set elements.
Please note that the separator has to be exactly one character long.
Individual readers can use a different value.
.. zeek:id:: Input::unset_field
:source-code: base/frameworks/input/main.zeek 49 49
:Type: :zeek:type:`string`
:Attributes: :zeek:attr:`&redef`
:Default: ``"-"``
String to use for an unset &optional field.
Individual readers can use a different value.
Types
#####
.. zeek:type:: Input::AnalysisDescription
:source-code: base/frameworks/input/main.zeek 180 204
:Type: :zeek:type:`record`
.. zeek:field:: source :zeek:type:`string`
String that allows the reader to find the source.
For :zeek:see:`Input::READER_ASCII`, this is the filename.
.. zeek:field:: reader :zeek:type:`Input::Reader` :zeek:attr:`&default` = ``Input::READER_BINARY`` :zeek:attr:`&optional`
Reader to use for this stream. Compatible readers must be
able to accept a filter of a single string type (i.e.
they read a byte stream).
.. zeek:field:: mode :zeek:type:`Input::Mode` :zeek:attr:`&default` = :zeek:see:`Input::default_mode` :zeek:attr:`&optional`
Read mode to use for this stream.
.. zeek:field:: name :zeek:type:`string`
Descriptive name that uniquely identifies the input source.
Can be used to remove a stream at a later time.
This will also be used for the unique *source* field of
:zeek:see:`fa_file`. Most of the time, the best choice for this
field will be the same value as the *source* field.
.. zeek:field:: config :zeek:type:`table` [:zeek:type:`string`] of :zeek:type:`string` :zeek:attr:`&default` = ``{ }`` :zeek:attr:`&optional`
A key/value table that will be passed to the reader.
Interpretation of the values is left to the reader, but
usually they will be used for configuration purposes.
A file analysis input stream type used to forward input data to the
file analysis framework.
.. zeek:type:: Input::Event
:source-code: base/frameworks/input/main.zeek 8 8
:Type: :zeek:type:`enum`
.. zeek:enum:: Input::EVENT_NEW Input::Event
New data has been imported.
.. zeek:enum:: Input::EVENT_CHANGED Input::Event
Existing data has been changed.
.. zeek:enum:: Input::EVENT_REMOVED Input::Event
Previously existing data has been removed.
Type that describes what kind of change occurred.
.. zeek:type:: Input::EventDescription
:source-code: base/frameworks/input/main.zeek 125 176
:Type: :zeek:type:`record`
.. zeek:field:: source :zeek:type:`string`
String that allows the reader to find the source.
For :zeek:see:`Input::READER_ASCII`, this is the filename.
.. zeek:field:: reader :zeek:type:`Input::Reader` :zeek:attr:`&default` = :zeek:see:`Input::default_reader` :zeek:attr:`&optional`
Reader to use for this stream.
.. zeek:field:: mode :zeek:type:`Input::Mode` :zeek:attr:`&default` = :zeek:see:`Input::default_mode` :zeek:attr:`&optional`
Read mode to use for this stream.
.. zeek:field:: name :zeek:type:`string`
Descriptive name. Used to remove a stream at a later time.
.. zeek:field:: fields :zeek:type:`any`
Record type describing the fields to be retrieved from the input
source.
.. zeek:field:: want_record :zeek:type:`bool` :zeek:attr:`&default` = ``T`` :zeek:attr:`&optional`
If this is false, the event receives each value in *fields* as a
separate argument.
If this is set to true (default), the event receives all fields in
a single record value.
.. zeek:field:: ev :zeek:type:`any`
The event that is raised each time a new line is received from the
reader. The event will receive an :zeek:see:`Input::EventDescription` record
as the first argument, an :zeek:see:`Input::Event` enum as the second
argument, and the fields (as specified in *fields*) as the following
arguments (this will either be a single record value containing
all fields, or each field value as a separate argument).
.. zeek:field:: error_ev :zeek:type:`any` :zeek:attr:`&optional`
Error event that is raised when an information, warning or error
is raised by the input stream. If the level is error, the stream will automatically
be closed.
The event receives the :zeek:see:`Input::EventDescription` as the first argument, the
message as the second argument and the :zeek:see:`Reporter::Level` as the third argument.
The event is raised like it had been declared as follows:
error_ev: function(desc: EventDescription, message: string, level: Reporter::Level) &optional;
The actual declaration uses the :zeek:type:`any` type because of deficiencies of the Zeek type system.
.. zeek:field:: config :zeek:type:`table` [:zeek:type:`string`] of :zeek:type:`string` :zeek:attr:`&default` = ``{ }`` :zeek:attr:`&optional`
A key/value table that will be passed to the reader.
Interpretation of the values is left to the reader, but
usually they will be used for configuration purposes.
An event input stream type used to send input data to a Zeek event.
.. zeek:type:: Input::Mode
:source-code: base/frameworks/input/main.zeek 18 26
:Type: :zeek:type:`enum`
.. zeek:enum:: Input::MANUAL Input::Mode
Do not automatically reread the file after it has been read.
.. zeek:enum:: Input::REREAD Input::Mode
Reread the entire file each time a change is found.
.. zeek:enum:: Input::STREAM Input::Mode
Read data from end of file each time new data is appended.
Type that defines the input stream read mode.
.. zeek:type:: Input::TableDescription
:source-code: base/frameworks/input/main.zeek 59 122
:Type: :zeek:type:`record`
.. zeek:field:: source :zeek:type:`string`
String that allows the reader to find the source of the data.
For :zeek:see:`Input::READER_ASCII`, this is the filename.
.. zeek:field:: reader :zeek:type:`Input::Reader` :zeek:attr:`&default` = :zeek:see:`Input::default_reader` :zeek:attr:`&optional`
Reader to use for this stream.
.. zeek:field:: mode :zeek:type:`Input::Mode` :zeek:attr:`&default` = :zeek:see:`Input::default_mode` :zeek:attr:`&optional`
Read mode to use for this stream.
.. zeek:field:: name :zeek:type:`string`
Name of the input stream. This is used by some functions to
manipulate the stream.
.. zeek:field:: destination :zeek:type:`any`
Table which will receive the data read by the input framework.
.. zeek:field:: idx :zeek:type:`any`
Record that defines the values used as the index of the table.
.. zeek:field:: val :zeek:type:`any` :zeek:attr:`&optional`
Record that defines the values used as the elements of the table.
If this is undefined, then *destination* must be a set.
.. zeek:field:: want_record :zeek:type:`bool` :zeek:attr:`&default` = ``T`` :zeek:attr:`&optional`
Defines if the value of the table is a record (default), or a single
value. When this is set to false, then *val* can only contain one
element.
.. zeek:field:: ev :zeek:type:`any` :zeek:attr:`&optional`
The event that is raised each time a value is added to, changed in,
or removed from the table. The event will receive an
Input::TableDescription as the first argument, an Input::Event
enum as the second argument, the *idx* record as the third argument
and the value (record) as the fourth argument.
.. zeek:field:: pred :zeek:type:`function` (typ: :zeek:type:`Input::Event`, left: :zeek:type:`any`, right: :zeek:type:`any`) : :zeek:type:`bool` :zeek:attr:`&optional`
Predicate function that can decide if an insertion, update or removal
should really be executed. Parameters have same meaning as for the
event.
If true is returned, the update is performed. If false is returned,
it is skipped.
.. zeek:field:: error_ev :zeek:type:`any` :zeek:attr:`&optional`
Error event that is raised when an information, warning or error
is raised by the input stream. If the level is error, the stream will automatically
be closed.
The event receives the Input::TableDescription as the first argument, the
message as the second argument and the Reporter::Level as the third argument.
The event is raised like if it had been declared as follows:
error_ev: function(desc: TableDescription, message: string, level: Reporter::Level) &optional;
The actual declaration uses the :zeek:type:`any` type because of deficiencies of the Zeek type system.
.. zeek:field:: config :zeek:type:`table` [:zeek:type:`string`] of :zeek:type:`string` :zeek:attr:`&default` = ``{ }`` :zeek:attr:`&optional`
A key/value table that will be passed to the reader.
Interpretation of the values is left to the reader, but
usually they will be used for configuration purposes.
A table input stream type used to send data to a Zeek table.
.. zeek:type:: Input::Reader
:Type: :zeek:type:`enum`
.. zeek:enum:: Input::READER_ASCII Input::Reader
.. zeek:enum:: Input::READER_BENCHMARK Input::Reader
.. zeek:enum:: Input::READER_BINARY Input::Reader
.. zeek:enum:: Input::READER_CONFIG Input::Reader
.. zeek:enum:: Input::READER_RAW Input::Reader
.. zeek:enum:: Input::READER_SQLITE Input::Reader
Events
######
.. zeek:id:: Input::end_of_data
:source-code: base/utils/exec.zeek 96 127
:Type: :zeek:type:`event` (name: :zeek:type:`string`, source: :zeek:type:`string`)
Event that is called when the end of a data source has been reached,
including after an update.
:param name: Name of the input stream.
:param source: String that identifies the data source (such as the filename).
Functions
#########
.. zeek:id:: Input::add_analysis
:source-code: base/frameworks/input/main.zeek 267 270
:Type: :zeek:type:`function` (description: :zeek:type:`Input::AnalysisDescription`) : :zeek:type:`bool`
Create a new file analysis input stream from a given source. Data read
from the source is automatically forwarded to the file analysis
framework.
:param description: A record describing the source.
:returns: true on success.
.. zeek:id:: Input::add_event
:source-code: base/frameworks/input/main.zeek 262 265
:Type: :zeek:type:`function` (description: :zeek:type:`Input::EventDescription`) : :zeek:type:`bool`
Create a new event input stream from a given source.
:param description: :zeek:see:`Input::EventDescription` record describing the source.
:returns: true on success.
.. zeek:id:: Input::add_table
:source-code: base/frameworks/input/main.zeek 257 260
:Type: :zeek:type:`function` (description: :zeek:type:`Input::TableDescription`) : :zeek:type:`bool`
Create a new table input stream from a given source.
:param description: :zeek:see:`Input::TableDescription` record describing the source.
:returns: true on success.
.. zeek:id:: Input::force_update
:source-code: base/frameworks/input/main.zeek 277 280
:Type: :zeek:type:`function` (id: :zeek:type:`string`) : :zeek:type:`bool`
Forces the current input to be checked for changes.
:param id: string value identifying the stream.
:returns: true on success and false if the named stream was not found.
.. zeek:id:: Input::remove
:source-code: base/frameworks/input/main.zeek 272 275
:Type: :zeek:type:`function` (id: :zeek:type:`string`) : :zeek:type:`bool`
Remove an input stream.
:param id: string value identifying the stream to be removed.
:returns: true on success and false if the named stream was not found.

124
doc/scripts/base/frameworks/input/readers/ascii.zeek.rst Normal file
View file

@ -0,0 +1,124 @@
:tocdepth: 3
base/frameworks/input/readers/ascii.zeek
========================================
.. zeek:namespace:: InputAscii
Interface for the ascii input reader.
The defaults are set to match Zeek's ASCII output.
:Namespace: InputAscii
Summary
~~~~~~~
Redefinable Options
###################
=================================================================================== ==================================================================
:zeek:id:`InputAscii::empty_field`: :zeek:type:`string` :zeek:attr:`&redef` String to use for empty fields.
:zeek:id:`InputAscii::fail_on_file_problem`: :zeek:type:`bool` :zeek:attr:`&redef` Fail on file read problems.
:zeek:id:`InputAscii::fail_on_invalid_lines`: :zeek:type:`bool` :zeek:attr:`&redef` Fail on invalid lines.
:zeek:id:`InputAscii::path_prefix`: :zeek:type:`string` :zeek:attr:`&redef` On input streams with a pathless or relative-path source filename,
prefix the following path.
:zeek:id:`InputAscii::separator`: :zeek:type:`string` :zeek:attr:`&redef` Separator between fields.
:zeek:id:`InputAscii::set_separator`: :zeek:type:`string` :zeek:attr:`&redef` Separator between set and vector elements.
:zeek:id:`InputAscii::unset_field`: :zeek:type:`string` :zeek:attr:`&redef` String to use for an unset &optional field.
=================================================================================== ==================================================================
Detailed Interface
~~~~~~~~~~~~~~~~~~
Redefinable Options
###################
.. zeek:id:: InputAscii::empty_field
:source-code: base/frameworks/input/readers/ascii.zeek 17 17
:Type: :zeek:type:`string`
:Attributes: :zeek:attr:`&redef`
:Default: ``"(empty)"``
String to use for empty fields.
.. zeek:id:: InputAscii::fail_on_file_problem
:source-code: base/frameworks/input/readers/ascii.zeek 49 49
:Type: :zeek:type:`bool`
:Attributes: :zeek:attr:`&redef`
:Default: ``F``
Fail on file read problems. If set to true, the ascii
input reader will fail when encountering any problems
while reading a file different from invalid lines.
Examples of such problems are permission problems, or
missing files.
When set to false, these problems will be ignored. This
has an especially big effect for the REREAD mode, which will
seamlessly recover from read errors when a file is
only temporarily inaccessible. For MANUAL or STREAM files,
errors will most likely still be fatal since no automatic
re-reading of the file is attempted.
Individual readers can use a different value using
the $config table.
fail_on_file_problem = T was the default behavior
until Bro 2.6.
.. zeek:id:: InputAscii::fail_on_invalid_lines
:source-code: base/frameworks/input/readers/ascii.zeek 32 32
:Type: :zeek:type:`bool`
:Attributes: :zeek:attr:`&redef`
:Default: ``F``
Fail on invalid lines. If set to false, the ascii
input reader will jump over invalid lines, reporting
warnings in reporter.log. If set to true, errors in
input lines will be handled as fatal errors for the
reader thread; reading will abort immediately and
an error will be logged to reporter.log.
Individual readers can use a different value using
the $config table.
fail_on_invalid_lines = T was the default behavior
until Bro 2.6.
.. zeek:id:: InputAscii::path_prefix
:source-code: base/frameworks/input/readers/ascii.zeek 55 55
:Type: :zeek:type:`string`
:Attributes: :zeek:attr:`&redef`
:Default: ``""``
On input streams with a pathless or relative-path source filename,
prefix the following path. This prefix can, but need not be, absolute.
The default is to leave any filenames unchanged. This prefix has no
effect if the source already is an absolute path.
.. zeek:id:: InputAscii::separator
:source-code: base/frameworks/input/readers/ascii.zeek 10 10
:Type: :zeek:type:`string`
:Attributes: :zeek:attr:`&redef`
:Default: ``"\x09"``
Separator between fields.
Please note that the separator has to be exactly one character long.
.. zeek:id:: InputAscii::set_separator
:source-code: base/frameworks/input/readers/ascii.zeek 14 14
:Type: :zeek:type:`string`
:Attributes: :zeek:attr:`&redef`
:Default: ``","``
Separator between set and vector elements.
Please note that the separator has to be exactly one character long.
.. zeek:id:: InputAscii::unset_field
:source-code: base/frameworks/input/readers/ascii.zeek 20 20
:Type: :zeek:type:`string`
:Attributes: :zeek:attr:`&redef`
:Default: ``"-"``
String to use for an unset &optional field.

83
doc/scripts/base/frameworks/input/readers/benchmark.zeek.rst Normal file
View file

@ -0,0 +1,83 @@
:tocdepth: 3
base/frameworks/input/readers/benchmark.zeek
============================================
.. zeek:namespace:: InputBenchmark
Interface for the benchmark input reader.
:Namespace: InputBenchmark
Summary
~~~~~~~
Redefinable Options
###################
=============================================================================== =========================================================
:zeek:id:`InputBenchmark::addfactor`: :zeek:type:`count` :zeek:attr:`&redef` Addition factor for each heartbeat.
:zeek:id:`InputBenchmark::autospread`: :zeek:type:`double` :zeek:attr:`&redef` Spreading where usleep = 1000000 / autospread * num_lines
:zeek:id:`InputBenchmark::factor`: :zeek:type:`double` :zeek:attr:`&redef` Multiplication factor for each second.
:zeek:id:`InputBenchmark::spread`: :zeek:type:`count` :zeek:attr:`&redef` Spread factor between lines.
:zeek:id:`InputBenchmark::stopspreadat`: :zeek:type:`count` :zeek:attr:`&redef` Stop spreading at x lines per heartbeat.
:zeek:id:`InputBenchmark::timedspread`: :zeek:type:`double` :zeek:attr:`&redef` 1 -> enable timed spreading.
=============================================================================== =========================================================
Detailed Interface
~~~~~~~~~~~~~~~~~~
Redefinable Options
###################
.. zeek:id:: InputBenchmark::addfactor
:source-code: base/frameworks/input/readers/benchmark.zeek 16 16
:Type: :zeek:type:`count`
:Attributes: :zeek:attr:`&redef`
:Default: ``0``
Addition factor for each heartbeat.
.. zeek:id:: InputBenchmark::autospread
:source-code: base/frameworks/input/readers/benchmark.zeek 13 13
:Type: :zeek:type:`double`
:Attributes: :zeek:attr:`&redef`
:Default: ``0.0``
Spreading where usleep = 1000000 / autospread * num_lines
.. zeek:id:: InputBenchmark::factor
:source-code: base/frameworks/input/readers/benchmark.zeek 7 7
:Type: :zeek:type:`double`
:Attributes: :zeek:attr:`&redef`
:Default: ``1.0``
Multiplication factor for each second.
.. zeek:id:: InputBenchmark::spread
:source-code: base/frameworks/input/readers/benchmark.zeek 10 10
:Type: :zeek:type:`count`
:Attributes: :zeek:attr:`&redef`
:Default: ``0``
Spread factor between lines.
.. zeek:id:: InputBenchmark::stopspreadat
:source-code: base/frameworks/input/readers/benchmark.zeek 19 19
:Type: :zeek:type:`count`
:Attributes: :zeek:attr:`&redef`
:Default: ``0``
Stop spreading at x lines per heartbeat.
.. zeek:id:: InputBenchmark::timedspread
:source-code: base/frameworks/input/readers/benchmark.zeek 22 22
:Type: :zeek:type:`double`
:Attributes: :zeek:attr:`&redef`
:Default: ``0.0``
1 -> enable timed spreading.

47
doc/scripts/base/frameworks/input/readers/binary.zeek.rst Normal file
View file

@ -0,0 +1,47 @@
:tocdepth: 3
base/frameworks/input/readers/binary.zeek
=========================================
.. zeek:namespace:: InputBinary
Interface for the binary input reader.
:Namespace: InputBinary
Summary
~~~~~~~
Redefinable Options
###################
============================================================================ ==================================================================
:zeek:id:`InputBinary::chunk_size`: :zeek:type:`count` :zeek:attr:`&redef` Size of data chunks to read from the input file at a time.
:zeek:id:`InputBinary::path_prefix`: :zeek:type:`string` :zeek:attr:`&redef` On input streams with a pathless or relative-path source filename,
prefix the following path.
============================================================================ ==================================================================
Detailed Interface
~~~~~~~~~~~~~~~~~~
Redefinable Options
###################
.. zeek:id:: InputBinary::chunk_size
:source-code: base/frameworks/input/readers/binary.zeek 7 7
:Type: :zeek:type:`count`
:Attributes: :zeek:attr:`&redef`
:Default: ``1024``
Size of data chunks to read from the input file at a time.
.. zeek:id:: InputBinary::path_prefix
:source-code: base/frameworks/input/readers/binary.zeek 13 13
:Type: :zeek:type:`string`
:Attributes: :zeek:attr:`&redef`
:Default: ``""``
On input streams with a pathless or relative-path source filename,
prefix the following path. This prefix can, but need not be, absolute.
The default is to leave any filenames unchanged. This prefix has no
effect if the source already is an absolute path.

99
doc/scripts/base/frameworks/input/readers/config.zeek.rst Normal file
View file

@ -0,0 +1,99 @@
:tocdepth: 3
base/frameworks/input/readers/config.zeek
=========================================
.. zeek:namespace:: InputConfig
Interface for the config input reader.
:Namespace: InputConfig
Summary
~~~~~~~
Redefinable Options
###################
=================================================================================== ==========================================
:zeek:id:`InputConfig::empty_field`: :zeek:type:`string` :zeek:attr:`&redef` String to use for empty fields.
:zeek:id:`InputConfig::fail_on_file_problem`: :zeek:type:`bool` :zeek:attr:`&redef` Fail on file read problems.
:zeek:id:`InputConfig::set_separator`: :zeek:type:`string` :zeek:attr:`&redef` Separator between set and vector elements.
=================================================================================== ==========================================
Events
######
===================================================== ==============================================================
:zeek:id:`InputConfig::new_value`: :zeek:type:`event` Event that is called when a config option is added or changes.
===================================================== ==============================================================
Detailed Interface
~~~~~~~~~~~~~~~~~~
Redefinable Options
###################
.. zeek:id:: InputConfig::empty_field
:source-code: base/frameworks/input/readers/config.zeek 13 13
:Type: :zeek:type:`string`
:Attributes: :zeek:attr:`&redef`
:Default: ``""``
String to use for empty fields.
By default this is the empty string, meaning that an empty input field
will result in an empty set.
.. zeek:id:: InputConfig::fail_on_file_problem
:source-code: base/frameworks/input/readers/config.zeek 28 28
:Type: :zeek:type:`bool`
:Attributes: :zeek:attr:`&redef`
:Default: ``F``
Fail on file read problems. If set to true, the config
input reader will fail when encountering any problems
while reading a file different from invalid lines.
Examples of such problems are permission problems, or
missing files.
When set to false, these problems will be ignored. This
has an especially big effect for the REREAD mode, which will
seamlessly recover from read errors when a file is
only temporarily inaccessible. For MANUAL or STREAM files,
errors will most likely still be fatal since no automatic
re-reading of the file is attempted.
Individual readers can use a different value using
the $config table.
.. zeek:id:: InputConfig::set_separator
:source-code: base/frameworks/input/readers/config.zeek 8 8
:Type: :zeek:type:`string`
:Attributes: :zeek:attr:`&redef`
:Default: ``","``
Separator between set and vector elements.
Please note that the separator has to be exactly one character long.
Events
######
.. zeek:id:: InputConfig::new_value
:source-code: base/frameworks/config/input.zeek 53 59
:Type: :zeek:type:`event` (name: :zeek:type:`string`, source: :zeek:type:`string`, id: :zeek:type:`string`, value: :zeek:type:`any`)
Event that is called when a config option is added or changes.
Note - this does not track the reason for a change (new, changed),
and also does not track removals. If you need this, combine the event
with a table reader.
:param name: Name of the input stream.
:param source: Source of the input stream.
:param id: ID of the configuration option being set.
:param value: New value of the configuration option being set.

60
doc/scripts/base/frameworks/input/readers/raw.zeek.rst Normal file
View file

@ -0,0 +1,60 @@
:tocdepth: 3
base/frameworks/input/readers/raw.zeek
======================================
.. zeek:namespace:: InputRaw
Interface for the raw input reader.
:Namespace: InputRaw
Summary
~~~~~~~
Redefinable Options
###################
============================================================================== ================================
:zeek:id:`InputRaw::record_separator`: :zeek:type:`string` :zeek:attr:`&redef` Separator between input records.
============================================================================== ================================
Events
######
========================================================= ====================================================================
:zeek:id:`InputRaw::process_finished`: :zeek:type:`event` Event that is called when a process created by the raw reader exits.
========================================================= ====================================================================
Detailed Interface
~~~~~~~~~~~~~~~~~~
Redefinable Options
###################
.. zeek:id:: InputRaw::record_separator
:source-code: base/frameworks/input/readers/raw.zeek 8 8
:Type: :zeek:type:`string`
:Attributes: :zeek:attr:`&redef`
:Default: ``"\x0a"``
Separator between input records.
Please note that the separator has to be exactly one character long.
Events
######
.. zeek:id:: InputRaw::process_finished
:source-code: base/utils/exec.zeek 129 151
:Type: :zeek:type:`event` (name: :zeek:type:`string`, source: :zeek:type:`string`, exit_code: :zeek:type:`count`, signal_exit: :zeek:type:`bool`)
Event that is called when a process created by the raw reader exits.
:param name: name of the input stream.
:param source: source of the input stream.
:param exit_code: exit code of the program, or number of the signal that forced
the program to exit.
:param signal_exit: false when program exited normally, true when program was
forced to exit by a signal.

62
doc/scripts/base/frameworks/input/readers/sqlite.zeek.rst Normal file
View file

@ -0,0 +1,62 @@
:tocdepth: 3
base/frameworks/input/readers/sqlite.zeek
=========================================
.. zeek:namespace:: InputSQLite
Interface for the SQLite input reader. Redefinable options are available
to tweak the input format of the SQLite reader.
See :doc:`/frameworks/logging-input-sqlite` for an introduction on how to
use the SQLite reader.
When using the SQLite reader, you have to specify the SQL query that returns
the desired data by setting ``query`` in the ``config`` table. See the
introduction mentioned above for an example.
:Namespace: InputSQLite
Summary
~~~~~~~
Redefinable Options
###################
============================================================================== ===========================================
:zeek:id:`InputSQLite::empty_field`: :zeek:type:`string` :zeek:attr:`&redef` String to use for empty fields.
:zeek:id:`InputSQLite::set_separator`: :zeek:type:`string` :zeek:attr:`&redef` Separator between set elements.
:zeek:id:`InputSQLite::unset_field`: :zeek:type:`string` :zeek:attr:`&redef` String to use for an unset &optional field.
============================================================================== ===========================================
Detailed Interface
~~~~~~~~~~~~~~~~~~
Redefinable Options
###################
.. zeek:id:: InputSQLite::empty_field
:source-code: base/frameworks/input/readers/sqlite.zeek 22 22
:Type: :zeek:type:`string`
:Attributes: :zeek:attr:`&redef`
:Default: ``"(empty)"``
String to use for empty fields.
.. zeek:id:: InputSQLite::set_separator
:source-code: base/frameworks/input/readers/sqlite.zeek 16 16
:Type: :zeek:type:`string`
:Attributes: :zeek:attr:`&redef`
:Default: ``","``
Separator between set elements.
Please note that the separator has to be exactly one character long.
.. zeek:id:: InputSQLite::unset_field
:source-code: base/frameworks/input/readers/sqlite.zeek 19 19
:Type: :zeek:type:`string`
:Attributes: :zeek:attr:`&redef`
:Default: ``"-"``
String to use for an unset &optional field.

14
doc/scripts/base/frameworks/intel/__load__.zeek.rst Normal file
View file

@ -0,0 +1,14 @@
:tocdepth: 3
base/frameworks/intel/__load__.zeek
===================================
:Imports: :doc:`base/frameworks/cluster </scripts/base/frameworks/cluster/index>`, :doc:`base/frameworks/intel/files.zeek </scripts/base/frameworks/intel/files.zeek>`, :doc:`base/frameworks/intel/input.zeek </scripts/base/frameworks/intel/input.zeek>`, :doc:`base/frameworks/intel/main.zeek </scripts/base/frameworks/intel/main.zeek>`
Summary
~~~~~~~
Detailed Interface
~~~~~~~~~~~~~~~~~~

55
doc/scripts/base/frameworks/intel/files.zeek.rst Normal file
View file

@ -0,0 +1,55 @@
:tocdepth: 3
base/frameworks/intel/files.zeek
================================
.. zeek:namespace:: Intel
File analysis framework integration for the intelligence framework. This
script manages file information in intelligence framework data structures.
:Namespace: Intel
:Imports: :doc:`base/frameworks/intel/main.zeek </scripts/base/frameworks/intel/main.zeek>`
Summary
~~~~~~~
Redefinitions
#############
============================================= ==============================================================================
:zeek:type:`Intel::Info`: :zeek:type:`record` Record used for the logging framework representing a positive
hit within the intelligence framework.
:New Fields: :zeek:type:`Intel::Info`
fuid: :zeek:type:`string` :zeek:attr:`&log` :zeek:attr:`&optional`
If a file was associated with this intelligence hit,
this is the uid for the file.
file_mime_type: :zeek:type:`string` :zeek:attr:`&log` :zeek:attr:`&optional`
A mime type if the intelligence hit is related to a file.
file_desc: :zeek:type:`string` :zeek:attr:`&log` :zeek:attr:`&optional`
Frequently files can be "described" to give a bit more context.
:zeek:type:`Intel::Seen`: :zeek:type:`record` Information about a piece of "seen" data.
:New Fields: :zeek:type:`Intel::Seen`
f: :zeek:type:`fa_file` :zeek:attr:`&optional`
If the data was discovered within a file, the file record
should go here to provide context to the data.
fuid: :zeek:type:`string` :zeek:attr:`&optional`
If the data was discovered within a file, the file uid should
go here to provide context to the data.
:zeek:type:`Intel::Type`: :zeek:type:`enum` Enum type to represent various types of intelligence data.
* :zeek:enum:`Intel::FILE_HASH`:
File hash which is non-hash type specific.
* :zeek:enum:`Intel::FILE_NAME`:
File name.
============================================= ==============================================================================
Detailed Interface
~~~~~~~~~~~~~~~~~~

29
doc/scripts/base/frameworks/intel/index.rst Normal file
View file

@ -0,0 +1,29 @@
:orphan:
Package: base/frameworks/intel
==============================
The intelligence framework provides a way to store and query intelligence
data (such as IP addresses or strings). Metadata can also be associated
with the intelligence.
:doc:`/scripts/base/frameworks/intel/__load__.zeek`
:doc:`/scripts/base/frameworks/intel/main.zeek`
The intelligence framework provides a way to store and query intelligence
data (e.g. IP addresses, URLs and hashes). The intelligence items can be
associated with metadata to allow informed decisions about matching and
handling.
:doc:`/scripts/base/frameworks/intel/files.zeek`
File analysis framework integration for the intelligence framework. This
script manages file information in intelligence framework data structures.
:doc:`/scripts/base/frameworks/intel/input.zeek`
Input handling for the intelligence framework. This script implements the
import of intelligence data from files using the input framework.

103
doc/scripts/base/frameworks/intel/input.zeek.rst Normal file
View file

@ -0,0 +1,103 @@
:tocdepth: 3
base/frameworks/intel/input.zeek
================================
.. zeek:namespace:: Intel
Input handling for the intelligence framework. This script implements the
import of intelligence data from files using the input framework.
:Namespace: Intel
:Imports: :doc:`base/frameworks/intel/main.zeek </scripts/base/frameworks/intel/main.zeek>`
Summary
~~~~~~~
Redefinable Options
###################
====================================================================== ==============================================
:zeek:id:`Intel::path_prefix`: :zeek:type:`string` :zeek:attr:`&redef` An optional path prefix for intel files.
:zeek:id:`Intel::read_files`: :zeek:type:`set` :zeek:attr:`&redef` Intelligence files that will be read off disk.
====================================================================== ==============================================
Events
######
================================================ ===================================================================
:zeek:id:`Intel::read_entry`: :zeek:type:`event` This event is raised each time the intel framework reads a new line
from an intel file.
:zeek:id:`Intel::read_error`: :zeek:type:`event` This event is raised each time the input framework detects an error
while reading the intel file.
================================================ ===================================================================
Detailed Interface
~~~~~~~~~~~~~~~~~~
Redefinable Options
###################
.. zeek:id:: Intel::path_prefix
:source-code: base/frameworks/intel/input.zeek 22 22
:Type: :zeek:type:`string`
:Attributes: :zeek:attr:`&redef`
:Default: ``""``
An optional path prefix for intel files. This prefix can, but
need not be, absolute. The default is to leave any filenames
unchanged. This prefix has no effect if a read_file entry is
an absolute path. This prefix gets applied _before_ entering
the input framework, so if the prefix is absolute, the input
framework won't munge it further. If it is relative, then
any path_prefix specified in the input framework will apply
additionally.
.. zeek:id:: Intel::read_files
:source-code: base/frameworks/intel/input.zeek 12 12
:Type: :zeek:type:`set` [:zeek:type:`string`]
:Attributes: :zeek:attr:`&redef`
:Default: ``{}``
Intelligence files that will be read off disk. The files are
reread every time they are updated so updates must be atomic
with "mv" instead of writing the file in place.
Events
######
.. zeek:id:: Intel::read_entry
:source-code: base/frameworks/intel/input.zeek 49 52
:Type: :zeek:type:`event` (desc: :zeek:type:`Input::EventDescription`, tpe: :zeek:type:`Input::Event`, item: :zeek:type:`Intel::Item`)
This event is raised each time the intel framework reads a new line
from an intel file. It is used in the intel framework but can
also be used in custom scripts for further checks.
:param desc: The :zeek:type:`Input::EventDescription` record which generated the event.
:param tpe: The type of input event.
:param item: The intel item being read (of type :zeek:type:`Intel::Item`).
.. zeek:id:: Intel::read_error
:source-code: base/frameworks/intel/input.zeek 46 46
:Type: :zeek:type:`event` (desc: :zeek:type:`Input::EventDescription`, message: :zeek:type:`string`, level: :zeek:type:`Reporter::Level`)
This event is raised each time the input framework detects an error
while reading the intel file. It can be used to implement further checks
in custom scripts. Errors can be of different levels (information, warning, errors).
:param desc: The :zeek:type:`Input::EventDescription` record which generated the error.
:param message: An error message.
:param level: The :zeek:type:`Reporter::Level` of the error.

754
doc/scripts/base/frameworks/intel/main.zeek.rst Normal file
View file

@ -0,0 +1,754 @@
:tocdepth: 3
base/frameworks/intel/main.zeek
===============================
.. zeek:namespace:: Intel
The intelligence framework provides a way to store and query intelligence
data (e.g. IP addresses, URLs and hashes). The intelligence items can be
associated with metadata to allow informed decisions about matching and
handling.
:Namespace: Intel
:Imports: :doc:`base/frameworks/notice </scripts/base/frameworks/notice/index>`
Summary
~~~~~~~
Redefinable Options
###################
============================================================================ ==============================================
:zeek:id:`Intel::item_expiration`: :zeek:type:`interval` :zeek:attr:`&redef` The expiration timeout for intelligence items.
============================================================================ ==============================================
Types
#####
================================================= ==============================================================
:zeek:type:`Intel::Info`: :zeek:type:`record` Record used for the logging framework representing a positive
hit within the intelligence framework.
:zeek:type:`Intel::Item`: :zeek:type:`record` Represents a piece of intelligence.
:zeek:type:`Intel::MetaData`: :zeek:type:`record` Data about an :zeek:type:`Intel::Item`.
:zeek:type:`Intel::Seen`: :zeek:type:`record` Information about a piece of "seen" data.
:zeek:type:`Intel::Type`: :zeek:type:`enum` Enum type to represent various types of intelligence data.
:zeek:type:`Intel::TypeSet`: :zeek:type:`set` Set of intelligence data types.
:zeek:type:`Intel::Where`: :zeek:type:`enum` Enum to represent where data came from when it was discovered.
================================================= ==============================================================
Redefinitions
#############
======================================= =========================
:zeek:type:`Log::ID`: :zeek:type:`enum`
* :zeek:enum:`Intel::LOG`
======================================= =========================
Events
######
=============================================== ==================================================================
:zeek:id:`Intel::log_intel`: :zeek:type:`event`
:zeek:id:`Intel::match`: :zeek:type:`event` Event to represent a match in the intelligence data from data that
was seen.
=============================================== ==================================================================
Hooks
#####
========================================================== =======================================================================
:zeek:id:`Intel::extend_match`: :zeek:type:`hook` This hook can be used to influence the logging of intelligence hits
(e.g.
:zeek:id:`Intel::filter_item`: :zeek:type:`hook` This hook can be used to filter intelligence items that are about to be
inserted into the internal data store.
:zeek:id:`Intel::indicator_inserted`: :zeek:type:`hook` This hook is invoked when a new indicator has been inserted into
the min data store for the first time.
:zeek:id:`Intel::indicator_removed`: :zeek:type:`hook` This hook is invoked when an indicator has been removed from
the min data store.
:zeek:id:`Intel::item_expired`: :zeek:type:`hook` This hook can be used to handle expiration of intelligence items.
:zeek:id:`Intel::log_policy`: :zeek:type:`Log::PolicyHook`
:zeek:id:`Intel::seen_policy`: :zeek:type:`hook` Hook to modify and intercept :zeek:see:`Intel::seen` behavior.
========================================================== =======================================================================
Functions
#########
=============================================== ==================================================================
:zeek:id:`Intel::insert`: :zeek:type:`function` Function to insert intelligence data.
:zeek:id:`Intel::remove`: :zeek:type:`function` Function to remove intelligence data.
:zeek:id:`Intel::seen`: :zeek:type:`function` Function to declare discovery of a piece of data in order to check
it against known intelligence for matches.
=============================================== ==================================================================
Detailed Interface
~~~~~~~~~~~~~~~~~~
Redefinable Options
###################
.. zeek:id:: Intel::item_expiration
:source-code: base/frameworks/intel/main.zeek 187 187
:Type: :zeek:type:`interval`
:Attributes: :zeek:attr:`&redef`
:Default: ``-1.0 min``
:Redefinition: from :doc:`/scripts/policy/frameworks/intel/do_expire.zeek`
``=``::
10.0 mins
The expiration timeout for intelligence items. Once an item expires, the
:zeek:id:`Intel::item_expired` hook is called. Reinsertion of an item
resets the timeout. A negative value disables expiration of intelligence
items.
Types
#####
.. zeek:type:: Intel::Info
:source-code: base/frameworks/intel/main.zeek 104 121
:Type: :zeek:type:`record`
.. zeek:field:: ts :zeek:type:`time` :zeek:attr:`&log`
Timestamp when the data was discovered.
.. zeek:field:: uid :zeek:type:`string` :zeek:attr:`&log` :zeek:attr:`&optional`
If a connection was associated with this intelligence hit,
this is the uid for the connection
.. zeek:field:: id :zeek:type:`conn_id` :zeek:attr:`&log` :zeek:attr:`&optional`
If a connection was associated with this intelligence hit,
this is the conn_id for the connection.
.. zeek:field:: seen :zeek:type:`Intel::Seen` :zeek:attr:`&log`
Where the data was seen.
.. zeek:field:: matched :zeek:type:`Intel::TypeSet` :zeek:attr:`&log`
Which indicator types matched.
.. zeek:field:: sources :zeek:type:`set` [:zeek:type:`string`] :zeek:attr:`&log` :zeek:attr:`&default` = ``{ }`` :zeek:attr:`&optional`
Sources which supplied data that resulted in this match.
.. zeek:field:: fuid :zeek:type:`string` :zeek:attr:`&log` :zeek:attr:`&optional`
(present if :doc:`/scripts/base/frameworks/intel/files.zeek` is loaded)
If a file was associated with this intelligence hit,
this is the uid for the file.
.. zeek:field:: file_mime_type :zeek:type:`string` :zeek:attr:`&log` :zeek:attr:`&optional`
(present if :doc:`/scripts/base/frameworks/intel/files.zeek` is loaded)
A mime type if the intelligence hit is related to a file.
If the $f field is provided this will be automatically filled
out.
.. zeek:field:: file_desc :zeek:type:`string` :zeek:attr:`&log` :zeek:attr:`&optional`
(present if :doc:`/scripts/base/frameworks/intel/files.zeek` is loaded)
Frequently files can be "described" to give a bit more context.
If the $f field is provided this field will be automatically
filled out.
.. zeek:field:: cif :zeek:type:`Intel::CIF` :zeek:attr:`&log` :zeek:attr:`&optional`
(present if :doc:`/scripts/policy/integration/collective-intel/main.zeek` is loaded)
Record used for the logging framework representing a positive
hit within the intelligence framework.
.. zeek:type:: Intel::Item
:source-code: base/frameworks/intel/main.zeek 54 64
:Type: :zeek:type:`record`
.. zeek:field:: indicator :zeek:type:`string`
The intelligence indicator.
.. zeek:field:: indicator_type :zeek:type:`Intel::Type`
The type of data that the indicator field represents.
.. zeek:field:: meta :zeek:type:`Intel::MetaData`
Metadata for the item. Typically represents more deeply
descriptive data for a piece of intelligence.
Represents a piece of intelligence.
.. zeek:type:: Intel::MetaData
:source-code: base/frameworks/intel/main.zeek 42 51
:Type: :zeek:type:`record`
.. zeek:field:: source :zeek:type:`string`
An arbitrary string value representing the data source. This
value is used as unique key to identify a metadata record in
the scope of a single intelligence item.
.. zeek:field:: desc :zeek:type:`string` :zeek:attr:`&optional`
A freeform description for the data.
.. zeek:field:: url :zeek:type:`string` :zeek:attr:`&optional`
A URL for more information about the data.
.. zeek:field:: do_notice :zeek:type:`bool` :zeek:attr:`&default` = ``F`` :zeek:attr:`&optional`
(present if :doc:`/scripts/policy/frameworks/intel/do_notice.zeek` is loaded)
A boolean value to allow the data itself to represent
if the indicator that this metadata is attached to
is notice worthy.
.. zeek:field:: if_in :zeek:type:`Intel::Where` :zeek:attr:`&optional`
(present if :doc:`/scripts/policy/frameworks/intel/do_notice.zeek` is loaded)
Restrictions on when notices are created to only create
them if the *do_notice* field is T and the notice was
seen in the indicated location.
.. zeek:field:: whitelist :zeek:type:`bool` :zeek:attr:`&default` = ``F`` :zeek:attr:`&optional`
(present if :doc:`/scripts/policy/frameworks/intel/whitelist.zeek` is loaded)
A boolean value to indicate whether the item is whitelisted.
.. zeek:field:: remove :zeek:type:`bool` :zeek:attr:`&default` = ``F`` :zeek:attr:`&optional`
(present if :doc:`/scripts/policy/frameworks/intel/removal.zeek` is loaded)
A boolean value to indicate whether the item should be removed.
.. zeek:field:: cif_tags :zeek:type:`string` :zeek:attr:`&optional`
(present if :doc:`/scripts/policy/integration/collective-intel/main.zeek` is loaded)
Maps to the 'tags' fields in CIF
.. zeek:field:: cif_confidence :zeek:type:`double` :zeek:attr:`&optional`
(present if :doc:`/scripts/policy/integration/collective-intel/main.zeek` is loaded)
Maps to the 'confidence' field in CIF
.. zeek:field:: cif_source :zeek:type:`string` :zeek:attr:`&optional`
(present if :doc:`/scripts/policy/integration/collective-intel/main.zeek` is loaded)
Maps to the 'source' field in CIF
.. zeek:field:: cif_description :zeek:type:`string` :zeek:attr:`&optional`
(present if :doc:`/scripts/policy/integration/collective-intel/main.zeek` is loaded)
Maps to the 'description' field in CIF
.. zeek:field:: cif_firstseen :zeek:type:`string` :zeek:attr:`&optional`
(present if :doc:`/scripts/policy/integration/collective-intel/main.zeek` is loaded)
Maps to the 'firstseen' field in CIF
.. zeek:field:: cif_lastseen :zeek:type:`string` :zeek:attr:`&optional`
(present if :doc:`/scripts/policy/integration/collective-intel/main.zeek` is loaded)
Maps to the 'lastseen' field in CIF
Data about an :zeek:type:`Intel::Item`.
.. zeek:type:: Intel::Seen
:source-code: base/frameworks/intel/main.zeek 74 100
:Type: :zeek:type:`record`
.. zeek:field:: indicator :zeek:type:`string` :zeek:attr:`&log` :zeek:attr:`&optional`
The string if the data is about a string.
.. zeek:field:: indicator_type :zeek:type:`Intel::Type` :zeek:attr:`&log` :zeek:attr:`&optional`
The type of data that the indicator represents.
.. zeek:field:: host :zeek:type:`addr` :zeek:attr:`&optional`
If the indicator type was :zeek:enum:`Intel::ADDR`, then this
field will be present.
.. zeek:field:: where :zeek:type:`Intel::Where` :zeek:attr:`&log`
Where the data was discovered.
.. zeek:field:: node :zeek:type:`string` :zeek:attr:`&optional` :zeek:attr:`&log`
The name of the node where the match was discovered.
.. zeek:field:: conn :zeek:type:`connection` :zeek:attr:`&optional`
If the data was discovered within a connection, the
connection record should go here to give context to the data.
.. zeek:field:: uid :zeek:type:`string` :zeek:attr:`&optional`
If the data was discovered within a connection, the
connection uid should go here to give context to the data.
If the *conn* field is provided, this will be automatically
filled out.
.. zeek:field:: f :zeek:type:`fa_file` :zeek:attr:`&optional`
(present if :doc:`/scripts/base/frameworks/intel/files.zeek` is loaded)
If the data was discovered within a file, the file record
should go here to provide context to the data.
.. zeek:field:: fuid :zeek:type:`string` :zeek:attr:`&optional`
(present if :doc:`/scripts/base/frameworks/intel/files.zeek` is loaded)
If the data was discovered within a file, the file uid should
go here to provide context to the data. If the file record *f*
is provided, this will be automatically filled out.
Information about a piece of "seen" data.
.. zeek:type:: Intel::Type
:source-code: base/frameworks/intel/main.zeek 16 37
:Type: :zeek:type:`enum`
.. zeek:enum:: Intel::ADDR Intel::Type
An IP address.
.. zeek:enum:: Intel::SUBNET Intel::Type
A subnet in CIDR notation.
.. zeek:enum:: Intel::URL Intel::Type
A complete URL without the prefix ``"http://"``.
.. zeek:enum:: Intel::SOFTWARE Intel::Type
Software name.
.. zeek:enum:: Intel::EMAIL Intel::Type
Email address.
.. zeek:enum:: Intel::DOMAIN Intel::Type
DNS domain name.
.. zeek:enum:: Intel::USER_NAME Intel::Type
A user name.
.. zeek:enum:: Intel::CERT_HASH Intel::Type
Certificate SHA-1 hash.
.. zeek:enum:: Intel::PUBKEY_HASH Intel::Type
Public key MD5 hash, formatted as hexadecimal digits delimited by colons.
(SSH server host keys are a good example.)
.. zeek:enum:: Intel::FILE_HASH Intel::Type
(present if :doc:`/scripts/base/frameworks/intel/files.zeek` is loaded)
File hash which is non-hash type specific. It's up to the
user to query for any relevant hash types.
.. zeek:enum:: Intel::FILE_NAME Intel::Type
(present if :doc:`/scripts/base/frameworks/intel/files.zeek` is loaded)
File name. Typically with protocols with definite
indications of a file name.
Enum type to represent various types of intelligence data.
.. zeek:type:: Intel::TypeSet
:source-code: base/frameworks/intel/main.zeek 39 39
:Type: :zeek:type:`set` [:zeek:type:`Intel::Type`]
Set of intelligence data types.
.. zeek:type:: Intel::Where
:source-code: base/frameworks/intel/main.zeek 68 72
:Type: :zeek:type:`enum`
.. zeek:enum:: Intel::IN_ANYWHERE Intel::Where
A catchall value to represent data of unknown provenance.
.. zeek:enum:: Conn::IN_ORIG Intel::Where
(present if :doc:`/scripts/policy/frameworks/intel/seen/where-locations.zeek` is loaded)
.. zeek:enum:: Conn::IN_RESP Intel::Where
(present if :doc:`/scripts/policy/frameworks/intel/seen/where-locations.zeek` is loaded)
.. zeek:enum:: Files::IN_HASH Intel::Where
(present if :doc:`/scripts/policy/frameworks/intel/seen/where-locations.zeek` is loaded)
.. zeek:enum:: Files::IN_NAME Intel::Where
(present if :doc:`/scripts/policy/frameworks/intel/seen/where-locations.zeek` is loaded)
.. zeek:enum:: DNS::IN_REQUEST Intel::Where
(present if :doc:`/scripts/policy/frameworks/intel/seen/where-locations.zeek` is loaded)
.. zeek:enum:: DNS::IN_RESPONSE Intel::Where
(present if :doc:`/scripts/policy/frameworks/intel/seen/where-locations.zeek` is loaded)
.. zeek:enum:: HTTP::IN_HOST_HEADER Intel::Where
(present if :doc:`/scripts/policy/frameworks/intel/seen/where-locations.zeek` is loaded)
.. zeek:enum:: HTTP::IN_REFERRER_HEADER Intel::Where
(present if :doc:`/scripts/policy/frameworks/intel/seen/where-locations.zeek` is loaded)
.. zeek:enum:: HTTP::IN_USER_AGENT_HEADER Intel::Where
(present if :doc:`/scripts/policy/frameworks/intel/seen/where-locations.zeek` is loaded)
.. zeek:enum:: HTTP::IN_X_FORWARDED_FOR_HEADER Intel::Where
(present if :doc:`/scripts/policy/frameworks/intel/seen/where-locations.zeek` is loaded)
.. zeek:enum:: HTTP::IN_URL Intel::Where
(present if :doc:`/scripts/policy/frameworks/intel/seen/where-locations.zeek` is loaded)
.. zeek:enum:: SMTP::IN_MAIL_FROM Intel::Where
(present if :doc:`/scripts/policy/frameworks/intel/seen/where-locations.zeek` is loaded)
.. zeek:enum:: SMTP::IN_RCPT_TO Intel::Where
(present if :doc:`/scripts/policy/frameworks/intel/seen/where-locations.zeek` is loaded)
.. zeek:enum:: SMTP::IN_FROM Intel::Where
(present if :doc:`/scripts/policy/frameworks/intel/seen/where-locations.zeek` is loaded)
.. zeek:enum:: SMTP::IN_TO Intel::Where
(present if :doc:`/scripts/policy/frameworks/intel/seen/where-locations.zeek` is loaded)
.. zeek:enum:: SMTP::IN_CC Intel::Where
(present if :doc:`/scripts/policy/frameworks/intel/seen/where-locations.zeek` is loaded)
.. zeek:enum:: SMTP::IN_RECEIVED_HEADER Intel::Where
(present if :doc:`/scripts/policy/frameworks/intel/seen/where-locations.zeek` is loaded)
.. zeek:enum:: SMTP::IN_REPLY_TO Intel::Where
(present if :doc:`/scripts/policy/frameworks/intel/seen/where-locations.zeek` is loaded)
.. zeek:enum:: SMTP::IN_X_ORIGINATING_IP_HEADER Intel::Where
(present if :doc:`/scripts/policy/frameworks/intel/seen/where-locations.zeek` is loaded)
.. zeek:enum:: SMTP::IN_MESSAGE Intel::Where
(present if :doc:`/scripts/policy/frameworks/intel/seen/where-locations.zeek` is loaded)
.. zeek:enum:: SSH::IN_SERVER_HOST_KEY Intel::Where
(present if :doc:`/scripts/policy/frameworks/intel/seen/where-locations.zeek` is loaded)
.. zeek:enum:: SSL::IN_SERVER_NAME Intel::Where
(present if :doc:`/scripts/policy/frameworks/intel/seen/where-locations.zeek` is loaded)
.. zeek:enum:: SMTP::IN_HEADER Intel::Where
(present if :doc:`/scripts/policy/frameworks/intel/seen/where-locations.zeek` is loaded)
.. zeek:enum:: X509::IN_CERT Intel::Where
(present if :doc:`/scripts/policy/frameworks/intel/seen/where-locations.zeek` is loaded)
.. zeek:enum:: SMB::IN_FILE_NAME Intel::Where
(present if :doc:`/scripts/policy/frameworks/intel/seen/where-locations.zeek` is loaded)
.. zeek:enum:: SSH::SUCCESSFUL_LOGIN Intel::Where
(present if :doc:`/scripts/policy/protocols/ssh/detect-bruteforcing.zeek` is loaded)
An indicator of the login for the intel framework.
Enum to represent where data came from when it was discovered.
The convention is to prefix the name with ``IN_``.
Events
######
.. zeek:id:: Intel::log_intel
:source-code: base/frameworks/intel/main.zeek 239 239
:Type: :zeek:type:`event` (rec: :zeek:type:`Intel::Info`)
.. zeek:id:: Intel::match
:source-code: base/frameworks/intel/main.zeek 146 146
:Type: :zeek:type:`event` (s: :zeek:type:`Intel::Seen`, items: :zeek:type:`set` [:zeek:type:`Intel::Item`])
Event to represent a match in the intelligence data from data that
was seen. On clusters there is no assurance as to when this event
will be generated so do not assume that arbitrary global state beyond
the given data will be available.
This is the primary mechanism where a user may take actions based on
data provided by the intelligence framework.
.. zeek::see:: Intel::seen_policy
Hooks
#####
.. zeek:id:: Intel::extend_match
:source-code: base/frameworks/intel/main.zeek 160 160
:Type: :zeek:type:`hook` (info: :zeek:type:`Intel::Info`, s: :zeek:type:`Intel::Seen`, items: :zeek:type:`set` [:zeek:type:`Intel::Item`]) : :zeek:type:`bool`
This hook can be used to influence the logging of intelligence hits
(e.g. by adding data to the Info record). The default information is
added with a priority of 5.
:param info: The Info record that will be logged.
:param s: Information about the data seen.
:param items: The intel items that match the seen data.
In case the hook execution is terminated using break, the match will
not be logged.
.. zeek:id:: Intel::filter_item
:source-code: policy/frameworks/intel/removal.zeek 14 22
:Type: :zeek:type:`hook` (item: :zeek:type:`Intel::Item`) : :zeek:type:`bool`
This hook can be used to filter intelligence items that are about to be
inserted into the internal data store. In case the hook execution is
terminated using break, the item will not be (re)added to the internal
data store.
:param item: The intel item that should be inserted.
.. zeek:id:: Intel::indicator_inserted
:source-code: policy/frameworks/intel/seen/manage-event-groups.zeek 42 57
:Type: :zeek:type:`hook` (indicator: :zeek:type:`string`, indiator_type: :zeek:type:`Intel::Type`) : :zeek:type:`bool`
This hook is invoked when a new indicator has been inserted into
the min data store for the first time.
Calls to :zeek:see:`Intel::seen` with a matching indicator value
and type will result in matches.
Subsequent inserts of the same indicator type and value do not
invoke this hook. Breaking from this hook has no effect.
:param indicator: The indicator value.
:param indicator_type: The indicator type.
.. zeek::see:: Intel::indicator_removed
.. zeek:id:: Intel::indicator_removed
:source-code: policy/frameworks/intel/seen/manage-event-groups.zeek 59 74
:Type: :zeek:type:`hook` (indicator: :zeek:type:`string`, indiator_type: :zeek:type:`Intel::Type`) : :zeek:type:`bool`
This hook is invoked when an indicator has been removed from
the min data store.
After this hooks runs, :zeek:see:`Intel::seen` for the indicator
will not return any matches. Breaking from this hook has no effect.
:param indicator: The indicator value.
:param indicator_type: The indicator type.
.. zeek::see:: Intel::indicator_inserted
.. zeek:id:: Intel::item_expired
:source-code: policy/frameworks/intel/do_expire.zeek 10 14
:Type: :zeek:type:`hook` (indicator: :zeek:type:`string`, indicator_type: :zeek:type:`Intel::Type`, metas: :zeek:type:`set` [:zeek:type:`Intel::MetaData`]) : :zeek:type:`bool`
This hook can be used to handle expiration of intelligence items.
:param indicator: The indicator of the expired item.
:param indicator_type: The indicator type of the expired item.
:param metas: The set of metadata describing the expired item.
If all hook handlers are executed, the expiration timeout will be reset.
Otherwise, if one of the handlers terminates using break, the item will
be removed.
.. zeek:id:: Intel::log_policy
:source-code: base/frameworks/intel/main.zeek 13 13
:Type: :zeek:type:`Log::PolicyHook`
.. zeek:id:: Intel::seen_policy
:source-code: base/frameworks/intel/main.zeek 181 181
:Type: :zeek:type:`hook` (s: :zeek:type:`Intel::Seen`, found: :zeek:type:`bool`) : :zeek:type:`bool`
Hook to modify and intercept :zeek:see:`Intel::seen` behavior.
This hook is invoked after the Intel datastore was searched for
a given :zeek:see:`Intel::Seen` instance. If a matching entry was
found, the *found* argument is set to ``T``, else ``F``.
Breaking from this hook suppresses :zeek:see:`Intel::match`
event generation and any subsequent logging.
Note that this hook only runs on the Zeek node where :zeek:see:`Intel::seen`
is invoked. In a cluster configuration that is usually on the worker nodes.
This is in contrast to :zeek:see:`Intel::match` that usually runs
centrally on the the manager node instead.
:param s: The :zeek:see:`Intel::Seen` instance passed to the :zeek:see:`Intel::seen` function.
:param found: ``T`` if Intel datastore contained *s*, else ``F``.
.. zeek::see:: Intel::match
Functions
#########
.. zeek:id:: Intel::insert
:source-code: base/frameworks/intel/main.zeek 596 603
:Type: :zeek:type:`function` (item: :zeek:type:`Intel::Item`) : :zeek:type:`void`
Function to insert intelligence data. If the indicator is already
present, the associated metadata will be added to the indicator. If
the indicator already contains a metadata record from the same source,
the existing metadata record will be updated.
.. zeek:id:: Intel::remove
:source-code: base/frameworks/intel/main.zeek 649 688
:Type: :zeek:type:`function` (item: :zeek:type:`Intel::Item`, purge_indicator: :zeek:type:`bool` :zeek:attr:`&default` = ``F`` :zeek:attr:`&optional`) : :zeek:type:`void`
Function to remove intelligence data. If purge_indicator is set, the
given metadata is ignored and the indicator is removed completely.
.. zeek:id:: Intel::seen
:source-code: base/frameworks/intel/main.zeek 405 433
:Type: :zeek:type:`function` (s: :zeek:type:`Intel::Seen`) : :zeek:type:`void`
Function to declare discovery of a piece of data in order to check
it against known intelligence for matches.

14
doc/scripts/base/frameworks/logging/__load__.zeek.rst Normal file
View file

@ -0,0 +1,14 @@
:tocdepth: 3
base/frameworks/logging/__load__.zeek
=====================================
:Imports: :doc:`base/frameworks/logging/main.zeek </scripts/base/frameworks/logging/main.zeek>`, :doc:`base/frameworks/logging/postprocessors </scripts/base/frameworks/logging/postprocessors/index>`, :doc:`base/frameworks/logging/writers/ascii.zeek </scripts/base/frameworks/logging/writers/ascii.zeek>`, :doc:`base/frameworks/logging/writers/none.zeek </scripts/base/frameworks/logging/writers/none.zeek>`, :doc:`base/frameworks/logging/writers/sqlite.zeek </scripts/base/frameworks/logging/writers/sqlite.zeek>`
Summary
~~~~~~~
Detailed Interface
~~~~~~~~~~~~~~~~~~

87
doc/scripts/base/frameworks/logging/index.rst Normal file
View file

@ -0,0 +1,87 @@
:orphan:
Package: base/frameworks/logging
================================
The logging framework provides a flexible key-value based logging interface.
:doc:`/scripts/base/frameworks/logging/__load__.zeek`
:doc:`/scripts/base/frameworks/logging/main.zeek`
The Zeek logging interface.
See :doc:`/frameworks/logging` for an introduction to Zeek's
logging framework.
:doc:`/scripts/base/frameworks/logging/postprocessors/__load__.zeek`
:doc:`/scripts/base/frameworks/logging/postprocessors/scp.zeek`
This script defines a postprocessing function that can be applied
to a logging filter in order to automatically SCP (secure copy)
a log stream (or a subset of it) to a remote host at configurable
rotation time intervals. Generally, to use this functionality
you must handle the :zeek:id:`zeek_init` event and do the following
in your handler:
1) Create a new :zeek:type:`Log::Filter` record that defines a name/path,
rotation interval, and set the ``postprocessor`` to
:zeek:id:`Log::scp_postprocessor`.
2) Add the filter to a logging stream using :zeek:id:`Log::add_filter`.
3) Add a table entry to :zeek:id:`Log::scp_destinations` for the filter's
writer/path pair which defines a set of :zeek:type:`Log::SCPDestination`
records.
:doc:`/scripts/base/frameworks/logging/postprocessors/sftp.zeek`
This script defines a postprocessing function that can be applied
to a logging filter in order to automatically SFTP
a log stream (or a subset of it) to a remote host at configurable
rotation time intervals. Generally, to use this functionality
you must handle the :zeek:id:`zeek_init` event and do the following
in your handler:
1) Create a new :zeek:type:`Log::Filter` record that defines a name/path,
rotation interval, and set the ``postprocessor`` to
:zeek:id:`Log::sftp_postprocessor`.
2) Add the filter to a logging stream using :zeek:id:`Log::add_filter`.
3) Add a table entry to :zeek:id:`Log::sftp_destinations` for the filter's
writer/path pair which defines a set of :zeek:type:`Log::SFTPDestination`
records.
:doc:`/scripts/base/frameworks/logging/writers/ascii.zeek`
Interface for the ASCII log writer. Redefinable options are available
to tweak the output format of ASCII logs.
The ASCII writer currently supports one writer-specific per-filter config
option: setting ``tsv`` to the string ``T`` turns the output into
"tab-separated-value" mode where only a single header row with the column
names is printed out as meta information, with no "# fields" prepended; no
other meta data gets included in that mode. Example filter using this::
local f = Log::Filter($name = "my-filter",
$writer = Log::WRITER_ASCII,
$config = table(["tsv"] = "T"));
:doc:`/scripts/base/frameworks/logging/writers/sqlite.zeek`
Interface for the SQLite log writer. Redefinable options are available
to tweak the output format of the SQLite reader.
See :doc:`/frameworks/logging-input-sqlite` for an introduction on how to
use the SQLite log writer.
The SQL writer currently supports one writer-specific filter option via
``config``: setting ``tablename`` sets the name of the table that is used
or created in the SQLite database. An example for this is given in the
introduction mentioned above.
:doc:`/scripts/base/frameworks/logging/writers/none.zeek`
Interface for the None log writer. This writer is mainly for debugging.

1825
doc/scripts/base/frameworks/logging/main.zeek.rst Normal file
View file

File diff suppressed because it is too large Load diff

14
doc/scripts/base/frameworks/logging/postprocessors/__load__.zeek.rst Normal file
View file

@ -0,0 +1,14 @@
:tocdepth: 3
base/frameworks/logging/postprocessors/__load__.zeek
====================================================
:Imports: :doc:`base/frameworks/logging/postprocessors/scp.zeek </scripts/base/frameworks/logging/postprocessors/scp.zeek>`, :doc:`base/frameworks/logging/postprocessors/sftp.zeek </scripts/base/frameworks/logging/postprocessors/sftp.zeek>`
Summary
~~~~~~~
Detailed Interface
~~~~~~~~~~~~~~~~~~

44
doc/scripts/base/frameworks/logging/postprocessors/index.rst Normal file
View file

@ -0,0 +1,44 @@
:orphan:
Package: base/frameworks/logging/postprocessors
===============================================
Support for postprocessors in the logging framework.
:doc:`/scripts/base/frameworks/logging/postprocessors/__load__.zeek`
:doc:`/scripts/base/frameworks/logging/postprocessors/scp.zeek`
This script defines a postprocessing function that can be applied
to a logging filter in order to automatically SCP (secure copy)
a log stream (or a subset of it) to a remote host at configurable
rotation time intervals. Generally, to use this functionality
you must handle the :zeek:id:`zeek_init` event and do the following
in your handler:
1) Create a new :zeek:type:`Log::Filter` record that defines a name/path,
rotation interval, and set the ``postprocessor`` to
:zeek:id:`Log::scp_postprocessor`.
2) Add the filter to a logging stream using :zeek:id:`Log::add_filter`.
3) Add a table entry to :zeek:id:`Log::scp_destinations` for the filter's
writer/path pair which defines a set of :zeek:type:`Log::SCPDestination`
records.
:doc:`/scripts/base/frameworks/logging/postprocessors/sftp.zeek`
This script defines a postprocessing function that can be applied
to a logging filter in order to automatically SFTP
a log stream (or a subset of it) to a remote host at configurable
rotation time intervals. Generally, to use this functionality
you must handle the :zeek:id:`zeek_init` event and do the following
in your handler:
1) Create a new :zeek:type:`Log::Filter` record that defines a name/path,
rotation interval, and set the ``postprocessor`` to
:zeek:id:`Log::sftp_postprocessor`.
2) Add the filter to a logging stream using :zeek:id:`Log::add_filter`.
3) Add a table entry to :zeek:id:`Log::sftp_destinations` for the filter's
writer/path pair which defines a set of :zeek:type:`Log::SFTPDestination`
records.

132
doc/scripts/base/frameworks/logging/postprocessors/scp.zeek.rst Normal file
View file

@ -0,0 +1,132 @@
:tocdepth: 3
base/frameworks/logging/postprocessors/scp.zeek
===============================================
.. zeek:namespace:: Log
This script defines a postprocessing function that can be applied
to a logging filter in order to automatically SCP (secure copy)
a log stream (or a subset of it) to a remote host at configurable
rotation time intervals. Generally, to use this functionality
you must handle the :zeek:id:`zeek_init` event and do the following
in your handler:
1) Create a new :zeek:type:`Log::Filter` record that defines a name/path,
rotation interval, and set the ``postprocessor`` to
:zeek:id:`Log::scp_postprocessor`.
2) Add the filter to a logging stream using :zeek:id:`Log::add_filter`.
3) Add a table entry to :zeek:id:`Log::scp_destinations` for the filter's
writer/path pair which defines a set of :zeek:type:`Log::SCPDestination`
records.
:Namespace: Log
Summary
~~~~~~~
Redefinable Options
###################
================================================================================= ================================================================
:zeek:id:`Log::scp_rotation_date_format`: :zeek:type:`string` :zeek:attr:`&redef` Default naming format for timestamps embedded into log filenames
that use the SCP rotator.
================================================================================= ================================================================
State Variables
###############
==================================================== =======================================================================
:zeek:id:`Log::scp_destinations`: :zeek:type:`table` A table indexed by a particular log writer and filter path, that yields
a set of remote destinations.
==================================================== =======================================================================
Types
#####
===================================================== =====================================================================
:zeek:type:`Log::SCPDestination`: :zeek:type:`record` A container that describes the remote destination for the SCP command
argument as ``user@host:path``.
===================================================== =====================================================================
Functions
#########
======================================================== ============================================================
:zeek:id:`Log::scp_postprocessor`: :zeek:type:`function` Secure-copies the rotated log to all the remote hosts
defined in :zeek:id:`Log::scp_destinations` and then deletes
the local copy of the rotated log.
======================================================== ============================================================
Detailed Interface
~~~~~~~~~~~~~~~~~~
Redefinable Options
###################
.. zeek:id:: Log::scp_rotation_date_format
:source-code: base/frameworks/logging/postprocessors/scp.zeek 53 53
:Type: :zeek:type:`string`
:Attributes: :zeek:attr:`&redef`
:Default: ``"%Y-%m-%d-%H-%M-%S"``
Default naming format for timestamps embedded into log filenames
that use the SCP rotator.
State Variables
###############
.. zeek:id:: Log::scp_destinations
:source-code: base/frameworks/logging/postprocessors/scp.zeek 49 49
:Type: :zeek:type:`table` [:zeek:type:`Log::Writer`, :zeek:type:`string`] of :zeek:type:`set` [:zeek:type:`Log::SCPDestination`]
:Default: ``{}``
A table indexed by a particular log writer and filter path, that yields
a set of remote destinations. The :zeek:id:`Log::scp_postprocessor`
function queries this table upon log rotation and performs a secure
copy of the rotated log to each destination in the set. This
table can be modified at run-time.
Types
#####
.. zeek:type:: Log::SCPDestination
:source-code: base/frameworks/logging/postprocessors/scp.zeek 34 42
:Type: :zeek:type:`record`
.. zeek:field:: user :zeek:type:`string`
The remote user to log in as. A trust mechanism should be
pre-established.
.. zeek:field:: host :zeek:type:`string`
The remote host to which to transfer logs.
.. zeek:field:: path :zeek:type:`string`
The path/directory on the remote host to send logs.
A container that describes the remote destination for the SCP command
argument as ``user@host:path``.
Functions
#########
.. zeek:id:: Log::scp_postprocessor
:source-code: base/frameworks/logging/postprocessors/scp.zeek 56 72
:Type: :zeek:type:`function` (info: :zeek:type:`Log::RotationInfo`) : :zeek:type:`bool`
Secure-copies the rotated log to all the remote hosts
defined in :zeek:id:`Log::scp_destinations` and then deletes
the local copy of the rotated log. It's not active when
reading from trace files.
:param info: A record holding meta-information about the log file to be
postprocessed.
:returns: True if secure-copy system command was initiated or
if no destination was configured for the log as described
by *info*.

137
doc/scripts/base/frameworks/logging/postprocessors/sftp.zeek.rst Normal file
View file

@ -0,0 +1,137 @@
:tocdepth: 3
base/frameworks/logging/postprocessors/sftp.zeek
================================================
.. zeek:namespace:: Log
This script defines a postprocessing function that can be applied
to a logging filter in order to automatically SFTP
a log stream (or a subset of it) to a remote host at configurable
rotation time intervals. Generally, to use this functionality
you must handle the :zeek:id:`zeek_init` event and do the following
in your handler:
1) Create a new :zeek:type:`Log::Filter` record that defines a name/path,
rotation interval, and set the ``postprocessor`` to
:zeek:id:`Log::sftp_postprocessor`.
2) Add the filter to a logging stream using :zeek:id:`Log::add_filter`.
3) Add a table entry to :zeek:id:`Log::sftp_destinations` for the filter's
writer/path pair which defines a set of :zeek:type:`Log::SFTPDestination`
records.
:Namespace: Log
Summary
~~~~~~~
Redefinable Options
###################
================================================================================== ================================================================
:zeek:id:`Log::sftp_rotation_date_format`: :zeek:type:`string` :zeek:attr:`&redef` Default naming format for timestamps embedded into log filenames
that use the SFTP rotator.
================================================================================== ================================================================
State Variables
###############
===================================================== =======================================================================
:zeek:id:`Log::sftp_destinations`: :zeek:type:`table` A table indexed by a particular log writer and filter path, that yields
a set of remote destinations.
===================================================== =======================================================================
Types
#####
====================================================== =======================================================================
:zeek:type:`Log::SFTPDestination`: :zeek:type:`record` A container that describes the remote destination for the SFTP command,
comprised of the username, host, and path at which to upload the file.
====================================================== =======================================================================
Functions
#########
========================================================= =============================================================
:zeek:id:`Log::sftp_postprocessor`: :zeek:type:`function` Securely transfers the rotated log to all the remote hosts
defined in :zeek:id:`Log::sftp_destinations` and then deletes
the local copy of the rotated log.
========================================================= =============================================================
Detailed Interface
~~~~~~~~~~~~~~~~~~
Redefinable Options
###################
.. zeek:id:: Log::sftp_rotation_date_format
:source-code: base/frameworks/logging/postprocessors/sftp.zeek 55 55
:Type: :zeek:type:`string`
:Attributes: :zeek:attr:`&redef`
:Default: ``"%Y-%m-%d-%H-%M-%S"``
Default naming format for timestamps embedded into log filenames
that use the SFTP rotator.
State Variables
###############
.. zeek:id:: Log::sftp_destinations
:source-code: base/frameworks/logging/postprocessors/sftp.zeek 51 51
:Type: :zeek:type:`table` [:zeek:type:`Log::Writer`, :zeek:type:`string`] of :zeek:type:`set` [:zeek:type:`Log::SFTPDestination`]
:Default: ``{}``
A table indexed by a particular log writer and filter path, that yields
a set of remote destinations. The :zeek:id:`Log::sftp_postprocessor`
function queries this table upon log rotation and performs a secure
transfer of the rotated log to each destination in the set. This
table can be modified at run-time.
Types
#####
.. zeek:type:: Log::SFTPDestination
:source-code: base/frameworks/logging/postprocessors/sftp.zeek 34 44
:Type: :zeek:type:`record`
.. zeek:field:: user :zeek:type:`string`
The remote user to log in as. A trust mechanism should be
pre-established.
.. zeek:field:: host :zeek:type:`string`
The remote host to which to transfer logs.
.. zeek:field:: host_port :zeek:type:`count` :zeek:attr:`&default` = ``22`` :zeek:attr:`&optional`
The port to connect to. Defaults to 22
.. zeek:field:: path :zeek:type:`string`
The path/directory on the remote host to send logs.
A container that describes the remote destination for the SFTP command,
comprised of the username, host, and path at which to upload the file.
Functions
#########
.. zeek:id:: Log::sftp_postprocessor
:source-code: base/frameworks/logging/postprocessors/sftp.zeek 58 75
:Type: :zeek:type:`function` (info: :zeek:type:`Log::RotationInfo`) : :zeek:type:`bool`
Securely transfers the rotated log to all the remote hosts
defined in :zeek:id:`Log::sftp_destinations` and then deletes
the local copy of the rotated log. It's not active when
reading from trace files.
:param info: A record holding meta-information about the log file to be
postprocessed.
:returns: True if sftp system command was initiated or
if no destination was configured for the log as described
by *info*.

229
doc/scripts/base/frameworks/logging/writers/ascii.zeek.rst Normal file
View file

@ -0,0 +1,229 @@
:tocdepth: 3
base/frameworks/logging/writers/ascii.zeek
==========================================
.. zeek:namespace:: LogAscii
Interface for the ASCII log writer. Redefinable options are available
to tweak the output format of ASCII logs.
The ASCII writer currently supports one writer-specific per-filter config
option: setting ``tsv`` to the string ``T`` turns the output into
"tab-separated-value" mode where only a single header row with the column
names is printed out as meta information, with no "# fields" prepended; no
other meta data gets included in that mode. Example filter using this::
local f = Log::Filter($name = "my-filter",
$writer = Log::WRITER_ASCII,
$config = table(["tsv"] = "T"));
:Namespace: LogAscii
Summary
~~~~~~~
Redefinable Options
###################
============================================================================================ =====================================================================
:zeek:id:`LogAscii::empty_field`: :zeek:type:`string` :zeek:attr:`&redef` String to use for empty fields.
:zeek:id:`LogAscii::enable_leftover_log_rotation`: :zeek:type:`bool` :zeek:attr:`&redef` If true, detect log files that did not get properly rotated
by a previous Zeek process (e.g.
:zeek:id:`LogAscii::enable_utf_8`: :zeek:type:`bool` :zeek:attr:`&redef` If true, valid UTF-8 sequences will pass through unescaped and be
written into logs.
:zeek:id:`LogAscii::gzip_file_extension`: :zeek:type:`string` :zeek:attr:`&redef` Define the file extension used when compressing log files when
they are created with the :zeek:see:`LogAscii::gzip_level` option.
:zeek:id:`LogAscii::gzip_level`: :zeek:type:`count` :zeek:attr:`&redef` Define the gzip level to compress the logs.
:zeek:id:`LogAscii::include_meta`: :zeek:type:`bool` :zeek:attr:`&redef` If true, include lines with log meta information such as column names
with types, the values of ASCII logging options that are in use, and
the time when the file was opened and closed (the latter at the end).
:zeek:id:`LogAscii::json_include_unset_fields`: :zeek:type:`bool` :zeek:attr:`&redef` Handling of optional fields when writing out JSON.
:zeek:id:`LogAscii::json_timestamps`: :zeek:type:`JSON::TimestampFormat` :zeek:attr:`&redef` Format of timestamps when writing out JSON.
:zeek:id:`LogAscii::meta_prefix`: :zeek:type:`string` :zeek:attr:`&redef` Prefix for lines with meta information.
:zeek:id:`LogAscii::output_to_stdout`: :zeek:type:`bool` :zeek:attr:`&redef` If true, output everything to stdout rather than
into files.
:zeek:id:`LogAscii::separator`: :zeek:type:`string` :zeek:attr:`&redef` Separator between fields.
:zeek:id:`LogAscii::set_separator`: :zeek:type:`string` :zeek:attr:`&redef` Separator between set elements.
:zeek:id:`LogAscii::unset_field`: :zeek:type:`string` :zeek:attr:`&redef` String to use for an unset &optional field.
:zeek:id:`LogAscii::use_json`: :zeek:type:`bool` :zeek:attr:`&redef` If true, the default will be to write logs in a JSON format.
============================================================================================ =====================================================================
Detailed Interface
~~~~~~~~~~~~~~~~~~
Redefinable Options
###################
.. zeek:id:: LogAscii::empty_field
:source-code: base/frameworks/logging/writers/ascii.zeek 95 95
:Type: :zeek:type:`string`
:Attributes: :zeek:attr:`&redef`
:Default: ``"(empty)"``
String to use for empty fields. This should be different from
*unset_field* to make the output unambiguous.
This option is also available as a per-filter ``$config`` option.
.. zeek:id:: LogAscii::enable_leftover_log_rotation
:source-code: base/frameworks/logging/writers/ascii.zeek 35 35
:Type: :zeek:type:`bool`
:Attributes: :zeek:attr:`&redef`
:Default: ``F``
If true, detect log files that did not get properly rotated
by a previous Zeek process (e.g. due to crash) and rotate them.
This requires a positive rotation interval to be configured
to have an effect. E.g. via :zeek:see:`Log::default_rotation_interval`
or the *interv* field of a :zeek:see:`Log::Filter`.
.. zeek:id:: LogAscii::enable_utf_8
:source-code: base/frameworks/logging/writers/ascii.zeek 41 41
:Type: :zeek:type:`bool`
:Attributes: :zeek:attr:`&redef`
:Default: ``T``
If true, valid UTF-8 sequences will pass through unescaped and be
written into logs.
This option is also available as a per-filter ``$config`` option.
.. zeek:id:: LogAscii::gzip_file_extension
:source-code: base/frameworks/logging/writers/ascii.zeek 55 55
:Type: :zeek:type:`string`
:Attributes: :zeek:attr:`&redef`
:Default: ``"gz"``
Define the file extension used when compressing log files when
they are created with the :zeek:see:`LogAscii::gzip_level` option.
This option is also available as a per-filter ``$config`` option.
.. zeek:id:: LogAscii::gzip_level
:source-code: base/frameworks/logging/writers/ascii.zeek 49 49
:Type: :zeek:type:`count`
:Attributes: :zeek:attr:`&redef`
:Default: ``0``
Define the gzip level to compress the logs. If 0, then no gzip
compression is performed. Enabling compression also changes
the log file name extension to include the value of
:zeek:see:`LogAscii::gzip_file_extension`.
This option is also available as a per-filter ``$config`` option.
.. zeek:id:: LogAscii::include_meta
:source-code: base/frameworks/logging/writers/ascii.zeek 74 74
:Type: :zeek:type:`bool`
:Attributes: :zeek:attr:`&redef`
:Default: ``T``
If true, include lines with log meta information such as column names
with types, the values of ASCII logging options that are in use, and
the time when the file was opened and closed (the latter at the end).
If writing in JSON format, this is implicitly disabled.
.. zeek:id:: LogAscii::json_include_unset_fields
:source-code: base/frameworks/logging/writers/ascii.zeek 67 67
:Type: :zeek:type:`bool`
:Attributes: :zeek:attr:`&redef`
:Default: ``F``
Handling of optional fields when writing out JSON. By default the
JSON formatter skips key and val when the field is absent. Setting
the following field to T includes the key, with a null value.
.. zeek:id:: LogAscii::json_timestamps
:source-code: base/frameworks/logging/writers/ascii.zeek 62 62
:Type: :zeek:type:`JSON::TimestampFormat`
:Attributes: :zeek:attr:`&redef`
:Default: ``JSON::TS_EPOCH``
Format of timestamps when writing out JSON. By default, the JSON
formatter will use double values for timestamps which represent the
number of seconds from the UNIX epoch.
This option is also available as a per-filter ``$config`` option.
.. zeek:id:: LogAscii::meta_prefix
:source-code: base/frameworks/logging/writers/ascii.zeek 79 79
:Type: :zeek:type:`string`
:Attributes: :zeek:attr:`&redef`
:Default: ``"#"``
Prefix for lines with meta information.
This option is also available as a per-filter ``$config`` option.
.. zeek:id:: LogAscii::output_to_stdout
:source-code: base/frameworks/logging/writers/ascii.zeek 22 22
:Type: :zeek:type:`bool`
:Attributes: :zeek:attr:`&redef`
:Default: ``F``
If true, output everything to stdout rather than
into files. This is primarily for debugging purposes.
This option is also available as a per-filter ``$config`` option.
.. zeek:id:: LogAscii::separator
:source-code: base/frameworks/logging/writers/ascii.zeek 84 84
:Type: :zeek:type:`string`
:Attributes: :zeek:attr:`&redef`
:Default: ``"\x09"``
Separator between fields.
This option is also available as a per-filter ``$config`` option.
.. zeek:id:: LogAscii::set_separator
:source-code: base/frameworks/logging/writers/ascii.zeek 89 89
:Type: :zeek:type:`string`
:Attributes: :zeek:attr:`&redef`
:Default: ``","``
Separator between set elements.
This option is also available as a per-filter ``$config`` option.
.. zeek:id:: LogAscii::unset_field
:source-code: base/frameworks/logging/writers/ascii.zeek 100 100
:Type: :zeek:type:`string`
:Attributes: :zeek:attr:`&redef`
:Default: ``"-"``
String to use for an unset &optional field.
This option is also available as a per-filter ``$config`` option.
.. zeek:id:: LogAscii::use_json
:source-code: base/frameworks/logging/writers/ascii.zeek 27 27
:Type: :zeek:type:`bool`
:Attributes: :zeek:attr:`&redef`
:Default: ``F``
:Redefinition: from :doc:`/scripts/policy/tuning/json-logs.zeek`
``=``::
T
If true, the default will be to write logs in a JSON format.
This option is also available as a per-filter ``$config`` option.

41
doc/scripts/base/frameworks/logging/writers/none.zeek.rst Normal file
View file

@ -0,0 +1,41 @@
:tocdepth: 3
base/frameworks/logging/writers/none.zeek
=========================================
.. zeek:namespace:: LogNone
Interface for the None log writer. This writer is mainly for debugging.
:Namespace: LogNone
Summary
~~~~~~~
Redefinable Options
###################
================================================================ ============================================================
:zeek:id:`LogNone::debug`: :zeek:type:`bool` :zeek:attr:`&redef` If true, output debugging output that can be useful for unit
testing the logging framework.
================================================================ ============================================================
Redefinitions
#############
======================================================================================= =
:zeek:id:`Log::default_rotation_postprocessors`: :zeek:type:`table` :zeek:attr:`&redef`
======================================================================================= =
Detailed Interface
~~~~~~~~~~~~~~~~~~
Redefinable Options
###################
.. zeek:id:: LogNone::debug
:source-code: base/frameworks/logging/writers/none.zeek 8 8
:Type: :zeek:type:`bool`
:Attributes: :zeek:attr:`&redef`
:Default: ``F``
If true, output debugging output that can be useful for unit
testing the logging framework.

141
doc/scripts/base/frameworks/logging/writers/sqlite.zeek.rst Normal file
View file

@ -0,0 +1,141 @@
:tocdepth: 3
base/frameworks/logging/writers/sqlite.zeek
===========================================
.. zeek:namespace:: LogSQLite
Interface for the SQLite log writer. Redefinable options are available
to tweak the output format of the SQLite reader.
See :doc:`/frameworks/logging-input-sqlite` for an introduction on how to
use the SQLite log writer.
The SQL writer currently supports one writer-specific filter option via
``config``: setting ``tablename`` sets the name of the table that is used
or created in the SQLite database. An example for this is given in the
introduction mentioned above.
:Namespace: LogSQLite
Summary
~~~~~~~
Redefinable Options
###################
================================================================================================= ==========================================================================
:zeek:id:`LogSQLite::empty_field`: :zeek:type:`string` :zeek:attr:`&redef` String to use for empty fields.
:zeek:id:`LogSQLite::journal_mode`: :zeek:type:`LogSQLite::SQLiteJournalMode` :zeek:attr:`&redef` If changed from SQLITE_JOURNAL_MODE_DEFAULT, runs the PRAGMA
journal_mode statement with the provided value after connecting to
the SQLite database.
:zeek:id:`LogSQLite::set_separator`: :zeek:type:`string` :zeek:attr:`&redef` Separator between set elements.
:zeek:id:`LogSQLite::synchronous`: :zeek:type:`LogSQLite::SQLiteSynchronous` :zeek:attr:`&redef` If changed from SQLITE_SYNCHRONOUS_DEFAULT, runs the PRAGMA synchronous
statement with the provided value after connecting to the SQLite database.
:zeek:id:`LogSQLite::unset_field`: :zeek:type:`string` :zeek:attr:`&redef` String to use for an unset &optional field.
================================================================================================= ==========================================================================
Types
#####
============================================================ ============================================================
:zeek:type:`LogSQLite::SQLiteJournalMode`: :zeek:type:`enum` Values supported for SQLite's PRAGMA journal_mode statement.
:zeek:type:`LogSQLite::SQLiteSynchronous`: :zeek:type:`enum` Values supported for SQLite's PRAGMA synchronous statement.
============================================================ ============================================================
Detailed Interface
~~~~~~~~~~~~~~~~~~
Redefinable Options
###################
.. zeek:id:: LogSQLite::empty_field
:source-code: base/frameworks/logging/writers/sqlite.zeek 23 23
:Type: :zeek:type:`string`
:Attributes: :zeek:attr:`&redef`
:Default: ``"(empty)"``
String to use for empty fields. This should be different from
*unset_field* to make the output unambiguous.
.. zeek:id:: LogSQLite::journal_mode
:source-code: base/frameworks/logging/writers/sqlite.zeek 57 57
:Type: :zeek:type:`LogSQLite::SQLiteJournalMode`
:Attributes: :zeek:attr:`&redef`
:Default: ``LogSQLite::SQLITE_JOURNAL_MODE_DEFAULT``
If changed from SQLITE_JOURNAL_MODE_DEFAULT, runs the PRAGMA
journal_mode statement with the provided value after connecting to
the SQLite database.
`SQLite's journal_mode documentation <https://www.sqlite.org/pragma.html#pragma_journal_mode>`_
for more details around performance, data safety trade offs
and interaction with the PRAGMA synchronous statement.
.. zeek:id:: LogSQLite::set_separator
:source-code: base/frameworks/logging/writers/sqlite.zeek 16 16
:Type: :zeek:type:`string`
:Attributes: :zeek:attr:`&redef`
:Default: ``","``
Separator between set elements.
.. zeek:id:: LogSQLite::synchronous
:source-code: base/frameworks/logging/writers/sqlite.zeek 49 49
:Type: :zeek:type:`LogSQLite::SQLiteSynchronous`
:Attributes: :zeek:attr:`&redef`
:Default: ``LogSQLite::SQLITE_SYNCHRONOUS_DEFAULT``
If changed from SQLITE_SYNCHRONOUS_DEFAULT, runs the PRAGMA synchronous
statement with the provided value after connecting to the SQLite database. See
`SQLite's synchronous documentation <https://www.sqlite.org/pragma.html#pragma_synchronous>`_
for more details around performance and data safety trade offs.
.. zeek:id:: LogSQLite::unset_field
:source-code: base/frameworks/logging/writers/sqlite.zeek 19 19
:Type: :zeek:type:`string`
:Attributes: :zeek:attr:`&redef`
:Default: ``"-"``
String to use for an unset &optional field.
Types
#####
.. zeek:type:: LogSQLite::SQLiteJournalMode
:source-code: base/frameworks/logging/writers/sqlite.zeek 35 35
:Type: :zeek:type:`enum`
.. zeek:enum:: LogSQLite::SQLITE_JOURNAL_MODE_DEFAULT LogSQLite::SQLiteJournalMode
.. zeek:enum:: LogSQLite::SQLITE_JOURNAL_MODE_DELETE LogSQLite::SQLiteJournalMode
.. zeek:enum:: LogSQLite::SQLITE_JOURNAL_MODE_TRUNCATE LogSQLite::SQLiteJournalMode
.. zeek:enum:: LogSQLite::SQLITE_JOURNAL_MODE_PERSIST LogSQLite::SQLiteJournalMode
.. zeek:enum:: LogSQLite::SQLITE_JOURNAL_MODE_MEMORY LogSQLite::SQLiteJournalMode
.. zeek:enum:: LogSQLite::SQLITE_JOURNAL_MODE_WAL LogSQLite::SQLiteJournalMode
.. zeek:enum:: LogSQLite::SQLITE_JOURNAL_MODE_OFF LogSQLite::SQLiteJournalMode
Values supported for SQLite's PRAGMA journal_mode statement.
.. zeek:type:: LogSQLite::SQLiteSynchronous
:source-code: base/frameworks/logging/writers/sqlite.zeek 26 26
:Type: :zeek:type:`enum`
.. zeek:enum:: LogSQLite::SQLITE_SYNCHRONOUS_DEFAULT LogSQLite::SQLiteSynchronous
.. zeek:enum:: LogSQLite::SQLITE_SYNCHRONOUS_OFF LogSQLite::SQLiteSynchronous
.. zeek:enum:: LogSQLite::SQLITE_SYNCHRONOUS_NORMAL LogSQLite::SQLiteSynchronous
.. zeek:enum:: LogSQLite::SQLITE_SYNCHRONOUS_FULL LogSQLite::SQLiteSynchronous
.. zeek:enum:: LogSQLite::SQLITE_SYNCHRONOUS_EXTRA LogSQLite::SQLiteSynchronous
Values supported for SQLite's PRAGMA synchronous statement.

14
doc/scripts/base/frameworks/netcontrol/__load__.zeek.rst Normal file
View file

@ -0,0 +1,14 @@
:tocdepth: 3
base/frameworks/netcontrol/__load__.zeek
========================================
:Imports: :doc:`base/frameworks/cluster </scripts/base/frameworks/cluster/index>`, :doc:`base/frameworks/netcontrol/drop.zeek </scripts/base/frameworks/netcontrol/drop.zeek>`, :doc:`base/frameworks/netcontrol/main.zeek </scripts/base/frameworks/netcontrol/main.zeek>`, :doc:`base/frameworks/netcontrol/non-cluster.zeek </scripts/base/frameworks/netcontrol/non-cluster.zeek>`, :doc:`base/frameworks/netcontrol/plugins </scripts/base/frameworks/netcontrol/plugins/index>`, :doc:`base/frameworks/netcontrol/shunt.zeek </scripts/base/frameworks/netcontrol/shunt.zeek>`, :doc:`base/frameworks/netcontrol/types.zeek </scripts/base/frameworks/netcontrol/types.zeek>`
Summary
~~~~~~~
Detailed Interface
~~~~~~~~~~~~~~~~~~

171
doc/scripts/base/frameworks/netcontrol/drop.zeek.rst Normal file
View file

@ -0,0 +1,171 @@
:tocdepth: 3
base/frameworks/netcontrol/drop.zeek
====================================
.. zeek:namespace:: NetControl
Implementation of the drop functionality for NetControl.
:Namespace: NetControl
:Imports: :doc:`base/frameworks/netcontrol/main.zeek </scripts/base/frameworks/netcontrol/main.zeek>`
Summary
~~~~~~~
Types
#####
====================================================== =
:zeek:type:`NetControl::DropInfo`: :zeek:type:`record`
====================================================== =
Redefinitions
#############
======================================= ===================================
:zeek:type:`Log::ID`: :zeek:type:`enum`
* :zeek:enum:`NetControl::DROP_LOG`
======================================= ===================================
Events
######
============================================================== ==========================================================================
:zeek:id:`NetControl::log_netcontrol_drop`: :zeek:type:`event` Event that can be handled to access the :zeek:type:`NetControl::ShuntInfo`
record as it is sent on to the logging framework.
============================================================== ==========================================================================
Hooks
#####
==================================================================== =======================================================================
:zeek:id:`NetControl::drop_rule_policy`: :zeek:type:`hook` Hook that allows the modification of rules passed to drop_* before they
are passed on.
:zeek:id:`NetControl::log_policy_drop`: :zeek:type:`Log::PolicyHook`
==================================================================== =======================================================================
Functions
#########
============================================================= ======================================================================
:zeek:id:`NetControl::drop_address`: :zeek:type:`function` Stops all packets involving an IP address from being forwarded.
:zeek:id:`NetControl::drop_connection`: :zeek:type:`function` Stops all packets involving a connection address from being forwarded.
============================================================= ======================================================================
Detailed Interface
~~~~~~~~~~~~~~~~~~
Types
#####
.. zeek:type:: NetControl::DropInfo
:source-code: base/frameworks/netcontrol/drop.zeek 34 47
:Type: :zeek:type:`record`
.. zeek:field:: ts :zeek:type:`time` :zeek:attr:`&log`
Time at which the recorded activity occurred.
.. zeek:field:: rule_id :zeek:type:`string` :zeek:attr:`&log`
ID of the rule; unique during each Zeek run.
.. zeek:field:: orig_h :zeek:type:`addr` :zeek:attr:`&log`
The originator's IP address.
.. zeek:field:: orig_p :zeek:type:`port` :zeek:attr:`&log` :zeek:attr:`&optional`
The originator's port number.
.. zeek:field:: resp_h :zeek:type:`addr` :zeek:attr:`&log` :zeek:attr:`&optional`
The responder's IP address.
.. zeek:field:: resp_p :zeek:type:`port` :zeek:attr:`&log` :zeek:attr:`&optional`
The responder's port number.
.. zeek:field:: expire :zeek:type:`interval` :zeek:attr:`&log`
Expiry time of the shunt.
.. zeek:field:: location :zeek:type:`string` :zeek:attr:`&log` :zeek:attr:`&optional`
Location where the underlying action was triggered.
Events
######
.. zeek:id:: NetControl::log_netcontrol_drop
:source-code: base/frameworks/netcontrol/drop.zeek 57 57
:Type: :zeek:type:`event` (rec: :zeek:type:`NetControl::DropInfo`)
Event that can be handled to access the :zeek:type:`NetControl::ShuntInfo`
record as it is sent on to the logging framework.
Hooks
#####
.. zeek:id:: NetControl::drop_rule_policy
:source-code: base/frameworks/netcontrol/drop.zeek 53 53
:Type: :zeek:type:`hook` (r: :zeek:type:`NetControl::Rule`) : :zeek:type:`bool`
Hook that allows the modification of rules passed to drop_* before they
are passed on. If one of the hooks uses break, the rule is ignored.
:param r: The rule to be added.
.. zeek:id:: NetControl::log_policy_drop
:source-code: base/frameworks/netcontrol/drop.zeek 10 10
:Type: :zeek:type:`Log::PolicyHook`
Functions
#########
.. zeek:id:: NetControl::drop_address
:source-code: base/frameworks/netcontrol/drop.zeek 89 111
:Type: :zeek:type:`function` (a: :zeek:type:`addr`, t: :zeek:type:`interval`, location: :zeek:type:`string` :zeek:attr:`&default` = ``""`` :zeek:attr:`&optional`) : :zeek:type:`string`
Stops all packets involving an IP address from being forwarded.
:param a: The address to be dropped.
:param t: How long to drop it, with 0 being indefinitely.
:param location: An optional string describing where the drop was triggered.
:returns: The id of the inserted rule on success and zero on failure.
.. zeek:id:: NetControl::drop_connection
:source-code: base/frameworks/netcontrol/drop.zeek 65 87
:Type: :zeek:type:`function` (c: :zeek:type:`conn_id`, t: :zeek:type:`interval`, location: :zeek:type:`string` :zeek:attr:`&default` = ``""`` :zeek:attr:`&optional`) : :zeek:type:`string`
Stops all packets involving a connection address from being forwarded.
:param c: The connection to be dropped.
:param t: How long to drop it, with 0 being indefinitely.
:param location: An optional string describing where the drop was triggered.
:returns: The id of the inserted rule on success and zero on failure.

77
doc/scripts/base/frameworks/netcontrol/index.rst Normal file
View file

@ -0,0 +1,77 @@
:orphan:
Package: base/frameworks/netcontrol
===================================
The NetControl framework provides a way for Zeek to interact with networking
hard- and software, e.g. for dropping and shunting IP addresses/connections,
etc.
:doc:`/scripts/base/frameworks/netcontrol/__load__.zeek`
:doc:`/scripts/base/frameworks/netcontrol/types.zeek`
This file defines the types that are used by the NetControl framework.
The most important type defined in this file is :zeek:see:`NetControl::Rule`,
which is used to describe all rules that can be expressed by the NetControl framework.
:doc:`/scripts/base/frameworks/netcontrol/main.zeek`
Zeek's NetControl framework.
This plugin-based framework allows to control the traffic that Zeek monitors
as well as, if having access to the forwarding path, the traffic the network
forwards. By default, the framework lets everything through, to both Zeek
itself as well as on the network. Scripts can then add rules to impose
restrictions on entities, such as specific connections or IP addresses.
This framework has two APIs: a high-level and low-level. The high-level API
provides convenience functions for a set of common operations. The
low-level API provides full flexibility.
:doc:`/scripts/base/frameworks/netcontrol/plugin.zeek`
This file defines the plugin interface for NetControl.
:doc:`/scripts/base/frameworks/netcontrol/plugins/__load__.zeek`
:doc:`/scripts/base/frameworks/netcontrol/plugins/debug.zeek`
Debugging plugin for the NetControl framework, providing insight into
executed operations.
:doc:`/scripts/base/frameworks/netcontrol/plugins/openflow.zeek`
OpenFlow plugin for the NetControl framework.
:doc:`/scripts/base/frameworks/netcontrol/plugins/packetfilter.zeek`
NetControl plugin for the process-level PacketFilter that comes with
Zeek. Since the PacketFilter in Zeek is quite limited in scope
and can only add/remove filters for addresses, this is quite
limited in scope at the moment.
:doc:`/scripts/base/frameworks/netcontrol/plugins/broker.zeek`
Broker plugin for the NetControl framework. Sends the raw data structures
used in NetControl on to Broker to allow for easy handling, e.g., of
command-line scripts.
:doc:`/scripts/base/frameworks/netcontrol/plugins/acld.zeek`
Acld plugin for the netcontrol framework.
:doc:`/scripts/base/frameworks/netcontrol/drop.zeek`
Implementation of the drop functionality for NetControl.
:doc:`/scripts/base/frameworks/netcontrol/shunt.zeek`
Implementation of the shunt functionality for NetControl.
:doc:`/scripts/base/frameworks/netcontrol/non-cluster.zeek`

649
doc/scripts/base/frameworks/netcontrol/main.zeek.rst Normal file
View file

@ -0,0 +1,649 @@
:tocdepth: 3
base/frameworks/netcontrol/main.zeek
====================================
.. zeek:namespace:: NetControl
Zeek's NetControl framework.
This plugin-based framework allows to control the traffic that Zeek monitors
as well as, if having access to the forwarding path, the traffic the network
forwards. By default, the framework lets everything through, to both Zeek
itself as well as on the network. Scripts can then add rules to impose
restrictions on entities, such as specific connections or IP addresses.
This framework has two APIs: a high-level and low-level. The high-level API
provides convenience functions for a set of common operations. The
low-level API provides full flexibility.
:Namespace: NetControl
:Imports: :doc:`base/frameworks/netcontrol/plugin.zeek </scripts/base/frameworks/netcontrol/plugin.zeek>`, :doc:`base/frameworks/netcontrol/types.zeek </scripts/base/frameworks/netcontrol/types.zeek>`
Summary
~~~~~~~
Types
#####
======================================================== =================================================================
:zeek:type:`NetControl::Info`: :zeek:type:`record` The record type defining the column fields of the NetControl log.
:zeek:type:`NetControl::InfoCategory`: :zeek:type:`enum` Type of an entry in the NetControl log.
:zeek:type:`NetControl::InfoState`: :zeek:type:`enum` State of an entry in the NetControl log.
======================================================== =================================================================
Redefinitions
#############
================================================== ===================================================================================================================
:zeek:type:`Log::ID`: :zeek:type:`enum` The framework's logging stream identifier.
* :zeek:enum:`NetControl::LOG`
:zeek:type:`NetControl::Rule`: :zeek:type:`record`
:New Fields: :zeek:type:`NetControl::Rule`
_plugin_ids: :zeek:type:`set` [:zeek:type:`count`] :zeek:attr:`&default` = ``{ }`` :zeek:attr:`&optional`
Internally set to the plugins handling the rule.
_active_plugin_ids: :zeek:type:`set` [:zeek:type:`count`] :zeek:attr:`&default` = ``{ }`` :zeek:attr:`&optional`
Internally set to the plugins on which the rule is currently active.
_no_expire_plugins: :zeek:type:`set` [:zeek:type:`count`] :zeek:attr:`&default` = ``{ }`` :zeek:attr:`&optional`
Internally set to plugins where the rule should not be removed upon timeout.
_added: :zeek:type:`bool` :zeek:attr:`&default` = ``F`` :zeek:attr:`&optional`
Track if the rule was added successfully by all responsible plugins.
================================================== ===================================================================================================================
Events
######
========================================================= ===========================================================================
:zeek:id:`NetControl::init`: :zeek:type:`event` Event that is used to initialize plugins.
:zeek:id:`NetControl::init_done`: :zeek:type:`event` Event that is raised once all plugins activated in ``NetControl::init``
have finished their initialization.
:zeek:id:`NetControl::log_netcontrol`: :zeek:type:`event` Event that can be handled to access the :zeek:type:`NetControl::Info`
record as it is sent on to the logging framework.
:zeek:id:`NetControl::rule_added`: :zeek:type:`event` Confirms that a rule was put in place by a plugin.
:zeek:id:`NetControl::rule_destroyed`: :zeek:type:`event` This event is raised when a rule is deleted from the NetControl framework,
because it is no longer in use.
:zeek:id:`NetControl::rule_error`: :zeek:type:`event` Reports an error when operating on a rule.
:zeek:id:`NetControl::rule_exists`: :zeek:type:`event` Signals that a rule that was supposed to be put in place was already
existing at the specified plugin.
:zeek:id:`NetControl::rule_new`: :zeek:type:`event` This event is raised when a new rule is created by the NetControl framework
due to a call to add_rule.
:zeek:id:`NetControl::rule_removed`: :zeek:type:`event` Reports that a plugin reports a rule was removed due to a
remove_rule function call.
:zeek:id:`NetControl::rule_timeout`: :zeek:type:`event` Reports that a rule was removed from a plugin due to a timeout.
========================================================= ===========================================================================
Hooks
#####
=============================================================== =============================================================================
:zeek:id:`NetControl::log_policy`: :zeek:type:`Log::PolicyHook` A default logging policy hook for the stream.
:zeek:id:`NetControl::rule_added_policy`: :zeek:type:`hook` Hook that allows the modification of rule states after they are returned from
the plugins and have been added to the rules database.
:zeek:id:`NetControl::rule_policy`: :zeek:type:`hook` Hook that allows the modification of rules passed to add_rule before they
are passed on to the plugins.
=============================================================== =============================================================================
Functions
#########
=============================================================== ===============================================================================================
:zeek:id:`NetControl::activate`: :zeek:type:`function` Activates a plugin.
:zeek:id:`NetControl::add_rule`: :zeek:type:`function` Installs a rule.
:zeek:id:`NetControl::clear`: :zeek:type:`function` Flushes all state by calling :zeek:see:`NetControl::remove_rule` on all currently active rules.
:zeek:id:`NetControl::delete_rule`: :zeek:type:`function` Deletes a rule without removing it from the backends to which it has been
added before.
:zeek:id:`NetControl::find_rules_addr`: :zeek:type:`function` Searches all rules affecting a certain IP address.
:zeek:id:`NetControl::find_rules_subnet`: :zeek:type:`function` Searches all rules affecting a certain subnet.
:zeek:id:`NetControl::plugin_activated`: :zeek:type:`function` Function called by plugins once they finished their activation.
:zeek:id:`NetControl::quarantine_host`: :zeek:type:`function` Quarantines a host.
:zeek:id:`NetControl::redirect_flow`: :zeek:type:`function` Redirects a uni-directional flow to another port.
:zeek:id:`NetControl::remove_rule`: :zeek:type:`function` Removes a rule.
:zeek:id:`NetControl::whitelist_address`: :zeek:type:`function` Allows all traffic involving a specific IP address to be forwarded.
:zeek:id:`NetControl::whitelist_subnet`: :zeek:type:`function` Allows all traffic involving a specific IP subnet to be forwarded.
=============================================================== ===============================================================================================
Detailed Interface
~~~~~~~~~~~~~~~~~~
Types
#####
.. zeek:type:: NetControl::Info
:source-code: base/frameworks/netcontrol/main.zeek 308 339
:Type: :zeek:type:`record`
.. zeek:field:: ts :zeek:type:`time` :zeek:attr:`&log`
Time at which the recorded activity occurred.
.. zeek:field:: rule_id :zeek:type:`string` :zeek:attr:`&log` :zeek:attr:`&optional`
ID of the rule; unique during each Zeek run.
.. zeek:field:: category :zeek:type:`NetControl::InfoCategory` :zeek:attr:`&log` :zeek:attr:`&optional`
Type of the log entry.
.. zeek:field:: cmd :zeek:type:`string` :zeek:attr:`&log` :zeek:attr:`&optional`
The command the log entry is about.
.. zeek:field:: state :zeek:type:`NetControl::InfoState` :zeek:attr:`&log` :zeek:attr:`&optional`
State the log entry reflects.
.. zeek:field:: action :zeek:type:`string` :zeek:attr:`&log` :zeek:attr:`&optional`
String describing an action the entry is about.
.. zeek:field:: target :zeek:type:`NetControl::TargetType` :zeek:attr:`&log` :zeek:attr:`&optional`
The target type of the action.
.. zeek:field:: entity_type :zeek:type:`string` :zeek:attr:`&log` :zeek:attr:`&optional`
Type of the entity the log entry is about.
.. zeek:field:: entity :zeek:type:`string` :zeek:attr:`&log` :zeek:attr:`&optional`
String describing the entity the log entry is about.
.. zeek:field:: mod :zeek:type:`string` :zeek:attr:`&log` :zeek:attr:`&optional`
String describing the optional modification of the entry (e.h. redirect)
.. zeek:field:: msg :zeek:type:`string` :zeek:attr:`&log` :zeek:attr:`&optional`
String with an additional message.
.. zeek:field:: priority :zeek:type:`int` :zeek:attr:`&log` :zeek:attr:`&optional`
Number describing the priority of the log entry.
.. zeek:field:: expire :zeek:type:`interval` :zeek:attr:`&log` :zeek:attr:`&optional`
Expiry time of the log entry.
.. zeek:field:: location :zeek:type:`string` :zeek:attr:`&log` :zeek:attr:`&optional`
Location where the underlying action was triggered.
.. zeek:field:: plugin :zeek:type:`string` :zeek:attr:`&log` :zeek:attr:`&optional`
Plugin triggering the log entry.
The record type defining the column fields of the NetControl log.
.. zeek:type:: NetControl::InfoCategory
:source-code: base/frameworks/netcontrol/main.zeek 288 296
:Type: :zeek:type:`enum`
.. zeek:enum:: NetControl::MESSAGE NetControl::InfoCategory
A log entry reflecting a framework message.
.. zeek:enum:: NetControl::ERROR NetControl::InfoCategory
A log entry reflecting a framework message.
.. zeek:enum:: NetControl::RULE NetControl::InfoCategory
A log entry about a rule.
Type of an entry in the NetControl log.
.. zeek:type:: NetControl::InfoState
:source-code: base/frameworks/netcontrol/main.zeek 298 306
:Type: :zeek:type:`enum`
.. zeek:enum:: NetControl::REQUESTED NetControl::InfoState
The request to add/remove a rule was sent to the respective backend.
.. zeek:enum:: NetControl::SUCCEEDED NetControl::InfoState
A rule was successfully added by a backend.
.. zeek:enum:: NetControl::EXISTS NetControl::InfoState
A backend reported that a rule was already existing.
.. zeek:enum:: NetControl::FAILED NetControl::InfoState
A rule addition failed.
.. zeek:enum:: NetControl::REMOVED NetControl::InfoState
A rule was successfully removed by a backend.
.. zeek:enum:: NetControl::TIMEOUT NetControl::InfoState
A rule timeout was triggered by the NetControl framework or a backend.
State of an entry in the NetControl log.
Events
######
.. zeek:id:: NetControl::init
:source-code: base/frameworks/netcontrol/main.zeek 642 650
:Type: :zeek:type:`event` ()
Event that is used to initialize plugins. Place all plugin initialization
related functionality in this event.
.. zeek:id:: NetControl::init_done
:source-code: base/frameworks/netcontrol/main.zeek 43 43
:Type: :zeek:type:`event` ()
Event that is raised once all plugins activated in ``NetControl::init``
have finished their initialization.
.. zeek:id:: NetControl::log_netcontrol
:source-code: base/frameworks/netcontrol/main.zeek 343 343
:Type: :zeek:type:`event` (rec: :zeek:type:`NetControl::Info`)
Event that can be handled to access the :zeek:type:`NetControl::Info`
record as it is sent on to the logging framework.
.. zeek:id:: NetControl::rule_added
:source-code: base/frameworks/netcontrol/main.zeek 191 191
:Type: :zeek:type:`event` (r: :zeek:type:`NetControl::Rule`, p: :zeek:type:`NetControl::PluginState`, msg: :zeek:type:`string` :zeek:attr:`&default` = ``""`` :zeek:attr:`&optional`)
Confirms that a rule was put in place by a plugin.
:param r: The rule now in place.
:param p: The state for the plugin that put it into place.
:param msg: An optional informational message by the plugin.
.. zeek:id:: NetControl::rule_destroyed
:source-code: base/frameworks/netcontrol/main.zeek 256 256
:Type: :zeek:type:`event` (r: :zeek:type:`NetControl::Rule`)
This event is raised when a rule is deleted from the NetControl framework,
because it is no longer in use. This can be caused by the fact that a rule
was removed by all plugins to which it was added, by the fact that it timed out
or due to rule errors.
To get the cause of a rule remove, catch the rule_removed, rule_timeout and
rule_error events.
.. zeek:id:: NetControl::rule_error
:source-code: base/frameworks/netcontrol/main.zeek 236 236
:Type: :zeek:type:`event` (r: :zeek:type:`NetControl::Rule`, p: :zeek:type:`NetControl::PluginState`, msg: :zeek:type:`string` :zeek:attr:`&default` = ``""`` :zeek:attr:`&optional`)
Reports an error when operating on a rule.
:param r: The rule that encountered an error.
:param p: The state for the plugin that reported the error.
:param msg: An optional informational message by the plugin.
.. zeek:id:: NetControl::rule_exists
:source-code: base/frameworks/netcontrol/main.zeek 204 204
:Type: :zeek:type:`event` (r: :zeek:type:`NetControl::Rule`, p: :zeek:type:`NetControl::PluginState`, msg: :zeek:type:`string` :zeek:attr:`&default` = ``""`` :zeek:attr:`&optional`)
Signals that a rule that was supposed to be put in place was already
existing at the specified plugin. Rules that already have been existing
continue to be tracked like normal, but no timeout calls will be sent
to the specified plugins. Removal of the rule from the hardware can
still be forced by manually issuing a remove_rule call.
:param r: The rule that was already in place.
:param p: The plugin that reported that the rule already was in place.
:param msg: An optional informational message by the plugin.
.. zeek:id:: NetControl::rule_new
:source-code: base/frameworks/netcontrol/main.zeek 247 247
:Type: :zeek:type:`event` (r: :zeek:type:`NetControl::Rule`)
This event is raised when a new rule is created by the NetControl framework
due to a call to add_rule. From this moment, until the rule_destroyed event
is raised, the rule is tracked internally by the NetControl framework.
Note that this event does not mean that a rule was successfully added by
any backend; it just means that the rule has been accepted and addition
to the specified backend is queued. To get information when rules are actually
installed by the hardware, use the rule_added, rule_exists, rule_removed, rule_timeout
and rule_error events.
.. zeek:id:: NetControl::rule_removed
:source-code: base/frameworks/netcontrol/non-cluster.zeek 47 50
:Type: :zeek:type:`event` (r: :zeek:type:`NetControl::Rule`, p: :zeek:type:`NetControl::PluginState`, msg: :zeek:type:`string` :zeek:attr:`&default` = ``""`` :zeek:attr:`&optional`)
Reports that a plugin reports a rule was removed due to a
remove_rule function call.
:param r: The rule now removed.
:param p: The state for the plugin that had the rule in place and now
removed it.
:param msg: An optional informational message by the plugin.
.. zeek:id:: NetControl::rule_timeout
:source-code: base/frameworks/netcontrol/main.zeek 227 227
:Type: :zeek:type:`event` (r: :zeek:type:`NetControl::Rule`, i: :zeek:type:`NetControl::FlowInfo`, p: :zeek:type:`NetControl::PluginState`)
Reports that a rule was removed from a plugin due to a timeout.
:param r: The rule now removed.
:param i: Additional flow information, if supported by the protocol.
:param p: The state for the plugin that had the rule in place and now
removed it.
:param msg: An optional informational message by the plugin.
Hooks
#####
.. zeek:id:: NetControl::log_policy
:source-code: base/frameworks/netcontrol/main.zeek 23 23
:Type: :zeek:type:`Log::PolicyHook`
A default logging policy hook for the stream.
.. zeek:id:: NetControl::rule_added_policy
:source-code: base/frameworks/netcontrol/main.zeek 277 277
:Type: :zeek:type:`hook` (r: :zeek:type:`NetControl::Rule`, p: :zeek:type:`NetControl::PluginState`, exists: :zeek:type:`bool`, msg: :zeek:type:`string`) : :zeek:type:`bool`
Hook that allows the modification of rule states after they are returned from
the plugins and have been added to the rules database. This allows low-level
modification of the handling of rules like, e.g., changing rule expiration depending
on context.
:param r: The rule now in place.
:param p: The state for the plugin that put it into place.
:param exists: If the adding plugin flagged the rule as already existing.
:param msg: An optional informational message by the plugin.
.. zeek:id:: NetControl::rule_policy
:source-code: base/frameworks/netcontrol/main.zeek 263 263
:Type: :zeek:type:`hook` (r: :zeek:type:`NetControl::Rule`) : :zeek:type:`bool`
Hook that allows the modification of rules passed to add_rule before they
are passed on to the plugins. If one of the hooks uses break, the rule is
ignored and not passed on to any plugin.
:param r: The rule to be added.
Functions
#########
.. zeek:id:: NetControl::activate
:source-code: base/frameworks/netcontrol/non-cluster.zeek 6 9
:Type: :zeek:type:`function` (p: :zeek:type:`NetControl::PluginState`, priority: :zeek:type:`int`) : :zeek:type:`void`
Activates a plugin.
:param p: The plugin to activate.
:param priority: The higher the priority, the earlier this plugin will be checked
whether it supports an operation, relative to other plugins.
.. zeek:id:: NetControl::add_rule
:source-code: base/frameworks/netcontrol/non-cluster.zeek 11 14
:Type: :zeek:type:`function` (r: :zeek:type:`NetControl::Rule`) : :zeek:type:`string`
Installs a rule.
:param r: The rule to install.
:returns: If successful, returns an ID string unique to the rule that can
later be used to refer to it. If unsuccessful, returns an empty
string. The ID is also assigned to ``r$id``. Note that
"successful" means "a plugin knew how to handle the rule", it
doesn't necessarily mean that it was indeed successfully put in
place, because that might happen asynchronously and thus fail
only later.
.. zeek:id:: NetControl::clear
:source-code: base/frameworks/netcontrol/main.zeek 1075 1079
:Type: :zeek:type:`function` () : :zeek:type:`void`
Flushes all state by calling :zeek:see:`NetControl::remove_rule` on all currently active rules.
.. zeek:id:: NetControl::delete_rule
:source-code: base/frameworks/netcontrol/non-cluster.zeek 16 19
:Type: :zeek:type:`function` (id: :zeek:type:`string`, reason: :zeek:type:`string` :zeek:attr:`&default` = ``""`` :zeek:attr:`&optional`) : :zeek:type:`bool`
Deletes a rule without removing it from the backends to which it has been
added before. This means that no messages will be sent to the switches to which
the rule has been added; if it is not removed from them by a separate mechanism,
it will stay installed and not be removed later.
:param id: The rule to delete, specified as the ID returned by :zeek:see:`NetControl::add_rule`.
:param reason: Optional string argument giving information on why the rule was deleted.
:returns: True if removal is successful, or sent to manager.
False if the rule could not be found.
.. zeek:id:: NetControl::find_rules_addr
:source-code: base/frameworks/netcontrol/main.zeek 763 766
:Type: :zeek:type:`function` (ip: :zeek:type:`addr`) : :zeek:type:`vector` of :zeek:type:`NetControl::Rule`
Searches all rules affecting a certain IP address.
This function works on both the manager and workers of a cluster. Note that on
the worker, the internal rule variables (starting with _) will not reflect the
current state.
:param ip: The ip address to search for.
:returns: vector of all rules affecting the IP address.
.. zeek:id:: NetControl::find_rules_subnet
:source-code: base/frameworks/netcontrol/main.zeek 741 761
:Type: :zeek:type:`function` (sn: :zeek:type:`subnet`) : :zeek:type:`vector` of :zeek:type:`NetControl::Rule`
Searches all rules affecting a certain subnet.
A rule affects a subnet, if it covers the whole subnet. Note especially that
this function will not reveal all rules that are covered by a subnet.
For example, a search for 192.168.17.0/8 will reveal a rule that exists for
192.168.0.0/16, since this rule affects the subnet. However, it will not reveal
a more specific rule for 192.168.17.1/32, which does not directly affect the whole
subnet.
This function works on both the manager and workers of a cluster. Note that on
the worker, the internal rule variables (starting with _) will not reflect the
current state.
:param sn: The subnet to search for.
:returns: vector of all rules affecting the subnet.
.. zeek:id:: NetControl::plugin_activated
:source-code: base/frameworks/netcontrol/main.zeek 617 635
:Type: :zeek:type:`function` (p: :zeek:type:`NetControl::PluginState`) : :zeek:type:`void`
Function called by plugins once they finished their activation. After all
plugins defined in zeek_init finished to activate, rules will start to be sent
to the plugins. Rules that scripts try to set before the backends are ready
will be discarded.
.. zeek:id:: NetControl::quarantine_host
:source-code: base/frameworks/netcontrol/main.zeek 570 590
:Type: :zeek:type:`function` (infected: :zeek:type:`addr`, dns: :zeek:type:`addr`, quarantine: :zeek:type:`addr`, t: :zeek:type:`interval`, location: :zeek:type:`string` :zeek:attr:`&default` = ``""`` :zeek:attr:`&optional`) : :zeek:type:`vector` of :zeek:type:`string`
Quarantines a host. This requires a special quarantine server, which runs a HTTP server explaining
the quarantine and a DNS server which resolves all requests to the quarantine server. DNS queries
from the host to the network DNS server will be rewritten and will be sent to the quarantine server
instead. Only http communication infected to quarantinehost is allowed. All other network communication
is blocked.
:param infected: the host to quarantine.
:param dns: the network dns server.
:param quarantine: the quarantine server running a dns and a web server.
:param t: how long to leave the quarantine in place.
:returns: Vector of inserted rules on success, empty list on failure.
.. zeek:id:: NetControl::redirect_flow
:source-code: base/frameworks/netcontrol/main.zeek 556 568
:Type: :zeek:type:`function` (f: :zeek:type:`flow_id`, out_port: :zeek:type:`count`, t: :zeek:type:`interval`, location: :zeek:type:`string` :zeek:attr:`&default` = ``""`` :zeek:attr:`&optional`) : :zeek:type:`string`
Redirects a uni-directional flow to another port.
:param f: The flow to redirect.
:param out_port: Port to redirect the flow to.
:param t: How long to leave the redirect in place, with 0 being indefinitely.
:param location: An optional string describing where the redirect was triggered.
:returns: The id of the inserted rule on success and zero on failure.
.. zeek:id:: NetControl::remove_rule
:source-code: base/frameworks/netcontrol/non-cluster.zeek 21 24
:Type: :zeek:type:`function` (id: :zeek:type:`string`, reason: :zeek:type:`string` :zeek:attr:`&default` = ``""`` :zeek:attr:`&optional`) : :zeek:type:`bool`
Removes a rule.
:param id: The rule to remove, specified as the ID returned by :zeek:see:`NetControl::add_rule`.
:param reason: Optional string argument giving information on why the rule was removed.
:returns: True if successful, the relevant plugin indicated that it knew
how to handle the removal. Note that again "success" means the
plugin accepted the removal. It might still fail to put it
into effect, as that might happen asynchronously and thus go
wrong at that point.
.. zeek:id:: NetControl::whitelist_address
:source-code: base/frameworks/netcontrol/main.zeek 539 545
:Type: :zeek:type:`function` (a: :zeek:type:`addr`, t: :zeek:type:`interval`, location: :zeek:type:`string` :zeek:attr:`&default` = ``""`` :zeek:attr:`&optional`) : :zeek:type:`string`
Allows all traffic involving a specific IP address to be forwarded.
:param a: The address to be whitelisted.
:param t: How long to whitelist it, with 0 being indefinitely.
:param location: An optional string describing whitelist was triggered.
:returns: The id of the inserted rule on success and zero on failure.
.. zeek:id:: NetControl::whitelist_subnet
:source-code: base/frameworks/netcontrol/main.zeek 547 553
:Type: :zeek:type:`function` (s: :zeek:type:`subnet`, t: :zeek:type:`interval`, location: :zeek:type:`string` :zeek:attr:`&default` = ``""`` :zeek:attr:`&optional`) : :zeek:type:`string`
Allows all traffic involving a specific IP subnet to be forwarded.
:param s: The subnet to be whitelisted.
:param t: How long to whitelist it, with 0 being indefinitely.
:param location: An optional string describing whitelist was triggered.
:returns: The id of the inserted rule on success and zero on failure.

16
doc/scripts/base/frameworks/netcontrol/non-cluster.zeek.rst Normal file
View file

@ -0,0 +1,16 @@
:tocdepth: 3
base/frameworks/netcontrol/non-cluster.zeek
===========================================
.. zeek:namespace:: NetControl
:Namespace: NetControl
:Imports: :doc:`base/frameworks/netcontrol/main.zeek </scripts/base/frameworks/netcontrol/main.zeek>`
Summary
~~~~~~~
Detailed Interface
~~~~~~~~~~~~~~~~~~

179
doc/scripts/base/frameworks/netcontrol/plugin.zeek.rst Normal file
View file

@ -0,0 +1,179 @@
:tocdepth: 3
base/frameworks/netcontrol/plugin.zeek
======================================
.. zeek:namespace:: NetControl
This file defines the plugin interface for NetControl.
:Namespace: NetControl
:Imports: :doc:`base/frameworks/netcontrol/types.zeek </scripts/base/frameworks/netcontrol/types.zeek>`
Summary
~~~~~~~
Types
#####
========================================================= =====================================================
:zeek:type:`NetControl::Plugin`: :zeek:type:`record` Definition of a plugin.
:zeek:type:`NetControl::PluginState`: :zeek:type:`record` This record keeps the per instance state of a plugin.
========================================================= =====================================================
Redefinitions
#############
========================================================= ========================================================================
:zeek:type:`NetControl::PluginState`: :zeek:type:`record` Table for a plugin to store instance-specific configuration information.
:New Fields: :zeek:type:`NetControl::PluginState`
plugin: :zeek:type:`NetControl::Plugin` :zeek:attr:`&optional`
The plugin that the state belongs to.
========================================================= ========================================================================
Detailed Interface
~~~~~~~~~~~~~~~~~~
Types
#####
.. zeek:type:: NetControl::Plugin
:source-code: base/frameworks/netcontrol/plugin.zeek 38 72
:Type: :zeek:type:`record`
.. zeek:field:: name :zeek:type:`function` (state: :zeek:type:`NetControl::PluginState`) : :zeek:type:`string`
Returns a descriptive name of the plugin instance, suitable for use in logging
messages. Note that this function is not optional.
.. zeek:field:: can_expire :zeek:type:`bool`
If true, plugin can expire rules itself. If false, the NetControl
framework will manage rule expiration.
.. zeek:field:: init :zeek:type:`function` (state: :zeek:type:`NetControl::PluginState`) : :zeek:type:`void` :zeek:attr:`&optional`
One-time initialization function called when plugin gets registered, and
before any other methods are called.
If this function is provided, NetControl assumes that the plugin has to
perform, potentially lengthy, initialization before the plugin will become
active. In this case, the plugin has to call ``NetControl::plugin_activated``,
once initialization finishes.
.. zeek:field:: done :zeek:type:`function` (state: :zeek:type:`NetControl::PluginState`) : :zeek:type:`void` :zeek:attr:`&optional`
One-time finalization function called when a plugin is shutdown; no further
functions will be called afterwards.
.. zeek:field:: add_rule :zeek:type:`function` (state: :zeek:type:`NetControl::PluginState`, r: :zeek:type:`NetControl::Rule`) : :zeek:type:`bool` :zeek:attr:`&optional`
Implements the add_rule() operation. If the plugin accepts the rule,
it returns true, false otherwise. The rule will already have its
``id`` field set, which the plugin may use for identification
purposes.
.. zeek:field:: remove_rule :zeek:type:`function` (state: :zeek:type:`NetControl::PluginState`, r: :zeek:type:`NetControl::Rule`, reason: :zeek:type:`string`) : :zeek:type:`bool` :zeek:attr:`&optional`
Implements the remove_rule() operation. This will only be called for
rules that the plugin has previously accepted with add_rule(). The
``id`` field will match that of the add_rule() call. Generally,
a plugin that accepts an add_rule() should also accept the
remove_rule().
Definition of a plugin.
Generally a plugin needs to implement only what it can support. By
returning failure, it indicates that it can't support something and
the framework will then try another plugin, if available; or inform the
that the operation failed. If a function isn't implemented by a plugin,
that's considered an implicit failure to support the operation.
If plugin accepts a rule operation, it *must* generate one of the reporting
events ``rule_{added,remove,error}`` to signal if it indeed worked out;
this is separate from accepting the operation because often a plugin
will only know later (i.e., asynchronously) if that was an error for
something it thought it could handle.
.. zeek:type:: NetControl::PluginState
:source-code: base/frameworks/netcontrol/plugin.zeek 11 23
:Type: :zeek:type:`record`
.. zeek:field:: config :zeek:type:`table` [:zeek:type:`string`] of :zeek:type:`string` :zeek:attr:`&default` = ``{ }`` :zeek:attr:`&optional`
Table for a plugin to store custom, instance-specific state.
.. zeek:field:: _id :zeek:type:`count` :zeek:attr:`&optional`
Unique plugin identifier -- used for backlookup of plugins from Rules. Set internally.
.. zeek:field:: _priority :zeek:type:`int` :zeek:attr:`&default` = ``0`` :zeek:attr:`&optional`
Set internally.
.. zeek:field:: _activated :zeek:type:`bool` :zeek:attr:`&default` = ``F`` :zeek:attr:`&optional`
Set internally. Signifies if the plugin has returned that it has activated successfully.
.. zeek:field:: plugin :zeek:type:`NetControl::Plugin` :zeek:attr:`&optional`
The plugin that the state belongs to. (Defined separately
because of cyclic type dependency.)
.. zeek:field:: of_controller :zeek:type:`OpenFlow::Controller` :zeek:attr:`&optional`
(present if :doc:`/scripts/base/frameworks/netcontrol/plugins/openflow.zeek` is loaded)
OpenFlow controller for NetControl OpenFlow plugin.
.. zeek:field:: of_config :zeek:type:`NetControl::OfConfig` :zeek:attr:`&optional`
(present if :doc:`/scripts/base/frameworks/netcontrol/plugins/openflow.zeek` is loaded)
OpenFlow configuration record that is passed on initialization.
.. zeek:field:: broker_config :zeek:type:`NetControl::BrokerConfig` :zeek:attr:`&optional`
(present if :doc:`/scripts/base/frameworks/netcontrol/plugins/broker.zeek` is loaded)
OpenFlow controller for NetControl Broker plugin.
.. zeek:field:: broker_id :zeek:type:`count` :zeek:attr:`&optional`
(present if :doc:`/scripts/base/frameworks/netcontrol/plugins/broker.zeek` is loaded)
The ID of this broker instance - for the mapping to PluginStates.
.. zeek:field:: acld_config :zeek:type:`NetControl::AcldConfig` :zeek:attr:`&optional`
(present if :doc:`/scripts/base/frameworks/netcontrol/plugins/acld.zeek` is loaded)
.. zeek:field:: acld_id :zeek:type:`count` :zeek:attr:`&optional`
(present if :doc:`/scripts/base/frameworks/netcontrol/plugins/acld.zeek` is loaded)
The ID of this acld instance - for the mapping to PluginStates.
This record keeps the per instance state of a plugin.
Individual plugins commonly extend this record to suit their needs.

14
doc/scripts/base/frameworks/netcontrol/plugins/__load__.zeek.rst Normal file
View file

@ -0,0 +1,14 @@
:tocdepth: 3
base/frameworks/netcontrol/plugins/__load__.zeek
================================================
:Imports: :doc:`base/frameworks/netcontrol/plugins/acld.zeek </scripts/base/frameworks/netcontrol/plugins/acld.zeek>`, :doc:`base/frameworks/netcontrol/plugins/broker.zeek </scripts/base/frameworks/netcontrol/plugins/broker.zeek>`, :doc:`base/frameworks/netcontrol/plugins/debug.zeek </scripts/base/frameworks/netcontrol/plugins/debug.zeek>`, :doc:`base/frameworks/netcontrol/plugins/openflow.zeek </scripts/base/frameworks/netcontrol/plugins/openflow.zeek>`, :doc:`base/frameworks/netcontrol/plugins/packetfilter.zeek </scripts/base/frameworks/netcontrol/plugins/packetfilter.zeek>`
Summary
~~~~~~~
Detailed Interface
~~~~~~~~~~~~~~~~~~

197
doc/scripts/base/frameworks/netcontrol/plugins/acld.zeek.rst Normal file
View file

@ -0,0 +1,197 @@
:tocdepth: 3
base/frameworks/netcontrol/plugins/acld.zeek
============================================
.. zeek:namespace:: NetControl
Acld plugin for the netcontrol framework.
:Namespace: NetControl
:Imports: :doc:`base/frameworks/broker </scripts/base/frameworks/broker/index>`, :doc:`base/frameworks/netcontrol/main.zeek </scripts/base/frameworks/netcontrol/main.zeek>`, :doc:`base/frameworks/netcontrol/plugin.zeek </scripts/base/frameworks/netcontrol/plugin.zeek>`
Summary
~~~~~~~
Types
#####
======================================================== =
:zeek:type:`NetControl::AclRule`: :zeek:type:`record`
:zeek:type:`NetControl::AcldConfig`: :zeek:type:`record`
======================================================== =
Redefinitions
#############
========================================================= =========================================================================
:zeek:type:`NetControl::PluginState`: :zeek:type:`record`
:New Fields: :zeek:type:`NetControl::PluginState`
acld_config: :zeek:type:`NetControl::AcldConfig` :zeek:attr:`&optional`
acld_id: :zeek:type:`count` :zeek:attr:`&optional`
The ID of this acld instance - for the mapping to PluginStates.
========================================================= =========================================================================
Events
######
============================================================ =======================================
:zeek:id:`NetControl::acld_add_rule`: :zeek:type:`event` Events that are sent from us to Broker.
:zeek:id:`NetControl::acld_remove_rule`: :zeek:type:`event`
:zeek:id:`NetControl::acld_rule_added`: :zeek:type:`event` Events that are sent from Broker to us.
:zeek:id:`NetControl::acld_rule_error`: :zeek:type:`event`
:zeek:id:`NetControl::acld_rule_exists`: :zeek:type:`event`
:zeek:id:`NetControl::acld_rule_removed`: :zeek:type:`event`
============================================================ =======================================
Hooks
#####
========================================================== ==============================================================
:zeek:id:`NetControl::acld_rule_policy`: :zeek:type:`hook` Hook that is called after a rule is converted to an acld rule.
========================================================== ==============================================================
Functions
#########
========================================================= =============================
:zeek:id:`NetControl::create_acld`: :zeek:type:`function` Instantiates the acld plugin.
========================================================= =============================
Detailed Interface
~~~~~~~~~~~~~~~~~~
Types
#####
.. zeek:type:: NetControl::AclRule
:source-code: base/frameworks/netcontrol/plugins/acld.zeek 10 15
:Type: :zeek:type:`record`
.. zeek:field:: command :zeek:type:`string`
.. zeek:field:: cookie :zeek:type:`count`
.. zeek:field:: arg :zeek:type:`string`
.. zeek:field:: comment :zeek:type:`string` :zeek:attr:`&optional`
.. zeek:type:: NetControl::AcldConfig
:source-code: base/frameworks/netcontrol/plugins/acld.zeek 17 37
:Type: :zeek:type:`record`
.. zeek:field:: acld_topic :zeek:type:`string`
The acld topic to send events to.
.. zeek:field:: acld_host :zeek:type:`addr`
Broker host to connect to.
.. zeek:field:: acld_port :zeek:type:`port`
Broker port to connect to.
.. zeek:field:: monitor :zeek:type:`bool` :zeek:attr:`&default` = ``F`` :zeek:attr:`&optional`
Do we accept rules for the monitor path? Default false.
.. zeek:field:: forward :zeek:type:`bool` :zeek:attr:`&default` = ``T`` :zeek:attr:`&optional`
Do we accept rules for the forward path? Default true.
.. zeek:field:: check_pred :zeek:type:`function` (p: :zeek:type:`NetControl::PluginState`, r: :zeek:type:`NetControl::Rule`) : :zeek:type:`bool` :zeek:attr:`&optional`
Predicate that is called on rule insertion or removal.
:param p: Current plugin state.
:param r: The rule to be inserted or removed.
:returns: T if the rule can be handled by the current backend, F otherwise.
Events
######
.. zeek:id:: NetControl::acld_add_rule
:source-code: base/frameworks/netcontrol/plugins/acld.zeek 61 61
:Type: :zeek:type:`event` (id: :zeek:type:`count`, r: :zeek:type:`NetControl::Rule`, ar: :zeek:type:`NetControl::AclRule`)
Events that are sent from us to Broker.
.. zeek:id:: NetControl::acld_remove_rule
:source-code: base/frameworks/netcontrol/plugins/acld.zeek 62 62
:Type: :zeek:type:`event` (id: :zeek:type:`count`, r: :zeek:type:`NetControl::Rule`, ar: :zeek:type:`NetControl::AclRule`)
.. zeek:id:: NetControl::acld_rule_added
:source-code: base/frameworks/netcontrol/plugins/acld.zeek 90 101
:Type: :zeek:type:`event` (id: :zeek:type:`count`, r: :zeek:type:`NetControl::Rule`, msg: :zeek:type:`string`)
Events that are sent from Broker to us.
.. zeek:id:: NetControl::acld_rule_error
:source-code: base/frameworks/netcontrol/plugins/acld.zeek 129 140
:Type: :zeek:type:`event` (id: :zeek:type:`count`, r: :zeek:type:`NetControl::Rule`, msg: :zeek:type:`string`)
.. zeek:id:: NetControl::acld_rule_exists
:source-code: base/frameworks/netcontrol/plugins/acld.zeek 103 114
:Type: :zeek:type:`event` (id: :zeek:type:`count`, r: :zeek:type:`NetControl::Rule`, msg: :zeek:type:`string`)
.. zeek:id:: NetControl::acld_rule_removed
:source-code: base/frameworks/netcontrol/plugins/acld.zeek 116 127
:Type: :zeek:type:`event` (id: :zeek:type:`count`, r: :zeek:type:`NetControl::Rule`, msg: :zeek:type:`string`)
Hooks
#####
.. zeek:id:: NetControl::acld_rule_policy
:source-code: base/frameworks/netcontrol/plugins/acld.zeek 58 58
:Type: :zeek:type:`hook` (p: :zeek:type:`NetControl::PluginState`, r: :zeek:type:`NetControl::Rule`, ar: :zeek:type:`NetControl::AclRule`) : :zeek:type:`bool`
Hook that is called after a rule is converted to an acld rule.
The hook may modify the rule before it is sent to acld.
Setting the acld command to F will cause the rule to be rejected
by the plugin.
:param p: Current plugin state.
:param r: The rule to be inserted or removed.
:param ar: The acld rule to be inserted or removed.
Functions
#########
.. zeek:id:: NetControl::create_acld
:source-code: base/frameworks/netcontrol/plugins/acld.zeek 298 317
:Type: :zeek:type:`function` (config: :zeek:type:`NetControl::AcldConfig`) : :zeek:type:`NetControl::PluginState`
Instantiates the acld plugin.

159
doc/scripts/base/frameworks/netcontrol/plugins/broker.zeek.rst Normal file
View file

@ -0,0 +1,159 @@
:tocdepth: 3
base/frameworks/netcontrol/plugins/broker.zeek
==============================================
.. zeek:namespace:: NetControl
Broker plugin for the NetControl framework. Sends the raw data structures
used in NetControl on to Broker to allow for easy handling, e.g., of
command-line scripts.
:Namespace: NetControl
:Imports: :doc:`base/frameworks/broker </scripts/base/frameworks/broker/index>`, :doc:`base/frameworks/netcontrol/main.zeek </scripts/base/frameworks/netcontrol/main.zeek>`, :doc:`base/frameworks/netcontrol/plugin.zeek </scripts/base/frameworks/netcontrol/plugin.zeek>`
Summary
~~~~~~~
Types
#####
========================================================== ================================================================================================
:zeek:type:`NetControl::BrokerConfig`: :zeek:type:`record` This record specifies the configuration that is passed to :zeek:see:`NetControl::create_broker`.
========================================================== ================================================================================================
Redefinitions
#############
========================================================= =============================================================================
:zeek:type:`NetControl::PluginState`: :zeek:type:`record`
:New Fields: :zeek:type:`NetControl::PluginState`
broker_config: :zeek:type:`NetControl::BrokerConfig` :zeek:attr:`&optional`
OpenFlow controller for NetControl Broker plugin.
broker_id: :zeek:type:`count` :zeek:attr:`&optional`
The ID of this broker instance - for the mapping to PluginStates.
========================================================= =============================================================================
Events
######
============================================================== =
:zeek:id:`NetControl::broker_add_rule`: :zeek:type:`event`
:zeek:id:`NetControl::broker_remove_rule`: :zeek:type:`event`
:zeek:id:`NetControl::broker_rule_added`: :zeek:type:`event`
:zeek:id:`NetControl::broker_rule_error`: :zeek:type:`event`
:zeek:id:`NetControl::broker_rule_exists`: :zeek:type:`event`
:zeek:id:`NetControl::broker_rule_removed`: :zeek:type:`event`
:zeek:id:`NetControl::broker_rule_timeout`: :zeek:type:`event`
============================================================== =
Functions
#########
=========================================================== ===============================
:zeek:id:`NetControl::create_broker`: :zeek:type:`function` Instantiates the broker plugin.
=========================================================== ===============================
Detailed Interface
~~~~~~~~~~~~~~~~~~
Types
#####
.. zeek:type:: NetControl::BrokerConfig
:source-code: base/frameworks/netcontrol/plugins/broker.zeek 13 34
:Type: :zeek:type:`record`
.. zeek:field:: topic :zeek:type:`string` :zeek:attr:`&optional`
The broker topic to send events to.
.. zeek:field:: host :zeek:type:`addr` :zeek:attr:`&optional`
Broker host to connect to.
.. zeek:field:: bport :zeek:type:`port` :zeek:attr:`&optional`
Broker port to connect to.
.. zeek:field:: monitor :zeek:type:`bool` :zeek:attr:`&default` = ``T`` :zeek:attr:`&optional`
Do we accept rules for the monitor path? Default true.
.. zeek:field:: forward :zeek:type:`bool` :zeek:attr:`&default` = ``T`` :zeek:attr:`&optional`
Do we accept rules for the forward path? Default true.
.. zeek:field:: check_pred :zeek:type:`function` (p: :zeek:type:`NetControl::PluginState`, r: :zeek:type:`NetControl::Rule`) : :zeek:type:`bool` :zeek:attr:`&optional`
Predicate that is called on rule insertion or removal.
:param p: Current plugin state.
:param r: The rule to be inserted or removed.
:returns: T if the rule can be handled by the current backend, F otherwise.
This record specifies the configuration that is passed to :zeek:see:`NetControl::create_broker`.
Events
######
.. zeek:id:: NetControl::broker_add_rule
:source-code: base/frameworks/netcontrol/plugins/broker.zeek 46 46
:Type: :zeek:type:`event` (id: :zeek:type:`count`, r: :zeek:type:`NetControl::Rule`)
.. zeek:id:: NetControl::broker_remove_rule
:source-code: base/frameworks/netcontrol/plugins/broker.zeek 47 47
:Type: :zeek:type:`event` (id: :zeek:type:`count`, r: :zeek:type:`NetControl::Rule`, reason: :zeek:type:`string`)
.. zeek:id:: NetControl::broker_rule_added
:source-code: base/frameworks/netcontrol/plugins/broker.zeek 61 72
:Type: :zeek:type:`event` (id: :zeek:type:`count`, r: :zeek:type:`NetControl::Rule`, msg: :zeek:type:`string`)
.. zeek:id:: NetControl::broker_rule_error
:source-code: base/frameworks/netcontrol/plugins/broker.zeek 100 111
:Type: :zeek:type:`event` (id: :zeek:type:`count`, r: :zeek:type:`NetControl::Rule`, msg: :zeek:type:`string`)
.. zeek:id:: NetControl::broker_rule_exists
:source-code: base/frameworks/netcontrol/plugins/broker.zeek 74 85
:Type: :zeek:type:`event` (id: :zeek:type:`count`, r: :zeek:type:`NetControl::Rule`, msg: :zeek:type:`string`)
.. zeek:id:: NetControl::broker_rule_removed
:source-code: base/frameworks/netcontrol/plugins/broker.zeek 87 98
:Type: :zeek:type:`event` (id: :zeek:type:`count`, r: :zeek:type:`NetControl::Rule`, msg: :zeek:type:`string`)
.. zeek:id:: NetControl::broker_rule_timeout
:source-code: base/frameworks/netcontrol/plugins/broker.zeek 113 124
:Type: :zeek:type:`event` (id: :zeek:type:`count`, r: :zeek:type:`NetControl::Rule`, i: :zeek:type:`NetControl::FlowInfo`)
Functions
#########
.. zeek:id:: NetControl::create_broker
:source-code: base/frameworks/netcontrol/plugins/broker.zeek 198 220
:Type: :zeek:type:`function` (config: :zeek:type:`NetControl::BrokerConfig`, can_expire: :zeek:type:`bool`) : :zeek:type:`NetControl::PluginState`
Instantiates the broker plugin.

65
doc/scripts/base/frameworks/netcontrol/plugins/debug.zeek.rst Normal file
View file

@ -0,0 +1,65 @@
:tocdepth: 3
base/frameworks/netcontrol/plugins/debug.zeek
=============================================
.. zeek:namespace:: NetControl
Debugging plugin for the NetControl framework, providing insight into
executed operations.
:Namespace: NetControl
:Imports: :doc:`base/frameworks/netcontrol/main.zeek </scripts/base/frameworks/netcontrol/main.zeek>`, :doc:`base/frameworks/netcontrol/plugin.zeek </scripts/base/frameworks/netcontrol/plugin.zeek>`
Summary
~~~~~~~
Functions
#########
================================================================= =========================================================
:zeek:id:`NetControl::create_debug`: :zeek:type:`function` Instantiates a debug plugin for the NetControl framework.
:zeek:id:`NetControl::create_debug_error`: :zeek:type:`function` Instantiates a debug plugin for the NetControl framework.
:zeek:id:`NetControl::create_debug_exists`: :zeek:type:`function` Instantiates a debug plugin for the NetControl framework.
================================================================= =========================================================
Detailed Interface
~~~~~~~~~~~~~~~~~~
Functions
#########
.. zeek:id:: NetControl::create_debug
:source-code: base/frameworks/netcontrol/plugins/debug.zeek 118 131
:Type: :zeek:type:`function` (do_something: :zeek:type:`bool`, name: :zeek:type:`string` :zeek:attr:`&default` = ``""`` :zeek:attr:`&optional`) : :zeek:type:`NetControl::PluginState`
Instantiates a debug plugin for the NetControl framework. The debug
plugin simply logs the operations it receives.
:param do_something: If true, the plugin will claim it supports all operations; if
false, it will indicate it doesn't support any.
:param name: Optional name that for the plugin.
.. zeek:id:: NetControl::create_debug_error
:source-code: base/frameworks/netcontrol/plugins/debug.zeek 133 140
:Type: :zeek:type:`function` (name: :zeek:type:`string`) : :zeek:type:`NetControl::PluginState`
Instantiates a debug plugin for the NetControl framework. This variation
of the plugin will return "error" to any rule operations.
:param name: Name of this plugin.
.. zeek:id:: NetControl::create_debug_exists
:source-code: base/frameworks/netcontrol/plugins/debug.zeek 142 149
:Type: :zeek:type:`function` (name: :zeek:type:`string`) : :zeek:type:`NetControl::PluginState`
Instantiates a debug plugin for the NetControl framework. This variation
of the plugin will return "exists" to any rule operations.
:param name: Name of this plugin.

36
doc/scripts/base/frameworks/netcontrol/plugins/index.rst Normal file
View file

@ -0,0 +1,36 @@
:orphan:
Package: base/frameworks/netcontrol/plugins
===========================================
Plugins for the NetControl framework.
:doc:`/scripts/base/frameworks/netcontrol/plugins/__load__.zeek`
:doc:`/scripts/base/frameworks/netcontrol/plugins/debug.zeek`
Debugging plugin for the NetControl framework, providing insight into
executed operations.
:doc:`/scripts/base/frameworks/netcontrol/plugins/openflow.zeek`
OpenFlow plugin for the NetControl framework.
:doc:`/scripts/base/frameworks/netcontrol/plugins/packetfilter.zeek`
NetControl plugin for the process-level PacketFilter that comes with
Zeek. Since the PacketFilter in Zeek is quite limited in scope
and can only add/remove filters for addresses, this is quite
limited in scope at the moment.
:doc:`/scripts/base/frameworks/netcontrol/plugins/broker.zeek`
Broker plugin for the NetControl framework. Sends the raw data structures
used in NetControl on to Broker to allow for easy handling, e.g., of
command-line scripts.
:doc:`/scripts/base/frameworks/netcontrol/plugins/acld.zeek`
Acld plugin for the netcontrol framework.

195
doc/scripts/base/frameworks/netcontrol/plugins/openflow.zeek.rst Normal file
View file

@ -0,0 +1,195 @@
:tocdepth: 3
base/frameworks/netcontrol/plugins/openflow.zeek
================================================
.. zeek:namespace:: NetControl
OpenFlow plugin for the NetControl framework.
:Namespace: NetControl
:Imports: :doc:`base/frameworks/netcontrol/main.zeek </scripts/base/frameworks/netcontrol/main.zeek>`, :doc:`base/frameworks/netcontrol/plugin.zeek </scripts/base/frameworks/netcontrol/plugin.zeek>`, :doc:`base/frameworks/openflow </scripts/base/frameworks/openflow/index>`
Summary
~~~~~~~
Redefinable Options
###################
========================================================================================== ===============================================================================
:zeek:id:`NetControl::openflow_flow_timeout`: :zeek:type:`interval` :zeek:attr:`&redef` The time interval after we consider a flow timed out.
:zeek:id:`NetControl::openflow_message_timeout`: :zeek:type:`interval` :zeek:attr:`&redef` The time interval after which an openflow message is considered to be timed out
and we delete it from our internal tracking.
========================================================================================== ===============================================================================
Types
#####
====================================================== ==================================================================================================
:zeek:type:`NetControl::OfConfig`: :zeek:type:`record` This record specifies the configuration that is passed to :zeek:see:`NetControl::create_openflow`.
:zeek:type:`NetControl::OfTable`: :zeek:type:`record`
====================================================== ==================================================================================================
Redefinitions
#############
========================================================= =========================================================================
:zeek:type:`NetControl::PluginState`: :zeek:type:`record`
:New Fields: :zeek:type:`NetControl::PluginState`
of_controller: :zeek:type:`OpenFlow::Controller` :zeek:attr:`&optional`
OpenFlow controller for NetControl OpenFlow plugin.
of_config: :zeek:type:`NetControl::OfConfig` :zeek:attr:`&optional`
OpenFlow configuration record that is passed on initialization.
========================================================= =========================================================================
Functions
#########
============================================================= =============================================================
:zeek:id:`NetControl::create_openflow`: :zeek:type:`function` Instantiates an openflow plugin for the NetControl framework.
============================================================= =============================================================
Detailed Interface
~~~~~~~~~~~~~~~~~~
Redefinable Options
###################
.. zeek:id:: NetControl::openflow_flow_timeout
:source-code: base/frameworks/netcontrol/plugins/openflow.zeek 76 76
:Type: :zeek:type:`interval`
:Attributes: :zeek:attr:`&redef`
:Default: ``1.0 day``
The time interval after we consider a flow timed out. This should be fairly high (or
even disabled) if you expect a lot of long flows. However, one also will have state
buildup for quite a while if keeping this around...
.. zeek:id:: NetControl::openflow_message_timeout
:source-code: base/frameworks/netcontrol/plugins/openflow.zeek 71 71
:Type: :zeek:type:`interval`
:Attributes: :zeek:attr:`&redef`
:Default: ``20.0 secs``
The time interval after which an openflow message is considered to be timed out
and we delete it from our internal tracking.
Types
#####
.. zeek:type:: NetControl::OfConfig
:source-code: base/frameworks/netcontrol/plugins/openflow.zeek 11 51
:Type: :zeek:type:`record`
.. zeek:field:: monitor :zeek:type:`bool` :zeek:attr:`&default` = ``T`` :zeek:attr:`&optional`
Accept rules that target the monitor path.
.. zeek:field:: forward :zeek:type:`bool` :zeek:attr:`&default` = ``T`` :zeek:attr:`&optional`
Accept rules that target the forward path.
.. zeek:field:: idle_timeout :zeek:type:`count` :zeek:attr:`&default` = ``0`` :zeek:attr:`&optional`
Default OpenFlow idle timeout.
.. zeek:field:: table_id :zeek:type:`count` :zeek:attr:`&optional`
Default OpenFlow table ID.
.. zeek:field:: priority_offset :zeek:type:`int` :zeek:attr:`&default` = ``0`` :zeek:attr:`&optional`
Add this to all rule priorities. Can be useful if you want the openflow priorities be offset from the netcontrol priorities without having to write a filter function.
.. zeek:field:: check_pred :zeek:type:`function` (p: :zeek:type:`NetControl::PluginState`, r: :zeek:type:`NetControl::Rule`) : :zeek:type:`bool` :zeek:attr:`&optional`
Predicate that is called on rule insertion or removal.
:param p: Current plugin state.
:param r: The rule to be inserted or removed.
:returns: T if the rule can be handled by the current backend, F otherwise.
.. zeek:field:: match_pred :zeek:type:`function` (p: :zeek:type:`NetControl::PluginState`, e: :zeek:type:`NetControl::Entity`, m: :zeek:type:`vector` of :zeek:type:`OpenFlow::ofp_match`) : :zeek:type:`vector` of :zeek:type:`OpenFlow::ofp_match` :zeek:attr:`&optional`
This predicate is called each time an OpenFlow match record is created.
The predicate can modify the match structure before it is sent on to the
device.
:param p: Current plugin state.
:param r: The rule to be inserted or removed.
:param m: The openflow match structures that were generated for this rules.
:returns: The modified OpenFlow match structures that will be used in place of the structures passed in m.
.. zeek:field:: flow_mod_pred :zeek:type:`function` (p: :zeek:type:`NetControl::PluginState`, r: :zeek:type:`NetControl::Rule`, m: :zeek:type:`OpenFlow::ofp_flow_mod`) : :zeek:type:`OpenFlow::ofp_flow_mod` :zeek:attr:`&optional`
This predicate is called before a FlowMod message is sent to the OpenFlow
device. It can modify the FlowMod message before it is passed on.
:param p: Current plugin state.
:param r: The rule to be inserted or removed.
:param m: The OpenFlow FlowMod message.
:returns: The modified FlowMod message that is used in lieu of m.
This record specifies the configuration that is passed to :zeek:see:`NetControl::create_openflow`.
.. zeek:type:: NetControl::OfTable
:source-code: base/frameworks/netcontrol/plugins/openflow.zeek 60 67
:Type: :zeek:type:`record`
.. zeek:field:: p :zeek:type:`NetControl::PluginState`
.. zeek:field:: r :zeek:type:`NetControl::Rule`
.. zeek:field:: c :zeek:type:`count` :zeek:attr:`&default` = ``0`` :zeek:attr:`&optional`
.. zeek:field:: packet_count :zeek:type:`count` :zeek:attr:`&default` = ``0`` :zeek:attr:`&optional`
.. zeek:field:: byte_count :zeek:type:`count` :zeek:attr:`&default` = ``0`` :zeek:attr:`&optional`
.. zeek:field:: duration_sec :zeek:type:`double` :zeek:attr:`&default` = ``0.0`` :zeek:attr:`&optional`
Functions
#########
.. zeek:id:: NetControl::create_openflow
:source-code: base/frameworks/netcontrol/plugins/openflow.zeek 448 453
:Type: :zeek:type:`function` (controller: :zeek:type:`OpenFlow::Controller`, config: :zeek:type:`NetControl::OfConfig` :zeek:attr:`&default` = ``[]`` :zeek:attr:`&optional`) : :zeek:type:`NetControl::PluginState`
Instantiates an openflow plugin for the NetControl framework.

35
doc/scripts/base/frameworks/netcontrol/plugins/packetfilter.zeek.rst Normal file
View file

@ -0,0 +1,35 @@
:tocdepth: 3
base/frameworks/netcontrol/plugins/packetfilter.zeek
====================================================
.. zeek:namespace:: NetControl
NetControl plugin for the process-level PacketFilter that comes with
Zeek. Since the PacketFilter in Zeek is quite limited in scope
and can only add/remove filters for addresses, this is quite
limited in scope at the moment.
:Namespace: NetControl
:Imports: :doc:`base/frameworks/netcontrol/plugin.zeek </scripts/base/frameworks/netcontrol/plugin.zeek>`
Summary
~~~~~~~
Functions
#########
================================================================= =====================================
:zeek:id:`NetControl::create_packetfilter`: :zeek:type:`function` Instantiates the packetfilter plugin.
================================================================= =====================================
Detailed Interface
~~~~~~~~~~~~~~~~~~
Functions
#########
.. zeek:id:: NetControl::create_packetfilter
:source-code: base/frameworks/netcontrol/plugins/packetfilter.zeek 107 112
:Type: :zeek:type:`function` () : :zeek:type:`NetControl::PluginState`
Instantiates the packetfilter plugin.

123
doc/scripts/base/frameworks/netcontrol/shunt.zeek.rst Normal file
View file

@ -0,0 +1,123 @@
:tocdepth: 3
base/frameworks/netcontrol/shunt.zeek
=====================================
.. zeek:namespace:: NetControl
Implementation of the shunt functionality for NetControl.
:Namespace: NetControl
:Imports: :doc:`base/frameworks/netcontrol/main.zeek </scripts/base/frameworks/netcontrol/main.zeek>`
Summary
~~~~~~~
Types
#####
======================================================= =
:zeek:type:`NetControl::ShuntInfo`: :zeek:type:`record`
======================================================= =
Redefinitions
#############
======================================= ================================
:zeek:type:`Log::ID`: :zeek:type:`enum`
* :zeek:enum:`NetControl::SHUNT`
======================================= ================================
Events
######
=============================================================== ==========================================================================
:zeek:id:`NetControl::log_netcontrol_shunt`: :zeek:type:`event` Event that can be handled to access the :zeek:type:`NetControl::ShuntInfo`
record as it is sent on to the logging framework.
=============================================================== ==========================================================================
Hooks
#####
===================================================================== =
:zeek:id:`NetControl::log_policy_shunt`: :zeek:type:`Log::PolicyHook`
===================================================================== =
Functions
#########
======================================================== ==========================================================
:zeek:id:`NetControl::shunt_flow`: :zeek:type:`function` Stops forwarding a uni-directional flow's packets to Zeek.
======================================================== ==========================================================
Detailed Interface
~~~~~~~~~~~~~~~~~~
Types
#####
.. zeek:type:: NetControl::ShuntInfo
:source-code: base/frameworks/netcontrol/shunt.zeek 23 34
:Type: :zeek:type:`record`
.. zeek:field:: ts :zeek:type:`time` :zeek:attr:`&log`
Time at which the recorded activity occurred.
.. zeek:field:: rule_id :zeek:type:`string` :zeek:attr:`&log`
ID of the rule; unique during each Zeek run.
.. zeek:field:: f :zeek:type:`flow_id` :zeek:attr:`&log`
Flow ID of the shunted flow.
.. zeek:field:: expire :zeek:type:`interval` :zeek:attr:`&log`
Expiry time of the shunt.
.. zeek:field:: location :zeek:type:`string` :zeek:attr:`&log` :zeek:attr:`&optional`
Location where the underlying action was triggered.
Events
######
.. zeek:id:: NetControl::log_netcontrol_shunt
:source-code: base/frameworks/netcontrol/shunt.zeek 38 38
:Type: :zeek:type:`event` (rec: :zeek:type:`NetControl::ShuntInfo`)
Event that can be handled to access the :zeek:type:`NetControl::ShuntInfo`
record as it is sent on to the logging framework.
Hooks
#####
.. zeek:id:: NetControl::log_policy_shunt
:source-code: base/frameworks/netcontrol/shunt.zeek 10 10
:Type: :zeek:type:`Log::PolicyHook`
Functions
#########
.. zeek:id:: NetControl::shunt_flow
:source-code: base/frameworks/netcontrol/shunt.zeek 46 70
:Type: :zeek:type:`function` (f: :zeek:type:`flow_id`, t: :zeek:type:`interval`, location: :zeek:type:`string` :zeek:attr:`&default` = ``""`` :zeek:attr:`&optional`) : :zeek:type:`string`
Stops forwarding a uni-directional flow's packets to Zeek.
:param f: The flow to shunt.
:param t: How long to leave the shunt in place, with 0 being indefinitely.
:param location: An optional string describing where the shunt was triggered.
:returns: The id of the inserted rule on success and zero on failure.

380
doc/scripts/base/frameworks/netcontrol/types.zeek.rst Normal file
View file

@ -0,0 +1,380 @@
:tocdepth: 3
base/frameworks/netcontrol/types.zeek
=====================================
.. zeek:namespace:: NetControl
This file defines the types that are used by the NetControl framework.
The most important type defined in this file is :zeek:see:`NetControl::Rule`,
which is used to describe all rules that can be expressed by the NetControl framework.
:Namespace: NetControl
Summary
~~~~~~~
Runtime Options
###############
============================================================================= ======================================================
:zeek:id:`NetControl::default_priority`: :zeek:type:`int` :zeek:attr:`&redef` The default priority that is used when creating rules.
============================================================================= ======================================================
Redefinable Options
###################
=============================================================================== =====================================================================================
:zeek:id:`NetControl::whitelist_priority`: :zeek:type:`int` :zeek:attr:`&redef` The default priority that is used when using the high-level functions to
push whitelist entries to the backends (:zeek:see:`NetControl::whitelist_address` and
:zeek:see:`NetControl::whitelist_subnet`).
=============================================================================== =====================================================================================
Types
#####
====================================================== ======================================================================================================
:zeek:type:`NetControl::Entity`: :zeek:type:`record` Type defining the entity a rule is operating on.
:zeek:type:`NetControl::EntityType`: :zeek:type:`enum` Type defining the entity that a rule applies to.
:zeek:type:`NetControl::Flow`: :zeek:type:`record` Flow is used in :zeek:type:`NetControl::Entity` together with :zeek:enum:`NetControl::FLOW` to specify
a uni-directional flow that a rule applies to.
:zeek:type:`NetControl::FlowInfo`: :zeek:type:`record` Information of a flow that can be provided by switches when the flow times out.
:zeek:type:`NetControl::FlowMod`: :zeek:type:`record` Type for defining a flow modification action.
:zeek:type:`NetControl::Rule`: :zeek:type:`record` A rule for the framework to put in place.
:zeek:type:`NetControl::RuleType`: :zeek:type:`enum` Type of rules that the framework supports.
:zeek:type:`NetControl::TargetType`: :zeek:type:`enum` Type defining the target of a rule.
====================================================== ======================================================================================================
Detailed Interface
~~~~~~~~~~~~~~~~~~
Runtime Options
###############
.. zeek:id:: NetControl::default_priority
:source-code: base/frameworks/netcontrol/types.zeek 10 10
:Type: :zeek:type:`int`
:Attributes: :zeek:attr:`&redef`
:Default: ``0``
The default priority that is used when creating rules.
Redefinable Options
###################
.. zeek:id:: NetControl::whitelist_priority
:source-code: base/frameworks/netcontrol/types.zeek 18 18
:Type: :zeek:type:`int`
:Attributes: :zeek:attr:`&redef`
:Default: ``5``
The default priority that is used when using the high-level functions to
push whitelist entries to the backends (:zeek:see:`NetControl::whitelist_address` and
:zeek:see:`NetControl::whitelist_subnet`).
Note that this priority is not automatically used when manually creating rules
that have a :zeek:see:`NetControl::RuleType` of :zeek:enum:`NetControl::WHITELIST`.
Types
#####
.. zeek:type:: NetControl::Entity
:source-code: base/frameworks/netcontrol/types.zeek 42 48
:Type: :zeek:type:`record`
.. zeek:field:: ty :zeek:type:`NetControl::EntityType`
Type of entity.
.. zeek:field:: conn :zeek:type:`conn_id` :zeek:attr:`&optional`
Used with :zeek:enum:`NetControl::CONNECTION`.
.. zeek:field:: flow :zeek:type:`NetControl::Flow` :zeek:attr:`&optional`
Used with :zeek:enum:`NetControl::FLOW`.
.. zeek:field:: ip :zeek:type:`subnet` :zeek:attr:`&optional`
Used with :zeek:enum:`NetControl::ADDRESS` to specify a CIDR subnet.
.. zeek:field:: mac :zeek:type:`string` :zeek:attr:`&optional`
Used with :zeek:enum:`NetControl::MAC`.
Type defining the entity a rule is operating on.
.. zeek:type:: NetControl::EntityType
:source-code: base/frameworks/netcontrol/types.zeek 21 27
:Type: :zeek:type:`enum`
.. zeek:enum:: NetControl::ADDRESS NetControl::EntityType
Activity involving a specific IP address.
.. zeek:enum:: NetControl::CONNECTION NetControl::EntityType
Activity involving all of a bi-directional connection's activity.
.. zeek:enum:: NetControl::FLOW NetControl::EntityType
Activity involving a uni-directional flow's activity. Can contain wildcards.
.. zeek:enum:: NetControl::MAC NetControl::EntityType
Activity involving a MAC address.
Type defining the entity that a rule applies to.
.. zeek:type:: NetControl::Flow
:source-code: base/frameworks/netcontrol/types.zeek 32 39
:Type: :zeek:type:`record`
.. zeek:field:: src_h :zeek:type:`subnet` :zeek:attr:`&optional`
The source IP address/subnet.
.. zeek:field:: src_p :zeek:type:`port` :zeek:attr:`&optional`
The source port number.
.. zeek:field:: dst_h :zeek:type:`subnet` :zeek:attr:`&optional`
The destination IP address/subnet.
.. zeek:field:: dst_p :zeek:type:`port` :zeek:attr:`&optional`
The destination port number.
.. zeek:field:: src_m :zeek:type:`string` :zeek:attr:`&optional`
The source MAC address.
.. zeek:field:: dst_m :zeek:type:`string` :zeek:attr:`&optional`
The destination MAC address.
Flow is used in :zeek:type:`NetControl::Entity` together with :zeek:enum:`NetControl::FLOW` to specify
a uni-directional flow that a rule applies to.
If optional fields are not set, they are interpreted as wildcarded.
.. zeek:type:: NetControl::FlowInfo
:source-code: base/frameworks/netcontrol/types.zeek 122 126
:Type: :zeek:type:`record`
.. zeek:field:: duration :zeek:type:`interval` :zeek:attr:`&optional`
Total duration of the rule.
.. zeek:field:: packet_count :zeek:type:`count` :zeek:attr:`&optional`
Number of packets exchanged over connections matched by the rule.
.. zeek:field:: byte_count :zeek:type:`count` :zeek:attr:`&optional`
Total bytes exchanged over connections matched by the rule.
Information of a flow that can be provided by switches when the flow times out.
Currently this is heavily influenced by the data that OpenFlow returns by default.
That being said - their design makes sense and this is probably the data one
can expect to be available.
.. zeek:type:: NetControl::FlowMod
:source-code: base/frameworks/netcontrol/types.zeek 90 98
:Type: :zeek:type:`record`
.. zeek:field:: src_h :zeek:type:`addr` :zeek:attr:`&optional`
The source IP address.
.. zeek:field:: src_p :zeek:type:`count` :zeek:attr:`&optional`
The source port number.
.. zeek:field:: dst_h :zeek:type:`addr` :zeek:attr:`&optional`
The destination IP address.
.. zeek:field:: dst_p :zeek:type:`count` :zeek:attr:`&optional`
The destination port number.
.. zeek:field:: src_m :zeek:type:`string` :zeek:attr:`&optional`
The source MAC address.
.. zeek:field:: dst_m :zeek:type:`string` :zeek:attr:`&optional`
The destination MAC address.
.. zeek:field:: redirect_port :zeek:type:`count` :zeek:attr:`&optional`
Type for defining a flow modification action.
.. zeek:type:: NetControl::Rule
:source-code: base/frameworks/netcontrol/types.zeek 103 116
:Type: :zeek:type:`record`
.. zeek:field:: ty :zeek:type:`NetControl::RuleType`
Type of rule.
.. zeek:field:: target :zeek:type:`NetControl::TargetType`
Where to apply rule.
.. zeek:field:: entity :zeek:type:`NetControl::Entity`
Entity to apply rule to.
.. zeek:field:: expire :zeek:type:`interval` :zeek:attr:`&optional`
Timeout after which to expire the rule.
.. zeek:field:: priority :zeek:type:`int` :zeek:attr:`&default` = :zeek:see:`NetControl::default_priority` :zeek:attr:`&optional`
Priority if multiple rules match an entity (larger value is higher priority).
.. zeek:field:: location :zeek:type:`string` :zeek:attr:`&optional`
Optional string describing where/what installed the rule.
.. zeek:field:: out_port :zeek:type:`count` :zeek:attr:`&optional`
Argument for :zeek:enum:`NetControl::REDIRECT` rules.
.. zeek:field:: mod :zeek:type:`NetControl::FlowMod` :zeek:attr:`&optional`
Argument for :zeek:enum:`NetControl::MODIFY` rules.
.. zeek:field:: id :zeek:type:`string` :zeek:attr:`&default` = ``""`` :zeek:attr:`&optional`
Internally determined unique ID for this rule. Will be set when added.
.. zeek:field:: cid :zeek:type:`count` :zeek:attr:`&default` = ``0`` :zeek:attr:`&optional`
Internally determined unique numeric ID for this rule. Set when added.
.. zeek:field:: _plugin_ids :zeek:type:`set` [:zeek:type:`count`] :zeek:attr:`&default` = ``{ }`` :zeek:attr:`&optional`
(present if :doc:`/scripts/base/frameworks/netcontrol/main.zeek` is loaded)
Internally set to the plugins handling the rule.
.. zeek:field:: _active_plugin_ids :zeek:type:`set` [:zeek:type:`count`] :zeek:attr:`&default` = ``{ }`` :zeek:attr:`&optional`
(present if :doc:`/scripts/base/frameworks/netcontrol/main.zeek` is loaded)
Internally set to the plugins on which the rule is currently active.
.. zeek:field:: _no_expire_plugins :zeek:type:`set` [:zeek:type:`count`] :zeek:attr:`&default` = ``{ }`` :zeek:attr:`&optional`
(present if :doc:`/scripts/base/frameworks/netcontrol/main.zeek` is loaded)
Internally set to plugins where the rule should not be removed upon timeout.
.. zeek:field:: _added :zeek:type:`bool` :zeek:attr:`&default` = ``F`` :zeek:attr:`&optional`
(present if :doc:`/scripts/base/frameworks/netcontrol/main.zeek` is loaded)
Track if the rule was added successfully by all responsible plugins.
A rule for the framework to put in place. Of all rules currently in
place, the first match will be taken, sorted by priority. All
further rules will be ignored.
.. zeek:type:: NetControl::RuleType
:source-code: base/frameworks/netcontrol/types.zeek 65 88
:Type: :zeek:type:`enum`
.. zeek:enum:: NetControl::DROP NetControl::RuleType
Stop forwarding all packets matching the entity.
No additional arguments.
.. zeek:enum:: NetControl::MODIFY NetControl::RuleType
Modify all packets matching entity. The packets
will be modified according to the `mod` entry of
the rule.
.. zeek:enum:: NetControl::REDIRECT NetControl::RuleType
Redirect all packets matching entity to a different switch port,
given in the `out_port` argument of the rule.
.. zeek:enum:: NetControl::WHITELIST NetControl::RuleType
Whitelists all packets of an entity, meaning no restrictions will be applied.
While whitelisting is the default if no rule matches, this type can be
used to override lower-priority rules that would otherwise take effect for the
entity.
Type of rules that the framework supports. Each type lists the extra
:zeek:type:`NetControl::Rule` fields it uses, if any.
Plugins may extend this type to define their own.
.. zeek:type:: NetControl::TargetType
:source-code: base/frameworks/netcontrol/types.zeek 56 60
:Type: :zeek:type:`enum`
.. zeek:enum:: NetControl::FORWARD NetControl::TargetType
.. zeek:enum:: NetControl::MONITOR NetControl::TargetType
Type defining the target of a rule.
Rules can either be applied to the forward path, affecting all network traffic, or
on the monitor path, only affecting the traffic that is sent to Zeek. The second
is mostly used for shunting, which allows Zeek to tell the networking hardware that
it wants to no longer see traffic that it identified as benign.

14
doc/scripts/base/frameworks/notice/__load__.zeek.rst Normal file
View file

@ -0,0 +1,14 @@
:tocdepth: 3
base/frameworks/notice/__load__.zeek
====================================
:Imports: :doc:`base/frameworks/notice/actions/add-geodata.zeek </scripts/base/frameworks/notice/actions/add-geodata.zeek>`, :doc:`base/frameworks/notice/actions/email_admin.zeek </scripts/base/frameworks/notice/actions/email_admin.zeek>`, :doc:`base/frameworks/notice/actions/page.zeek </scripts/base/frameworks/notice/actions/page.zeek>`, :doc:`base/frameworks/notice/actions/pp-alarms.zeek </scripts/base/frameworks/notice/actions/pp-alarms.zeek>`, :doc:`base/frameworks/notice/main.zeek </scripts/base/frameworks/notice/main.zeek>`, :doc:`base/frameworks/notice/weird.zeek </scripts/base/frameworks/notice/weird.zeek>`
Summary
~~~~~~~
Detailed Interface
~~~~~~~~~~~~~~~~~~

56
doc/scripts/base/frameworks/notice/actions/add-geodata.zeek.rst Normal file
View file

@ -0,0 +1,56 @@
:tocdepth: 3
base/frameworks/notice/actions/add-geodata.zeek
===============================================
.. zeek:namespace:: Notice
This script adds geographic location data to notices for the "remote"
host in a connection. It does make the assumption that one of the
addresses in a connection is "local" and one is "remote" which is
probably a safe assumption to make in most cases. If both addresses
are remote, it will use the $src address.
:Namespace: Notice
:Imports: :doc:`base/frameworks/notice </scripts/base/frameworks/notice/index>`, :doc:`base/frameworks/notice/main.zeek </scripts/base/frameworks/notice/main.zeek>`, :doc:`base/utils/site.zeek </scripts/base/utils/site.zeek>`
Summary
~~~~~~~
Runtime Options
###############
============================================================================== ===============================================================
:zeek:id:`Notice::lookup_location_types`: :zeek:type:`set` :zeek:attr:`&redef` Notice types which should have the "remote" location looked up.
============================================================================== ===============================================================
Redefinitions
#############
============================================== =====================================================================================
:zeek:type:`Notice::Action`: :zeek:type:`enum`
* :zeek:enum:`Notice::ACTION_ADD_GEODATA`:
Indicates that the notice should have geodata added for the
"remote" host.
:zeek:type:`Notice::Info`: :zeek:type:`record`
:New Fields: :zeek:type:`Notice::Info`
remote_location: :zeek:type:`geo_location` :zeek:attr:`&log` :zeek:attr:`&optional`
If GeoIP support is built in, notices can have geographic
information attached to them.
============================================== =====================================================================================
Detailed Interface
~~~~~~~~~~~~~~~~~~
Runtime Options
###############
.. zeek:id:: Notice::lookup_location_types
:source-code: base/frameworks/notice/actions/add-geodata.zeek 29 29
:Type: :zeek:type:`set` [:zeek:type:`Notice::Type`]
:Attributes: :zeek:attr:`&redef`
:Default: ``{}``
Notice types which should have the "remote" location looked up.
If GeoIP support is not built in, this does nothing.

32
doc/scripts/base/frameworks/notice/actions/email_admin.zeek.rst Normal file
View file

@ -0,0 +1,32 @@
:tocdepth: 3
base/frameworks/notice/actions/email_admin.zeek
===============================================
.. zeek:namespace:: Notice
Adds a new notice action type which can be used to email notices
to the administrators of a particular address space as set by
:zeek:id:`Site::local_admins` if the notice contains a source
or destination address that lies within their space.
:Namespace: Notice
:Imports: :doc:`base/frameworks/notice/main.zeek </scripts/base/frameworks/notice/main.zeek>`, :doc:`base/utils/site.zeek </scripts/base/utils/site.zeek>`
Summary
~~~~~~~
Redefinitions
#############
============================================== ==============================================================
:zeek:type:`Notice::Action`: :zeek:type:`enum`
* :zeek:enum:`Notice::ACTION_EMAIL_ADMIN`:
Indicate that the generated email should be addressed to the
appropriate email addresses as found by the
:zeek:id:`Site::get_emails` function based on the relevant
address or addresses indicated in the notice.
============================================== ==============================================================
Detailed Interface
~~~~~~~~~~~~~~~~~~

47
doc/scripts/base/frameworks/notice/actions/page.zeek.rst Normal file
View file

@ -0,0 +1,47 @@
:tocdepth: 3
base/frameworks/notice/actions/page.zeek
========================================
.. zeek:namespace:: Notice
Allows configuration of a pager email address to which notices can be sent.
:Namespace: Notice
:Imports: :doc:`base/frameworks/notice/main.zeek </scripts/base/frameworks/notice/main.zeek>`
Summary
~~~~~~~
Runtime Options
###############
========================================================================== =======================================================================
:zeek:id:`Notice::mail_page_dest`: :zeek:type:`string` :zeek:attr:`&redef` Email address to send notices with the :zeek:enum:`Notice::ACTION_PAGE`
action.
========================================================================== =======================================================================
Redefinitions
#############
============================================== =============================================================
:zeek:type:`Notice::Action`: :zeek:type:`enum`
* :zeek:enum:`Notice::ACTION_PAGE`:
Indicates that the notice should be sent to the pager email
address configured in the :zeek:id:`Notice::mail_page_dest`
variable.
============================================== =============================================================
Detailed Interface
~~~~~~~~~~~~~~~~~~
Runtime Options
###############
.. zeek:id:: Notice::mail_page_dest
:source-code: base/frameworks/notice/actions/page.zeek 17 17
:Type: :zeek:type:`string`
:Attributes: :zeek:attr:`&redef`
:Default: ``""``
Email address to send notices with the :zeek:enum:`Notice::ACTION_PAGE`
action.

97
doc/scripts/base/frameworks/notice/actions/pp-alarms.zeek.rst Normal file
View file

@ -0,0 +1,97 @@
:tocdepth: 3
base/frameworks/notice/actions/pp-alarms.zeek
=============================================
.. zeek:namespace:: Notice
Notice extension that mails out a pretty-printed version of notice_alarm.log
in regular intervals, formatted for better human readability. If activated,
that replaces the default summary mail having the raw log output.
:Namespace: Notice
:Imports: :doc:`base/frameworks/cluster </scripts/base/frameworks/cluster/index>`, :doc:`base/frameworks/notice/main.zeek </scripts/base/frameworks/notice/main.zeek>`
Summary
~~~~~~~
Redefinable Options
###################
==================================================================================== ==============================================
:zeek:id:`Notice::mail_dest_pretty_printed`: :zeek:type:`string` :zeek:attr:`&redef` Address to send the pretty-printed reports to.
:zeek:id:`Notice::pretty_print_alarms`: :zeek:type:`bool` :zeek:attr:`&redef` Activate pretty-printed alarm summaries.
==================================================================================== ==============================================
State Variables
###############
=============================================================================== ==================================================================
:zeek:id:`Notice::flag_nets`: :zeek:type:`set` :zeek:attr:`&redef` If an address from one of these networks is reported, we mark
the entry with an additional quote symbol (i.e., ">").
:zeek:id:`Notice::force_email_summaries`: :zeek:type:`bool` :zeek:attr:`&redef` Force generating mail file, even if reading from traces or no mail
destination is defined.
=============================================================================== ==================================================================
Functions
#########
================================================================================ =====================================
:zeek:id:`Notice::pretty_print_alarm`: :zeek:type:`function` :zeek:attr:`&redef` Function that renders a single alarm.
================================================================================ =====================================
Detailed Interface
~~~~~~~~~~~~~~~~~~
Redefinable Options
###################
.. zeek:id:: Notice::mail_dest_pretty_printed
:source-code: base/frameworks/notice/actions/pp-alarms.zeek 18 18
:Type: :zeek:type:`string`
:Attributes: :zeek:attr:`&redef`
:Default: ``""``
Address to send the pretty-printed reports to. Default if not set is
:zeek:id:`Notice::mail_dest`.
Note that this is overridden by the ZeekControl MailAlarmsTo option.
.. zeek:id:: Notice::pretty_print_alarms
:source-code: base/frameworks/notice/actions/pp-alarms.zeek 12 12
:Type: :zeek:type:`bool`
:Attributes: :zeek:attr:`&redef`
:Default: ``T``
Activate pretty-printed alarm summaries.
State Variables
###############
.. zeek:id:: Notice::flag_nets
:source-code: base/frameworks/notice/actions/pp-alarms.zeek 22 22
:Type: :zeek:type:`set` [:zeek:type:`subnet`]
:Attributes: :zeek:attr:`&redef`
:Default: ``{}``
If an address from one of these networks is reported, we mark
the entry with an additional quote symbol (i.e., ">"). Many MUAs
then highlight such lines differently.
.. zeek:id:: Notice::force_email_summaries
:source-code: base/frameworks/notice/actions/pp-alarms.zeek 29 29
:Type: :zeek:type:`bool`
:Attributes: :zeek:attr:`&redef`
:Default: ``F``
Force generating mail file, even if reading from traces or no mail
destination is defined. This is mainly for testing.
Functions
#########
.. zeek:id:: Notice::pretty_print_alarm
:source-code: base/frameworks/notice/actions/pp-alarms.zeek 152 254
:Type: :zeek:type:`function` (out: :zeek:type:`file`, n: :zeek:type:`Notice::Info`) : :zeek:type:`void`
:Attributes: :zeek:attr:`&redef`
Function that renders a single alarm. Can be overridden.

57
doc/scripts/base/frameworks/notice/index.rst Normal file
View file

@ -0,0 +1,57 @@
:orphan:
Package: base/frameworks/notice
===============================
The notice framework enables Zeek to "notice" things which are odd or
potentially bad, leaving it to the local configuration to define which
of them are actionable. This decoupling of detection and reporting allows
Zeek to be customized to the different needs that sites have.
:doc:`/scripts/base/frameworks/notice/__load__.zeek`
:doc:`/scripts/base/frameworks/notice/main.zeek`
This is the notice framework which enables Zeek to "notice" things which
are odd or potentially bad. Decisions of the meaning of various notices
need to be done per site because Zeek does not ship with assumptions about
what is bad activity for sites. More extensive documentation about using
the notice framework can be found in :doc:`/frameworks/notice`.
:doc:`/scripts/base/frameworks/notice/weird.zeek`
This script provides a default set of actions to take for "weird activity"
events generated from Zeek's event engine. Weird activity is defined as
unusual or exceptional activity that can indicate malformed connections,
traffic that doesn't conform to a particular protocol, malfunctioning
or misconfigured hardware, or even an attacker attempting to avoid/confuse
a sensor. Without context, it's hard to judge whether a particular
category of weird activity is interesting, but this script provides
a starting point for the user.
:doc:`/scripts/base/frameworks/notice/actions/email_admin.zeek`
Adds a new notice action type which can be used to email notices
to the administrators of a particular address space as set by
:zeek:id:`Site::local_admins` if the notice contains a source
or destination address that lies within their space.
:doc:`/scripts/base/frameworks/notice/actions/page.zeek`
Allows configuration of a pager email address to which notices can be sent.
:doc:`/scripts/base/frameworks/notice/actions/add-geodata.zeek`
This script adds geographic location data to notices for the "remote"
host in a connection. It does make the assumption that one of the
addresses in a connection is "local" and one is "remote" which is
probably a safe assumption to make in most cases. If both addresses
are remote, it will use the $src address.
:doc:`/scripts/base/frameworks/notice/actions/pp-alarms.zeek`
Notice extension that mails out a pretty-printed version of notice_alarm.log
in regular intervals, formatted for better human readability. If activated,
that replaces the default summary mail having the raw log output.

1156
doc/scripts/base/frameworks/notice/main.zeek.rst Normal file
View file

File diff suppressed because it is too large Load diff

482
doc/scripts/base/frameworks/notice/weird.zeek.rst Normal file
View file

@ -0,0 +1,482 @@
:tocdepth: 3
base/frameworks/notice/weird.zeek
=================================
.. zeek:namespace:: Weird
This script provides a default set of actions to take for "weird activity"
events generated from Zeek's event engine. Weird activity is defined as
unusual or exceptional activity that can indicate malformed connections,
traffic that doesn't conform to a particular protocol, malfunctioning
or misconfigured hardware, or even an attacker attempting to avoid/confuse
a sensor. Without context, it's hard to judge whether a particular
category of weird activity is interesting, but this script provides
a starting point for the user.
:Namespace: Weird
:Imports: :doc:`base/frameworks/notice/main.zeek </scripts/base/frameworks/notice/main.zeek>`, :doc:`base/utils/conn-ids.zeek </scripts/base/utils/conn-ids.zeek>`, :doc:`base/utils/site.zeek </scripts/base/utils/site.zeek>`
Summary
~~~~~~~
Runtime Options
###############
=================================================================================== ==============================================================
:zeek:id:`Weird::ignore_hosts`: :zeek:type:`set` :zeek:attr:`&redef` To completely ignore a specific weird for a host, add the host
and weird name into this set.
:zeek:id:`Weird::weird_do_not_ignore_repeats`: :zeek:type:`set` :zeek:attr:`&redef` Don't ignore repeats for weirds in this set.
=================================================================================== ==============================================================
Redefinable Options
###################
=============================================================================================================== ==============================================================
:zeek:id:`Weird::actions`: :zeek:type:`table` :zeek:attr:`&default` = ``Weird::ACTION_LOG`` :zeek:attr:`&redef` A table specifying default/recommended actions per weird type.
=============================================================================================================== ==============================================================
State Variables
###############
================================================================================================================ ====================================================================
:zeek:id:`Weird::did_log`: :zeek:type:`set` :zeek:attr:`&create_expire` = ``1.0 day`` :zeek:attr:`&redef` A state set which tracks unique weirds solely by name to reduce
duplicate logging.
:zeek:id:`Weird::did_notice`: :zeek:type:`set` :zeek:attr:`&create_expire` = ``1.0 day`` :zeek:attr:`&redef` A state set which tracks unique weirds solely by name to reduce
duplicate notices from being raised.
:zeek:id:`Weird::weird_ignore`: :zeek:type:`set` :zeek:attr:`&create_expire` = ``10.0 mins`` :zeek:attr:`&redef` This table is used to track identifier and name pairs that should be
temporarily ignored because the problem has already been reported.
================================================================================================================ ====================================================================
Types
#####
============================================= =======================================================================
:zeek:type:`Weird::Action`: :zeek:type:`enum` Types of actions that may be taken when handling weird activity events.
:zeek:type:`Weird::Info`: :zeek:type:`record` The record which is used for representing and logging weirds.
============================================= =======================================================================
Redefinitions
#############
============================================ ===================================================
:zeek:type:`Log::ID`: :zeek:type:`enum` The weird logging stream identifier.
* :zeek:enum:`Weird::LOG`
:zeek:type:`Notice::Type`: :zeek:type:`enum`
* :zeek:enum:`Weird::Activity`:
Generic unusual but notice-worthy weird activity.
============================================ ===================================================
Events
######
=============================================== ==============================================================
:zeek:id:`Weird::log_weird`: :zeek:type:`event` Handlers of this event are invoked once per write to the weird
logging stream before the data is actually written.
=============================================== ==============================================================
Hooks
#####
========================================================== =============================================
:zeek:id:`Weird::log_policy`: :zeek:type:`Log::PolicyHook` A default logging policy hook for the stream.
========================================================== =============================================
Functions
#########
============================================== =
:zeek:id:`Weird::weird`: :zeek:type:`function`
============================================== =
Detailed Interface
~~~~~~~~~~~~~~~~~~
Runtime Options
###############
.. zeek:id:: Weird::ignore_hosts
:source-code: base/frameworks/notice/weird.zeek 266 266
:Type: :zeek:type:`set` [:zeek:type:`addr`, :zeek:type:`string`]
:Attributes: :zeek:attr:`&redef`
:Default: ``{}``
To completely ignore a specific weird for a host, add the host
and weird name into this set.
.. zeek:id:: Weird::weird_do_not_ignore_repeats
:source-code: base/frameworks/notice/weird.zeek 270 270
:Type: :zeek:type:`set` [:zeek:type:`string`]
:Attributes: :zeek:attr:`&redef`
:Default:
::
{
"bad_ICMP_checksum",
"bad_UDP_checksum",
"bad_TCP_checksum",
"bad_IP_checksum"
}
Don't ignore repeats for weirds in this set. For example,
it's handy keeping track of clustered checksum errors.
Redefinable Options
###################
.. zeek:id:: Weird::actions
:source-code: base/frameworks/notice/weird.zeek 95 95
:Type: :zeek:type:`table` [:zeek:type:`string`] of :zeek:type:`Weird::Action`
:Attributes: :zeek:attr:`&default` = ``Weird::ACTION_LOG`` :zeek:attr:`&redef`
:Default:
::
{
["bad_ICMP_checksum"] = Weird::ACTION_LOG_PER_ORIG,
["truncated_IP"] = Weird::ACTION_LOG,
["data_after_reset"] = Weird::ACTION_LOG,
["data_before_established"] = Weird::ACTION_LOG,
["irc_invalid_oper_message_format"] = Weird::ACTION_LOG,
["irc_invalid_whois_user_line"] = Weird::ACTION_LOG,
["FTP_too_many_pending_commands"] = Weird::ACTION_LOG_PER_CONN,
["DNS_label_len_gt_name_len"] = Weird::ACTION_LOG_PER_ORIG,
["FTP_max_command_length_exceeded"] = Weird::ACTION_LOG_PER_CONN,
["irc_invalid_dcc_message_format"] = Weird::ACTION_LOG,
["netbios_server_session_request"] = Weird::ACTION_LOG,
["RPC_rexmit_inconsistency"] = Weird::ACTION_LOG,
["unpaired_RPC_response"] = Weird::ACTION_LOG,
["SYN_inside_connection"] = Weird::ACTION_LOG,
["excessively_large_fragment"] = Weird::ACTION_LOG,
["truncated_header"] = Weird::ACTION_LOG,
["contentline_size_exceeded"] = Weird::ACTION_LOG,
["SMB_discarded_dce_rpc_analyzers"] = Weird::ACTION_LOG,
["unescaped_%_in_URI"] = Weird::ACTION_LOG,
["smtp_mail_transaction_invalid"] = Weird::ACTION_LOG_PER_CONN,
["fragment_overlap"] = Weird::ACTION_LOG_PER_ORIG,
["smb_tree_connect_andx_response_without_tree"] = Weird::ACTION_LOG_PER_CONN,
["bad_TCP_checksum"] = Weird::ACTION_LOG_PER_ORIG,
["pop3_client_command_unknown"] = Weird::ACTION_LOG,
["TCP_christmas"] = Weird::ACTION_LOG,
["irc_invalid_notice_message_format"] = Weird::ACTION_LOG,
["partial_RPC"] = Weird::ACTION_LOG_PER_ORIG,
["irc_invalid_kick_message_format"] = Weird::ACTION_LOG,
["excess_RPC"] = Weird::ACTION_LOG_PER_ORIG,
["invalid_irc_global_users_reply"] = Weird::ACTION_LOG,
["FTP_reply_msg_too_long"] = Weird::ACTION_LOG_PER_CONN,
["bad_rsh_prolog"] = Weird::ACTION_LOG,
["irc_line_too_short"] = Weird::ACTION_LOG,
["UDP_datagram_length_mismatch"] = Weird::ACTION_LOG_PER_ORIG,
["bad_ident_reply"] = Weird::ACTION_LOG,
["fragment_with_DF"] = Weird::ACTION_LOG,
["irc_invalid_invite_message_format"] = Weird::ACTION_LOG,
["premature_connection_reuse"] = Weird::ACTION_LOG,
["fragment_size_inconsistency"] = Weird::ACTION_LOG_PER_ORIG,
["smtp_excessive_invalid_mail_transactions"] = Weird::ACTION_LOG_PER_CONN,
["SYN_with_data"] = Weird::ACTION_LOG_PER_ORIG,
["DNS_RR_length_mismatch"] = Weird::ACTION_LOG,
["irc_invalid_whois_operator_line"] = Weird::ACTION_LOG,
["bad_ident_port"] = Weird::ACTION_LOG,
["base64_illegal_encoding"] = Weird::ACTION_LOG,
["simultaneous_open"] = Weird::ACTION_LOG_PER_CONN,
["ident_request_addendum"] = Weird::ACTION_LOG,
["DNS_truncated_quest_too_short"] = Weird::ACTION_LOG,
["illegal_%_at_end_of_URI"] = Weird::ACTION_LOG,
["pop3_server_sending_client_commands"] = Weird::ACTION_LOG,
["irc_invalid_mode_message_format"] = Weird::ACTION_LOG,
["SYN_after_close"] = Weird::ACTION_LOG,
["window_recision"] = Weird::ACTION_LOG,
["inflate_failed"] = Weird::ACTION_LOG,
["DNS_truncated_len_lt_hdr_len"] = Weird::ACTION_LOG,
["irc_invalid_privmsg_message_format"] = Weird::ACTION_LOG,
["pending_data_when_closed"] = Weird::ACTION_LOG,
["excessive_data_without_further_acks"] = Weird::ACTION_LOG,
["netbios_raw_session_msg"] = Weird::ACTION_LOG,
["responder_RPC_call"] = Weird::ACTION_LOG_PER_ORIG,
["SYN_after_reset"] = Weird::ACTION_LOG,
["bad_SYN_ack"] = Weird::ACTION_LOG,
["unsolicited_SYN_response"] = Weird::ACTION_IGNORE,
["line_terminated_with_single_LF"] = Weird::ACTION_LOG,
["irc_invalid_names_line"] = Weird::ACTION_LOG,
["DNS_RR_unknown_type"] = Weird::ACTION_LOG,
["repeated_SYN_reply_wo_ack"] = Weird::ACTION_LOG,
["bad_HTTP_reply"] = Weird::ACTION_LOG,
["FIN_storm"] = Weird::ACTION_NOTICE_PER_ORIG,
["HTTP_response_before_request"] = Weird::ACTION_LOG,
["NUL_in_line"] = Weird::ACTION_LOG,
["pop3_server_command_unknown"] = Weird::ACTION_LOG,
["fragment_inconsistency"] = Weird::ACTION_LOG_PER_ORIG,
["FIN_advanced_last_seq"] = Weird::ACTION_LOG,
["baroque_SYN"] = Weird::ACTION_LOG,
["unexpected_multiple_HTTP_requests"] = Weird::ACTION_LOG,
["unknown_netbios_type"] = Weird::ACTION_LOG,
["FTP_user_too_long"] = Weird::ACTION_LOG_PER_CONN,
["bad_RPC"] = Weird::ACTION_LOG_PER_ORIG,
["incompletely_captured_fragment"] = Weird::ACTION_LOG,
["irc_too_many_invalid"] = Weird::ACTION_LOG,
["DNS_AAAA_neg_length"] = Weird::ACTION_LOG,
["SYN_seq_jump"] = Weird::ACTION_LOG,
["malformed_ssh_version"] = Weird::ACTION_LOG,
["bad_rlogin_prolog"] = Weird::ACTION_LOG,
["unmatched_HTTP_reply"] = Weird::ACTION_LOG,
["possible_split_routing"] = Weird::ACTION_LOG,
["irc_invalid_who_message_format"] = Weird::ACTION_LOG,
["HTTP_chunked_transfer_for_multipart_message"] = Weird::ACTION_LOG,
["irc_invalid_njoin_line"] = Weird::ACTION_LOG,
["FTP_arg_too_long"] = Weird::ACTION_LOG_PER_CONN,
["line_terminated_with_single_CR"] = Weird::ACTION_LOG,
["HTTP_overlapping_messages"] = Weird::ACTION_LOG,
["deficit_netbios_hdr_len"] = Weird::ACTION_LOG,
["repeated_SYN_with_ack"] = Weird::ACTION_LOG,
["irc_invalid_reply_number"] = Weird::ACTION_LOG,
["malformed_ssh_identification"] = Weird::ACTION_LOG,
["RPC_underflow"] = Weird::ACTION_LOG,
["unexpected_server_HTTP_data"] = Weird::ACTION_LOG,
["DNS_RR_bad_length"] = Weird::ACTION_LOG,
["SSL_many_server_names"] = Weird::ACTION_LOG,
["irc_invalid_whois_channel_line"] = Weird::ACTION_LOG,
["DNS_label_len_gt_pkt"] = Weird::ACTION_LOG_PER_ORIG,
["irc_line_size_exceeded"] = Weird::ACTION_LOG,
["HTTP_excessive_pipelining"] = Weird::ACTION_LOG,
["DNS_Conn_count_too_large"] = Weird::ACTION_LOG,
["irc_invalid_command"] = Weird::ACTION_LOG,
["inappropriate_FIN"] = Weird::ACTION_LOG,
["transaction_subcmd_missing"] = Weird::ACTION_LOG,
["HTTP_version_mismatch"] = Weird::ACTION_LOG,
["irc_invalid_join_line"] = Weird::ACTION_LOG,
["multiple_HTTP_request_elements"] = Weird::ACTION_LOG,
["rlogin_text_after_rejected"] = Weird::ACTION_LOG,
["spontaneous_RST"] = Weird::ACTION_IGNORE,
["bad_IP_checksum"] = Weird::ACTION_LOG_PER_ORIG,
["no_smb_session_using_parsesambamsg"] = Weird::ACTION_LOG,
["unexpected_client_HTTP_data"] = Weird::ACTION_LOG,
["SMB_parsing_error"] = Weird::ACTION_LOG,
["partial_ftp_request"] = Weird::ACTION_LOG,
["double_%_in_URI"] = Weird::ACTION_LOG,
["truncated_NTP"] = Weird::ACTION_LOG,
["internally_truncated_header"] = Weird::ACTION_LOG,
["partial_finger_request"] = Weird::ACTION_LOG,
["irc_invalid_squery_message_format"] = Weird::ACTION_LOG,
["DNS_label_forward_compress_offset"] = Weird::ACTION_LOG_PER_ORIG,
["connection_originator_SYN_ack"] = Weird::ACTION_LOG_PER_ORIG,
["excessively_small_fragment"] = Weird::ACTION_LOG_PER_ORIG,
["irc_invalid_line"] = Weird::ACTION_LOG,
["DNS_label_too_long"] = Weird::ACTION_LOG_PER_ORIG,
["bad_ident_request"] = Weird::ACTION_LOG,
["irc_invalid_who_line"] = Weird::ACTION_LOG,
["DNS_truncated_ans_too_short"] = Weird::ACTION_LOG,
["RST_with_data"] = Weird::ACTION_LOG,
["FTP_password_too_long"] = Weird::ACTION_LOG_PER_CONN,
["above_hole_data_without_any_acks"] = Weird::ACTION_LOG,
["bad_UDP_checksum"] = Weird::ACTION_LOG_PER_ORIG,
["partial_ident_request"] = Weird::ACTION_LOG,
["DNS_truncated_RR_rdlength_lt_len"] = Weird::ACTION_LOG,
["bad_RPC_program"] = Weird::ACTION_LOG,
["irc_invalid_topic_reply"] = Weird::ACTION_LOG,
["unescaped_special_URI_char"] = Weird::ACTION_LOG,
["successful_RPC_reply_to_invalid_request"] = Weird::ACTION_NOTICE_PER_ORIG,
["smb_andx_command_failed_to_parse"] = Weird::ACTION_LOG,
["SMB_discarded_messages_state"] = Weird::ACTION_LOG,
["DNS_NAME_too_long"] = Weird::ACTION_LOG,
["crud_trailing_HTTP_request"] = Weird::ACTION_LOG,
["spontaneous_FIN"] = Weird::ACTION_IGNORE,
["rsh_text_after_rejected"] = Weird::ACTION_LOG,
["pop3_malformed_auth_plain"] = Weird::ACTION_LOG,
["bad_HTTP_version"] = Weird::ACTION_LOG,
["bad_TCP_header_len"] = Weird::ACTION_LOG,
["unknown_HTTP_method"] = Weird::ACTION_LOG,
["netbios_client_session_reply"] = Weird::ACTION_LOG,
["SYN_after_partial"] = Weird::ACTION_NOTICE_PER_ORIG,
["irc_invalid_whois_message_format"] = Weird::ACTION_LOG,
["excess_netbios_hdr_len"] = Weird::ACTION_LOG,
["pop3_client_sending_server_commands"] = Weird::ACTION_LOG,
["RST_storm"] = Weird::ACTION_LOG,
["fragment_protocol_inconsistency"] = Weird::ACTION_LOG,
["originator_RPC_reply"] = Weird::ACTION_LOG_PER_ORIG,
["FIN_after_reset"] = Weird::ACTION_IGNORE,
["pop3_bad_base64_encoding"] = Weird::ACTION_LOG,
["active_connection_reuse"] = Weird::ACTION_LOG,
["truncated_ARP"] = Weird::ACTION_LOG,
["HTTP_bad_chunk_size"] = Weird::ACTION_LOG
}
A table specifying default/recommended actions per weird type.
State Variables
###############
.. zeek:id:: Weird::did_log
:source-code: base/frameworks/notice/weird.zeek 284 284
:Type: :zeek:type:`set` [:zeek:type:`string`, :zeek:type:`string`]
:Attributes: :zeek:attr:`&create_expire` = ``1.0 day`` :zeek:attr:`&redef`
:Default: ``{}``
A state set which tracks unique weirds solely by name to reduce
duplicate logging. This is deliberately not synchronized because it
could cause overload during storms.
.. zeek:id:: Weird::did_notice
:source-code: base/frameworks/notice/weird.zeek 288 288
:Type: :zeek:type:`set` [:zeek:type:`string`, :zeek:type:`string`]
:Attributes: :zeek:attr:`&create_expire` = ``1.0 day`` :zeek:attr:`&redef`
:Default: ``{}``
A state set which tracks unique weirds solely by name to reduce
duplicate notices from being raised.
.. zeek:id:: Weird::weird_ignore
:source-code: base/frameworks/notice/weird.zeek 279 279
:Type: :zeek:type:`set` [:zeek:type:`string`, :zeek:type:`string`]
:Attributes: :zeek:attr:`&create_expire` = ``10.0 mins`` :zeek:attr:`&redef`
:Default: ``{}``
This table is used to track identifier and name pairs that should be
temporarily ignored because the problem has already been reported.
This helps reduce the volume of high volume weirds by only allowing
a unique weird every ``create_expire`` interval.
Types
#####
.. zeek:type:: Weird::Action
:source-code: base/frameworks/notice/weird.zeek 70 93
:Type: :zeek:type:`enum`
.. zeek:enum:: Weird::ACTION_UNSPECIFIED Weird::Action
A dummy action indicating the user does not care what
internal decision is made regarding a given type of weird.
.. zeek:enum:: Weird::ACTION_IGNORE Weird::Action
No action is to be taken.
.. zeek:enum:: Weird::ACTION_LOG Weird::Action
Log the weird event every time it occurs.
.. zeek:enum:: Weird::ACTION_LOG_ONCE Weird::Action
Log the weird event only once.
.. zeek:enum:: Weird::ACTION_LOG_PER_CONN Weird::Action
Log the weird event once per connection.
.. zeek:enum:: Weird::ACTION_LOG_PER_ORIG Weird::Action
Log the weird event once per originator host.
.. zeek:enum:: Weird::ACTION_NOTICE Weird::Action
Always generate a notice associated with the weird event.
.. zeek:enum:: Weird::ACTION_NOTICE_ONCE Weird::Action
Generate a notice associated with the weird event only once.
.. zeek:enum:: Weird::ACTION_NOTICE_PER_CONN Weird::Action
Generate a notice for the weird event once per connection.
.. zeek:enum:: Weird::ACTION_NOTICE_PER_ORIG Weird::Action
Generate a notice for the weird event once per originator host.
Types of actions that may be taken when handling weird activity events.
.. zeek:type:: Weird::Info
:source-code: base/frameworks/notice/weird.zeek 29 67
:Type: :zeek:type:`record`
.. zeek:field:: ts :zeek:type:`time` :zeek:attr:`&log`
The time when the weird occurred.
.. zeek:field:: uid :zeek:type:`string` :zeek:attr:`&log` :zeek:attr:`&optional`
If a connection is associated with this weird, this will be
the connection's unique ID.
.. zeek:field:: id :zeek:type:`conn_id` :zeek:attr:`&log` :zeek:attr:`&optional`
conn_id for the optional connection.
.. zeek:field:: conn :zeek:type:`connection` :zeek:attr:`&optional`
A shorthand way of giving the uid and id to a weird.
.. zeek:field:: name :zeek:type:`string` :zeek:attr:`&log`
The name of the weird that occurred.
.. zeek:field:: addl :zeek:type:`string` :zeek:attr:`&log` :zeek:attr:`&optional`
Additional information accompanying the weird if any.
.. zeek:field:: notice :zeek:type:`bool` :zeek:attr:`&log` :zeek:attr:`&default` = ``F`` :zeek:attr:`&optional`
Indicate if this weird was also turned into a notice.
.. zeek:field:: peer :zeek:type:`string` :zeek:attr:`&log` :zeek:attr:`&default` = :zeek:see:`peer_description` :zeek:attr:`&optional`
The peer that originated this weird. This is helpful in
cluster deployments if a particular cluster node is having
trouble to help identify which node is having trouble.
.. zeek:field:: source :zeek:type:`string` :zeek:attr:`&log` :zeek:attr:`&optional`
The source of the weird. When reported by an analyzer, this
should be the name of the analyzer.
.. zeek:field:: identifier :zeek:type:`string` :zeek:attr:`&optional`
This field is to be provided when a weird is generated for
the purpose of deduplicating weirds. The identifier string
should be unique for a single instance of the weird. This field
is used to define when a weird is conceptually a duplicate of
a previous weird.
The record which is used for representing and logging weirds.
Events
######
.. zeek:id:: Weird::log_weird
:source-code: base/frameworks/notice/weird.zeek 294 294
:Type: :zeek:type:`event` (rec: :zeek:type:`Weird::Info`)
Handlers of this event are invoked once per write to the weird
logging stream before the data is actually written.
:param rec: The weird columns about to be logged to the weird stream.
Hooks
#####
.. zeek:id:: Weird::log_policy
:source-code: base/frameworks/notice/weird.zeek 21 21
:Type: :zeek:type:`Log::PolicyHook`
A default logging policy hook for the stream.
Functions
#########
.. zeek:id:: Weird::weird
:source-code: base/frameworks/notice/weird.zeek 329 417
:Type: :zeek:type:`function` (w: :zeek:type:`Weird::Info`) : :zeek:type:`void`

14
doc/scripts/base/frameworks/openflow/__load__.zeek.rst Normal file
View file

@ -0,0 +1,14 @@
:tocdepth: 3
base/frameworks/openflow/__load__.zeek
======================================
:Imports: :doc:`base/frameworks/cluster </scripts/base/frameworks/cluster/index>`, :doc:`base/frameworks/openflow/consts.zeek </scripts/base/frameworks/openflow/consts.zeek>`, :doc:`base/frameworks/openflow/main.zeek </scripts/base/frameworks/openflow/main.zeek>`, :doc:`base/frameworks/openflow/non-cluster.zeek </scripts/base/frameworks/openflow/non-cluster.zeek>`, :doc:`base/frameworks/openflow/plugins </scripts/base/frameworks/openflow/plugins/index>`, :doc:`base/frameworks/openflow/types.zeek </scripts/base/frameworks/openflow/types.zeek>`
Summary
~~~~~~~
Detailed Interface
~~~~~~~~~~~~~~~~~~

623
doc/scripts/base/frameworks/openflow/consts.zeek.rst Normal file
View file

@ -0,0 +1,623 @@
:tocdepth: 3
base/frameworks/openflow/consts.zeek
====================================
.. zeek:namespace:: OpenFlow
Constants used by the OpenFlow framework.
:Namespace: OpenFlow
Summary
~~~~~~~
Constants
#########
=============================================================== ======================================================================
:zeek:id:`OpenFlow::ETH_APPLETALK`: :zeek:type:`count`
:zeek:id:`OpenFlow::ETH_APPLETALK_ARP`: :zeek:type:`count`
:zeek:id:`OpenFlow::ETH_ARP`: :zeek:type:`count`
:zeek:id:`OpenFlow::ETH_EAP_OVER_LAN`: :zeek:type:`count`
:zeek:id:`OpenFlow::ETH_ETHER_FLOW_CONTROL`: :zeek:type:`count`
:zeek:id:`OpenFlow::ETH_IPX`: :zeek:type:`count`
:zeek:id:`OpenFlow::ETH_IPX_OLD`: :zeek:type:`count`
:zeek:id:`OpenFlow::ETH_IPv4`: :zeek:type:`count`
:zeek:id:`OpenFlow::ETH_IPv6`: :zeek:type:`count`
:zeek:id:`OpenFlow::ETH_JUMBO_FRAMES`: :zeek:type:`count`
:zeek:id:`OpenFlow::ETH_MAC_SECURITY`: :zeek:type:`count`
:zeek:id:`OpenFlow::ETH_MPLS_MULTICAST`: :zeek:type:`count`
:zeek:id:`OpenFlow::ETH_MPLS_UNICAST`: :zeek:type:`count`
:zeek:id:`OpenFlow::ETH_PPPOE_DISCOVERY`: :zeek:type:`count`
:zeek:id:`OpenFlow::ETH_PPPOE_SESSION`: :zeek:type:`count`
:zeek:id:`OpenFlow::ETH_PROVIDER_BRIDING`: :zeek:type:`count`
:zeek:id:`OpenFlow::ETH_QINQ`: :zeek:type:`count`
:zeek:id:`OpenFlow::ETH_RARP`: :zeek:type:`count`
:zeek:id:`OpenFlow::ETH_VLAN`: :zeek:type:`count`
:zeek:id:`OpenFlow::ETH_WOL`: :zeek:type:`count`
:zeek:id:`OpenFlow::INVALID_COOKIE`: :zeek:type:`count` Return value for a cookie from a flow
which is not added, modified or deleted
from the Zeek openflow framework.
:zeek:id:`OpenFlow::IP_CBT`: :zeek:type:`count`
:zeek:id:`OpenFlow::IP_EGP`: :zeek:type:`count`
:zeek:id:`OpenFlow::IP_ETHERIP`: :zeek:type:`count`
:zeek:id:`OpenFlow::IP_FC`: :zeek:type:`count`
:zeek:id:`OpenFlow::IP_GGP`: :zeek:type:`count`
:zeek:id:`OpenFlow::IP_GRE`: :zeek:type:`count`
:zeek:id:`OpenFlow::IP_HOPOPT`: :zeek:type:`count`
:zeek:id:`OpenFlow::IP_ICMP`: :zeek:type:`count`
:zeek:id:`OpenFlow::IP_IGMP`: :zeek:type:`count`
:zeek:id:`OpenFlow::IP_IGP`: :zeek:type:`count`
:zeek:id:`OpenFlow::IP_IPIP`: :zeek:type:`count`
:zeek:id:`OpenFlow::IP_IPv6`: :zeek:type:`count`
:zeek:id:`OpenFlow::IP_ISIS`: :zeek:type:`count`
:zeek:id:`OpenFlow::IP_L2TP`: :zeek:type:`count`
:zeek:id:`OpenFlow::IP_MPLS`: :zeek:type:`count`
:zeek:id:`OpenFlow::IP_MTP`: :zeek:type:`count`
:zeek:id:`OpenFlow::IP_OSPF`: :zeek:type:`count`
:zeek:id:`OpenFlow::IP_RDP`: :zeek:type:`count`
:zeek:id:`OpenFlow::IP_RSVP`: :zeek:type:`count`
:zeek:id:`OpenFlow::IP_ST`: :zeek:type:`count`
:zeek:id:`OpenFlow::IP_TCP`: :zeek:type:`count`
:zeek:id:`OpenFlow::IP_UDP`: :zeek:type:`count`
:zeek:id:`OpenFlow::OFPFF_CHECK_OVERLAP`: :zeek:type:`count` Check for overlapping entries first.
:zeek:id:`OpenFlow::OFPFF_EMERG`: :zeek:type:`count` Remark this is for emergency.
:zeek:id:`OpenFlow::OFPFF_SEND_FLOW_REM`: :zeek:type:`count` Send flow removed message when flow
expires or is deleted.
:zeek:id:`OpenFlow::OFPP_ALL`: :zeek:type:`count` All physical ports except input port.
:zeek:id:`OpenFlow::OFPP_ANY`: :zeek:type:`count` Wildcard port used only for flow mod (delete) and flow stats requests.
:zeek:id:`OpenFlow::OFPP_CONTROLLER`: :zeek:type:`count` Send to controller.
:zeek:id:`OpenFlow::OFPP_FLOOD`: :zeek:type:`count` All physical ports except input port and
those disabled by STP.
:zeek:id:`OpenFlow::OFPP_IN_PORT`: :zeek:type:`count` Send the packet out the input port.
:zeek:id:`OpenFlow::OFPP_LOCAL`: :zeek:type:`count` Local openflow "port".
:zeek:id:`OpenFlow::OFPP_NORMAL`: :zeek:type:`count` Process with normal L2/L3 switching.
:zeek:id:`OpenFlow::OFPP_TABLE`: :zeek:type:`count` Perform actions in flow table.
:zeek:id:`OpenFlow::OFPTT_ALL`: :zeek:type:`count`
:zeek:id:`OpenFlow::OFP_NO_BUFFER`: :zeek:type:`count`
=============================================================== ======================================================================
Types
#####
============================================================== ======================================
:zeek:type:`OpenFlow::ofp_action_type`: :zeek:type:`enum` Openflow action_type definitions.
:zeek:type:`OpenFlow::ofp_config_flags`: :zeek:type:`enum` Openflow config flag definitions.
:zeek:type:`OpenFlow::ofp_flow_mod_command`: :zeek:type:`enum` Openflow flow_mod_command definitions.
============================================================== ======================================
Detailed Interface
~~~~~~~~~~~~~~~~~~
Constants
#########
.. zeek:id:: OpenFlow::ETH_APPLETALK
:source-code: base/frameworks/openflow/consts.zeek 38 38
:Type: :zeek:type:`count`
:Default: ``32923``
.. zeek:id:: OpenFlow::ETH_APPLETALK_ARP
:source-code: base/frameworks/openflow/consts.zeek 40 40
:Type: :zeek:type:`count`
:Default: ``33011``
.. zeek:id:: OpenFlow::ETH_ARP
:source-code: base/frameworks/openflow/consts.zeek 32 32
:Type: :zeek:type:`count`
:Default: ``2054``
.. zeek:id:: OpenFlow::ETH_EAP_OVER_LAN
:source-code: base/frameworks/openflow/consts.zeek 62 62
:Type: :zeek:type:`count`
:Default: ``34958``
.. zeek:id:: OpenFlow::ETH_ETHER_FLOW_CONTROL
:source-code: base/frameworks/openflow/consts.zeek 50 50
:Type: :zeek:type:`count`
:Default: ``34824``
.. zeek:id:: OpenFlow::ETH_IPX
:source-code: base/frameworks/openflow/consts.zeek 46 46
:Type: :zeek:type:`count`
:Default: ``33080``
.. zeek:id:: OpenFlow::ETH_IPX_OLD
:source-code: base/frameworks/openflow/consts.zeek 44 44
:Type: :zeek:type:`count`
:Default: ``33079``
.. zeek:id:: OpenFlow::ETH_IPv4
:source-code: base/frameworks/openflow/consts.zeek 30 30
:Type: :zeek:type:`count`
:Default: ``2048``
.. zeek:id:: OpenFlow::ETH_IPv6
:source-code: base/frameworks/openflow/consts.zeek 48 48
:Type: :zeek:type:`count`
:Default: ``34525``
.. zeek:id:: OpenFlow::ETH_JUMBO_FRAMES
:source-code: base/frameworks/openflow/consts.zeek 60 60
:Type: :zeek:type:`count`
:Default: ``34928``
.. zeek:id:: OpenFlow::ETH_MAC_SECURITY
:source-code: base/frameworks/openflow/consts.zeek 66 66
:Type: :zeek:type:`count`
:Default: ``35045``
.. zeek:id:: OpenFlow::ETH_MPLS_MULTICAST
:source-code: base/frameworks/openflow/consts.zeek 54 54
:Type: :zeek:type:`count`
:Default: ``34888``
.. zeek:id:: OpenFlow::ETH_MPLS_UNICAST
:source-code: base/frameworks/openflow/consts.zeek 52 52
:Type: :zeek:type:`count`
:Default: ``34887``
.. zeek:id:: OpenFlow::ETH_PPPOE_DISCOVERY
:source-code: base/frameworks/openflow/consts.zeek 56 56
:Type: :zeek:type:`count`
:Default: ``34915``
.. zeek:id:: OpenFlow::ETH_PPPOE_SESSION
:source-code: base/frameworks/openflow/consts.zeek 58 58
:Type: :zeek:type:`count`
:Default: ``34916``
.. zeek:id:: OpenFlow::ETH_PROVIDER_BRIDING
:source-code: base/frameworks/openflow/consts.zeek 64 64
:Type: :zeek:type:`count`
:Default: ``34984``
.. zeek:id:: OpenFlow::ETH_QINQ
:source-code: base/frameworks/openflow/consts.zeek 68 68
:Type: :zeek:type:`count`
:Default: ``37120``
.. zeek:id:: OpenFlow::ETH_RARP
:source-code: base/frameworks/openflow/consts.zeek 36 36
:Type: :zeek:type:`count`
:Default: ``32821``
.. zeek:id:: OpenFlow::ETH_VLAN
:source-code: base/frameworks/openflow/consts.zeek 42 42
:Type: :zeek:type:`count`
:Default: ``33024``
.. zeek:id:: OpenFlow::ETH_WOL
:source-code: base/frameworks/openflow/consts.zeek 34 34
:Type: :zeek:type:`count`
:Default: ``2114``
.. zeek:id:: OpenFlow::INVALID_COOKIE
:source-code: base/frameworks/openflow/consts.zeek 126 126
:Type: :zeek:type:`count`
:Default: ``9223372036854775807``
Return value for a cookie from a flow
which is not added, modified or deleted
from the Zeek openflow framework.
.. zeek:id:: OpenFlow::IP_CBT
:source-code: base/frameworks/openflow/consts.zeek 89 89
:Type: :zeek:type:`count`
:Default: ``7``
.. zeek:id:: OpenFlow::IP_EGP
:source-code: base/frameworks/openflow/consts.zeek 91 91
:Type: :zeek:type:`count`
:Default: ``8``
.. zeek:id:: OpenFlow::IP_ETHERIP
:source-code: base/frameworks/openflow/consts.zeek 112 112
:Type: :zeek:type:`count`
:Default: ``97``
.. zeek:id:: OpenFlow::IP_FC
:source-code: base/frameworks/openflow/consts.zeek 118 118
:Type: :zeek:type:`count`
:Default: ``133``
.. zeek:id:: OpenFlow::IP_GGP
:source-code: base/frameworks/openflow/consts.zeek 81 81
:Type: :zeek:type:`count`
:Default: ``3``
.. zeek:id:: OpenFlow::IP_GRE
:source-code: base/frameworks/openflow/consts.zeek 104 104
:Type: :zeek:type:`count`
:Default: ``47``
.. zeek:id:: OpenFlow::IP_HOPOPT
:source-code: base/frameworks/openflow/consts.zeek 75 75
:Type: :zeek:type:`count`
:Default: ``0``
.. zeek:id:: OpenFlow::IP_ICMP
:source-code: base/frameworks/openflow/consts.zeek 77 77
:Type: :zeek:type:`count`
:Default: ``1``
.. zeek:id:: OpenFlow::IP_IGMP
:source-code: base/frameworks/openflow/consts.zeek 79 79
:Type: :zeek:type:`count`
:Default: ``2``
.. zeek:id:: OpenFlow::IP_IGP
:source-code: base/frameworks/openflow/consts.zeek 94 94
:Type: :zeek:type:`count`
:Default: ``9``
.. zeek:id:: OpenFlow::IP_IPIP
:source-code: base/frameworks/openflow/consts.zeek 83 83
:Type: :zeek:type:`count`
:Default: ``4``
.. zeek:id:: OpenFlow::IP_IPv6
:source-code: base/frameworks/openflow/consts.zeek 100 100
:Type: :zeek:type:`count`
:Default: ``41``
.. zeek:id:: OpenFlow::IP_ISIS
:source-code: base/frameworks/openflow/consts.zeek 116 116
:Type: :zeek:type:`count`
:Default: ``124``
.. zeek:id:: OpenFlow::IP_L2TP
:source-code: base/frameworks/openflow/consts.zeek 114 114
:Type: :zeek:type:`count`
:Default: ``115``
.. zeek:id:: OpenFlow::IP_MPLS
:source-code: base/frameworks/openflow/consts.zeek 120 120
:Type: :zeek:type:`count`
:Default: ``137``
.. zeek:id:: OpenFlow::IP_MTP
:source-code: base/frameworks/openflow/consts.zeek 108 108
:Type: :zeek:type:`count`
:Default: ``92``
.. zeek:id:: OpenFlow::IP_OSPF
:source-code: base/frameworks/openflow/consts.zeek 106 106
:Type: :zeek:type:`count`
:Default: ``89``
.. zeek:id:: OpenFlow::IP_RDP
:source-code: base/frameworks/openflow/consts.zeek 98 98
:Type: :zeek:type:`count`
:Default: ``27``
.. zeek:id:: OpenFlow::IP_RSVP
:source-code: base/frameworks/openflow/consts.zeek 102 102
:Type: :zeek:type:`count`
:Default: ``46``
.. zeek:id:: OpenFlow::IP_ST
:source-code: base/frameworks/openflow/consts.zeek 85 85
:Type: :zeek:type:`count`
:Default: ``5``
.. zeek:id:: OpenFlow::IP_TCP
:source-code: base/frameworks/openflow/consts.zeek 87 87
:Type: :zeek:type:`count`
:Default: ``6``
.. zeek:id:: OpenFlow::IP_UDP
:source-code: base/frameworks/openflow/consts.zeek 96 96
:Type: :zeek:type:`count`
:Default: ``17``
.. zeek:id:: OpenFlow::OFPFF_CHECK_OVERLAP
:source-code: base/frameworks/openflow/consts.zeek 155 155
:Type: :zeek:type:`count`
:Default: ``2``
Check for overlapping entries first.
.. zeek:id:: OpenFlow::OFPFF_EMERG
:source-code: base/frameworks/openflow/consts.zeek 159 159
:Type: :zeek:type:`count`
:Default: ``4``
Remark this is for emergency.
Flows added with this are only used
when the controller is disconnected.
.. zeek:id:: OpenFlow::OFPFF_SEND_FLOW_REM
:source-code: base/frameworks/openflow/consts.zeek 153 153
:Type: :zeek:type:`count`
:Default: ``1``
Send flow removed message when flow
expires or is deleted.
.. zeek:id:: OpenFlow::OFPP_ALL
:source-code: base/frameworks/openflow/consts.zeek 142 142
:Type: :zeek:type:`count`
:Default: ``4294967292``
All physical ports except input port.
.. zeek:id:: OpenFlow::OFPP_ANY
:source-code: base/frameworks/openflow/consts.zeek 148 148
:Type: :zeek:type:`count`
:Default: ``4294967295``
Wildcard port used only for flow mod (delete) and flow stats requests.
.. zeek:id:: OpenFlow::OFPP_CONTROLLER
:source-code: base/frameworks/openflow/consts.zeek 144 144
:Type: :zeek:type:`count`
:Default: ``4294967293``
Send to controller.
.. zeek:id:: OpenFlow::OFPP_FLOOD
:source-code: base/frameworks/openflow/consts.zeek 140 140
:Type: :zeek:type:`count`
:Default: ``4294967291``
All physical ports except input port and
those disabled by STP.
.. zeek:id:: OpenFlow::OFPP_IN_PORT
:source-code: base/frameworks/openflow/consts.zeek 131 131
:Type: :zeek:type:`count`
:Default: ``4294967288``
Send the packet out the input port. This
virtual port must be explicitly used in
order to send back out of the input port.
.. zeek:id:: OpenFlow::OFPP_LOCAL
:source-code: base/frameworks/openflow/consts.zeek 146 146
:Type: :zeek:type:`count`
:Default: ``4294967294``
Local openflow "port".
.. zeek:id:: OpenFlow::OFPP_NORMAL
:source-code: base/frameworks/openflow/consts.zeek 137 137
:Type: :zeek:type:`count`
:Default: ``4294967290``
Process with normal L2/L3 switching.
.. zeek:id:: OpenFlow::OFPP_TABLE
:source-code: base/frameworks/openflow/consts.zeek 135 135
:Type: :zeek:type:`count`
:Default: ``4294967289``
Perform actions in flow table.
NB: This can only be the destination port
for packet-out messages.
.. zeek:id:: OpenFlow::OFPTT_ALL
:source-code: base/frameworks/openflow/consts.zeek 163 163
:Type: :zeek:type:`count`
:Default: ``255``
.. zeek:id:: OpenFlow::OFP_NO_BUFFER
:source-code: base/frameworks/openflow/consts.zeek 150 150
:Type: :zeek:type:`count`
:Default: ``4294967295``
Types
#####
.. zeek:type:: OpenFlow::ofp_action_type
:source-code: base/frameworks/openflow/consts.zeek 170 198
:Type: :zeek:type:`enum`
.. zeek:enum:: OpenFlow::OFPAT_OUTPUT OpenFlow::ofp_action_type
Output to switch port.
.. zeek:enum:: OpenFlow::OFPAT_SET_VLAN_VID OpenFlow::ofp_action_type
Set the 802.1q VLAN id.
.. zeek:enum:: OpenFlow::OFPAT_SET_VLAN_PCP OpenFlow::ofp_action_type
Set the 802.1q priority.
.. zeek:enum:: OpenFlow::OFPAT_STRIP_VLAN OpenFlow::ofp_action_type
Strip the 802.1q header.
.. zeek:enum:: OpenFlow::OFPAT_SET_DL_SRC OpenFlow::ofp_action_type
Ethernet source address.
.. zeek:enum:: OpenFlow::OFPAT_SET_DL_DST OpenFlow::ofp_action_type
Ethernet destination address.
.. zeek:enum:: OpenFlow::OFPAT_SET_NW_SRC OpenFlow::ofp_action_type
IP source address.
.. zeek:enum:: OpenFlow::OFPAT_SET_NW_DST OpenFlow::ofp_action_type
IP destination address.
.. zeek:enum:: OpenFlow::OFPAT_SET_NW_TOS OpenFlow::ofp_action_type
IP ToS (DSCP field, 6 bits).
.. zeek:enum:: OpenFlow::OFPAT_SET_TP_SRC OpenFlow::ofp_action_type
TCP/UDP source port.
.. zeek:enum:: OpenFlow::OFPAT_SET_TP_DST OpenFlow::ofp_action_type
TCP/UDP destination port.
.. zeek:enum:: OpenFlow::OFPAT_ENQUEUE OpenFlow::ofp_action_type
Output to queue.
.. zeek:enum:: OpenFlow::OFPAT_VENDOR OpenFlow::ofp_action_type
Vendor specific.
Openflow action_type definitions.
The openflow action type defines
what actions openflow can take
to modify a packet
.. zeek:type:: OpenFlow::ofp_config_flags
:source-code: base/frameworks/openflow/consts.zeek 219 228
:Type: :zeek:type:`enum`
.. zeek:enum:: OpenFlow::OFPC_FRAG_NORMAL OpenFlow::ofp_config_flags
No special handling for fragments.
.. zeek:enum:: OpenFlow::OFPC_FRAG_DROP OpenFlow::ofp_config_flags
Drop fragments.
.. zeek:enum:: OpenFlow::OFPC_FRAG_REASM OpenFlow::ofp_config_flags
Reassemble (only if OFPC_IP_REASM set).
.. zeek:enum:: OpenFlow::OFPC_FRAG_MASK OpenFlow::ofp_config_flags
Openflow config flag definitions.
TODO: describe
.. zeek:type:: OpenFlow::ofp_flow_mod_command
:source-code: base/frameworks/openflow/consts.zeek 203 215
:Type: :zeek:type:`enum`
.. zeek:enum:: OpenFlow::OFPFC_ADD OpenFlow::ofp_flow_mod_command
New flow.
.. zeek:enum:: OpenFlow::OFPFC_MODIFY OpenFlow::ofp_flow_mod_command
Modify all matching flows.
.. zeek:enum:: OpenFlow::OFPFC_MODIFY_STRICT OpenFlow::ofp_flow_mod_command
Modify entry strictly matching wildcards.
.. zeek:enum:: OpenFlow::OFPFC_DELETE OpenFlow::ofp_flow_mod_command
Delete all matching flows.
.. zeek:enum:: OpenFlow::OFPFC_DELETE_STRICT OpenFlow::ofp_flow_mod_command
Strictly matching wildcards and priority.
Openflow flow_mod_command definitions.
The openflow flow_mod_command describes
of what kind an action is.

50
doc/scripts/base/frameworks/openflow/index.rst Normal file
View file

@ -0,0 +1,50 @@
:orphan:
Package: base/frameworks/openflow
=================================
The OpenFlow framework exposes the data structures and functions
necessary to interface to OpenFlow capable hardware.
:doc:`/scripts/base/frameworks/openflow/__load__.zeek`
:doc:`/scripts/base/frameworks/openflow/consts.zeek`
Constants used by the OpenFlow framework.
:doc:`/scripts/base/frameworks/openflow/types.zeek`
Types used by the OpenFlow framework.
:doc:`/scripts/base/frameworks/openflow/main.zeek`
Zeek's OpenFlow control framework.
This plugin-based framework allows to control OpenFlow capable
switches by implementing communication to an OpenFlow controller
via plugins. The framework has to be instantiated via the new function
in one of the plugins. This framework only offers very low-level
functionality; if you want to use OpenFlow capable switches, e.g.,
for shunting, please look at the NetControl framework, which provides higher
level functions and can use the OpenFlow framework as a backend.
:doc:`/scripts/base/frameworks/openflow/plugins/__load__.zeek`
:doc:`/scripts/base/frameworks/openflow/plugins/ryu.zeek`
OpenFlow plugin for the Ryu controller.
:doc:`/scripts/base/frameworks/openflow/plugins/log.zeek`
OpenFlow plugin that outputs flow-modification commands
to a Zeek log file.
:doc:`/scripts/base/frameworks/openflow/plugins/broker.zeek`
OpenFlow plugin for interfacing to controllers via Broker.
:doc:`/scripts/base/frameworks/openflow/non-cluster.zeek`

279
doc/scripts/base/frameworks/openflow/main.zeek.rst Normal file
View file

@ -0,0 +1,279 @@
:tocdepth: 3
base/frameworks/openflow/main.zeek
==================================
.. zeek:namespace:: OpenFlow
Zeek's OpenFlow control framework.
This plugin-based framework allows to control OpenFlow capable
switches by implementing communication to an OpenFlow controller
via plugins. The framework has to be instantiated via the new function
in one of the plugins. This framework only offers very low-level
functionality; if you want to use OpenFlow capable switches, e.g.,
for shunting, please look at the NetControl framework, which provides higher
level functions and can use the OpenFlow framework as a backend.
:Namespace: OpenFlow
:Imports: :doc:`base/frameworks/openflow/consts.zeek </scripts/base/frameworks/openflow/consts.zeek>`, :doc:`base/frameworks/openflow/types.zeek </scripts/base/frameworks/openflow/types.zeek>`
Summary
~~~~~~~
Events
######
============================================================= =============================================================================================
:zeek:id:`OpenFlow::controller_activated`: :zeek:type:`event` Event that is raised once a controller finishes initialization
and is completely activated.
:zeek:id:`OpenFlow::flow_mod_failure`: :zeek:type:`event` Reports an error while installing a flow Rule.
:zeek:id:`OpenFlow::flow_mod_success`: :zeek:type:`event` Event confirming successful modification of a flow rule.
:zeek:id:`OpenFlow::flow_removed`: :zeek:type:`event` Reports that a flow was removed by the switch because of either the hard or the idle timeout.
============================================================= =============================================================================================
Functions
#########
================================================================= =====================================================================
:zeek:id:`OpenFlow::controller_init_done`: :zeek:type:`function` Function to signal that a controller finished activation and is
ready to use.
:zeek:id:`OpenFlow::flow_clear`: :zeek:type:`function` Clear the current flow table of the controller.
:zeek:id:`OpenFlow::flow_mod`: :zeek:type:`function` Global flow_mod function.
:zeek:id:`OpenFlow::generate_cookie`: :zeek:type:`function` Function to generate a new cookie using our group id.
:zeek:id:`OpenFlow::get_cookie_gid`: :zeek:type:`function` Function to get the group id out of a given cookie.
:zeek:id:`OpenFlow::get_cookie_uid`: :zeek:type:`function` Function to get the unique id out of a given cookie.
:zeek:id:`OpenFlow::lookup_controller`: :zeek:type:`function` Function to lookup a controller instance by name.
:zeek:id:`OpenFlow::match_conn`: :zeek:type:`function` Convert a conn_id record into an ofp_match record that can be used to
create match objects for OpenFlow.
:zeek:id:`OpenFlow::register_controller`: :zeek:type:`function` Function to register a controller instance.
:zeek:id:`OpenFlow::unregister_controller`: :zeek:type:`function` Function to unregister a controller instance.
================================================================= =====================================================================
Detailed Interface
~~~~~~~~~~~~~~~~~~
Events
######
.. zeek:id:: OpenFlow::controller_activated
:source-code: base/frameworks/netcontrol/plugins/openflow.zeek 433 437
:Type: :zeek:type:`event` (name: :zeek:type:`string`, controller: :zeek:type:`OpenFlow::Controller`)
Event that is raised once a controller finishes initialization
and is completely activated.
:param name: Unique name of this controller instance.
:param controller: The controller that finished activation.
.. zeek:id:: OpenFlow::flow_mod_failure
:source-code: base/frameworks/netcontrol/plugins/openflow.zeek 383 394
:Type: :zeek:type:`event` (name: :zeek:type:`string`, match: :zeek:type:`OpenFlow::ofp_match`, flow_mod: :zeek:type:`OpenFlow::ofp_flow_mod`, msg: :zeek:type:`string` :zeek:attr:`&default` = ``""`` :zeek:attr:`&optional`)
Reports an error while installing a flow Rule.
:param name: The unique name of the OpenFlow controller from which this event originated.
:param match: The ofp_match record which describes the flow to match.
:param flow_mod: The openflow flow_mod record which describes the action to take.
:param msg: Message to describe the event.
.. zeek:id:: OpenFlow::flow_mod_success
:source-code: base/frameworks/netcontrol/plugins/openflow.zeek 356 381
:Type: :zeek:type:`event` (name: :zeek:type:`string`, match: :zeek:type:`OpenFlow::ofp_match`, flow_mod: :zeek:type:`OpenFlow::ofp_flow_mod`, msg: :zeek:type:`string` :zeek:attr:`&default` = ``""`` :zeek:attr:`&optional`)
Event confirming successful modification of a flow rule.
:param name: The unique name of the OpenFlow controller from which this event originated.
:param match: The ofp_match record which describes the flow to match.
:param flow_mod: The openflow flow_mod record which describes the action to take.
:param msg: An optional informational message by the plugin.
.. zeek:id:: OpenFlow::flow_removed
:source-code: base/frameworks/netcontrol/plugins/openflow.zeek 396 418
:Type: :zeek:type:`event` (name: :zeek:type:`string`, match: :zeek:type:`OpenFlow::ofp_match`, cookie: :zeek:type:`count`, priority: :zeek:type:`count`, reason: :zeek:type:`count`, duration_sec: :zeek:type:`count`, idle_timeout: :zeek:type:`count`, packet_count: :zeek:type:`count`, byte_count: :zeek:type:`count`)
Reports that a flow was removed by the switch because of either the hard or the idle timeout.
This message is only generated by controllers that indicate that they support flow removal
in supports_flow_removed.
:param name: The unique name of the OpenFlow controller from which this event originated.
:param match: The ofp_match record which was used to create the flow.
:param cookie: The cookie that was specified when creating the flow.
:param priority: The priority that was specified when creating the flow.
:param reason: The reason for flow removal (OFPRR_*).
:param duration_sec: Duration of the flow in seconds.
:param packet_count: Packet count of the flow.
:param byte_count: Byte count of the flow.
Functions
#########
.. zeek:id:: OpenFlow::controller_init_done
:source-code: base/frameworks/openflow/main.zeek 242 252
:Type: :zeek:type:`function` (controller: :zeek:type:`OpenFlow::Controller`) : :zeek:type:`void`
Function to signal that a controller finished activation and is
ready to use. Will throw the ``OpenFlow::controller_activated``
event.
.. zeek:id:: OpenFlow::flow_clear
:source-code: base/frameworks/openflow/non-cluster.zeek 17 26
:Type: :zeek:type:`function` (controller: :zeek:type:`OpenFlow::Controller`) : :zeek:type:`bool`
Clear the current flow table of the controller.
:param controller: The controller which should execute the flow modification.
:returns: F on error or if the plugin does not support the operation, T when the operation was queued.
.. zeek:id:: OpenFlow::flow_mod
:source-code: base/frameworks/openflow/non-cluster.zeek 6 15
:Type: :zeek:type:`function` (controller: :zeek:type:`OpenFlow::Controller`, match: :zeek:type:`OpenFlow::ofp_match`, flow_mod: :zeek:type:`OpenFlow::ofp_flow_mod`) : :zeek:type:`bool`
Global flow_mod function.
:param controller: The controller which should execute the flow modification.
:param match: The ofp_match record which describes the flow to match.
:param flow_mod: The openflow flow_mod record which describes the action to take.
:returns: F on error or if the plugin does not support the operation, T when the operation was queued.
.. zeek:id:: OpenFlow::generate_cookie
:source-code: base/frameworks/openflow/main.zeek 199 209
:Type: :zeek:type:`function` (cookie: :zeek:type:`count` :zeek:attr:`&default` = ``0`` :zeek:attr:`&optional`) : :zeek:type:`count`
Function to generate a new cookie using our group id.
:param cookie: The openflow match cookie.
:returns: The cookie group id.
.. zeek:id:: OpenFlow::get_cookie_gid
:source-code: base/frameworks/openflow/main.zeek 230 240
:Type: :zeek:type:`function` (cookie: :zeek:type:`count`) : :zeek:type:`count`
Function to get the group id out of a given cookie.
:param cookie: The openflow match cookie.
:returns: The cookie group id.
.. zeek:id:: OpenFlow::get_cookie_uid
:source-code: base/frameworks/openflow/main.zeek 222 228
:Type: :zeek:type:`function` (cookie: :zeek:type:`count`) : :zeek:type:`count`
Function to get the unique id out of a given cookie.
:param cookie: The openflow match cookie.
:returns: The cookie unique id.
.. zeek:id:: OpenFlow::lookup_controller
:source-code: base/frameworks/openflow/non-cluster.zeek 41 44
:Type: :zeek:type:`function` (name: :zeek:type:`string`) : :zeek:type:`vector` of :zeek:type:`OpenFlow::Controller`
Function to lookup a controller instance by name.
:param name: Unique name of the controller to look up.
:returns: One element vector with controller, if found. Empty vector otherwise.
.. zeek:id:: OpenFlow::match_conn
:source-code: base/frameworks/openflow/main.zeek 153 194
:Type: :zeek:type:`function` (id: :zeek:type:`conn_id`, reverse: :zeek:type:`bool` :zeek:attr:`&default` = ``F`` :zeek:attr:`&optional`) : :zeek:type:`OpenFlow::ofp_match`
Convert a conn_id record into an ofp_match record that can be used to
create match objects for OpenFlow.
:param id: The conn_id record that describes the record.
:param reverse: Reverse the sources and destinations when creating the match record (default F).
:returns: ofp_match object for the conn_id record.
.. zeek:id:: OpenFlow::register_controller
:source-code: base/frameworks/openflow/non-cluster.zeek 28 34
:Type: :zeek:type:`function` (tpe: :zeek:type:`OpenFlow::Plugin`, name: :zeek:type:`string`, controller: :zeek:type:`OpenFlow::Controller`) : :zeek:type:`void`
Function to register a controller instance. This function
is called automatically by the plugin _new functions.
:param tpe: Type of this plugin.
:param name: Unique name of this controller instance.
:param controller: The controller to register.
.. zeek:id:: OpenFlow::unregister_controller
:source-code: base/frameworks/openflow/non-cluster.zeek 36 39
:Type: :zeek:type:`function` (controller: :zeek:type:`OpenFlow::Controller`) : :zeek:type:`void`
Function to unregister a controller instance. This function
should be called when a specific controller should no longer
be used.
:param controller: The controller to unregister.

16
doc/scripts/base/frameworks/openflow/non-cluster.zeek.rst Normal file
View file

@ -0,0 +1,16 @@
:tocdepth: 3
base/frameworks/openflow/non-cluster.zeek
=========================================
.. zeek:namespace:: OpenFlow
:Namespace: OpenFlow
:Imports: :doc:`base/frameworks/openflow/main.zeek </scripts/base/frameworks/openflow/main.zeek>`
Summary
~~~~~~~
Detailed Interface
~~~~~~~~~~~~~~~~~~

14
doc/scripts/base/frameworks/openflow/plugins/__load__.zeek.rst Normal file
View file

@ -0,0 +1,14 @@
:tocdepth: 3
base/frameworks/openflow/plugins/__load__.zeek
==============================================
:Imports: :doc:`base/frameworks/openflow/plugins/broker.zeek </scripts/base/frameworks/openflow/plugins/broker.zeek>`, :doc:`base/frameworks/openflow/plugins/log.zeek </scripts/base/frameworks/openflow/plugins/log.zeek>`, :doc:`base/frameworks/openflow/plugins/ryu.zeek </scripts/base/frameworks/openflow/plugins/ryu.zeek>`
Summary
~~~~~~~
Detailed Interface
~~~~~~~~~~~~~~~~~~

91
doc/scripts/base/frameworks/openflow/plugins/broker.zeek.rst Normal file
View file

@ -0,0 +1,91 @@
:tocdepth: 3
base/frameworks/openflow/plugins/broker.zeek
============================================
.. zeek:namespace:: OpenFlow
OpenFlow plugin for interfacing to controllers via Broker.
:Namespace: OpenFlow
:Imports: :doc:`base/frameworks/broker </scripts/base/frameworks/broker/index>`, :doc:`base/frameworks/openflow </scripts/base/frameworks/openflow/index>`
Summary
~~~~~~~
Redefinitions
#############
=============================================================================== ==========================================================
:zeek:type:`OpenFlow::ControllerState`: :zeek:type:`record` :zeek:attr:`&redef`
:New Fields: :zeek:type:`OpenFlow::ControllerState`
broker_host: :zeek:type:`addr` :zeek:attr:`&optional`
Controller ip.
broker_port: :zeek:type:`port` :zeek:attr:`&optional`
Controller listen port.
broker_dpid: :zeek:type:`count` :zeek:attr:`&optional`
OpenFlow switch datapath id.
broker_topic: :zeek:type:`string` :zeek:attr:`&optional`
Topic to send events for this controller to.
:zeek:type:`OpenFlow::Plugin`: :zeek:type:`enum`
* :zeek:enum:`OpenFlow::BROKER`
=============================================================================== ==========================================================
Events
######
========================================================== =
:zeek:id:`OpenFlow::broker_flow_clear`: :zeek:type:`event`
:zeek:id:`OpenFlow::broker_flow_mod`: :zeek:type:`event`
========================================================== =
Functions
#########
====================================================== ==============================
:zeek:id:`OpenFlow::broker_new`: :zeek:type:`function` Broker controller constructor.
====================================================== ==============================
Detailed Interface
~~~~~~~~~~~~~~~~~~
Events
######
.. zeek:id:: OpenFlow::broker_flow_clear
:source-code: base/frameworks/openflow/plugins/broker.zeek 38 38
:Type: :zeek:type:`event` (name: :zeek:type:`string`, dpid: :zeek:type:`count`)
.. zeek:id:: OpenFlow::broker_flow_mod
:source-code: base/frameworks/openflow/plugins/broker.zeek 37 37
:Type: :zeek:type:`event` (name: :zeek:type:`string`, dpid: :zeek:type:`count`, match: :zeek:type:`OpenFlow::ofp_match`, flow_mod: :zeek:type:`OpenFlow::ofp_flow_mod`)
Functions
#########
.. zeek:id:: OpenFlow::broker_new
:source-code: base/frameworks/openflow/plugins/broker.zeek 82 95
:Type: :zeek:type:`function` (name: :zeek:type:`string`, host: :zeek:type:`addr`, host_port: :zeek:type:`port`, topic: :zeek:type:`string`, dpid: :zeek:type:`count`) : :zeek:type:`OpenFlow::Controller`
Broker controller constructor.
:param host: Controller ip.
:param host_port: Controller listen port.
:param topic: Broker topic to send messages to.
:param dpid: OpenFlow switch datapath id.
:returns: OpenFlow::Controller record.

23
doc/scripts/base/frameworks/openflow/plugins/index.rst Normal file
View file

@ -0,0 +1,23 @@
:orphan:
Package: base/frameworks/openflow/plugins
=========================================
Plugins for the OpenFlow framework.
:doc:`/scripts/base/frameworks/openflow/plugins/__load__.zeek`
:doc:`/scripts/base/frameworks/openflow/plugins/ryu.zeek`
OpenFlow plugin for the Ryu controller.
:doc:`/scripts/base/frameworks/openflow/plugins/log.zeek`
OpenFlow plugin that outputs flow-modification commands
to a Zeek log file.
:doc:`/scripts/base/frameworks/openflow/plugins/broker.zeek`
OpenFlow plugin for interfacing to controllers via Broker.

129
doc/scripts/base/frameworks/openflow/plugins/log.zeek.rst Normal file
View file

@ -0,0 +1,129 @@
:tocdepth: 3
base/frameworks/openflow/plugins/log.zeek
=========================================
.. zeek:namespace:: OpenFlow
OpenFlow plugin that outputs flow-modification commands
to a Zeek log file.
:Namespace: OpenFlow
:Imports: :doc:`base/frameworks/logging </scripts/base/frameworks/logging/index>`, :doc:`base/frameworks/openflow </scripts/base/frameworks/openflow/index>`
Summary
~~~~~~~
Types
#####
================================================ =================================================================
:zeek:type:`OpenFlow::Info`: :zeek:type:`record` The record type which contains column fields of the OpenFlow log.
================================================ =================================================================
Redefinitions
#############
=============================================================================== =============================================================
:zeek:type:`Log::ID`: :zeek:type:`enum`
* :zeek:enum:`OpenFlow::LOG`
:zeek:type:`OpenFlow::ControllerState`: :zeek:type:`record` :zeek:attr:`&redef`
:New Fields: :zeek:type:`OpenFlow::ControllerState`
log_dpid: :zeek:type:`count` :zeek:attr:`&optional`
OpenFlow switch datapath id.
log_success_event: :zeek:type:`bool` :zeek:attr:`&optional`
Raise or do not raise success event.
:zeek:type:`OpenFlow::Plugin`: :zeek:type:`enum`
* :zeek:enum:`OpenFlow::OFLOG`
=============================================================================== =============================================================
Events
######
===================================================== ===================================================================
:zeek:id:`OpenFlow::log_openflow`: :zeek:type:`event` Event that can be handled to access the :zeek:type:`OpenFlow::Info`
record as it is sent on to the logging framework.
===================================================== ===================================================================
Hooks
#####
============================================================= =
:zeek:id:`OpenFlow::log_policy`: :zeek:type:`Log::PolicyHook`
============================================================= =
Functions
#########
=================================================== ===========================
:zeek:id:`OpenFlow::log_new`: :zeek:type:`function` Log controller constructor.
=================================================== ===========================
Detailed Interface
~~~~~~~~~~~~~~~~~~
Types
#####
.. zeek:type:: OpenFlow::Info
:source-code: base/frameworks/openflow/plugins/log.zeek 35 44
:Type: :zeek:type:`record`
.. zeek:field:: ts :zeek:type:`time` :zeek:attr:`&log`
Network time.
.. zeek:field:: dpid :zeek:type:`count` :zeek:attr:`&log`
OpenFlow switch datapath id.
.. zeek:field:: match :zeek:type:`OpenFlow::ofp_match` :zeek:attr:`&log`
OpenFlow match fields.
.. zeek:field:: flow_mod :zeek:type:`OpenFlow::ofp_flow_mod` :zeek:attr:`&log`
OpenFlow modify flow entry message.
The record type which contains column fields of the OpenFlow log.
Events
######
.. zeek:id:: OpenFlow::log_openflow
:source-code: base/frameworks/openflow/plugins/log.zeek 48 48
:Type: :zeek:type:`event` (rec: :zeek:type:`OpenFlow::Info`)
Event that can be handled to access the :zeek:type:`OpenFlow::Info`
record as it is sent on to the logging framework.
Hooks
#####
.. zeek:id:: OpenFlow::log_policy
:source-code: base/frameworks/openflow/plugins/log.zeek 16 16
:Type: :zeek:type:`Log::PolicyHook`
Functions
#########
.. zeek:id:: OpenFlow::log_new
:source-code: base/frameworks/openflow/plugins/log.zeek 70 78
:Type: :zeek:type:`function` (dpid: :zeek:type:`count`, success_event: :zeek:type:`bool` :zeek:attr:`&default` = ``T`` :zeek:attr:`&optional`) : :zeek:type:`OpenFlow::Controller`
Log controller constructor.
:param dpid: OpenFlow switch datapath id.
:param success_event: If true, flow_mod_success is raised for each logged line.
:returns: OpenFlow::Controller record.

67
doc/scripts/base/frameworks/openflow/plugins/ryu.zeek.rst Normal file
View file

@ -0,0 +1,67 @@
:tocdepth: 3
base/frameworks/openflow/plugins/ryu.zeek
=========================================
.. zeek:namespace:: OpenFlow
OpenFlow plugin for the Ryu controller.
:Namespace: OpenFlow
:Imports: :doc:`base/frameworks/openflow </scripts/base/frameworks/openflow/index>`, :doc:`base/utils/active-http.zeek </scripts/base/utils/active-http.zeek>`, :doc:`base/utils/exec.zeek </scripts/base/utils/exec.zeek>`
Summary
~~~~~~~
Redefinitions
#############
=============================================================================== ===================================================================================
:zeek:type:`OpenFlow::ControllerState`: :zeek:type:`record` :zeek:attr:`&redef`
:New Fields: :zeek:type:`OpenFlow::ControllerState`
ryu_host: :zeek:type:`addr` :zeek:attr:`&optional`
Controller ip.
ryu_port: :zeek:type:`count` :zeek:attr:`&optional`
Controller listen port.
ryu_dpid: :zeek:type:`count` :zeek:attr:`&optional`
OpenFlow switch datapath id.
ryu_debug: :zeek:type:`bool` :zeek:attr:`&default` = ``F`` :zeek:attr:`&optional`
Enable debug mode - output JSON to stdout; do not perform actions.
:zeek:type:`OpenFlow::Plugin`: :zeek:type:`enum`
* :zeek:enum:`OpenFlow::RYU`
=============================================================================== ===================================================================================
Functions
#########
=================================================== ===========================
:zeek:id:`OpenFlow::ryu_new`: :zeek:type:`function` Ryu controller constructor.
=================================================== ===========================
Detailed Interface
~~~~~~~~~~~~~~~~~~
Functions
#########
.. zeek:id:: OpenFlow::ryu_new
:source-code: base/frameworks/openflow/plugins/ryu.zeek 181 189
:Type: :zeek:type:`function` (host: :zeek:type:`addr`, host_port: :zeek:type:`count`, dpid: :zeek:type:`count`) : :zeek:type:`OpenFlow::Controller`
Ryu controller constructor.
:param host: Controller ip.
:param host_port: Controller listen port.
:param dpid: OpenFlow switch datapath id.
:returns: OpenFlow::Controller record.

368
doc/scripts/base/frameworks/openflow/types.zeek.rst Normal file
View file

@ -0,0 +1,368 @@
:tocdepth: 3
base/frameworks/openflow/types.zeek
===================================
.. zeek:namespace:: OpenFlow
Types used by the OpenFlow framework.
:Namespace: OpenFlow
:Imports: :doc:`base/frameworks/openflow/consts.zeek </scripts/base/frameworks/openflow/consts.zeek>`
Summary
~~~~~~~
Types
#####
=============================================================================== ===============================================================
:zeek:type:`OpenFlow::Controller`: :zeek:type:`record` Controller record representing an openflow controller.
:zeek:type:`OpenFlow::ControllerState`: :zeek:type:`record` :zeek:attr:`&redef` Controller related state.
:zeek:type:`OpenFlow::Plugin`: :zeek:type:`enum` Available openflow plugins.
:zeek:type:`OpenFlow::ofp_flow_action`: :zeek:type:`record` :zeek:attr:`&log` The actions that can be taken in a flow.
:zeek:type:`OpenFlow::ofp_flow_mod`: :zeek:type:`record` :zeek:attr:`&log` Openflow flow_mod definition, describing the action to perform.
:zeek:type:`OpenFlow::ofp_match`: :zeek:type:`record` :zeek:attr:`&log` Openflow match definition.
=============================================================================== ===============================================================
Detailed Interface
~~~~~~~~~~~~~~~~~~
Types
#####
.. zeek:type:: OpenFlow::Controller
:source-code: base/frameworks/openflow/types.zeek 116 131
:Type: :zeek:type:`record`
.. zeek:field:: state :zeek:type:`OpenFlow::ControllerState`
Controller related state.
.. zeek:field:: supports_flow_removed :zeek:type:`bool`
Does the controller support the flow_removed event?
.. zeek:field:: describe :zeek:type:`function` (state: :zeek:type:`OpenFlow::ControllerState`) : :zeek:type:`string`
Function that describes the controller. Has to be implemented.
.. zeek:field:: init :zeek:type:`function` (state: :zeek:type:`OpenFlow::ControllerState`) : :zeek:type:`void` :zeek:attr:`&optional`
One-time initialization function. If defined, controller_init_done has to be called once initialization finishes.
.. zeek:field:: destroy :zeek:type:`function` (state: :zeek:type:`OpenFlow::ControllerState`) : :zeek:type:`void` :zeek:attr:`&optional`
One-time destruction function.
.. zeek:field:: flow_mod :zeek:type:`function` (state: :zeek:type:`OpenFlow::ControllerState`, match: :zeek:type:`OpenFlow::ofp_match`, flow_mod: :zeek:type:`OpenFlow::ofp_flow_mod`) : :zeek:type:`bool` :zeek:attr:`&optional`
flow_mod function.
.. zeek:field:: flow_clear :zeek:type:`function` (state: :zeek:type:`OpenFlow::ControllerState`) : :zeek:type:`bool` :zeek:attr:`&optional`
flow_clear function.
Controller record representing an openflow controller.
.. zeek:type:: OpenFlow::ControllerState
:source-code: base/frameworks/openflow/types.zeek 17 24
:Type: :zeek:type:`record`
.. zeek:field:: _plugin :zeek:type:`OpenFlow::Plugin` :zeek:attr:`&optional`
Internally set to the type of plugin used.
.. zeek:field:: _name :zeek:type:`string` :zeek:attr:`&optional`
Internally set to the unique name of the controller.
.. zeek:field:: _activated :zeek:type:`bool` :zeek:attr:`&default` = ``F`` :zeek:attr:`&optional`
Internally set to true once the controller is activated.
.. zeek:field:: ryu_host :zeek:type:`addr` :zeek:attr:`&optional`
(present if :doc:`/scripts/base/frameworks/openflow/plugins/ryu.zeek` is loaded)
Controller ip.
.. zeek:field:: ryu_port :zeek:type:`count` :zeek:attr:`&optional`
(present if :doc:`/scripts/base/frameworks/openflow/plugins/ryu.zeek` is loaded)
Controller listen port.
.. zeek:field:: ryu_dpid :zeek:type:`count` :zeek:attr:`&optional`
(present if :doc:`/scripts/base/frameworks/openflow/plugins/ryu.zeek` is loaded)
OpenFlow switch datapath id.
.. zeek:field:: ryu_debug :zeek:type:`bool` :zeek:attr:`&default` = ``F`` :zeek:attr:`&optional`
(present if :doc:`/scripts/base/frameworks/openflow/plugins/ryu.zeek` is loaded)
Enable debug mode - output JSON to stdout; do not perform actions.
.. zeek:field:: log_dpid :zeek:type:`count` :zeek:attr:`&optional`
(present if :doc:`/scripts/base/frameworks/openflow/plugins/log.zeek` is loaded)
OpenFlow switch datapath id.
.. zeek:field:: log_success_event :zeek:type:`bool` :zeek:attr:`&optional`
(present if :doc:`/scripts/base/frameworks/openflow/plugins/log.zeek` is loaded)
Raise or do not raise success event.
.. zeek:field:: broker_host :zeek:type:`addr` :zeek:attr:`&optional`
(present if :doc:`/scripts/base/frameworks/openflow/plugins/broker.zeek` is loaded)
Controller ip.
.. zeek:field:: broker_port :zeek:type:`port` :zeek:attr:`&optional`
(present if :doc:`/scripts/base/frameworks/openflow/plugins/broker.zeek` is loaded)
Controller listen port.
.. zeek:field:: broker_dpid :zeek:type:`count` :zeek:attr:`&optional`
(present if :doc:`/scripts/base/frameworks/openflow/plugins/broker.zeek` is loaded)
OpenFlow switch datapath id.
.. zeek:field:: broker_topic :zeek:type:`string` :zeek:attr:`&optional`
(present if :doc:`/scripts/base/frameworks/openflow/plugins/broker.zeek` is loaded)
Topic to send events for this controller to.
:Attributes: :zeek:attr:`&redef`
Controller related state.
Can be redefined by plugins to
add state.
.. zeek:type:: OpenFlow::Plugin
:source-code: base/frameworks/openflow/types.zeek 9 13
:Type: :zeek:type:`enum`
.. zeek:enum:: OpenFlow::INVALID OpenFlow::Plugin
Internal placeholder plugin.
.. zeek:enum:: OpenFlow::RYU OpenFlow::Plugin
(present if :doc:`/scripts/base/frameworks/openflow/plugins/ryu.zeek` is loaded)
.. zeek:enum:: OpenFlow::OFLOG OpenFlow::Plugin
(present if :doc:`/scripts/base/frameworks/openflow/plugins/log.zeek` is loaded)
.. zeek:enum:: OpenFlow::BROKER OpenFlow::Plugin
(present if :doc:`/scripts/base/frameworks/openflow/plugins/broker.zeek` is loaded)
Available openflow plugins.
.. zeek:type:: OpenFlow::ofp_flow_action
:source-code: base/frameworks/openflow/types.zeek 62 85
:Type: :zeek:type:`record`
.. zeek:field:: out_ports :zeek:type:`vector` of :zeek:type:`count` :zeek:attr:`&default` = ``[]`` :zeek:attr:`&optional` :zeek:attr:`&log`
Output ports to send data to.
.. zeek:field:: vlan_vid :zeek:type:`count` :zeek:attr:`&optional` :zeek:attr:`&log`
Set vlan vid to this value.
.. zeek:field:: vlan_pcp :zeek:type:`count` :zeek:attr:`&optional` :zeek:attr:`&log`
Set vlan priority to this value.
.. zeek:field:: vlan_strip :zeek:type:`bool` :zeek:attr:`&default` = ``F`` :zeek:attr:`&optional` :zeek:attr:`&log`
Strip vlan tag.
.. zeek:field:: dl_src :zeek:type:`string` :zeek:attr:`&optional` :zeek:attr:`&log`
Set ethernet source address.
.. zeek:field:: dl_dst :zeek:type:`string` :zeek:attr:`&optional` :zeek:attr:`&log`
Set ethernet destination address.
.. zeek:field:: nw_tos :zeek:type:`count` :zeek:attr:`&optional` :zeek:attr:`&log`
Set ip tos to this value.
.. zeek:field:: nw_src :zeek:type:`addr` :zeek:attr:`&optional` :zeek:attr:`&log`
Set source to this ip.
.. zeek:field:: nw_dst :zeek:type:`addr` :zeek:attr:`&optional` :zeek:attr:`&log`
Set destination to this ip.
.. zeek:field:: tp_src :zeek:type:`count` :zeek:attr:`&optional` :zeek:attr:`&log`
Set tcp/udp source port.
.. zeek:field:: tp_dst :zeek:type:`count` :zeek:attr:`&optional` :zeek:attr:`&log`
Set tcp/udp destination port.
:Attributes: :zeek:attr:`&log`
The actions that can be taken in a flow.
(Separate record to make ofp_flow_mod less crowded)
.. zeek:type:: OpenFlow::ofp_flow_mod
:source-code: base/frameworks/openflow/types.zeek 88 113
:Type: :zeek:type:`record`
.. zeek:field:: cookie :zeek:type:`count` :zeek:attr:`&log`
Opaque controller-issued identifier.
.. zeek:field:: table_id :zeek:type:`count` :zeek:attr:`&optional` :zeek:attr:`&log`
Table to put the flow in. OFPTT_ALL can be used for delete,
to delete flows from all matching tables.
.. zeek:field:: command :zeek:type:`OpenFlow::ofp_flow_mod_command` :zeek:attr:`&log`
One of OFPFC_*.
.. zeek:field:: idle_timeout :zeek:type:`count` :zeek:attr:`&default` = ``0`` :zeek:attr:`&optional` :zeek:attr:`&log`
Idle time before discarding (seconds).
.. zeek:field:: hard_timeout :zeek:type:`count` :zeek:attr:`&default` = ``0`` :zeek:attr:`&optional` :zeek:attr:`&log`
Max time before discarding (seconds).
.. zeek:field:: priority :zeek:type:`count` :zeek:attr:`&default` = ``0`` :zeek:attr:`&optional` :zeek:attr:`&log`
Priority level of flow entry.
.. zeek:field:: out_port :zeek:type:`count` :zeek:attr:`&optional` :zeek:attr:`&log`
For OFPFC_DELETE* commands, require matching entry to include
this as an output port/group. OFPP_ANY/OFPG_ANY means no restrictions.
.. zeek:field:: out_group :zeek:type:`count` :zeek:attr:`&optional` :zeek:attr:`&log`
.. zeek:field:: flags :zeek:type:`count` :zeek:attr:`&default` = ``0`` :zeek:attr:`&optional` :zeek:attr:`&log`
Bitmap of the OFPFF_* flags
.. zeek:field:: actions :zeek:type:`OpenFlow::ofp_flow_action` :zeek:attr:`&default` = *...* :zeek:attr:`&optional` :zeek:attr:`&log`
Actions to take on match
:Attributes: :zeek:attr:`&log`
Openflow flow_mod definition, describing the action to perform.
.. zeek:type:: OpenFlow::ofp_match
:source-code: base/frameworks/openflow/types.zeek 31 58
:Type: :zeek:type:`record`
.. zeek:field:: in_port :zeek:type:`count` :zeek:attr:`&optional` :zeek:attr:`&log`
.. zeek:field:: dl_src :zeek:type:`string` :zeek:attr:`&optional` :zeek:attr:`&log`
.. zeek:field:: dl_dst :zeek:type:`string` :zeek:attr:`&optional` :zeek:attr:`&log`
.. zeek:field:: dl_vlan :zeek:type:`count` :zeek:attr:`&optional` :zeek:attr:`&log`
.. zeek:field:: dl_vlan_pcp :zeek:type:`count` :zeek:attr:`&optional` :zeek:attr:`&log`
.. zeek:field:: dl_type :zeek:type:`count` :zeek:attr:`&optional` :zeek:attr:`&log`
.. zeek:field:: nw_tos :zeek:type:`count` :zeek:attr:`&optional` :zeek:attr:`&log`
.. zeek:field:: nw_proto :zeek:type:`count` :zeek:attr:`&optional` :zeek:attr:`&log`
.. zeek:field:: nw_src :zeek:type:`subnet` :zeek:attr:`&optional` :zeek:attr:`&log`
.. zeek:field:: nw_dst :zeek:type:`subnet` :zeek:attr:`&optional` :zeek:attr:`&log`
.. zeek:field:: tp_src :zeek:type:`count` :zeek:attr:`&optional` :zeek:attr:`&log`
.. zeek:field:: tp_dst :zeek:type:`count` :zeek:attr:`&optional` :zeek:attr:`&log`
:Attributes: :zeek:attr:`&log`
Openflow match definition.
The openflow match record describes
which packets match to a specific
rule in a flow table.

14
doc/scripts/base/frameworks/packet-filter/__load__.zeek.rst Normal file
View file

@ -0,0 +1,14 @@
:tocdepth: 3
base/frameworks/packet-filter/__load__.zeek
===========================================
:Imports: :doc:`base/frameworks/cluster </scripts/base/frameworks/cluster/index>`, :doc:`base/frameworks/packet-filter/main.zeek </scripts/base/frameworks/packet-filter/main.zeek>`, :doc:`base/frameworks/packet-filter/netstats.zeek </scripts/base/frameworks/packet-filter/netstats.zeek>`, :doc:`base/frameworks/packet-filter/utils.zeek </scripts/base/frameworks/packet-filter/utils.zeek>`
Summary
~~~~~~~
Detailed Interface
~~~~~~~~~~~~~~~~~~

27
doc/scripts/base/frameworks/packet-filter/index.rst Normal file
View file

@ -0,0 +1,27 @@
:orphan:
Package: base/frameworks/packet-filter
======================================
The packet filter framework supports how Zeek sets its BPF capture filter.
:doc:`/scripts/base/frameworks/packet-filter/utils.zeek`
:doc:`/scripts/base/frameworks/packet-filter/__load__.zeek`
:doc:`/scripts/base/frameworks/packet-filter/main.zeek`
This script supports how Zeek sets its BPF capture filter. By default
Zeek sets a capture filter that allows all traffic. If a filter
is set on the command line, that filter takes precedence over the default
open filter and all filters defined in Zeek scripts with the
:zeek:id:`capture_filters` and :zeek:id:`restrict_filters` variables.
:doc:`/scripts/base/frameworks/packet-filter/netstats.zeek`
This script reports on packet loss from the various packet sources.
When Zeek is reading input from trace files, this script will not
report any packet loss statistics.

301
doc/scripts/base/frameworks/packet-filter/main.zeek.rst Normal file
View file

@ -0,0 +1,301 @@
:tocdepth: 3
base/frameworks/packet-filter/main.zeek
=======================================
.. zeek:namespace:: PacketFilter
This script supports how Zeek sets its BPF capture filter. By default
Zeek sets a capture filter that allows all traffic. If a filter
is set on the command line, that filter takes precedence over the default
open filter and all filters defined in Zeek scripts with the
:zeek:id:`capture_filters` and :zeek:id:`restrict_filters` variables.
:Namespace: PacketFilter
:Imports: :doc:`base/frameworks/analyzer </scripts/base/frameworks/analyzer/index>`, :doc:`base/frameworks/notice </scripts/base/frameworks/notice/index>`, :doc:`base/frameworks/packet-filter/utils.zeek </scripts/base/frameworks/packet-filter/utils.zeek>`
Summary
~~~~~~~
Redefinable Options
###################
==================================================================================================== ===============================================================================
:zeek:id:`PacketFilter::default_capture_filter`: :zeek:type:`string` :zeek:attr:`&redef` The BPF filter that is used by default to define what traffic should
be captured.
:zeek:id:`PacketFilter::enable_auto_protocol_capture_filters`: :zeek:type:`bool` :zeek:attr:`&redef` Enables the old filtering approach of "only watch common ports for
analyzed protocols".
:zeek:id:`PacketFilter::max_filter_compile_time`: :zeek:type:`interval` :zeek:attr:`&redef` The maximum amount of time that you'd like to allow for BPF filters to compile.
:zeek:id:`PacketFilter::restricted_filter`: :zeek:type:`string` :zeek:attr:`&redef` Filter string which is unconditionally and'ed to the beginning of
every dynamically built filter.
:zeek:id:`PacketFilter::unrestricted_filter`: :zeek:type:`string` :zeek:attr:`&redef` Filter string which is unconditionally or'ed to the beginning of
every dynamically built filter.
==================================================================================================== ===============================================================================
State Variables
###############
============================================================ ===================================================================
:zeek:id:`PacketFilter::current_filter`: :zeek:type:`string` This is where the default packet filter is stored and it should not
normally be modified by users.
============================================================ ===================================================================
Types
#####
============================================================ ==================================================================
:zeek:type:`PacketFilter::FilterPlugin`: :zeek:type:`record` A data structure to represent filter generating plugins.
:zeek:type:`PacketFilter::Info`: :zeek:type:`record` The record type defining columns to be logged in the packet filter
logging stream.
============================================================ ==================================================================
Redefinitions
#############
============================================ =================================================================
:zeek:type:`Log::ID`: :zeek:type:`enum` Add the packet filter logging stream.
* :zeek:enum:`PacketFilter::LOG`
:zeek:type:`Notice::Type`: :zeek:type:`enum` Add notice types related to packet filter errors.
* :zeek:enum:`PacketFilter::Compile_Failure`:
This notice is generated if a packet filter cannot be compiled.
* :zeek:enum:`PacketFilter::Install_Failure`:
Generated if a packet filter fails to install.
* :zeek:enum:`PacketFilter::Too_Long_To_Compile_Filter`:
Generated when a notice takes too long to compile.
:zeek:type:`PcapFilterID`: :zeek:type:`enum`
* :zeek:enum:`PacketFilter::DefaultPcapFilter`
* :zeek:enum:`PacketFilter::FilterTester`
============================================ =================================================================
Hooks
#####
================================================================= =============================================
:zeek:id:`PacketFilter::log_policy`: :zeek:type:`Log::PolicyHook` A default logging policy hook for the stream.
================================================================= =============================================
Functions
#########
====================================================================== ======================================================================
:zeek:id:`PacketFilter::exclude`: :zeek:type:`function` Install a BPF filter to exclude some traffic.
:zeek:id:`PacketFilter::exclude_for`: :zeek:type:`function` Install a temporary filter to traffic which should not be passed
through the BPF filter.
:zeek:id:`PacketFilter::install`: :zeek:type:`function` Call this function to build and install a new dynamically built
packet filter.
:zeek:id:`PacketFilter::register_filter_plugin`: :zeek:type:`function` API function to register a new plugin for dynamic restriction filters.
:zeek:id:`PacketFilter::remove_exclude`: :zeek:type:`function` Remove a previously added exclude filter fragment by name.
====================================================================== ======================================================================
Detailed Interface
~~~~~~~~~~~~~~~~~~
Redefinable Options
###################
.. zeek:id:: PacketFilter::default_capture_filter
:source-code: base/frameworks/packet-filter/main.zeek 59 59
:Type: :zeek:type:`string`
:Attributes: :zeek:attr:`&redef`
:Default: ``"ip or not ip"``
The BPF filter that is used by default to define what traffic should
be captured. Filters defined in :zeek:id:`restrict_filters` will
still be applied to reduce the captured traffic.
.. zeek:id:: PacketFilter::enable_auto_protocol_capture_filters
:source-code: base/frameworks/packet-filter/main.zeek 131 131
:Type: :zeek:type:`bool`
:Attributes: :zeek:attr:`&redef`
:Default: ``F``
Enables the old filtering approach of "only watch common ports for
analyzed protocols".
Unless you know what you are doing, leave this set to F.
.. zeek:id:: PacketFilter::max_filter_compile_time
:source-code: base/frameworks/packet-filter/main.zeek 74 74
:Type: :zeek:type:`interval`
:Attributes: :zeek:attr:`&redef`
:Default: ``100.0 msecs``
The maximum amount of time that you'd like to allow for BPF filters to compile.
If this time is exceeded, compensation measures may be taken by the framework
to reduce the filter size. This threshold being crossed also results
in the :zeek:see:`PacketFilter::Too_Long_To_Compile_Filter` notice.
.. zeek:id:: PacketFilter::restricted_filter
:source-code: base/frameworks/packet-filter/main.zeek 68 68
:Type: :zeek:type:`string`
:Attributes: :zeek:attr:`&redef`
:Default: ``""``
Filter string which is unconditionally and'ed to the beginning of
every dynamically built filter. This is mostly used when a custom
filter is being used but MPLS or VLAN tags are on the traffic.
.. zeek:id:: PacketFilter::unrestricted_filter
:source-code: base/frameworks/packet-filter/main.zeek 63 63
:Type: :zeek:type:`string`
:Attributes: :zeek:attr:`&redef`
:Default: ``""``
Filter string which is unconditionally or'ed to the beginning of
every dynamically built filter.
State Variables
###############
.. zeek:id:: PacketFilter::current_filter
:source-code: base/frameworks/packet-filter/main.zeek 135 135
:Type: :zeek:type:`string`
:Default: ``"<not set yet>"``
This is where the default packet filter is stored and it should not
normally be modified by users.
Types
#####
.. zeek:type:: PacketFilter::FilterPlugin
:source-code: base/frameworks/packet-filter/main.zeek 119 122
:Type: :zeek:type:`record`
.. zeek:field:: func :zeek:type:`function` () : :zeek:type:`void`
A function that is directly called when generating the complete filter.
A data structure to represent filter generating plugins.
.. zeek:type:: PacketFilter::Info
:source-code: base/frameworks/packet-filter/main.zeek 34 54
:Type: :zeek:type:`record`
.. zeek:field:: ts :zeek:type:`time` :zeek:attr:`&log`
The time at which the packet filter installation attempt was made.
.. zeek:field:: node :zeek:type:`string` :zeek:attr:`&log` :zeek:attr:`&optional`
This is a string representation of the node that applied this
packet filter. It's mostly useful in the context of
dynamically changing filters on clusters.
.. zeek:field:: filter :zeek:type:`string` :zeek:attr:`&log`
The packet filter that is being set.
.. zeek:field:: init :zeek:type:`bool` :zeek:attr:`&log` :zeek:attr:`&default` = ``F`` :zeek:attr:`&optional`
Indicate if this is the filter set during initialization.
.. zeek:field:: success :zeek:type:`bool` :zeek:attr:`&log` :zeek:attr:`&default` = ``T`` :zeek:attr:`&optional`
Indicate if the filter was applied successfully.
.. zeek:field:: failure_reason :zeek:type:`string` :zeek:attr:`&log` :zeek:attr:`&optional`
A string reason why the filter failed to be created/installed.
The record type defining columns to be logged in the packet filter
logging stream.
Hooks
#####
.. zeek:id:: PacketFilter::log_policy
:source-code: base/frameworks/packet-filter/main.zeek 18 18
:Type: :zeek:type:`Log::PolicyHook`
A default logging policy hook for the stream.
Functions
#########
.. zeek:id:: PacketFilter::exclude
:source-code: base/frameworks/packet-filter/main.zeek 222 230
:Type: :zeek:type:`function` (filter_id: :zeek:type:`string`, filter: :zeek:type:`string`) : :zeek:type:`bool`
Install a BPF filter to exclude some traffic. The filter should
positively match what is to be excluded, it will be wrapped in
a "not".
:param filter_id: An arbitrary string that can be used to identify
the filter.
:param filter: A BPF expression of traffic that should be excluded.
:returns: A boolean value to indicate if the filter was successfully
installed or not.
.. zeek:id:: PacketFilter::exclude_for
:source-code: base/frameworks/packet-filter/main.zeek 232 240
:Type: :zeek:type:`function` (filter_id: :zeek:type:`string`, filter: :zeek:type:`string`, span: :zeek:type:`interval`) : :zeek:type:`bool`
Install a temporary filter to traffic which should not be passed
through the BPF filter. The filter should match the traffic you
don't want to see (it will be wrapped in a "not" condition).
:param filter_id: An arbitrary string that can be used to identify
the filter.
:param filter: A BPF expression of traffic that should be excluded.
:param length: The duration for which this filter should be put in place.
:returns: A boolean value to indicate if the filter was successfully
installed or not.
.. zeek:id:: PacketFilter::install
:source-code: base/frameworks/packet-filter/main.zeek 287 364
:Type: :zeek:type:`function` () : :zeek:type:`bool`
Call this function to build and install a new dynamically built
packet filter.
.. zeek:id:: PacketFilter::register_filter_plugin
:source-code: base/frameworks/packet-filter/main.zeek 201 204
:Type: :zeek:type:`function` (fp: :zeek:type:`PacketFilter::FilterPlugin`) : :zeek:type:`void`
API function to register a new plugin for dynamic restriction filters.
.. zeek:id:: PacketFilter::remove_exclude
:source-code: base/frameworks/packet-filter/main.zeek 211 220
:Type: :zeek:type:`function` (filter_id: :zeek:type:`string`) : :zeek:type:`bool`
Remove a previously added exclude filter fragment by name. The
traffic that was being filtered will be allowed through the filter
after calling this function.
:param filter_id: The name given to the filter fragment which you'd like to remove.
:returns: A boolean value to indicate if a filter fragment with the given name
actually installed.

44
doc/scripts/base/frameworks/packet-filter/netstats.zeek.rst Normal file
View file

@ -0,0 +1,44 @@
:tocdepth: 3
base/frameworks/packet-filter/netstats.zeek
===========================================
.. zeek:namespace:: PacketFilter
This script reports on packet loss from the various packet sources.
When Zeek is reading input from trace files, this script will not
report any packet loss statistics.
:Namespace: PacketFilter
:Imports: :doc:`base/frameworks/notice </scripts/base/frameworks/notice/index>`
Summary
~~~~~~~
Constants
#########
========================================================================= ==============================================================
:zeek:id:`PacketFilter::stats_collection_interval`: :zeek:type:`interval` This is the interval between individual statistics collection.
========================================================================= ==============================================================
Redefinitions
#############
============================================ ======================================================
:zeek:type:`Notice::Type`: :zeek:type:`enum`
* :zeek:enum:`PacketFilter::Dropped_Packets`:
Indicates packets were dropped by the packet filter.
============================================ ======================================================
Detailed Interface
~~~~~~~~~~~~~~~~~~
Constants
#########
.. zeek:id:: PacketFilter::stats_collection_interval
:source-code: base/frameworks/packet-filter/netstats.zeek 16 16
:Type: :zeek:type:`interval`
:Default: ``5.0 mins``
This is the interval between individual statistics collection.

76
doc/scripts/base/frameworks/packet-filter/utils.zeek.rst Normal file
View file

@ -0,0 +1,76 @@
:tocdepth: 3
base/frameworks/packet-filter/utils.zeek
========================================
.. zeek:namespace:: PacketFilter
:Namespace: PacketFilter
Summary
~~~~~~~
Functions
#########
=============================================================== ==================================================================
:zeek:id:`PacketFilter::combine_filters`: :zeek:type:`function` Combines two valid BPF filter strings with a string based operator
to form a new filter.
:zeek:id:`PacketFilter::port_to_bpf`: :zeek:type:`function` Takes a :zeek:type:`port` and returns a BPF expression which will
match the port.
:zeek:id:`PacketFilter::sampling_filter`: :zeek:type:`function` Create a BPF filter to sample IPv4 and IPv6 traffic.
=============================================================== ==================================================================
Detailed Interface
~~~~~~~~~~~~~~~~~~
Functions
#########
.. zeek:id:: PacketFilter::combine_filters
:source-code: base/frameworks/packet-filter/utils.zeek 40 50
:Type: :zeek:type:`function` (lfilter: :zeek:type:`string`, op: :zeek:type:`string`, rfilter: :zeek:type:`string`) : :zeek:type:`string`
Combines two valid BPF filter strings with a string based operator
to form a new filter.
:param lfilter: Filter which will go on the left side.
:param op: Operation being applied (typically "or" or "and").
:param rfilter: Filter which will go on the right side.
:returns: A new string representing the two filters combined with
the operator. Either filter being an empty string will
still result in a valid filter.
.. zeek:id:: PacketFilter::port_to_bpf
:source-code: base/frameworks/packet-filter/utils.zeek 34 38
:Type: :zeek:type:`function` (p: :zeek:type:`port`) : :zeek:type:`string`
Takes a :zeek:type:`port` and returns a BPF expression which will
match the port.
:param p: The port.
:returns: A valid BPF filter string for matching the port.
.. zeek:id:: PacketFilter::sampling_filter
:source-code: base/frameworks/packet-filter/utils.zeek 52 58
:Type: :zeek:type:`function` (num_parts: :zeek:type:`count`, this_part: :zeek:type:`count`) : :zeek:type:`string`
Create a BPF filter to sample IPv4 and IPv6 traffic.
:param num_parts: The number of parts the traffic should be split into.
:param this_part: The part of the traffic this filter will accept (0-based).

14
doc/scripts/base/frameworks/reporter/__load__.zeek.rst Normal file
View file

@ -0,0 +1,14 @@
:tocdepth: 3
base/frameworks/reporter/__load__.zeek
======================================
:Imports: :doc:`base/frameworks/reporter/main.zeek </scripts/base/frameworks/reporter/main.zeek>`
Summary
~~~~~~~
Detailed Interface
~~~~~~~~~~~~~~~~~~

26
doc/scripts/base/frameworks/reporter/index.rst Normal file
View file

@ -0,0 +1,26 @@
:orphan:
Package: base/frameworks/reporter
=================================
This framework is intended to create an output and filtering path for
internally generated messages/warnings/errors.
:doc:`/scripts/base/frameworks/reporter/__load__.zeek`
:doc:`/scripts/base/frameworks/reporter/main.zeek`
This framework is intended to create an output and filtering path for
internal messages/warnings/errors. It should typically be loaded to
log such messages to a file in a standard way. For the options to
toggle whether messages are additionally written to STDERR, see
:zeek:see:`Reporter::info_to_stderr`,
:zeek:see:`Reporter::warnings_to_stderr`, and
:zeek:see:`Reporter::errors_to_stderr`.
Note that this framework deals with the handling of internally generated
reporter messages, for the interface
into actually creating reporter messages from the scripting layer, use
the built-in functions in :doc:`/scripts/base/bif/reporter.bif.zeek`.

91
doc/scripts/base/frameworks/reporter/main.zeek.rst Normal file
View file

@ -0,0 +1,91 @@
:tocdepth: 3
base/frameworks/reporter/main.zeek
==================================
.. zeek:namespace:: Reporter
This framework is intended to create an output and filtering path for
internal messages/warnings/errors. It should typically be loaded to
log such messages to a file in a standard way. For the options to
toggle whether messages are additionally written to STDERR, see
:zeek:see:`Reporter::info_to_stderr`,
:zeek:see:`Reporter::warnings_to_stderr`, and
:zeek:see:`Reporter::errors_to_stderr`.
Note that this framework deals with the handling of internally generated
reporter messages, for the interface
into actually creating reporter messages from the scripting layer, use
the built-in functions in :doc:`/scripts/base/bif/reporter.bif.zeek`.
:Namespace: Reporter
Summary
~~~~~~~
Types
#####
================================================ =====================================================================
:zeek:type:`Reporter::Info`: :zeek:type:`record` The record type which contains the column fields of the reporter log.
================================================ =====================================================================
Redefinitions
#############
======================================= =======================================
:zeek:type:`Log::ID`: :zeek:type:`enum` The reporter logging stream identifier.
* :zeek:enum:`Reporter::LOG`
======================================= =======================================
Hooks
#####
============================================================= =============================================
:zeek:id:`Reporter::log_policy`: :zeek:type:`Log::PolicyHook` A default logging policy hook for the stream.
============================================================= =============================================
Detailed Interface
~~~~~~~~~~~~~~~~~~
Types
#####
.. zeek:type:: Reporter::Info
:source-code: base/frameworks/reporter/main.zeek 24 38
:Type: :zeek:type:`record`
.. zeek:field:: ts :zeek:type:`time` :zeek:attr:`&log`
The network time at which the reporter event was generated.
.. zeek:field:: level :zeek:type:`Reporter::Level` :zeek:attr:`&log`
The severity of the reporter message. Levels are INFO for informational
messages, not needing specific attention; WARNING for warning of a potential
problem, and ERROR for a non-fatal error that should be addressed, but doesn't
terminate program execution.
.. zeek:field:: message :zeek:type:`string` :zeek:attr:`&log`
An info/warning/error message that could have either been
generated from the internal Zeek core or at the scripting-layer.
.. zeek:field:: location :zeek:type:`string` :zeek:attr:`&log` :zeek:attr:`&optional`
This is the location in a Zeek script where the message originated.
Not all reporter messages will have locations in them though.
The record type which contains the column fields of the reporter log.
Hooks
#####
.. zeek:id:: Reporter::log_policy
:source-code: base/frameworks/reporter/main.zeek 21 21
:Type: :zeek:type:`Log::PolicyHook`
A default logging policy hook for the stream.

14
doc/scripts/base/frameworks/signatures/__load__.zeek.rst Normal file
View file

@ -0,0 +1,14 @@
:tocdepth: 3
base/frameworks/signatures/__load__.zeek
========================================
:Imports: :doc:`base/frameworks/signatures/main.zeek </scripts/base/frameworks/signatures/main.zeek>`
Summary
~~~~~~~
Detailed Interface
~~~~~~~~~~~~~~~~~~

19
doc/scripts/base/frameworks/signatures/index.rst Normal file
View file

@ -0,0 +1,19 @@
:orphan:
Package: base/frameworks/signatures
===================================
The signature framework provides for doing low-level pattern matching. While
signatures are not Zeek's preferred detection tool, they sometimes come in
handy and are closer to what many people are familiar with from using
other NIDS.
:doc:`/scripts/base/frameworks/signatures/__load__.zeek`
:doc:`/scripts/base/frameworks/signatures/main.zeek`
Script level signature support. See the
:doc:`signature documentation </frameworks/signatures>` for more
information about Zeek's signature engine.

364
doc/scripts/base/frameworks/signatures/main.zeek.rst Normal file
View file

@ -0,0 +1,364 @@
:tocdepth: 3
base/frameworks/signatures/main.zeek
====================================
.. zeek:namespace:: Signatures
Script level signature support. See the
:doc:`signature documentation </frameworks/signatures>` for more
information about Zeek's signature engine.
:Namespace: Signatures
:Imports: :doc:`base/frameworks/notice </scripts/base/frameworks/notice/index>`
Summary
~~~~~~~
Runtime Options
###############
================================================================================== ====================================================================
:zeek:id:`Signatures::ignored_ids`: :zeek:type:`pattern` :zeek:attr:`&redef` Signature IDs that should always be ignored.
:zeek:id:`Signatures::summary_interval`: :zeek:type:`interval` :zeek:attr:`&redef` The interval between when :zeek:enum:`Signatures::Signature_Summary`
notices are generated.
================================================================================== ====================================================================
Redefinable Options
###################
================================================================================== ====================================================================
:zeek:id:`Signatures::count_thresholds`: :zeek:type:`set` :zeek:attr:`&redef` Generate a notice if a :zeek:enum:`Signatures::SIG_COUNT_PER_RESP`
signature is triggered as often as given by one of these thresholds.
:zeek:id:`Signatures::horiz_scan_thresholds`: :zeek:type:`set` :zeek:attr:`&redef` Generate a notice if, for a pair [orig, signature], the number of
different responders has reached one of the thresholds.
:zeek:id:`Signatures::vert_scan_thresholds`: :zeek:type:`set` :zeek:attr:`&redef` Generate a notice if, for a pair [orig, resp], the number of
different signature matches has reached one of the thresholds.
================================================================================== ====================================================================
State Variables
###############
======================================================================================================================== ========================
:zeek:id:`Signatures::actions`: :zeek:type:`table` :zeek:attr:`&redef` :zeek:attr:`&default` = ``Signatures::SIG_ALARM`` Actions for a signature.
======================================================================================================================== ========================
Types
#####
================================================== ======================================================================
:zeek:type:`Signatures::Action`: :zeek:type:`enum` These are the default actions you can apply to signature matches.
:zeek:type:`Signatures::Info`: :zeek:type:`record` The record type which contains the column fields of the signature log.
================================================== ======================================================================
Redefinitions
#############
============================================ =============================================================
:zeek:type:`Log::ID`: :zeek:type:`enum` The signature logging stream identifier.
* :zeek:enum:`Signatures::LOG`
:zeek:type:`Notice::Type`: :zeek:type:`enum` Add various signature-related notice types.
* :zeek:enum:`Signatures::Count_Signature`:
The same signature has triggered multiple times for a host.
* :zeek:enum:`Signatures::Multiple_Sig_Responders`:
Host has triggered the same signature on multiple hosts as
defined by the :zeek:id:`Signatures::horiz_scan_thresholds`
variable.
* :zeek:enum:`Signatures::Multiple_Signatures`:
Host has triggered many signatures on the same host.
* :zeek:enum:`Signatures::Sensitive_Signature`:
Generic notice type for notice-worthy signature matches.
* :zeek:enum:`Signatures::Signature_Summary`:
Summarize the number of times a host triggered a signature.
============================================ =============================================================
Events
######
======================================================== =================================================================
:zeek:id:`Signatures::log_signature`: :zeek:type:`event` This event can be handled to access/alter data about to be logged
to the signature logging stream.
======================================================== =================================================================
Hooks
#####
=============================================================== =============================================
:zeek:id:`Signatures::log_policy`: :zeek:type:`Log::PolicyHook` A default logging policy hook for the stream.
=============================================================== =============================================
Detailed Interface
~~~~~~~~~~~~~~~~~~
Runtime Options
###############
.. zeek:id:: Signatures::ignored_ids
:source-code: base/frameworks/signatures/main.zeek 110 110
:Type: :zeek:type:`pattern`
:Attributes: :zeek:attr:`&redef`
:Default:
::
/^?(NO_DEFAULT_MATCHES)$?/
:Redefinition: from :doc:`/scripts/policy/misc/detect-traceroute/main.zeek`
``+=``::
/^?(traceroute-detector.*)$?/
:Redefinition: from :doc:`/scripts/policy/protocols/http/detect-webapps.zeek`
``+=``::
/^?(^webapp-)$?/
Signature IDs that should always be ignored.
.. zeek:id:: Signatures::summary_interval
:source-code: base/frameworks/signatures/main.zeek 126 126
:Type: :zeek:type:`interval`
:Attributes: :zeek:attr:`&redef`
:Default: ``1.0 day``
The interval between when :zeek:enum:`Signatures::Signature_Summary`
notices are generated.
Redefinable Options
###################
.. zeek:id:: Signatures::count_thresholds
:source-code: base/frameworks/signatures/main.zeek 122 122
:Type: :zeek:type:`set` [:zeek:type:`count`]
:Attributes: :zeek:attr:`&redef`
:Default:
::
{
1000,
10000,
500,
5,
10,
100,
50,
1000000
}
Generate a notice if a :zeek:enum:`Signatures::SIG_COUNT_PER_RESP`
signature is triggered as often as given by one of these thresholds.
.. zeek:id:: Signatures::horiz_scan_thresholds
:source-code: base/frameworks/signatures/main.zeek 114 114
:Type: :zeek:type:`set` [:zeek:type:`count`]
:Attributes: :zeek:attr:`&redef`
:Default:
::
{
1000,
500,
5,
10,
100,
50
}
Generate a notice if, for a pair [orig, signature], the number of
different responders has reached one of the thresholds.
.. zeek:id:: Signatures::vert_scan_thresholds
:source-code: base/frameworks/signatures/main.zeek 118 118
:Type: :zeek:type:`set` [:zeek:type:`count`]
:Attributes: :zeek:attr:`&redef`
:Default:
::
{
1000,
500,
5,
10,
100,
50
}
Generate a notice if, for a pair [orig, resp], the number of
different signature matches has reached one of the thresholds.
State Variables
###############
.. zeek:id:: Signatures::actions
:source-code: base/frameworks/signatures/main.zeek 105 105
:Type: :zeek:type:`table` [:zeek:type:`string`] of :zeek:type:`Signatures::Action`
:Attributes: :zeek:attr:`&redef` :zeek:attr:`&default` = ``Signatures::SIG_ALARM``
:Default:
::
{
["unspecified"] = Signatures::SIG_IGNORE
}
Actions for a signature. Can be updated dynamically.
Types
#####
.. zeek:type:: Signatures::Action
:source-code: base/frameworks/signatures/main.zeek 44 70
:Type: :zeek:type:`enum`
.. zeek:enum:: Signatures::SIG_IGNORE Signatures::Action
Ignore this signature completely (even for scan detection).
Don't write to the signatures logging stream.
.. zeek:enum:: Signatures::SIG_QUIET Signatures::Action
Process through the various aggregate techniques, but don't
report individually and don't write to the signatures logging
stream.
.. zeek:enum:: Signatures::SIG_LOG Signatures::Action
Generate a notice.
.. zeek:enum:: Signatures::SIG_FILE_BUT_NO_SCAN Signatures::Action
The same as :zeek:enum:`Signatures::SIG_LOG`, but ignore for
aggregate/scan processing.
.. zeek:enum:: Signatures::SIG_ALARM Signatures::Action
Generate a notice and set it to be alarmed upon.
.. zeek:enum:: Signatures::SIG_ALARM_PER_ORIG Signatures::Action
Alarm once per originator.
.. zeek:enum:: Signatures::SIG_ALARM_ONCE Signatures::Action
Alarm once and then never again.
.. zeek:enum:: Signatures::SIG_COUNT_PER_RESP Signatures::Action
Count signatures per responder host and alarm with the
:zeek:enum:`Signatures::Count_Signature` notice if a threshold
defined by :zeek:id:`Signatures::count_thresholds` is reached.
.. zeek:enum:: Signatures::SIG_SUMMARY Signatures::Action
Don't alarm, but generate per-orig summary.
These are the default actions you can apply to signature matches.
All of them write the signature record to the logging stream unless
declared otherwise.
.. zeek:type:: Signatures::Info
:source-code: base/frameworks/signatures/main.zeek 72 102
:Type: :zeek:type:`record`
.. zeek:field:: ts :zeek:type:`time` :zeek:attr:`&log`
The network time at which a signature matching type of event
to be logged has occurred.
.. zeek:field:: uid :zeek:type:`string` :zeek:attr:`&log` :zeek:attr:`&optional`
A unique identifier of the connection which triggered the
signature match event.
.. zeek:field:: src_addr :zeek:type:`addr` :zeek:attr:`&log` :zeek:attr:`&optional`
The host which triggered the signature match event.
.. zeek:field:: src_port :zeek:type:`port` :zeek:attr:`&log` :zeek:attr:`&optional`
The host port on which the signature-matching activity
occurred.
.. zeek:field:: dst_addr :zeek:type:`addr` :zeek:attr:`&log` :zeek:attr:`&optional`
The destination host which was sent the payload that
triggered the signature match.
.. zeek:field:: dst_port :zeek:type:`port` :zeek:attr:`&log` :zeek:attr:`&optional`
The destination host port which was sent the payload that
triggered the signature match.
.. zeek:field:: note :zeek:type:`Notice::Type` :zeek:attr:`&log`
Notice associated with signature event.
.. zeek:field:: sig_id :zeek:type:`string` :zeek:attr:`&log` :zeek:attr:`&optional`
The name of the signature that matched.
.. zeek:field:: event_msg :zeek:type:`string` :zeek:attr:`&log` :zeek:attr:`&optional`
A more descriptive message of the signature-matching event.
.. zeek:field:: sub_msg :zeek:type:`string` :zeek:attr:`&log` :zeek:attr:`&optional`
Extracted payload data or extra message.
.. zeek:field:: sig_count :zeek:type:`count` :zeek:attr:`&log` :zeek:attr:`&optional`
Number of sigs, usually from summary count.
.. zeek:field:: host_count :zeek:type:`count` :zeek:attr:`&log` :zeek:attr:`&optional`
Number of hosts, from a summary count.
The record type which contains the column fields of the signature log.
Events
######
.. zeek:id:: Signatures::log_signature
:source-code: base/frameworks/signatures/main.zeek 132 132
:Type: :zeek:type:`event` (rec: :zeek:type:`Signatures::Info`)
This event can be handled to access/alter data about to be logged
to the signature logging stream.
:param rec: The record of signature data about to be logged.
Hooks
#####
.. zeek:id:: Signatures::log_policy
:source-code: base/frameworks/signatures/main.zeek 39 39
:Type: :zeek:type:`Log::PolicyHook`
A default logging policy hook for the stream.

14
doc/scripts/base/frameworks/software/__load__.zeek.rst Normal file
View file

@ -0,0 +1,14 @@
:tocdepth: 3
base/frameworks/software/__load__.zeek
======================================
:Imports: :doc:`base/frameworks/software/main.zeek </scripts/base/frameworks/software/main.zeek>`
Summary
~~~~~~~
Detailed Interface
~~~~~~~~~~~~~~~~~~

21
doc/scripts/base/frameworks/software/index.rst Normal file
View file

@ -0,0 +1,21 @@
:orphan:
Package: base/frameworks/software
=================================
The software framework provides infrastructure for maintaining a table
of software versions seen on the network. The version parsing itself
is carried out by external protocol-specific scripts that feed into
this framework.
:doc:`/scripts/base/frameworks/software/__load__.zeek`
:doc:`/scripts/base/frameworks/software/main.zeek`
This script provides the framework for software version detection and
parsing but doesn't actually do any detection on it's own. It relies on
other protocol specific scripts to parse out software from the protocols
that they analyze. The entry point for providing new software detections
to this framework is through the :zeek:id:`Software::found` function.

Some files were not shown because too many files have changed in this diff Show more