diff --git a/CHANGES b/CHANGES index 6ce7b8d0bb..75c53c4c46 100644 --- a/CHANGES +++ b/CHANGES @@ -1,3 +1,18 @@ +7.1.0-dev.732 | 2024-12-09 23:28:30 -0800 + + * Support for Broker I/O backpressure overflow policies (Christian Kreibich, Corelight, and Dominik Charousset) + + - Add sleep() BiF + - Add backpressure disconnect notification to cluster.log and via telemetry + - Remove unneeded @loads from base/misc/version.zeek + - Add Cluster::nodeid_to_node() helper function + - Support re-peering with Broker peers that fall behind + - Add Zeek-level configurability of Broker slow-peer disconnects + - Bump Broker to pull in disconnect feature and infinite-loop fix + - No need to namespace Cluster:: functions in their own namespace + + * Update doc submodule [nomail] [skip ci] (zeek-bot) + 7.1.0-dev.720 | 2024-12-09 12:22:44 -0700 * Add missing copyright line to headers and cc files (Arne Welzel, Corelight) diff --git a/NEWS b/NEWS index 142f63c9ab..95b41a5528 100644 --- a/NEWS +++ b/NEWS @@ -55,6 +55,54 @@ New Functionality If you observe PostgreSQL traffic in your environment, please provide feedback about the analyzer and structure of the new log. +- Broker's message I/O buffering now operates on per-peering granularity at the + sender (it was previously global) and provides configurable overflow handling + when a fast sender overwhelms a slow receiver, via the following new tunables + in the ``Broker`` module: + + const peer_buffer_size = 2048 &redef; + const peer_overflow_policy = "disconnect" &redef; + const web_socket_buffer_size = 512 &redef; + const web_socket_overflow_policy = "disconnect" &redef; + + When a send buffer overflows (i.e., it is full when a node tries to transmit + another message), the sender may drop the message and unpeer the slow receiver + (policy ``disconnect``, the default), drop the newest message in the buffer + (``drop_newest``), or drop the oldest (``drop_oldest``). Buffer sizes are + measured in number of messages, not bytes. Note that "sender" and "receiver" + are independent of the direction in which Zeek established the peering. After + disconnects Zeek automatically tries to re-establish peering with the slow + node, in case it recovers. + + Zeek notifies you in two ways of such disconnects: + + * A cluster.log entry for the sending node indicates that a slow peered node + has been removed. Here node ``worker01`` has removed a peered ``proxy01`: + + 1733468802.626622 worker01 removed due to backpressure overflow: 127.0.0.1:42204/tcp (proxy01) + + * The labeled counter metric ``zeek_broker_backpressure_disconnects_total`` + in the telemetry framework tracks the number of times such disconnects + happen between respective nodes. The following scraped telemetry indicates + the same disconnect as above: + + zeek_broker_backpressure_disconnects_total{endpoint="worker01",peer="proxy01"} 1 + + To implement custom handling of a backpressure-induced disconnect, add a + ``Broker::peer_removed`` event handler, as follows: + + event Broker::peer_removed(endpoint: Broker::EndpointInfo, msg: string) + { + if ( "caf::sec::backpressure_overflow" !in msg ) + return; + + # The local node has disconnected the given endpoint, + # add your logic here. + } + + These new policies fix a problem in which misbehaving nodes could trigger + cascading "lockups" of nodes, each ceasing to transmit any messages. + * The LDAP analyzer now supports handling of non-sealed GSS-API WRAP tokens. * StartTLS support was added to the LDAP analyzer. The SSL analyzer is enabled diff --git a/VERSION b/VERSION index 2f27e8c4aa..ab8f782116 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -7.1.0-dev.720 +7.1.0-dev.732 diff --git a/auxil/broker b/auxil/broker index 2a6e6201f7..28cdb7524f 160000 --- a/auxil/broker +++ b/auxil/broker @@ -1 +1 @@ -Subproject commit 2a6e6201f7b43e213f2bac3863ca571b659e8a16 +Subproject commit 28cdb7524f73ffa37315f4058f4f48948fe1683a diff --git a/scripts/base/frameworks/broker/__load__.zeek b/scripts/base/frameworks/broker/__load__.zeek index 77dd69d554..a30468a776 100644 --- a/scripts/base/frameworks/broker/__load__.zeek +++ b/scripts/base/frameworks/broker/__load__.zeek @@ -1,3 +1,4 @@ @load ./main @load ./store @load ./log +@load ./backpressure diff --git a/scripts/base/frameworks/broker/backpressure.zeek b/scripts/base/frameworks/broker/backpressure.zeek new file mode 100644 index 0000000000..652935eed9 --- /dev/null +++ b/scripts/base/frameworks/broker/backpressure.zeek @@ -0,0 +1,35 @@ +##! This handles Broker peers that fall so far behind in handling messages that +##! this node sends it that the local Broker endpoint decides to unpeer them. +##! Zeek captures this as follows: +##! +##! - In broker.log, with a regular "peer-removed" entry indicating CAF's reason. +##! - Via eventing through :zeek:see:`Broker::peer_removed` as done in this script. +##! +##! The cluster framework additionally captures the unpeering as follows: +##! +##! - In cluster.log, with a higher-level message indicating the node names involved. +##! - Via telemetry, using a labeled counter. + +event Broker::peer_removed(endpoint: Broker::EndpointInfo, msg: string) + { + if ( "caf::sec::backpressure_overflow" !in msg ) { + return; + } + + if ( ! endpoint?$network ) { + Reporter::error(fmt("Missing network info to re-peer with %s", endpoint$id)); + return; + } + + # Re-establish the peering so Broker's reconnect behavior kicks in once + # the other endpoint catches up. Broker will periodically re-try + # connecting as necessary. If the other endpoint originally connected to + # us, our attempt will fail (since we attempt to connect to the peer's + # ephemeral port), but in that case the peer will reconnect with us once + # it recovers. + # + # We could do this more cleanly by leveraging information from the + # cluster framework (since it knows who connects to whom), but that + # would further entangle Broker into it. + Broker::peer(endpoint$network$address, endpoint$network$bound_port); +} diff --git a/scripts/base/frameworks/broker/main.zeek b/scripts/base/frameworks/broker/main.zeek index d41f64ab2e..2990f3f297 100644 --- a/scripts/base/frameworks/broker/main.zeek +++ b/scripts/base/frameworks/broker/main.zeek @@ -86,6 +86,24 @@ export { ## ZEEK_BROKER_MAX_THREADS environment variable overrides this setting. const max_threads = 1 &redef; + ## Max number of items we buffer at most per peer. What action to take when + ## the buffer reaches its maximum size is determined by + ## `peer_overflow_policy`. + const peer_buffer_size = 2048 &redef; + + ## Configures how Broker responds to peers that cannot keep up with the + ## incoming message rate. Available strategies: + ## - disconnect: drop the connection to the unresponsive peer + ## - drop_newest: replace the newest message in the buffer + ## - drop_oldest: removed the olsted message from the buffer, then append + const peer_overflow_policy = "disconnect" &redef; + + ## Same as `peer_buffer_size` but for WebSocket clients. + const web_socket_buffer_size = 512 &redef; + + ## Same as `peer_overflow_policy` but for WebSocket clients. + const web_socket_overflow_policy = "disconnect" &redef; + ## The CAF scheduling policy to use. Available options are "sharing" and ## "stealing". The "sharing" policy uses a single, global work queue along ## with mutex and condition variable used for accessing it, which may be diff --git a/scripts/base/frameworks/cluster/__load__.zeek b/scripts/base/frameworks/cluster/__load__.zeek index a854302636..0d6372e3d4 100644 --- a/scripts/base/frameworks/cluster/__load__.zeek +++ b/scripts/base/frameworks/cluster/__load__.zeek @@ -14,6 +14,9 @@ redef Broker::log_topic = Cluster::rr_log_topic; # Add a cluster prefix. @prefixes += cluster +# This should soon condition on loading only when Broker is in use. +@load ./broker-backpressure + @if ( Supervisor::is_supervised() ) # When running a supervised cluster, populate Cluster::nodes from the node table # the Supervisor provides to new Zeek nodes. The management framework configures diff --git a/scripts/base/frameworks/cluster/broker-backpressure.zeek b/scripts/base/frameworks/cluster/broker-backpressure.zeek new file mode 100644 index 0000000000..e3fe4c9cdd --- /dev/null +++ b/scripts/base/frameworks/cluster/broker-backpressure.zeek @@ -0,0 +1,29 @@ +# Notifications for Broker-reported backpressure overflow. +# See base/frameworks/broker/backpressure.zeek for context. + +@load base/frameworks/telemetry + +module Cluster; + +global broker_backpressure_disconnects_cf = Telemetry::register_counter_family([ + $prefix="zeek", + $name="broker-backpressure-disconnects", + $unit="", + $label_names=vector("peer"), + $help_text="Number of Broker peerings dropped due to a neighbor falling behind in message I/O", +]); + +event Broker::peer_removed(endpoint: Broker::EndpointInfo, msg: string) + { + if ( ! endpoint?$network || "caf::sec::backpressure_overflow" !in msg ) + return; + + local nn = nodeid_to_node(endpoint$id); + + Cluster::log(fmt("removed due to backpressure overflow: %s%s:%s (%s)", + nn$name != "" ? "" : "non-cluster peer ", + endpoint$network$address, endpoint$network$bound_port, + nn$name != "" ? nn$name : endpoint$id)); + Telemetry::counter_family_inc(broker_backpressure_disconnects_cf, + vector(nn$name != "" ? nn$name : "unknown")); + } diff --git a/scripts/base/frameworks/cluster/main.zeek b/scripts/base/frameworks/cluster/main.zeek index caf2e6a11d..0427d6adcd 100644 --- a/scripts/base/frameworks/cluster/main.zeek +++ b/scripts/base/frameworks/cluster/main.zeek @@ -281,6 +281,15 @@ export { ## a given cluster node. global nodeid_topic: function(id: string): string; + ## Retrieve the cluster-level naming of a node based on its node ID, + ## a backend-specific identifier. + ## + ## id: the node ID of a peer. + ## + ## Returns: the :zeek:see:`Cluster::NamedNode` for the requested node, if + ## known, otherwise a "null" instance with an empty name field. + global nodeid_to_node: function(id: string): NamedNode; + ## Initialize the cluster backend. ## ## Cluster backends usually invoke this from a :zeek:see:`zeek_init` handler. @@ -336,7 +345,7 @@ function nodes_with_type(node_type: NodeType): vector of NamedNode { return strcmp(n1$name, n2$name); }); } -function Cluster::get_node_count(node_type: NodeType): count +function get_node_count(node_type: NodeType): count { local cnt = 0; @@ -349,7 +358,7 @@ function Cluster::get_node_count(node_type: NodeType): count return cnt; } -function Cluster::get_active_node_count(node_type: NodeType): count +function get_active_node_count(node_type: NodeType): count { return node_type in active_node_ids ? |active_node_ids[node_type]| : 0; } @@ -394,6 +403,17 @@ function nodeid_topic(id: string): string return nodeid_topic_prefix + id + "/"; } +function nodeid_to_node(id: string): NamedNode + { + for ( name, n in nodes ) + { + if ( n?$id && n$id == id ) + return NamedNode($name=name, $node=n); + } + + return NamedNode($name="", $node=[$node_type=NONE, $ip=0.0.0.0]); + } + event Cluster::hello(name: string, id: string) &priority=10 { if ( name !in nodes ) diff --git a/scripts/base/misc/version.zeek b/scripts/base/misc/version.zeek index 14e3d4c2a7..36d8ab1fc3 100644 --- a/scripts/base/misc/version.zeek +++ b/scripts/base/misc/version.zeek @@ -2,9 +2,6 @@ ##! The most convenient way to access this are the Version::number ##! and Version::info constants. -@load base/frameworks/reporter -@load base/utils/strings - module Version; export { diff --git a/src/broker/Manager.cc b/src/broker/Manager.cc index 7584c05fd4..9cf1a4481d 100644 --- a/src/broker/Manager.cc +++ b/src/broker/Manager.cc @@ -257,6 +257,36 @@ void Manager::DoInitPostScript() { options.disable_forwarding = ! get_option("Broker::forward_messages")->AsBool(); options.use_real_time = use_real_time; + options.peer_buffer_size = get_option("Broker::peer_buffer_size")->AsCount(); + auto peer_overflow_policy = get_option("Broker::peer_overflow_policy")->AsString()->CheckString(); + if ( util::streq(peer_overflow_policy, "disconnect") ) { + options.peer_overflow_policy = broker::overflow_policy::disconnect; + } + else if ( util::streq(peer_overflow_policy, "drop_oldest") ) { + options.peer_overflow_policy = broker::overflow_policy::drop_oldest; + } + else if ( util::streq(peer_overflow_policy, "drop_newest") ) { + options.peer_overflow_policy = broker::overflow_policy::drop_newest; + } + else { + reporter->FatalError("Invalid Broker::peer_overflow_policy: %s", peer_overflow_policy); + } + + options.web_socket_buffer_size = get_option("Broker::web_socket_buffer_size")->AsCount(); + auto web_socket_overflow_policy = get_option("Broker::web_socket_overflow_policy")->AsString()->CheckString(); + if ( util::streq(web_socket_overflow_policy, "disconnect") ) { + options.web_socket_overflow_policy = broker::overflow_policy::disconnect; + } + else if ( util::streq(web_socket_overflow_policy, "drop_oldest") ) { + options.web_socket_overflow_policy = broker::overflow_policy::drop_oldest; + } + else if ( util::streq(web_socket_overflow_policy, "drop_newest") ) { + options.web_socket_overflow_policy = broker::overflow_policy::drop_newest; + } + else { + reporter->FatalError("Invalid Broker::web_socket_overflow_policy: %s", web_socket_overflow_policy); + } + broker::configuration config{std::move(options)}; config.openssl_cafile(get_option("Broker::ssl_cafile")->AsString()->CheckString()); diff --git a/src/broker/comm.bif b/src/broker/comm.bif index b6d6292539..5cc2e89f8c 100644 --- a/src/broker/comm.bif +++ b/src/broker/comm.bif @@ -7,16 +7,56 @@ module Broker; -## Generated when something changes in the Broker sub-system. -event Broker::status%(endpoint: EndpointInfo, msg: string%); - -## Generated when a new peering has been established. +## Generated when a new peering has been established. Both sides of the peering +## receive this event, created independently in each endpoint. For the endpoint +## establishing the peering, the added endpoint's network information will match +## the address and port provided to :zeek:see:`Broker::peer`; for the listening +## endpoint it's the peer's TCP client's address and (likely ephemeral) TCP +## port. +## +## endpoint: the added endpoint's Broker ID and connection information. +## +## msg: a message providing additional context. +## +## .. zeek:see:: Broker::peer_removed Broker::peer_lost +## Broker::endpoint_discovered Broker::endpoint_unreachable +## Broker::status Broker::error event Broker::peer_added%(endpoint: EndpointInfo, msg: string%); -## Generated when an existing peer has been removed. +## Generated when the local endpoint has removed its peering with another +## endpoint. This event can fire for multiple reasons, such as a local call to +## :zeek:see:`Broker::unpeer`, or because Broker autonomously decides to +## unpeer. One reason it might do this is message I/O backpressure overflow, +## meaning that the remote peer cannot keep up with the stream of messages the +## local endpoint sends it. Regardless of the cause, the remote endpoint will +## locally trigger a corresponding :zeek:see:`Broker::peer_lost` event once the +## peering ends. These events are independent of the original directionality of +## TCP connection establishment and only reflect which endpoint terminates the +## peering. +## +## endpoint: the removed endpoint's Broker ID and connection information. +## +## msg: a message providing additional context. If backpressure overflow +## caused this unpeering, the message contains the string +## *caf::sec::backpressure_overflow*. +## +## .. zeek:see:: Broker::peer_added Broker::peer_lost +## Broker::endpoint_discovered Broker::endpoint_unreachable +## Broker::status Broker::error event Broker::peer_removed%(endpoint: EndpointInfo, msg: string%); -## Generated when an existing peering has been lost. +## Generated when the local endpoint has lost its peering with another +## endpoint. This event fires when the other endpoint stops or removes the +## peering for some other reason. This event is independent of the original +## directionality of connection establishment. +## +## endpoint: the lost endpoint's Broker ID and connection information. +## +## msg: a message providing additional context. +## +## .. zeek:see:: Broker::peer_added Broker::peer_removed +## Broker::endpoint_discovered Broker::endpoint_unreachable +## Broker::status Broker::error event Broker::peer_lost%(endpoint: EndpointInfo, msg: string%); ## Generated when a new Broker endpoint appeared. @@ -25,7 +65,29 @@ event Broker::endpoint_discovered%(endpoint: EndpointInfo, msg: string%); ## Generated when the last path to a Broker endpoint has been lost. event Broker::endpoint_unreachable%(endpoint: EndpointInfo, msg: string%); -## Generated when an error occurs in the Broker sub-system. +## Generated when an unspecified change occurs in Broker. This event only fires +## when the status change isn't covered by more specific Broker events. The +## provided message string may be empty. +## +## endpoint: the Broker ID and connection information, if available, +## of the endpoint the update relates to. +## +## msg: a message providing additional context. +## +## .. zeek:see:: Broker::peer_added Broker::peer_removed Broker::peer_lost +## Broker::endpoint_discovered Broker::endpoint_unreachable Broker::error +event Broker::status%(endpoint: EndpointInfo, msg: string%); + +## Generated when an error occurs in the Broker sub-system. This event +## reports local errors in Broker, as indicated by the provided +## :zeek:type:`Broker::ErrorCode`. +## +## code: the type of error that triggered this event. +## +## msg: a message providing additional context. +## +## .. zeek:see:: Broker::peer_added Broker::peer_removed Broker::peer_lost +## Broker::endpoint_discovered Broker::endpoint_unreachable Broker::status event Broker::error%(code: ErrorCode, msg: string%); ## Enumerates the possible error types. diff --git a/src/script_opt/FuncInfo.cc b/src/script_opt/FuncInfo.cc index 2516d1bd44..545f278f96 100644 --- a/src/script_opt/FuncInfo.cc +++ b/src/script_opt/FuncInfo.cc @@ -431,6 +431,7 @@ static std::unordered_map func_attrs = { {"skip_further_processing", ATTR_NO_SCRIPT_SIDE_EFFECTS}, {"skip_http_entity_data", ATTR_NO_SCRIPT_SIDE_EFFECTS}, {"skip_smtp_data", ATTR_NO_SCRIPT_SIDE_EFFECTS}, + {"sleep", ATTR_NO_SCRIPT_SIDE_EFFECTS}, {"split_string", ATTR_FOLDABLE}, {"split_string1", ATTR_FOLDABLE}, {"split_string_all", ATTR_FOLDABLE}, diff --git a/src/zeek.bif b/src/zeek.bif index ee14c0ddce..fbcf322c44 100644 --- a/src/zeek.bif +++ b/src/zeek.bif @@ -600,6 +600,27 @@ function piped_exec%(program: string, to_write: string%): bool return zeek::val_mgr->True(); %} +## Sleeps for the given amount of time. +## +## i: The time interval to sleep for. +## +## Returns: The :zeek:type:`interval` Zeek actually slept for. +## +## .. note:: +## +## This is a blocking sleep! Zeek will not run most of its processing +## during that time. You almost certainly DO NOT WANT THIS outside +## of specific testing/troubleshooting scenarios. To sleep asynchronously, +## :zeek:see:`schedule` an event, or consider :zeek:id:`Exec::run`. +function sleep%(i: interval%): interval + %{ + const auto start = std::chrono::high_resolution_clock::now(); + std::this_thread::sleep_for(std::chrono::duration(i)); + const auto end = std::chrono::high_resolution_clock::now(); + const auto slept = std::chrono::duration(end - start).count(); + return zeek::make_intrusive(slept); + %} + %%{ #include "zeek/OpaqueVal.h" %%} diff --git a/testing/btest/Baseline.zam/opt.ZAM-bif-tracking/output b/testing/btest/Baseline.zam/opt.ZAM-bif-tracking/output index 6f645f9525..0138aa7bfe 100644 --- a/testing/btest/Baseline.zam/opt.ZAM-bif-tracking/output +++ b/testing/btest/Baseline.zam/opt.ZAM-bif-tracking/output @@ -1,2 +1,2 @@ ### BTest baseline data generated by btest-diff. Do not edit. Use "btest -U/-u" to update. Requires BTest >= 0.63. -539 seen BiFs, 0 unseen BiFs (), 0 new BiFs () +540 seen BiFs, 0 unseen BiFs (), 0 new BiFs () diff --git a/testing/btest/Baseline/bifs.sleep/out b/testing/btest/Baseline/bifs.sleep/out new file mode 100644 index 0000000000..49d861c74c --- /dev/null +++ b/testing/btest/Baseline/bifs.sleep/out @@ -0,0 +1 @@ +### BTest baseline data generated by btest-diff. Do not edit. Use "btest -U/-u" to update. Requires BTest >= 0.63. diff --git a/testing/btest/Baseline/coverage.bare-load-baseline/canonified_loaded_scripts.log b/testing/btest/Baseline/coverage.bare-load-baseline/canonified_loaded_scripts.log index 5e88f9d327..443d35b00e 100644 --- a/testing/btest/Baseline/coverage.bare-load-baseline/canonified_loaded_scripts.log +++ b/testing/btest/Baseline/coverage.bare-load-baseline/canonified_loaded_scripts.log @@ -119,6 +119,7 @@ scripts/base/init-frameworks-and-bifs.zeek build/scripts/base/bif/data.bif.zeek build/scripts/base/bif/store.bif.zeek scripts/base/frameworks/broker/log.zeek + scripts/base/frameworks/broker/backpressure.zeek scripts/base/frameworks/supervisor/__load__.zeek scripts/base/frameworks/supervisor/control.zeek scripts/base/frameworks/supervisor/main.zeek diff --git a/testing/btest/Baseline/coverage.default-load-baseline/canonified_loaded_scripts.log b/testing/btest/Baseline/coverage.default-load-baseline/canonified_loaded_scripts.log index a3f06f9db9..8a23826c17 100644 --- a/testing/btest/Baseline/coverage.default-load-baseline/canonified_loaded_scripts.log +++ b/testing/btest/Baseline/coverage.default-load-baseline/canonified_loaded_scripts.log @@ -119,6 +119,7 @@ scripts/base/init-frameworks-and-bifs.zeek build/scripts/base/bif/data.bif.zeek build/scripts/base/bif/store.bif.zeek scripts/base/frameworks/broker/log.zeek + scripts/base/frameworks/broker/backpressure.zeek scripts/base/frameworks/supervisor/__load__.zeek scripts/base/frameworks/supervisor/control.zeek scripts/base/frameworks/supervisor/main.zeek diff --git a/testing/btest/Baseline/coverage.init-default/missing_loads b/testing/btest/Baseline/coverage.init-default/missing_loads index e16624e1fb..9997ec4fd8 100644 --- a/testing/btest/Baseline/coverage.init-default/missing_loads +++ b/testing/btest/Baseline/coverage.init-default/missing_loads @@ -1,4 +1,5 @@ ### BTest baseline data generated by btest-diff. Do not edit. Use "btest -U/-u" to update. Requires BTest >= 0.63. +-./frameworks/cluster/broker-backpressure.zeek -./frameworks/cluster/broker-stores.zeek -./frameworks/cluster/nodes/logger.zeek -./frameworks/cluster/nodes/manager.zeek diff --git a/testing/btest/Baseline/plugins.hooks/output b/testing/btest/Baseline/plugins.hooks/output index c799317bd1..9eadabd9ad 100644 --- a/testing/btest/Baseline/plugins.hooks/output +++ b/testing/btest/Baseline/plugins.hooks/output @@ -461,6 +461,7 @@ 0.000000 MetaHookPost LoadFile(0, ./addrs, <...>/addrs.zeek) -> -1 0.000000 MetaHookPost LoadFile(0, ./analyzer.bif.zeek, <...>/analyzer.bif.zeek) -> -1 0.000000 MetaHookPost LoadFile(0, ./api, <...>/api.zeek) -> -1 +0.000000 MetaHookPost LoadFile(0, ./backpressure, <...>/backpressure.zeek) -> -1 0.000000 MetaHookPost LoadFile(0, ./bloom-filter.bif.zeek, <...>/bloom-filter.bif.zeek) -> -1 0.000000 MetaHookPost LoadFile(0, ./cardinality-counter.bif.zeek, <...>/cardinality-counter.bif.zeek) -> -1 0.000000 MetaHookPost LoadFile(0, ./cluster.bif.zeek, <...>/cluster.bif.zeek) -> -1 @@ -766,6 +767,7 @@ 0.000000 MetaHookPost LoadFileExtended(0, ./addrs, <...>/addrs.zeek) -> (-1, ) 0.000000 MetaHookPost LoadFileExtended(0, ./analyzer.bif.zeek, <...>/analyzer.bif.zeek) -> (-1, ) 0.000000 MetaHookPost LoadFileExtended(0, ./api, <...>/api.zeek) -> (-1, ) +0.000000 MetaHookPost LoadFileExtended(0, ./backpressure, <...>/backpressure.zeek) -> (-1, ) 0.000000 MetaHookPost LoadFileExtended(0, ./bloom-filter.bif.zeek, <...>/bloom-filter.bif.zeek) -> (-1, ) 0.000000 MetaHookPost LoadFileExtended(0, ./cardinality-counter.bif.zeek, <...>/cardinality-counter.bif.zeek) -> (-1, ) 0.000000 MetaHookPost LoadFileExtended(0, ./cluster.bif.zeek, <...>/cluster.bif.zeek) -> (-1, ) @@ -1403,6 +1405,7 @@ 0.000000 MetaHookPre LoadFile(0, ./addrs, <...>/addrs.zeek) 0.000000 MetaHookPre LoadFile(0, ./analyzer.bif.zeek, <...>/analyzer.bif.zeek) 0.000000 MetaHookPre LoadFile(0, ./api, <...>/api.zeek) +0.000000 MetaHookPre LoadFile(0, ./backpressure, <...>/backpressure.zeek) 0.000000 MetaHookPre LoadFile(0, ./bloom-filter.bif.zeek, <...>/bloom-filter.bif.zeek) 0.000000 MetaHookPre LoadFile(0, ./cardinality-counter.bif.zeek, <...>/cardinality-counter.bif.zeek) 0.000000 MetaHookPre LoadFile(0, ./cluster.bif.zeek, <...>/cluster.bif.zeek) @@ -1708,6 +1711,7 @@ 0.000000 MetaHookPre LoadFileExtended(0, ./addrs, <...>/addrs.zeek) 0.000000 MetaHookPre LoadFileExtended(0, ./analyzer.bif.zeek, <...>/analyzer.bif.zeek) 0.000000 MetaHookPre LoadFileExtended(0, ./api, <...>/api.zeek) +0.000000 MetaHookPre LoadFileExtended(0, ./backpressure, <...>/backpressure.zeek) 0.000000 MetaHookPre LoadFileExtended(0, ./bloom-filter.bif.zeek, <...>/bloom-filter.bif.zeek) 0.000000 MetaHookPre LoadFileExtended(0, ./cardinality-counter.bif.zeek, <...>/cardinality-counter.bif.zeek) 0.000000 MetaHookPre LoadFileExtended(0, ./cluster.bif.zeek, <...>/cluster.bif.zeek) @@ -2346,6 +2350,7 @@ 0.000000 | HookLoadFile ./api <...>/api.zeek 0.000000 | HookLoadFile ./archive <...>/archive.sig 0.000000 | HookLoadFile ./audio <...>/audio.sig +0.000000 | HookLoadFile ./backpressure <...>/backpressure.zeek 0.000000 | HookLoadFile ./bloom-filter.bif.zeek <...>/bloom-filter.bif.zeek 0.000000 | HookLoadFile ./cardinality-counter.bif.zeek <...>/cardinality-counter.bif.zeek 0.000000 | HookLoadFile ./cluster.bif.zeek <...>/cluster.bif.zeek @@ -2651,6 +2656,7 @@ 0.000000 | HookLoadFileExtended ./api <...>/api.zeek 0.000000 | HookLoadFileExtended ./archive <...>/archive.sig 0.000000 | HookLoadFileExtended ./audio <...>/audio.sig +0.000000 | HookLoadFileExtended ./backpressure <...>/backpressure.zeek 0.000000 | HookLoadFileExtended ./bloom-filter.bif.zeek <...>/bloom-filter.bif.zeek 0.000000 | HookLoadFileExtended ./cardinality-counter.bif.zeek <...>/cardinality-counter.bif.zeek 0.000000 | HookLoadFileExtended ./cluster.bif.zeek <...>/cluster.bif.zeek diff --git a/testing/btest/Baseline/scripts.base.misc.version/.stderr b/testing/btest/Baseline/scripts.base.misc.version/.stderr index 4cd9e25cb2..c7e80a0f4c 100644 --- a/testing/btest/Baseline/scripts.base.misc.version/.stderr +++ b/testing/btest/Baseline/scripts.base.misc.version/.stderr @@ -1,4 +1,4 @@ ### BTest baseline data generated by btest-diff. Do not edit. Use "btest -U/-u" to update. Requires BTest >= 0.63. -error in <...>/version.zeek, line 63: Version string 1 cannot be parsed -error in <...>/version.zeek, line 63: Version string 1.12-beta-drunk-too-much cannot be parsed -error in <...>/version.zeek, line 63: Version string JustARandomString cannot be parsed +error in <...>/version.zeek, line 60: Version string 1 cannot be parsed +error in <...>/version.zeek, line 60: Version string 1.12-beta-drunk-too-much cannot be parsed +error in <...>/version.zeek, line 60: Version string JustARandomString cannot be parsed diff --git a/testing/btest/bifs/sleep.zeek b/testing/btest/bifs/sleep.zeek new file mode 100644 index 0000000000..7fbd3e6b46 --- /dev/null +++ b/testing/btest/bifs/sleep.zeek @@ -0,0 +1,21 @@ +# Verifies sleep()'s reported latencies. +# +# @TEST-EXEC: zeek -b %INPUT 2>out +# @TEST-EXEC: btest-diff out + +function test_sleep(i: interval) + { + local start = current_time(); + local sleep_delay = sleep(i); + local script_delay = current_time() - start; + + assert script_delay >= i, fmt("sleep() took %s, less than %s", script_delay, i); + assert sleep_delay >= i, fmt("slept for %s, less than %s", script_delay, i); + assert sleep_delay <= script_delay, fmt("sleep() claims %s, longer than %s", sleep_delay, script_delay); + } + +event zeek_init() + { + test_sleep(100msec); + test_sleep(1sec); + } diff --git a/testing/btest/opt/ZAM-bif-tracking.zeek b/testing/btest/opt/ZAM-bif-tracking.zeek index 627c21f444..e059f1f839 100644 --- a/testing/btest/opt/ZAM-bif-tracking.zeek +++ b/testing/btest/opt/ZAM-bif-tracking.zeek @@ -464,6 +464,7 @@ global known_BiFs = set( "skip_further_processing", "skip_http_entity_data", "skip_smtp_data", + "sleep", "sort", "split_string", "split_string1", diff --git a/testing/external/commit-hash.zeek-testing-cluster b/testing/external/commit-hash.zeek-testing-cluster index da9c310b2e..b129fa1026 100644 --- a/testing/external/commit-hash.zeek-testing-cluster +++ b/testing/external/commit-hash.zeek-testing-cluster @@ -1 +1 @@ -d2987b0bc07cb70bd2f8f707b372fb852147b71f +aa361fc9f5fba202a9df68717a1d403be5f1e6b9