diff --git a/CHANGES b/CHANGES index 38e25cd07d..266582dd0b 100644 --- a/CHANGES +++ b/CHANGES @@ -1,4 +1,137 @@ +2.4-569 | 2016-05-18 07:39:35 -0700 + + * DTLS: Use magix constant from RFC 5389 for STUN detection. + (Johanna Amann) + + * DTLS: Fix binpac bug with DTLSv1.2 client hellos. (Johanna Amann) + + * DTLS: Fix interaction with STUN. Now the DTLS analyzer cleanly + skips all STUN messages. (Johanna Amann) + + * Fix the way that child analyzers are added. (Johanna Amann) + +2.4-563 | 2016-05-17 16:25:21 -0700 + + * Fix duplication of new_connection_contents event. Addresses + BIT-1602 (Johanna Amann) + + * SMTP: Support SSL upgrade via X-ANONYMOUSTLS This seems to be a + non-standardized microsoft extension that, besides having a + different name, works pretty much the same as StartTLS. We just + treat it as such. (Johanna Amann) + + * Fixing control framework's net_stats and peer_status commands. For + the latter, this removes most of the values returned, as we don't + have access to them anymore. (Robin Sommer) + +2.4-555 | 2016-05-16 20:10:15 -0700 + + * Fix failing plugin tests on OS X 10.11. (Daniel Thayer) + + * Fix failing test on Debian/FreeBSD. (Johanna Amann) + +2.4-552 | 2016-05-12 08:04:33 -0700 + + * Fix a bug in receiving remote logs via broker. (Daniel Thayer) + + * Fix Bro and unit tests when broker is not enabled. (Daniel Thayer) + + * Added interpreter error for local event variables. (Jan Grashoefer) + +2.4-544 | 2016-05-07 12:19:07 -0700 + + * Switching all use of gmtime and localtime to use reentrant + variants. (Seth Hall) + +2.4-541 | 2016-05-06 17:58:45 -0700 + + * A set of new built-in function for gathering execution statistics: + + get_net_stats(), get_conn_stats(), get_proc_stats(), + get_event_stats(), get_reassembler_stats(), get_dns_stats(), + get_timer_stats(), get_file_analysis_stats(), get_thread_stats(), + get_gap_stats(), get_matcher_stats(). + + net_stats() resource_usage() have been superseded by these. (Seth + Hall) + + * New policy script misc/stats.bro that records Bro execution + statistics in a standard Bro log file. (Seth Hall) + + * A series of documentation improvements. (Daniel Thayer) + + * Rudimentary XMPP StartTLS analyzer. It parses certificates out of + XMPP connections using StartTLS. It aborts processing if StartTLS + is not found. (Johanna Amann) + +2.4-507 | 2016-05-03 11:18:16 -0700 + + * Fix incorrect type tags in Bro broker source code. These are just + used for error reporting. (Daniel Thayer) + + * Update docs and tests of the fmt() function. (Daniel Thayer) + +2.4-500 | 2016-05-03 11:16:50 -0700 + + * Updating submodule(s). + +2.4-498 | 2016-04-28 11:34:52 -0700 + + * Rename Broker::print to Broker::send_print and Broker::event to + Broker::send_event to avoid using reserved keywords as function + names. (Daniel Thayer) + + * Add script wrapper functions for Broker BIFs. This faciliates + documenting them through Broxygen. (Daniel Thayer) + + * Extend, update, and clean up Broker tests. (Daniel Thayer) + + * Intel: Allow to provide uid/fuid instead of conn/file. (Johanna + Amann) + + * Provide file IDs for hostname matches in certificates. (Johanna + Amann) + + * Rudimentary IMAP StartTLS analyzer. It parses certificates out of + IMAP connections using StartTLS. It aborts processing if StartTLS + is not found. (Johanna Amann) + +2.4-478 | 2016-04-28 09:56:24 + + * Fix parsing of x509 pre-y2k dates. (Johanna Amann) + + * Fix small error in bif documentation. (Johanna Amann) + + * Fix unknown data link type error message. (Vitaly Repin) + + * Correcting spelling errors. (Jeannette Dopheide) + + * Minor cleanup in ARP analyzer. (Johanna Amann) + + * Fix parsing of pre-y2k dates in X509 certificates. (Johanna Amann) + + * Fix small error in get_current_packet documentation. (Johanna Amann) + +2.4-471 | 2016-04-25 15:37:15 -0700 + + * Add DNS tests for huge TLLs and CAA. (Johanna Amann) + + * Add DNS "CAA" RR type and event. (Mark Taylor) + + * Fix DNS response parsing: TTLs are unsigned. (Mark Taylor) + +2.4-466 | 2016-04-22 16:25:33 -0700 + + * Rename BrokerStore and BrokerComm to Broker. Also split broker main.bro + into two scripts. (Daniel Thayer) + + * Add get_current_packet_header bif. (Jan Grashoefer) + +2.4-457 | 2016-04-22 08:36:27 -0700 + + * Fix Intel framework not checking the CERT_HASH indicator type. (Johanna Amann) + 2.4-454 | 2016-04-14 10:06:58 -0400 * Additional mime types for file identification and a few fixes. (Seth Hall) diff --git a/NEWS b/NEWS index 7e66ace7aa..5c0579a626 100644 --- a/NEWS +++ b/NEWS @@ -33,9 +33,20 @@ New Functionality - Bro now supports the Radiotap header for 802.11 frames. +- Bro now has rudimentary IMAP and XMPP analyzers examinig the initial + phases of the protocol. Right now these analyzer only identify + STARTTLS sessions, handing them over to TLS analysis. The analyzer + does not yet analyze any further IMAP/XMPP content. + - Bro now tracks VLAN IDs. To record them inside the connection log, load protocols/conn/vlan-logging.bro. +- The new misc/stats.bro records Bro executions statistics in a + standard Bro log file. + +- A new dns_CAA_reply event gives access to DNS Certification Authority + Authorization replies. + - A new per-packet event raw_packet() provides access to layer 2 information. Use with care, generating events per packet is expensive. @@ -45,6 +56,9 @@ New Functionality argument that will be used for decoding errors into weird.log (instead of reporter.log). +- A new get_current_packet_header bif returns the headers of the current + packet. + - Two new built-in functions for handling set[subnet] and table[subnet]: - check_subnet(subnet, table) checks if a specific subnet is a member @@ -72,6 +86,13 @@ New Functionality - The IRC analyzer now recognizes StartTLS sessions and enable the SSL analyzer for them. +- A set of new built-in function for gathering execution statistics: + + get_net_stats(), get_conn_stats(), get_proc_stats(), + get_event_stats(), get_reassembler_stats(), get_dns_stats(), + get_timer_stats(), get_file_analysis_stats(), get_thread_stats(), + get_gap_stats(), get_matcher_stats(), + - New Bro plugins in aux/plugins: - af_packet: Native AF_PACKET support. @@ -84,9 +105,16 @@ New Functionality Changed Functionality --------------------- +- The BrokerComm and BrokerStore namespaces were renamed to Broker. + The Broker "print" function was renamed to Broker::send_print, and + "event" to "Broker::send_event". + - ``SSH::skip_processing_after_detection`` was removed. The functionality was replaced by ``SSH::disable_analyzer_after_detection``. +- ``net_stats()`` and ``resource_usage()`` have been superseded by the + new execution statistics functions (see above). + - Some script-level identifier have changed their names: snaplen -> Pcap::snaplen diff --git a/VERSION b/VERSION index 532e871b23..de9a360a7a 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -2.4-454 +2.4-569 diff --git a/aux/binpac b/aux/binpac index 424d40c1e8..4179f9f00f 160000 --- a/aux/binpac +++ b/aux/binpac @@ -1 +1 @@ -Subproject commit 424d40c1e8d5888311b50c0e5a9dfc9c5f818b66 +Subproject commit 4179f9f00f4df21e4bcfece0323ec3468f688e8a diff --git a/aux/bro-aux b/aux/bro-aux index 105dfe4ad6..50d33db5d1 160000 --- a/aux/bro-aux +++ b/aux/bro-aux @@ -1 +1 @@ -Subproject commit 105dfe4ad6c4ae4563b21cb0466ee350f0af0d43 +Subproject commit 50d33db5d12b81187ea127a08903b444a3c4bd04 diff --git a/aux/broccoli b/aux/broccoli index f83038b17f..b4d1686cdd 160000 --- a/aux/broccoli +++ b/aux/broccoli @@ -1 +1 @@ -Subproject commit f83038b17fc83788415a58d77f75ad182ca6a9b7 +Subproject commit b4d1686cdd3f5505e405667b1083e8335cae6928 diff --git a/aux/broctl b/aux/broctl index 583f3a3ff1..9cce8be1a9 160000 --- a/aux/broctl +++ b/aux/broctl @@ -1 +1 @@ -Subproject commit 583f3a3ff1847cf96a87f865d5cf0f36fae9dd67 +Subproject commit 9cce8be1a9c02b275f8a51d175e4729bdb0afee4 diff --git a/aux/broker b/aux/broker index 6684ab5109..bb3f55f198 160000 --- a/aux/broker +++ b/aux/broker @@ -1 +1 @@ -Subproject commit 6684ab5109f526fb535013760f17a4c8dff093ae +Subproject commit bb3f55f198f9cfd5e545345dd6425dd08ca1d45e diff --git a/aux/plugins b/aux/plugins index ab61be0c4f..ebab672fa4 160000 --- a/aux/plugins +++ b/aux/plugins @@ -1 +1 @@ -Subproject commit ab61be0c4f128c976f72dfa5a09a87cd842f387a +Subproject commit ebab672fa404b26944a6df6fbfb1aaab95ec5d48 diff --git a/bro-config.h.in b/bro-config.h.in index 755a9eee98..0937950604 100644 --- a/bro-config.h.in +++ b/bro-config.h.in @@ -14,6 +14,9 @@ /* We are on a Linux system */ #cmakedefine HAVE_LINUX +/* We are on a Mac OS X (Darwin) system */ +#cmakedefine HAVE_DARWIN + /* Define if you have the `mallinfo' function. */ #cmakedefine HAVE_MALLINFO diff --git a/cmake b/cmake index 537e45afe1..0a2b36874a 160000 --- a/cmake +++ b/cmake @@ -1 +1 @@ -Subproject commit 537e45afe1006a10f73847fab5f13d28ce43fc4d +Subproject commit 0a2b36874ad5c1a22829135f8aeeac534469053f diff --git a/doc/cluster/index.rst b/doc/cluster/index.rst index 544ca5e0f8..6e426c005e 100644 --- a/doc/cluster/index.rst +++ b/doc/cluster/index.rst @@ -96,13 +96,13 @@ logging is done remotely to the manager, and normally very little is written to disk. The rule of thumb we have followed recently is to allocate approximately 1 -core for every 80Mbps of traffic that is being analyzed. However, this +core for every 250Mbps of traffic that is being analyzed. However, this estimate could be extremely traffic mix-specific. It has generally worked for mixed traffic with many users and servers. For example, if your traffic peaks around 2Gbps (combined) and you want to handle traffic at peak load, -you may want to have 26 cores available (2048 / 80 == 25.6). If the 80Mbps -estimate works for your traffic, this could be handled by 3 physical hosts -dedicated to being workers with each one containing dual 6-core processors. +you may want to have 8 cores available (2048 / 250 == 8.2). If the 250Mbps +estimate works for your traffic, this could be handled by 2 physical hosts +dedicated to being workers with each one containing a quad-core processor. Once a flow-based load balancer is put into place this model is extremely easy to scale. It is recommended that you estimate the amount of diff --git a/doc/components/bro-plugins/kafka/README.rst b/doc/components/bro-plugins/kafka/README.rst new file mode 120000 index 0000000000..6ca2195f17 --- /dev/null +++ b/doc/components/bro-plugins/kafka/README.rst @@ -0,0 +1 @@ +../../../../aux/plugins/kafka/README \ No newline at end of file diff --git a/doc/frameworks/broker.rst b/doc/frameworks/broker.rst index 8c5ed24e25..9c9ed89514 100644 --- a/doc/frameworks/broker.rst +++ b/doc/frameworks/broker.rst @@ -17,20 +17,20 @@ Connecting to Peers =================== Communication via Broker must first be turned on via -:bro:see:`BrokerComm::enable`. +:bro:see:`Broker::enable`. -Bro can accept incoming connections by calling :bro:see:`BrokerComm::listen` +Bro can accept incoming connections by calling :bro:see:`Broker::listen` and then monitor connection status updates via the -:bro:see:`BrokerComm::incoming_connection_established` and -:bro:see:`BrokerComm::incoming_connection_broken` events. +:bro:see:`Broker::incoming_connection_established` and +:bro:see:`Broker::incoming_connection_broken` events. .. btest-include:: ${DOC_ROOT}/frameworks/broker/connecting-listener.bro -Bro can initiate outgoing connections by calling :bro:see:`BrokerComm::connect` +Bro can initiate outgoing connections by calling :bro:see:`Broker::connect` and then monitor connection status updates via the -:bro:see:`BrokerComm::outgoing_connection_established`, -:bro:see:`BrokerComm::outgoing_connection_broken`, and -:bro:see:`BrokerComm::outgoing_connection_incompatible` events. +:bro:see:`Broker::outgoing_connection_established`, +:bro:see:`Broker::outgoing_connection_broken`, and +:bro:see:`Broker::outgoing_connection_incompatible` events. .. btest-include:: ${DOC_ROOT}/frameworks/broker/connecting-connector.bro @@ -38,14 +38,14 @@ Remote Printing =============== To receive remote print messages, first use the -:bro:see:`BrokerComm::subscribe_to_prints` function to advertise to peers a +:bro:see:`Broker::subscribe_to_prints` function to advertise to peers a topic prefix of interest and then create an event handler for -:bro:see:`BrokerComm::print_handler` to handle any print messages that are +:bro:see:`Broker::print_handler` to handle any print messages that are received. .. btest-include:: ${DOC_ROOT}/frameworks/broker/printing-listener.bro -To send remote print messages, just call :bro:see:`BrokerComm::print`. +To send remote print messages, just call :bro:see:`Broker::send_print`. .. btest-include:: ${DOC_ROOT}/frameworks/broker/printing-connector.bro @@ -69,14 +69,14 @@ Remote Events ============= Receiving remote events is similar to remote prints. Just use the -:bro:see:`BrokerComm::subscribe_to_events` function and possibly define any +:bro:see:`Broker::subscribe_to_events` function and possibly define any new events along with handlers that peers may want to send. .. btest-include:: ${DOC_ROOT}/frameworks/broker/events-listener.bro There are two different ways to send events. The first is to call the -:bro:see:`BrokerComm::event` function directly. The second option is to call -the :bro:see:`BrokerComm::auto_event` function where you specify a +:bro:see:`Broker::send_event` function directly. The second option is to call +the :bro:see:`Broker::auto_event` function where you specify a particular event that will be automatically sent to peers whenever the event is called locally via the normal event invocation syntax. @@ -104,14 +104,14 @@ Remote Logging .. btest-include:: ${DOC_ROOT}/frameworks/broker/testlog.bro -Use the :bro:see:`BrokerComm::subscribe_to_logs` function to advertise interest +Use the :bro:see:`Broker::subscribe_to_logs` function to advertise interest in logs written by peers. The topic names that Bro uses are implicitly of the form "bro/log/". .. btest-include:: ${DOC_ROOT}/frameworks/broker/logs-listener.bro To send remote logs either redef :bro:see:`Log::enable_remote_logging` or -use the :bro:see:`BrokerComm::enable_remote_logs` function. The former +use the :bro:see:`Broker::enable_remote_logs` function. The former allows any log stream to be sent to peers while the latter enables remote logging for particular streams. @@ -137,24 +137,24 @@ Tuning Access Control By default, endpoints do not restrict the message topics that it sends to peers and do not restrict what message topics and data store identifiers get advertised to peers. These are the default -:bro:see:`BrokerComm::EndpointFlags` supplied to :bro:see:`BrokerComm::enable`. +:bro:see:`Broker::EndpointFlags` supplied to :bro:see:`Broker::enable`. If not using the ``auto_publish`` flag, one can use the -:bro:see:`BrokerComm::publish_topic` and :bro:see:`BrokerComm::unpublish_topic` +:bro:see:`Broker::publish_topic` and :bro:see:`Broker::unpublish_topic` functions to manipulate the set of message topics (must match exactly) that are allowed to be sent to peer endpoints. These settings take precedence over the per-message ``peers`` flag supplied to functions -that take a :bro:see:`BrokerComm::SendFlags` such as :bro:see:`BrokerComm::print`, -:bro:see:`BrokerComm::event`, :bro:see:`BrokerComm::auto_event` or -:bro:see:`BrokerComm::enable_remote_logs`. +that take a :bro:see:`Broker::SendFlags` such as :bro:see:`Broker::send_print`, +:bro:see:`Broker::send_event`, :bro:see:`Broker::auto_event` or +:bro:see:`Broker::enable_remote_logs`. If not using the ``auto_advertise`` flag, one can use the -:bro:see:`BrokerComm::advertise_topic` and -:bro:see:`BrokerComm::unadvertise_topic` functions +:bro:see:`Broker::advertise_topic` and +:bro:see:`Broker::unadvertise_topic` functions to manipulate the set of topic prefixes that are allowed to be advertised to peers. If an endpoint does not advertise a topic prefix, then the only way peers can send messages to it is via the ``unsolicited`` -flag of :bro:see:`BrokerComm::SendFlags` and choosing a topic with a matching +flag of :bro:see:`Broker::SendFlags` and choosing a topic with a matching prefix (i.e. full topic may be longer than receivers prefix, just the prefix needs to match). @@ -192,8 +192,8 @@ last modification time. .. btest-include:: ${DOC_ROOT}/frameworks/broker/stores-connector.bro In the above example, if a local copy of the store contents isn't -needed, just replace the :bro:see:`BrokerStore::create_clone` call with -:bro:see:`BrokerStore::create_frontend`. Queries will then be made against +needed, just replace the :bro:see:`Broker::create_clone` call with +:bro:see:`Broker::create_frontend`. Queries will then be made against the remote master store instead of the local clone. Note that all data store queries must be made within Bro's asynchronous diff --git a/doc/frameworks/broker/connecting-connector.bro b/doc/frameworks/broker/connecting-connector.bro index cd5c74add8..adf901ea6a 100644 --- a/doc/frameworks/broker/connecting-connector.bro +++ b/doc/frameworks/broker/connecting-connector.bro @@ -1,18 +1,18 @@ const broker_port: port = 9999/tcp &redef; redef exit_only_after_terminate = T; -redef BrokerComm::endpoint_name = "connector"; +redef Broker::endpoint_name = "connector"; event bro_init() { - BrokerComm::enable(); - BrokerComm::connect("127.0.0.1", broker_port, 1sec); + Broker::enable(); + Broker::connect("127.0.0.1", broker_port, 1sec); } -event BrokerComm::outgoing_connection_established(peer_address: string, +event Broker::outgoing_connection_established(peer_address: string, peer_port: port, peer_name: string) { - print "BrokerComm::outgoing_connection_established", + print "Broker::outgoing_connection_established", peer_address, peer_port, peer_name; terminate(); } diff --git a/doc/frameworks/broker/connecting-listener.bro b/doc/frameworks/broker/connecting-listener.bro index 21c67f9696..aa2b945dbe 100644 --- a/doc/frameworks/broker/connecting-listener.bro +++ b/doc/frameworks/broker/connecting-listener.bro @@ -1,20 +1,20 @@ const broker_port: port = 9999/tcp &redef; redef exit_only_after_terminate = T; -redef BrokerComm::endpoint_name = "listener"; +redef Broker::endpoint_name = "listener"; event bro_init() { - BrokerComm::enable(); - BrokerComm::listen(broker_port, "127.0.0.1"); + Broker::enable(); + Broker::listen(broker_port, "127.0.0.1"); } -event BrokerComm::incoming_connection_established(peer_name: string) +event Broker::incoming_connection_established(peer_name: string) { - print "BrokerComm::incoming_connection_established", peer_name; + print "Broker::incoming_connection_established", peer_name; } -event BrokerComm::incoming_connection_broken(peer_name: string) +event Broker::incoming_connection_broken(peer_name: string) { - print "BrokerComm::incoming_connection_broken", peer_name; + print "Broker::incoming_connection_broken", peer_name; terminate(); } diff --git a/doc/frameworks/broker/events-connector.bro b/doc/frameworks/broker/events-connector.bro index 1ad458c245..437e197925 100644 --- a/doc/frameworks/broker/events-connector.bro +++ b/doc/frameworks/broker/events-connector.bro @@ -1,30 +1,30 @@ const broker_port: port = 9999/tcp &redef; redef exit_only_after_terminate = T; -redef BrokerComm::endpoint_name = "connector"; +redef Broker::endpoint_name = "connector"; global my_event: event(msg: string, c: count); global my_auto_event: event(msg: string, c: count); event bro_init() { - BrokerComm::enable(); - BrokerComm::connect("127.0.0.1", broker_port, 1sec); - BrokerComm::auto_event("bro/event/my_auto_event", my_auto_event); + Broker::enable(); + Broker::connect("127.0.0.1", broker_port, 1sec); + Broker::auto_event("bro/event/my_auto_event", my_auto_event); } -event BrokerComm::outgoing_connection_established(peer_address: string, +event Broker::outgoing_connection_established(peer_address: string, peer_port: port, peer_name: string) { - print "BrokerComm::outgoing_connection_established", + print "Broker::outgoing_connection_established", peer_address, peer_port, peer_name; - BrokerComm::event("bro/event/my_event", BrokerComm::event_args(my_event, "hi", 0)); + Broker::send_event("bro/event/my_event", Broker::event_args(my_event, "hi", 0)); event my_auto_event("stuff", 88); - BrokerComm::event("bro/event/my_event", BrokerComm::event_args(my_event, "...", 1)); + Broker::send_event("bro/event/my_event", Broker::event_args(my_event, "...", 1)); event my_auto_event("more stuff", 51); - BrokerComm::event("bro/event/my_event", BrokerComm::event_args(my_event, "bye", 2)); + Broker::send_event("bro/event/my_event", Broker::event_args(my_event, "bye", 2)); } -event BrokerComm::outgoing_connection_broken(peer_address: string, +event Broker::outgoing_connection_broken(peer_address: string, peer_port: port) { terminate(); diff --git a/doc/frameworks/broker/events-listener.bro b/doc/frameworks/broker/events-listener.bro index dc18795903..b803e646ec 100644 --- a/doc/frameworks/broker/events-listener.bro +++ b/doc/frameworks/broker/events-listener.bro @@ -1,20 +1,20 @@ const broker_port: port = 9999/tcp &redef; redef exit_only_after_terminate = T; -redef BrokerComm::endpoint_name = "listener"; +redef Broker::endpoint_name = "listener"; global msg_count = 0; global my_event: event(msg: string, c: count); global my_auto_event: event(msg: string, c: count); event bro_init() { - BrokerComm::enable(); - BrokerComm::subscribe_to_events("bro/event/"); - BrokerComm::listen(broker_port, "127.0.0.1"); + Broker::enable(); + Broker::subscribe_to_events("bro/event/"); + Broker::listen(broker_port, "127.0.0.1"); } -event BrokerComm::incoming_connection_established(peer_name: string) +event Broker::incoming_connection_established(peer_name: string) { - print "BrokerComm::incoming_connection_established", peer_name; + print "Broker::incoming_connection_established", peer_name; } event my_event(msg: string, c: count) diff --git a/doc/frameworks/broker/logs-connector.bro b/doc/frameworks/broker/logs-connector.bro index 6089419cab..9c5df335b9 100644 --- a/doc/frameworks/broker/logs-connector.bro +++ b/doc/frameworks/broker/logs-connector.bro @@ -2,16 +2,16 @@ const broker_port: port = 9999/tcp &redef; redef exit_only_after_terminate = T; -redef BrokerComm::endpoint_name = "connector"; +redef Broker::endpoint_name = "connector"; redef Log::enable_local_logging = F; redef Log::enable_remote_logging = F; global n = 0; event bro_init() { - BrokerComm::enable(); - BrokerComm::enable_remote_logs(Test::LOG); - BrokerComm::connect("127.0.0.1", broker_port, 1sec); + Broker::enable(); + Broker::enable_remote_logs(Test::LOG); + Broker::connect("127.0.0.1", broker_port, 1sec); } event do_write() @@ -24,16 +24,16 @@ event do_write() event do_write(); } -event BrokerComm::outgoing_connection_established(peer_address: string, +event Broker::outgoing_connection_established(peer_address: string, peer_port: port, peer_name: string) { - print "BrokerComm::outgoing_connection_established", + print "Broker::outgoing_connection_established", peer_address, peer_port, peer_name; event do_write(); } -event BrokerComm::outgoing_connection_broken(peer_address: string, +event Broker::outgoing_connection_broken(peer_address: string, peer_port: port) { terminate(); diff --git a/doc/frameworks/broker/logs-listener.bro b/doc/frameworks/broker/logs-listener.bro index 5c807f08b7..34d475512a 100644 --- a/doc/frameworks/broker/logs-listener.bro +++ b/doc/frameworks/broker/logs-listener.bro @@ -2,18 +2,18 @@ const broker_port: port = 9999/tcp &redef; redef exit_only_after_terminate = T; -redef BrokerComm::endpoint_name = "listener"; +redef Broker::endpoint_name = "listener"; event bro_init() { - BrokerComm::enable(); - BrokerComm::subscribe_to_logs("bro/log/Test::LOG"); - BrokerComm::listen(broker_port, "127.0.0.1"); + Broker::enable(); + Broker::subscribe_to_logs("bro/log/Test::LOG"); + Broker::listen(broker_port, "127.0.0.1"); } -event BrokerComm::incoming_connection_established(peer_name: string) +event Broker::incoming_connection_established(peer_name: string) { - print "BrokerComm::incoming_connection_established", peer_name; + print "Broker::incoming_connection_established", peer_name; } event Test::log_test(rec: Test::Info) diff --git a/doc/frameworks/broker/printing-connector.bro b/doc/frameworks/broker/printing-connector.bro index 2a504ffba0..42d961669a 100644 --- a/doc/frameworks/broker/printing-connector.bro +++ b/doc/frameworks/broker/printing-connector.bro @@ -1,25 +1,25 @@ const broker_port: port = 9999/tcp &redef; redef exit_only_after_terminate = T; -redef BrokerComm::endpoint_name = "connector"; +redef Broker::endpoint_name = "connector"; event bro_init() { - BrokerComm::enable(); - BrokerComm::connect("127.0.0.1", broker_port, 1sec); + Broker::enable(); + Broker::connect("127.0.0.1", broker_port, 1sec); } -event BrokerComm::outgoing_connection_established(peer_address: string, +event Broker::outgoing_connection_established(peer_address: string, peer_port: port, peer_name: string) { - print "BrokerComm::outgoing_connection_established", + print "Broker::outgoing_connection_established", peer_address, peer_port, peer_name; - BrokerComm::print("bro/print/hi", "hello"); - BrokerComm::print("bro/print/stuff", "..."); - BrokerComm::print("bro/print/bye", "goodbye"); + Broker::send_print("bro/print/hi", "hello"); + Broker::send_print("bro/print/stuff", "..."); + Broker::send_print("bro/print/bye", "goodbye"); } -event BrokerComm::outgoing_connection_broken(peer_address: string, +event Broker::outgoing_connection_broken(peer_address: string, peer_port: port) { terminate(); diff --git a/doc/frameworks/broker/printing-listener.bro b/doc/frameworks/broker/printing-listener.bro index f55c5b9bad..4630a7e6d7 100644 --- a/doc/frameworks/broker/printing-listener.bro +++ b/doc/frameworks/broker/printing-listener.bro @@ -1,21 +1,21 @@ const broker_port: port = 9999/tcp &redef; redef exit_only_after_terminate = T; -redef BrokerComm::endpoint_name = "listener"; +redef Broker::endpoint_name = "listener"; global msg_count = 0; event bro_init() { - BrokerComm::enable(); - BrokerComm::subscribe_to_prints("bro/print/"); - BrokerComm::listen(broker_port, "127.0.0.1"); + Broker::enable(); + Broker::subscribe_to_prints("bro/print/"); + Broker::listen(broker_port, "127.0.0.1"); } -event BrokerComm::incoming_connection_established(peer_name: string) +event Broker::incoming_connection_established(peer_name: string) { - print "BrokerComm::incoming_connection_established", peer_name; + print "Broker::incoming_connection_established", peer_name; } -event BrokerComm::print_handler(msg: string) +event Broker::print_handler(msg: string) { ++msg_count; print "got print message", msg; diff --git a/doc/frameworks/broker/stores-connector.bro b/doc/frameworks/broker/stores-connector.bro index 5db8657a68..d50807cc89 100644 --- a/doc/frameworks/broker/stores-connector.bro +++ b/doc/frameworks/broker/stores-connector.bro @@ -1,42 +1,42 @@ const broker_port: port = 9999/tcp &redef; redef exit_only_after_terminate = T; -global h: opaque of BrokerStore::Handle; +global h: opaque of Broker::Handle; -function dv(d: BrokerComm::Data): BrokerComm::DataVector +function dv(d: Broker::Data): Broker::DataVector { - local rval: BrokerComm::DataVector; + local rval: Broker::DataVector; rval[0] = d; return rval; } global ready: event(); -event BrokerComm::outgoing_connection_broken(peer_address: string, +event Broker::outgoing_connection_broken(peer_address: string, peer_port: port) { terminate(); } -event BrokerComm::outgoing_connection_established(peer_address: string, +event Broker::outgoing_connection_established(peer_address: string, peer_port: port, peer_name: string) { local myset: set[string] = {"a", "b", "c"}; local myvec: vector of string = {"alpha", "beta", "gamma"}; - h = BrokerStore::create_master("mystore"); - BrokerStore::insert(h, BrokerComm::data("one"), BrokerComm::data(110)); - BrokerStore::insert(h, BrokerComm::data("two"), BrokerComm::data(223)); - BrokerStore::insert(h, BrokerComm::data("myset"), BrokerComm::data(myset)); - BrokerStore::insert(h, BrokerComm::data("myvec"), BrokerComm::data(myvec)); - BrokerStore::increment(h, BrokerComm::data("one")); - BrokerStore::decrement(h, BrokerComm::data("two")); - BrokerStore::add_to_set(h, BrokerComm::data("myset"), BrokerComm::data("d")); - BrokerStore::remove_from_set(h, BrokerComm::data("myset"), BrokerComm::data("b")); - BrokerStore::push_left(h, BrokerComm::data("myvec"), dv(BrokerComm::data("delta"))); - BrokerStore::push_right(h, BrokerComm::data("myvec"), dv(BrokerComm::data("omega"))); + h = Broker::create_master("mystore"); + Broker::insert(h, Broker::data("one"), Broker::data(110)); + Broker::insert(h, Broker::data("two"), Broker::data(223)); + Broker::insert(h, Broker::data("myset"), Broker::data(myset)); + Broker::insert(h, Broker::data("myvec"), Broker::data(myvec)); + Broker::increment(h, Broker::data("one")); + Broker::decrement(h, Broker::data("two")); + Broker::add_to_set(h, Broker::data("myset"), Broker::data("d")); + Broker::remove_from_set(h, Broker::data("myset"), Broker::data("b")); + Broker::push_left(h, Broker::data("myvec"), dv(Broker::data("delta"))); + Broker::push_right(h, Broker::data("myvec"), dv(Broker::data("omega"))); - when ( local res = BrokerStore::size(h) ) + when ( local res = Broker::size(h) ) { print "master size", res; event ready(); @@ -47,7 +47,7 @@ event BrokerComm::outgoing_connection_established(peer_address: string, event bro_init() { - BrokerComm::enable(); - BrokerComm::connect("127.0.0.1", broker_port, 1secs); - BrokerComm::auto_event("bro/event/ready", ready); + Broker::enable(); + Broker::connect("127.0.0.1", broker_port, 1secs); + Broker::auto_event("bro/event/ready", ready); } diff --git a/doc/frameworks/broker/stores-listener.bro b/doc/frameworks/broker/stores-listener.bro index 454e41a8c2..3dac30deca 100644 --- a/doc/frameworks/broker/stores-listener.bro +++ b/doc/frameworks/broker/stores-listener.bro @@ -1,13 +1,13 @@ const broker_port: port = 9999/tcp &redef; redef exit_only_after_terminate = T; -global h: opaque of BrokerStore::Handle; +global h: opaque of Broker::Handle; global expected_key_count = 4; global key_count = 0; function do_lookup(key: string) { - when ( local res = BrokerStore::lookup(h, BrokerComm::data(key)) ) + when ( local res = Broker::lookup(h, Broker::data(key)) ) { ++key_count; print "lookup", key, res; @@ -21,15 +21,15 @@ function do_lookup(key: string) event ready() { - h = BrokerStore::create_clone("mystore"); + h = Broker::create_clone("mystore"); - when ( local res = BrokerStore::keys(h) ) + when ( local res = Broker::keys(h) ) { print "clone keys", res; - do_lookup(BrokerComm::refine_to_string(BrokerComm::vector_lookup(res$result, 0))); - do_lookup(BrokerComm::refine_to_string(BrokerComm::vector_lookup(res$result, 1))); - do_lookup(BrokerComm::refine_to_string(BrokerComm::vector_lookup(res$result, 2))); - do_lookup(BrokerComm::refine_to_string(BrokerComm::vector_lookup(res$result, 3))); + do_lookup(Broker::refine_to_string(Broker::vector_lookup(res$result, 0))); + do_lookup(Broker::refine_to_string(Broker::vector_lookup(res$result, 1))); + do_lookup(Broker::refine_to_string(Broker::vector_lookup(res$result, 2))); + do_lookup(Broker::refine_to_string(Broker::vector_lookup(res$result, 3))); } timeout 10sec { print "timeout"; } @@ -37,7 +37,7 @@ event ready() event bro_init() { - BrokerComm::enable(); - BrokerComm::subscribe_to_events("bro/event/ready"); - BrokerComm::listen(broker_port, "127.0.0.1"); + Broker::enable(); + Broker::subscribe_to_events("bro/event/ready"); + Broker::listen(broker_port, "127.0.0.1"); } diff --git a/doc/frameworks/broker/testlog.bro b/doc/frameworks/broker/testlog.bro index 506d359bb7..0099671e6d 100644 --- a/doc/frameworks/broker/testlog.bro +++ b/doc/frameworks/broker/testlog.bro @@ -13,6 +13,6 @@ export { event bro_init() &priority=5 { - BrokerComm::enable(); + Broker::enable(); Log::create_stream(Test::LOG, [$columns=Test::Info, $ev=log_test, $path="test"]); } diff --git a/doc/script-reference/log-files.rst b/doc/script-reference/log-files.rst index c3fbca95a0..3c1720afd1 100644 --- a/doc/script-reference/log-files.rst +++ b/doc/script-reference/log-files.rst @@ -39,6 +39,8 @@ Network Protocols +----------------------------+---------------------------------------+---------------------------------+ | rdp.log | RDP | :bro:type:`RDP::Info` | +----------------------------+---------------------------------------+---------------------------------+ +| rfb.log | Remote Framebuffer (RFB) | :bro:type:`RFB::Info` | ++----------------------------+---------------------------------------+---------------------------------+ | sip.log | SIP | :bro:type:`SIP::Info` | +----------------------------+---------------------------------------+---------------------------------+ | smtp.log | SMTP transactions | :bro:type:`SMTP::Info` | diff --git a/doc/script-reference/statements.rst b/doc/script-reference/statements.rst index e2f93a5627..14e0cc3c32 100644 --- a/doc/script-reference/statements.rst +++ b/doc/script-reference/statements.rst @@ -277,16 +277,25 @@ Here are the statements that the Bro scripting language supports. .. bro:keyword:: delete The "delete" statement is used to remove an element from a - :bro:type:`set` or :bro:type:`table`. Nothing happens if the - specified element does not exist in the set or table. + :bro:type:`set` or :bro:type:`table`, or to remove a value from + a :bro:type:`record` field that has the :bro:attr:`&optional` attribute. + When attempting to remove an element from a set or table, + nothing happens if the specified index does not exist. + When attempting to remove a value from an "&optional" record field, + nothing happens if that field doesn't have a value. Example:: local myset = set("this", "test"); local mytable = table(["key1"] = 80/tcp, ["key2"] = 53/udp); + local myrec = MyRecordType($a = 1, $b = 2); + delete myset["test"]; delete mytable["key1"]; + # In this example, "b" must have the "&optional" attribute + delete myrec$b; + .. bro:keyword:: event The "event" statement immediately queues invocation of an event handler. @@ -306,30 +315,33 @@ Here are the statements that the Bro scripting language supports. .. bro:keyword:: for A "for" loop iterates over each element in a string, set, vector, or - table and executes a statement for each iteration. Currently, - modifying a container's membership while iterating over it may - result in undefined behavior, so avoid adding or removing elements - inside the loop. + table and executes a statement for each iteration (note that the order + in which the loop iterates over the elements in a set or a table is + nondeterministic). However, no loop iterations occur if the string, + set, vector, or table is empty. For each iteration of the loop, a loop variable will be assigned to an element if the expression evaluates to a string or set, or an index if the expression evaluates to a vector or table. Then the statement - is executed. However, the statement will not be executed if the expression - evaluates to an object with no elements. + is executed. If the expression is a table or a set with more than one index, then the loop variable must be specified as a comma-separated list of different loop variables (one for each index), enclosed in brackets. - A :bro:keyword:`break` statement can be used at any time to immediately - terminate the "for" loop, and a :bro:keyword:`next` statement can be - used to skip to the next loop iteration. - Note that the loop variable in a "for" statement is not allowed to be a global variable, and it does not need to be declared prior to the "for" statement. The type will be inferred from the elements of the expression. + Currently, modifying a container's membership while iterating over it may + result in undefined behavior, so do not add or remove elements + inside the loop. + + A :bro:keyword:`break` statement will immediately terminate the "for" + loop, and a :bro:keyword:`next` statement will skip to the next loop + iteration. + Example:: local myset = set(80/tcp, 81/tcp); @@ -532,8 +544,6 @@ Here are the statements that the Bro scripting language supports. end with either a :bro:keyword:`break`, :bro:keyword:`fallthrough`, or :bro:keyword:`return` statement (although "return" is allowed only if the "switch" statement is inside a function, hook, or event handler). - If a "case" (or "default") block contain more than one statement, then - there is no need to wrap them in braces. Note that the braces in a "switch" statement are always required (these do not indicate the presence of a `compound statement`_), and that no @@ -604,12 +614,9 @@ Here are the statements that the Bro scripting language supports. if ( skip_ahead() ) next; - [...] - if ( finish_up ) break; - [...] } .. _compound statement: diff --git a/doc/scripting/data_type_record.bro b/doc/scripting/data_type_record.bro new file mode 100644 index 0000000000..2380137cac --- /dev/null +++ b/doc/scripting/data_type_record.bro @@ -0,0 +1,25 @@ +module Conn; + +export { + ## The record type which contains column fields of the connection log. + type Info: record { + ts: time &log; + uid: string &log; + id: conn_id &log; + proto: transport_proto &log; + service: string &log &optional; + duration: interval &log &optional; + orig_bytes: count &log &optional; + resp_bytes: count &log &optional; + conn_state: string &log &optional; + local_orig: bool &log &optional; + local_resp: bool &log &optional; + missed_bytes: count &log &default=0; + history: string &log &optional; + orig_pkts: count &log &optional; + orig_ip_bytes: count &log &optional; + resp_pkts: count &log &optional; + resp_ip_bytes: count &log &optional; + tunnel_parents: set[string] &log; + }; +} diff --git a/doc/scripting/http_main.bro b/doc/scripting/http_main.bro new file mode 100644 index 0000000000..5182accb35 --- /dev/null +++ b/doc/scripting/http_main.bro @@ -0,0 +1,7 @@ +module HTTP; + +export { + ## This setting changes if passwords used in Basic-Auth are captured or + ## not. + const default_capture_password = F &redef; +} diff --git a/doc/scripting/index.rst b/doc/scripting/index.rst index a776fc0ad3..597d8ec41a 100644 --- a/doc/scripting/index.rst +++ b/doc/scripting/index.rst @@ -362,8 +362,7 @@ decrypted from HTTP streams is stored in :bro:see:`HTTP::default_capture_password` as shown in the stripped down excerpt from :doc:`/scripts/base/protocols/http/main.bro` below. -.. btest-include:: ${BRO_SRC_ROOT}/scripts/base/protocols/http/main.bro - :lines: 9-11,20-22,125 +.. btest-include:: ${DOC_ROOT}/scripting/http_main.bro Because the constant was declared with the ``&redef`` attribute, if we needed to turn this option on globally, we could do so by adding the @@ -825,8 +824,7 @@ example of the ``record`` data type in the earlier sections, the :bro:type:`Conn::Info`, which corresponds to the fields logged into ``conn.log``, is shown by the excerpt below. -.. btest-include:: ${BRO_SRC_ROOT}/scripts/base/protocols/conn/main.bro - :lines: 10-12,16-17,19,21,23,25,28,31,35,38,57,63,69,75,98,101,105,108,112,116-117,122 +.. btest-include:: ${DOC_ROOT}/scripting/data_type_record.bro Looking at the structure of the definition, a new collection of data types is being defined as a type called ``Info``. Since this type diff --git a/scripts/base/files/x509/main.bro b/scripts/base/files/x509/main.bro index c097b84560..bbf99f6a4d 100644 --- a/scripts/base/files/x509/main.bro +++ b/scripts/base/files/x509/main.bro @@ -6,6 +6,7 @@ module X509; export { redef enum Log::ID += { LOG }; + ## The record type which contains the fields of the X.509 log. type Info: record { ## Current timestamp. ts: time &log; diff --git a/scripts/base/frameworks/broker/__load__.bro b/scripts/base/frameworks/broker/__load__.bro index a10fe855df..018d772f4f 100644 --- a/scripts/base/frameworks/broker/__load__.bro +++ b/scripts/base/frameworks/broker/__load__.bro @@ -1 +1,2 @@ @load ./main +@load ./store diff --git a/scripts/base/frameworks/broker/main.bro b/scripts/base/frameworks/broker/main.bro index e8b57d57d9..0818855d8f 100644 --- a/scripts/base/frameworks/broker/main.bro +++ b/scripts/base/frameworks/broker/main.bro @@ -1,11 +1,20 @@ ##! Various data structure definitions for use with Bro's communication system. -module BrokerComm; +module Log; + +export { + type Log::ID: enum { + ## Dummy place-holder. + UNKNOWN + }; +} + +module Broker; export { ## A name used to identify this endpoint to peers. - ## .. bro:see:: BrokerComm::connect BrokerComm::listen + ## .. bro:see:: Broker::connect Broker::listen const endpoint_name = "" &redef; ## Change communication behavior. @@ -32,11 +41,11 @@ export { ## Opaque communication data. type Data: record { - d: opaque of BrokerComm::Data &optional; + d: opaque of Broker::Data &optional; }; ## Opaque communication data. - type DataVector: vector of BrokerComm::Data; + type DataVector: vector of Broker::Data; ## Opaque event communication data. type EventArgs: record { @@ -49,55 +58,318 @@ export { ## Opaque communication data used as a convenient way to wrap key-value ## pairs that comprise table entries. type TableItem : record { - key: BrokerComm::Data; - val: BrokerComm::Data; + key: Broker::Data; + val: Broker::Data; }; + + ## Enable use of communication. + ## + ## flags: used to tune the local Broker endpoint behavior. + ## + ## Returns: true if communication is successfully initialized. + global enable: function(flags: EndpointFlags &default = EndpointFlags()): bool; + + ## Changes endpoint flags originally supplied to :bro:see:`Broker::enable`. + ## + ## flags: the new endpoint behavior flags to use. + ## + ## Returns: true if flags were changed. + global set_endpoint_flags: function(flags: EndpointFlags &default = EndpointFlags()): bool; + + ## Allow sending messages to peers if associated with the given topic. + ## This has no effect if auto publication behavior is enabled via the flags + ## supplied to :bro:see:`Broker::enable` or :bro:see:`Broker::set_endpoint_flags`. + ## + ## topic: a topic to allow messages to be published under. + ## + ## Returns: true if successful. + global publish_topic: function(topic: string): bool; + + ## Disallow sending messages to peers if associated with the given topic. + ## This has no effect if auto publication behavior is enabled via the flags + ## supplied to :bro:see:`Broker::enable` or :bro:see:`Broker::set_endpoint_flags`. + ## + ## topic: a topic to disallow messages to be published under. + ## + ## Returns: true if successful. + global unpublish_topic: function(topic: string): bool; + + ## Listen for remote connections. + ## + ## p: the TCP port to listen on. + ## + ## a: an address string on which to accept connections, e.g. + ## "127.0.0.1". An empty string refers to @p INADDR_ANY. + ## + ## reuse: equivalent to behavior of SO_REUSEADDR. + ## + ## Returns: true if the local endpoint is now listening for connections. + ## + ## .. bro:see:: Broker::incoming_connection_established + global listen: function(p: port, a: string &default = "", reuse: bool &default = T): bool; + + ## Initiate a remote connection. + ## + ## a: an address to connect to, e.g. "localhost" or "127.0.0.1". + ## + ## p: the TCP port on which the remote side is listening. + ## + ## retry: an interval at which to retry establishing the + ## connection with the remote peer if it cannot be made initially, or + ## if it ever becomes disconnected. + ## + ## Returns: true if it's possible to try connecting with the peer and + ## it's a new peer. The actual connection may not be established + ## until a later point in time. + ## + ## .. bro:see:: Broker::outgoing_connection_established + global connect: function(a: string, p: port, retry: interval): bool; + + ## Remove a remote connection. + ## + ## a: the address used in previous successful call to :bro:see:`Broker::connect`. + ## + ## p: the port used in previous successful call to :bro:see:`Broker::connect`. + ## + ## Returns: true if the arguments match a previously successful call to + ## :bro:see:`Broker::connect`. + global disconnect: function(a: string, p: port): bool; + + ## Print a simple message to any interested peers. The receiver can use + ## :bro:see:`Broker::print_handler` to handle messages. + ## + ## topic: a topic associated with the printed message. + ## + ## msg: the print message to send to peers. + ## + ## flags: tune the behavior of how the message is sent. + ## + ## Returns: true if the message is sent. + global send_print: function(topic: string, msg: string, flags: SendFlags &default = SendFlags()): bool; + + ## Register interest in all peer print messages that use a certain topic + ## prefix. Use :bro:see:`Broker::print_handler` to handle received + ## messages. + ## + ## topic_prefix: a prefix to match against remote message topics. + ## e.g. an empty prefix matches everything and "a" matches + ## "alice" and "amy" but not "bob". + ## + ## Returns: true if it's a new print subscription and it is now registered. + global subscribe_to_prints: function(topic_prefix: string): bool; + + ## Unregister interest in all peer print messages that use a topic prefix. + ## + ## topic_prefix: a prefix previously supplied to a successful call to + ## :bro:see:`Broker::subscribe_to_prints`. + ## + ## Returns: true if interest in the topic prefix is no longer advertised. + global unsubscribe_to_prints: function(topic_prefix: string): bool; + + ## Send an event to any interested peers. + ## + ## topic: a topic associated with the event message. + ## + ## args: event arguments as made by :bro:see:`Broker::event_args`. + ## + ## flags: tune the behavior of how the message is sent. + ## + ## Returns: true if the message is sent. + global send_event: function(topic: string, args: EventArgs, flags: SendFlags &default = SendFlags()): bool; + + ## Automatically send an event to any interested peers whenever it is + ## locally dispatched (e.g. using "event my_event(...);" in a script). + ## + ## topic: a topic string associated with the event message. + ## Peers advertise interest by registering a subscription to some + ## prefix of this topic name. + ## + ## ev: a Bro event value. + ## + ## flags: tune the behavior of how the message is sent. + ## + ## Returns: true if automatic event sending is now enabled. + global auto_event: function(topic: string, ev: any, flags: SendFlags &default = SendFlags()): bool; + + ## Stop automatically sending an event to peers upon local dispatch. + ## + ## topic: a topic originally given to :bro:see:`Broker::auto_event`. + ## + ## ev: an event originally given to :bro:see:`Broker::auto_event`. + ## + ## Returns: true if automatic events will not occur for the topic/event + ## pair. + global auto_event_stop: function(topic: string, ev: any): bool; + + ## Register interest in all peer event messages that use a certain topic + ## prefix. + ## + ## topic_prefix: a prefix to match against remote message topics. + ## e.g. an empty prefix matches everything and "a" matches + ## "alice" and "amy" but not "bob". + ## + ## Returns: true if it's a new event subscription and it is now registered. + global subscribe_to_events: function(topic_prefix: string): bool; + + ## Unregister interest in all peer event messages that use a topic prefix. + ## + ## topic_prefix: a prefix previously supplied to a successful call to + ## :bro:see:`Broker::subscribe_to_events`. + ## + ## Returns: true if interest in the topic prefix is no longer advertised. + global unsubscribe_to_events: function(topic_prefix: string): bool; + + ## Enable remote logs for a given log stream. + ## + ## id: the log stream to enable remote logs for. + ## + ## flags: tune the behavior of how log entry messages are sent. + ## + ## Returns: true if remote logs are enabled for the stream. + global enable_remote_logs: function(id: Log::ID, flags: SendFlags &default = SendFlags()): bool; + + ## Disable remote logs for a given log stream. + ## + ## id: the log stream to disable remote logs for. + ## + ## Returns: true if remote logs are disabled for the stream. + global disable_remote_logs: function(id: Log::ID): bool; + + ## Check if remote logs are enabled for a given log stream. + ## + ## id: the log stream to check. + ## + ## Returns: true if remote logs are enabled for the given stream. + global remote_logs_enabled: function(id: Log::ID): bool; + + ## Register interest in all peer log messages that use a certain topic + ## prefix. Logs are implicitly sent with topic "bro/log/" and + ## the receiving side processes them through the logging framework as usual. + ## + ## topic_prefix: a prefix to match against remote message topics. + ## e.g. an empty prefix matches everything and "a" matches + ## "alice" and "amy" but not "bob". + ## + ## Returns: true if it's a new log subscription and it is now registered. + global subscribe_to_logs: function(topic_prefix: string): bool; + + ## Unregister interest in all peer log messages that use a topic prefix. + ## Logs are implicitly sent with topic "bro/log/" and the + ## receiving side processes them through the logging framework as usual. + ## + ## topic_prefix: a prefix previously supplied to a successful call to + ## :bro:see:`Broker::subscribe_to_logs`. + ## + ## Returns: true if interest in the topic prefix is no longer advertised. + global unsubscribe_to_logs: function(topic_prefix: string): bool; + } -module BrokerStore; +@load base/bif/comm.bif +@load base/bif/messaging.bif -export { +module Broker; - ## Whether a data store query could be completed or not. - type QueryStatus: enum { - SUCCESS, - FAILURE, - }; +@ifdef ( Broker::__enable ) - ## An expiry time for a key-value pair inserted in to a data store. - type ExpiryTime: record { - ## Absolute point in time at which to expire the entry. - absolute: time &optional; - ## A point in time relative to the last modification time at which - ## to expire the entry. New modifications will delay the expiration. - since_last_modification: interval &optional; - }; +function enable(flags: EndpointFlags &default = EndpointFlags()) : bool + { + return __enable(flags); + } - ## The result of a data store query. - type QueryResult: record { - ## Whether the query completed or not. - status: BrokerStore::QueryStatus; - ## The result of the query. Certain queries may use a particular - ## data type (e.g. querying store size always returns a count, but - ## a lookup may return various data types). - result: BrokerComm::Data; - }; +function set_endpoint_flags(flags: EndpointFlags &default = EndpointFlags()): bool + { + return __set_endpoint_flags(flags); + } - ## Options to tune the SQLite storage backend. - type SQLiteOptions: record { - ## File system path of the database. - path: string &default = "store.sqlite"; - }; +function publish_topic(topic: string): bool + { + return __publish_topic(topic); + } - ## Options to tune the RocksDB storage backend. - type RocksDBOptions: record { - ## File system path of the database. - path: string &default = "store.rocksdb"; - }; +function unpublish_topic(topic: string): bool + { + return __unpublish_topic(topic); + } - ## Options to tune the particular storage backends. - type BackendOptions: record { - sqlite: SQLiteOptions &default = SQLiteOptions(); - rocksdb: RocksDBOptions &default = RocksDBOptions(); - }; -} +function listen(p: port, a: string &default = "", reuse: bool &default = T): bool + { + return __listen(p, a, reuse); + } + +function connect(a: string, p: port, retry: interval): bool + { + return __connect(a, p, retry); + } + +function disconnect(a: string, p: port): bool + { + return __disconnect(a, p); + } + +function send_print(topic: string, msg: string, flags: SendFlags &default = SendFlags()): bool + { + return __send_print(topic, msg, flags); + } + +function subscribe_to_prints(topic_prefix: string): bool + { + return __subscribe_to_prints(topic_prefix); + } + +function unsubscribe_to_prints(topic_prefix: string): bool + { + return __unsubscribe_to_prints(topic_prefix); + } + +function send_event(topic: string, args: EventArgs, flags: SendFlags &default = SendFlags()): bool + { + return __event(topic, args, flags); + } + +function auto_event(topic: string, ev: any, flags: SendFlags &default = SendFlags()): bool + { + return __auto_event(topic, ev, flags); + } + +function auto_event_stop(topic: string, ev: any): bool + { + return __auto_event_stop(topic, ev); + } + +function subscribe_to_events(topic_prefix: string): bool + { + return __subscribe_to_events(topic_prefix); + } + +function unsubscribe_to_events(topic_prefix: string): bool + { + return __unsubscribe_to_events(topic_prefix); + } + +function enable_remote_logs(id: Log::ID, flags: SendFlags &default = SendFlags()): bool + { + return __enable_remote_logs(id, flags); + } + +function disable_remote_logs(id: Log::ID): bool + { + return __disable_remote_logs(id); + } + +function remote_logs_enabled(id: Log::ID): bool + { + return __remote_logs_enabled(id); + } + +function subscribe_to_logs(topic_prefix: string): bool + { + return __subscribe_to_logs(topic_prefix); + } + +function unsubscribe_to_logs(topic_prefix: string): bool + { + return __unsubscribe_to_logs(topic_prefix); + } + +@endif diff --git a/scripts/base/frameworks/broker/store.bro b/scripts/base/frameworks/broker/store.bro new file mode 100644 index 0000000000..8640e80648 --- /dev/null +++ b/scripts/base/frameworks/broker/store.bro @@ -0,0 +1,1105 @@ +##! Various data structure definitions for use with Bro's communication system. + +@load ./main +@load base/bif/data.bif + +module Broker; + +export { + + ## Whether a data store query could be completed or not. + type QueryStatus: enum { + SUCCESS, + FAILURE, + }; + + ## An expiry time for a key-value pair inserted in to a data store. + type ExpiryTime: record { + ## Absolute point in time at which to expire the entry. + absolute: time &optional; + ## A point in time relative to the last modification time at which + ## to expire the entry. New modifications will delay the expiration. + since_last_modification: interval &optional; + }; + + ## The result of a data store query. + type QueryResult: record { + ## Whether the query completed or not. + status: Broker::QueryStatus; + ## The result of the query. Certain queries may use a particular + ## data type (e.g. querying store size always returns a count, but + ## a lookup may return various data types). + result: Broker::Data; + }; + + ## Enumerates the possible storage backends. + type BackendType: enum { + MEMORY, + SQLITE, + ROCKSDB, + }; + + ## Options to tune the SQLite storage backend. + type SQLiteOptions: record { + ## File system path of the database. + path: string &default = "store.sqlite"; + }; + + ## Options to tune the RocksDB storage backend. + type RocksDBOptions: record { + ## File system path of the database. + path: string &default = "store.rocksdb"; + }; + + ## Options to tune the particular storage backends. + type BackendOptions: record { + sqlite: SQLiteOptions &default = SQLiteOptions(); + rocksdb: RocksDBOptions &default = RocksDBOptions(); + }; + +@ifdef ( Broker::__enable ) + + ## Create a master data store which contains key-value pairs. + ## + ## id: a unique name for the data store. + ## + ## b: the storage backend to use. + ## + ## options: tunes how some storage backends operate. + ## + ## Returns: a handle to the data store. + global create_master: function(id: string, b: BackendType &default = MEMORY, + options: BackendOptions &default = BackendOptions()): opaque of Broker::Handle; + + ## Create a clone of a master data store which may live with a remote peer. + ## A clone automatically synchronizes to the master by automatically + ## receiving modifications and applying them locally. Direct modifications + ## are not possible, they must be sent through the master store, which then + ## automatically broadcasts the changes out to clones. But queries may be + ## made directly against the local cloned copy, which may be resolved + ## quicker than reaching out to a remote master store. + ## + ## id: the unique name which identifies the master data store. + ## + ## b: the storage backend to use. + ## + ## options: tunes how some storage backends operate. + ## + ## resync: the interval at which to re-attempt synchronizing with the master + ## store should the connection be lost. If the clone has not yet + ## synchronized for the first time, updates and queries queue up + ## until the synchronization completes. After, if the connection + ## to the master store is lost, queries continue to use the clone's + ## version, but updates will be lost until the master is once again + ## available. + ## + ## Returns: a handle to the data store. + global create_clone: function(id: string, b: BackendType &default = MEMORY, + options: BackendOptions &default = BackendOptions(), + resync: interval &default = 1sec): opaque of Broker::Handle; + + ## Create a frontend interface to an existing master data store that allows + ## querying and updating its contents. + ## + ## id: the unique name which identifies the master data store. + ## + ## Returns: a handle to the data store. + global create_frontend: function(id: string): opaque of Broker::Handle; + + ## Close a data store. + ## + ## h: a data store handle. + ## + ## Returns: true if store was valid and is now closed. The handle can no + ## longer be used for data store operations. + global close_by_handle: function(h: opaque of Broker::Handle): bool; + + ########################### + # non-blocking update API # + ########################### + + ## Insert a key-value pair in to the store. + ## + ## h: the handle of the store to modify. + ## + ## k: the key to insert. + ## + ## v: the value to insert. + ## + ## e: the expiration time of the key-value pair. + ## + ## Returns: false if the store handle was not valid. + global insert: function(h: opaque of Broker::Handle, + k: Broker::Data, v: Broker::Data, + e: Broker::ExpiryTime &default = Broker::ExpiryTime()): bool; + + ## Remove a key-value pair from the store. + ## + ## h: the handle of the store to modify. + ## + ## k: the key to remove. + ## + ## Returns: false if the store handle was not valid. + global erase: function(h: opaque of Broker::Handle, k: Broker::Data): bool; + + ## Remove all key-value pairs from the store. + ## + ## h: the handle of the store to modify. + ## + ## Returns: false if the store handle was not valid. + global clear: function(h: opaque of Broker::Handle): bool; + + ## Increment an integer value in a data store. + ## + ## h: the handle of the store to modify. + ## + ## k: the key whose associated value is to be modified. + ## + ## by: the amount to increment the value by. A non-existent key will first + ## create it with an implicit value of zero before incrementing. + ## + ## Returns: false if the store handle was not valid. + global increment: function(h: opaque of Broker::Handle, + k: Broker::Data, by: int &default = +1): bool; + + ## Decrement an integer value in a data store. + ## + ## h: the handle of the store to modify. + ## + ## k: the key whose associated value is to be modified. + ## + ## by: the amount to decrement the value by. A non-existent key will first + ## create it with an implicit value of zero before decrementing. + ## + ## Returns: false if the store handle was not valid. + global decrement: function(h: opaque of Broker::Handle, + k: Broker::Data, by: int &default = +1): bool; + + ## Add an element to a set value in a data store. + ## + ## h: the handle of the store to modify. + ## + ## k: the key whose associated value is to be modified. + ## + ## element: the element to add to the set. A non-existent key will first + ## create it with an implicit empty set value before modifying. + ## + ## Returns: false if the store handle was not valid. + global add_to_set: function(h: opaque of Broker::Handle, + k: Broker::Data, element: Broker::Data): bool; + + ## Remove an element from a set value in a data store. + ## + ## h: the handle of the store to modify. + ## + ## k: the key whose associated value is to be modified. + ## + ## element: the element to remove from the set. A non-existent key will + ## implicitly create an empty set value associated with the key. + ## + ## Returns: false if the store handle was not valid. + global remove_from_set: function(h: opaque of Broker::Handle, + k: Broker::Data, element: Broker::Data): bool; + + ## Add a new item to the head of a vector value in a data store. + ## + ## h: the handle of store to modify. + ## + ## k: the key whose associated value is to be modified. + ## + ## items: the element to insert in to the vector. A non-existent key will + ## first create an empty vector value before modifying. + ## + ## Returns: false if the store handle was not valid. + global push_left: function(h: opaque of Broker::Handle, k: Broker::Data, + items: Broker::DataVector): bool; + + ## Add a new item to the tail of a vector value in a data store. + ## + ## h: the handle of store to modify. + ## + ## k: the key whose associated value is to be modified. + ## + ## items: the element to insert in to the vector. A non-existent key will + ## first create an empty vector value before modifying. + ## + ## Returns: false if the store handle was not valid. + global push_right: function(h: opaque of Broker::Handle, k: Broker::Data, + items: Broker::DataVector): bool; + + ########################## + # non-blocking query API # + ########################## + + ## Pop the head of a data store vector value. + ## + ## h: the handle of the store to query. + ## + ## k: the key associated with the vector to modify. + ## + ## Returns: the result of the query. + global pop_left: function(h: opaque of Broker::Handle, + k: Broker::Data): QueryResult; + + ## Pop the tail of a data store vector value. + ## + ## h: the handle of the store to query. + ## + ## k: the key associated with the vector to modify. + ## + ## Returns: the result of the query. + global pop_right: function(h: opaque of Broker::Handle, + k: Broker::Data): QueryResult; + + ## Lookup the value associated with a key in a data store. + ## + ## h: the handle of the store to query. + ## + ## k: the key to lookup. + ## + ## Returns: the result of the query. + global lookup: function(h: opaque of Broker::Handle, + k: Broker::Data): QueryResult; + + ## Check if a data store contains a given key. + ## + ## h: the handle of the store to query. + ## + ## k: the key to check for existence. + ## + ## Returns: the result of the query (uses :bro:see:`Broker::BOOL`). + global exists: function(h: opaque of Broker::Handle, + k: Broker::Data): QueryResult; + + ## Retrieve all keys in a data store. + ## + ## h: the handle of the store to query. + ## + ## Returns: the result of the query (uses :bro:see:`Broker::VECTOR`). + global keys: function(h: opaque of Broker::Handle): QueryResult; + + ## Get the number of key-value pairs in a data store. + ## + ## h: the handle of the store to query. + ## + ## Returns: the result of the query (uses :bro:see:`Broker::COUNT`). + global size: function(h: opaque of Broker::Handle): QueryResult; + + ########################## + # data API # + ########################## + + ## Convert any Bro value to communication data. + ## + ## d: any Bro value to attempt to convert (not all types are supported). + ## + ## Returns: the converted communication data. The returned record's optional + ## field will not be set if the conversion was not possible (this can + ## happen if the Bro data type does not support being converted to + ## communication data). + global data: function(d: any): Broker::Data; + + ## Retrieve the type of data associated with communication data. + ## + ## d: the communication data. + ## + ## Returns: the data type associated with the communication data. + global data_type: function(d: Broker::Data): Broker::DataType; + + ## Convert communication data with a type of :bro:see:`Broker::BOOL` to + ## an actual Bro value. + ## + ## d: the communication data to convert. + ## + ## Returns: the value retrieved from the communication data. + global refine_to_bool: function(d: Broker::Data): bool; + + ## Convert communication data with a type of :bro:see:`Broker::INT` to + ## an actual Bro value. + ## + ## d: the communication data to convert. + ## + ## Returns: the value retrieved from the communication data. + global refine_to_int: function(d: Broker::Data): int; + + ## Convert communication data with a type of :bro:see:`Broker::COUNT` to + ## an actual Bro value. + ## + ## d: the communication data to convert. + ## + ## Returns: the value retrieved from the communication data. + global refine_to_count: function(d: Broker::Data): count; + + ## Convert communication data with a type of :bro:see:`Broker::DOUBLE` to + ## an actual Bro value. + ## + ## d: the communication data to convert. + ## + ## Returns: the value retrieved from the communication data. + global refine_to_double: function(d: Broker::Data): double; + + ## Convert communication data with a type of :bro:see:`Broker::STRING` to + ## an actual Bro value. + ## + ## d: the communication data to convert. + ## + ## Returns: the value retrieved from the communication data. + global refine_to_string: function(d: Broker::Data): string; + + ## Convert communication data with a type of :bro:see:`Broker::ADDR` to + ## an actual Bro value. + ## + ## d: the communication data to convert. + ## + ## Returns: the value retrieved from the communication data. + global refine_to_addr: function(d: Broker::Data): addr; + + ## Convert communication data with a type of :bro:see:`Broker::SUBNET` to + ## an actual Bro value. + ## + ## d: the communication data to convert. + ## + ## Returns: the value retrieved from the communication data. + global refine_to_subnet: function(d: Broker::Data): subnet; + + ## Convert communication data with a type of :bro:see:`Broker::PORT` to + ## an actual Bro value. + ## + ## d: the communication data to convert. + ## + ## Returns: the value retrieved from the communication data. + global refine_to_port: function(d: Broker::Data): port; + + ## Convert communication data with a type of :bro:see:`Broker::TIME` to + ## an actual Bro value. + ## + ## d: the communication data to convert. + ## + ## Returns: the value retrieved from the communication data. + global refine_to_time: function(d: Broker::Data): time; + + ## Convert communication data with a type of :bro:see:`Broker::INTERVAL` to + ## an actual Bro value. + ## + ## d: the communication data to convert. + ## + ## Returns: the value retrieved from the communication data. + global refine_to_interval: function(d: Broker::Data): interval; + + ## Convert communication data with a type of :bro:see:`Broker::ENUM` to + ## the name of the enum value. :bro:see:`lookup_ID` may be used to convert + ## the name to the actual enum value. + ## + ## d: the communication data to convert. + ## + ## Returns: the enum name retrieved from the communication data. + global refine_to_enum_name: function(d: Broker::Data): string; + + ## Create communication data of type "set". + global set_create: function(): Broker::Data; + + ## Remove all elements within a set. + ## + ## s: the set to clear. + ## + ## Returns: always true. + global set_clear: function(s: Broker::Data): bool; + + ## Get the number of elements within a set. + ## + ## s: the set to query. + ## + ## Returns: the number of elements in the set. + global set_size: function(s: Broker::Data): count; + + ## Check if a set contains a particular element. + ## + ## s: the set to query. + ## + ## key: the element to check for existence. + ## + ## Returns: true if the key exists in the set. + global set_contains: function(s: Broker::Data, key: Broker::Data): bool; + + ## Insert an element into a set. + ## + ## s: the set to modify. + ## + ## key: the element to insert. + ## + ## Returns: true if the key was inserted, or false if it already existed. + global set_insert: function(s: Broker::Data, key: Broker::Data): bool; + + ## Remove an element from a set. + ## + ## s: the set to modify. + ## + ## key: the element to remove. + ## + ## Returns: true if the element existed in the set and is now removed. + global set_remove: function(s: Broker::Data, key: Broker::Data): bool; + + ## Create an iterator for a set. Note that this makes a copy of the set + ## internally to ensure the iterator is always valid. + ## + ## s: the set to iterate over. + ## + ## Returns: an iterator. + global set_iterator: function(s: Broker::Data): opaque of Broker::SetIterator; + + ## Check if there are no more elements to iterate over. + ## + ## it: an iterator. + ## + ## Returns: true if there are no more elements to iterator over, i.e. + ## the iterator is one-past-the-final-element. + global set_iterator_last: function(it: opaque of Broker::SetIterator): bool; + + ## Advance an iterator. + ## + ## it: an iterator. + ## + ## Returns: true if the iterator, after advancing, still references an element + ## in the collection. False if the iterator, after advancing, is + ## one-past-the-final-element. + global set_iterator_next: function(it: opaque of Broker::SetIterator): bool; + + ## Retrieve the data at an iterator's current position. + ## + ## it: an iterator. + ## + ## Returns: element in the collection that the iterator currently references. + global set_iterator_value: function(it: opaque of Broker::SetIterator): Broker::Data; + + ## Create communication data of type "table". + global table_create: function(): Broker::Data; + + ## Remove all elements within a table. + ## + ## t: the table to clear. + ## + ## Returns: always true. + global table_clear: function(t: Broker::Data): bool; + + ## Get the number of elements within a table. + ## + ## t: the table to query. + ## + ## Returns: the number of elements in the table. + global table_size: function(t: Broker::Data): count; + + ## Check if a table contains a particular key. + ## + ## t: the table to query. + ## + ## key: the key to check for existence. + ## + ## Returns: true if the key exists in the table. + global table_contains: function(t: Broker::Data, key: Broker::Data): bool; + + ## Insert a key-value pair into a table. + ## + ## t: the table to modify. + ## + ## key: the key at which to insert the value. + ## + ## val: the value to insert. + ## + ## Returns: true if the key-value pair was inserted, or false if the key + ## already existed in the table. + global table_insert: function(t: Broker::Data, key: Broker::Data, val: Broker::Data): Broker::Data; + + ## Remove a key-value pair from a table. + ## + ## t: the table to modify. + ## + ## key: the key to remove from the table. + ## + ## Returns: the value associated with the key. If the key did not exist, then + ## the optional field of the returned record is not set. + global table_remove: function(t: Broker::Data, key: Broker::Data): Broker::Data; + + ## Retrieve a value from a table. + ## + ## t: the table to query. + ## + ## key: the key to lookup. + ## + ## Returns: the value associated with the key. If the key did not exist, then + ## the optional field of the returned record is not set. + global table_lookup: function(t: Broker::Data, key: Broker::Data): Broker::Data; + + ## Create an iterator for a table. Note that this makes a copy of the table + ## internally to ensure the iterator is always valid. + ## + ## t: the table to iterate over. + ## + ## Returns: an iterator. + global table_iterator: function(t: Broker::Data): opaque of Broker::TableIterator; + + ## Check if there are no more elements to iterate over. + ## + ## it: an iterator. + ## + ## Returns: true if there are no more elements to iterator over, i.e. + ## the iterator is one-past-the-final-element. + global table_iterator_last: function(it: opaque of Broker::TableIterator): bool; + + ## Advance an iterator. + ## + ## it: an iterator. + ## + ## Returns: true if the iterator, after advancing, still references an element + ## in the collection. False if the iterator, after advancing, is + ## one-past-the-final-element. + global table_iterator_next: function(it: opaque of Broker::TableIterator): bool; + + ## Retrieve the data at an iterator's current position. + ## + ## it: an iterator. + ## + ## Returns: element in the collection that the iterator currently references. + global table_iterator_value: function(it: opaque of Broker::TableIterator): Broker::TableItem; + + ## Create communication data of type "vector". + global vector_create: function(): Broker::Data; + + ## Remove all elements within a vector. + ## + ## v: the vector to clear. + ## + ## Returns: always true. + global vector_clear: function(v: Broker::Data): bool; + + ## Get the number of elements within a vector. + ## + ## v: the vector to query. + ## + ## Returns: the number of elements in the vector. + global vector_size: function(v: Broker::Data): count; + + ## Insert an element into a vector at a particular position, possibly displacing + ## existing elements (insertion always grows the size of the vector by one). + ## + ## v: the vector to modify. + ## + ## d: the element to insert. + ## + ## idx: the index at which to insert the data. If it is greater than the + ## current size of the vector, the element is inserted at the end. + ## + ## Returns: always true. + global vector_insert: function(v: Broker::Data, d: Broker::Data, idx: count): bool; + + ## Replace an element in a vector at a particular position. + ## + ## v: the vector to modify. + ## + ## d: the element to insert. + ## + ## idx: the index to replace. + ## + ## Returns: the value that was just evicted. If the index was larger than any + ## valid index, the optional field of the returned record is not set. + global vector_replace: function(v: Broker::Data, d: Broker::Data, idx: count): Broker::Data; + + ## Remove an element from a vector at a particular position. + ## + ## v: the vector to modify. + ## + ## idx: the index to remove. + ## + ## Returns: the value that was just evicted. If the index was larger than any + ## valid index, the optional field of the returned record is not set. + global vector_remove: function(v: Broker::Data, idx: count): Broker::Data; + + ## Lookup an element in a vector at a particular position. + ## + ## v: the vector to query. + ## + ## idx: the index to lookup. + ## + ## Returns: the value at the index. If the index was larger than any + ## valid index, the optional field of the returned record is not set. + global vector_lookup: function(v: Broker::Data, idx: count): Broker::Data; + + ## Create an iterator for a vector. Note that this makes a copy of the vector + ## internally to ensure the iterator is always valid. + ## + ## v: the vector to iterate over. + ## + ## Returns: an iterator. + global vector_iterator: function(v: Broker::Data): opaque of Broker::VectorIterator; + + ## Check if there are no more elements to iterate over. + ## + ## it: an iterator. + ## + ## Returns: true if there are no more elements to iterator over, i.e. + ## the iterator is one-past-the-final-element. + global vector_iterator_last: function(it: opaque of Broker::VectorIterator): bool; + + ## Advance an iterator. + ## + ## it: an iterator. + ## + ## Returns: true if the iterator, after advancing, still references an element + ## in the collection. False if the iterator, after advancing, is + ## one-past-the-final-element. + global vector_iterator_next: function(it: opaque of Broker::VectorIterator): bool; + + ## Retrieve the data at an iterator's current position. + ## + ## it: an iterator. + ## + ## Returns: element in the collection that the iterator currently references. + global vector_iterator_value: function(it: opaque of Broker::VectorIterator): Broker::Data; + + ## Create communication data of type "record". + ## + ## sz: the number of fields in the record. + ## + ## Returns: record data, with all fields uninitialized. + global record_create: function(sz: count): Broker::Data; + + ## Get the number of fields within a record. + ## + ## r: the record to query. + ## + ## Returns: the number of fields in the record. + global record_size: function(r: Broker::Data): count; + + ## Replace a field in a record at a particular position. + ## + ## r: the record to modify. + ## + ## d: the new field value to assign. + ## + ## idx: the index to replace. + ## + ## Returns: false if the index was larger than any valid index, else true. + global record_assign: function(r: Broker::Data, d: Broker::Data, idx: count): bool; + + ## Lookup a field in a record at a particular position. + ## + ## r: the record to query. + ## + ## idx: the index to lookup. + ## + ## Returns: the value at the index. The optional field of the returned record + ## may not be set if the field of the record has no value or if the + ## index was not valid. + global record_lookup: function(r: Broker::Data, idx: count): Broker::Data; + + ## Create an iterator for a record. Note that this makes a copy of the record + ## internally to ensure the iterator is always valid. + ## + ## r: the record to iterate over. + ## + ## Returns: an iterator. + global record_iterator: function(r: Broker::Data): opaque of Broker::RecordIterator; + + ## Check if there are no more elements to iterate over. + ## + ## it: an iterator. + ## + ## Returns: true if there are no more elements to iterator over, i.e. + ## the iterator is one-past-the-final-element. + global record_iterator_last: function(it: opaque of Broker::RecordIterator): bool; + + ## Advance an iterator. + ## + ## it: an iterator. + ## + ## Returns: true if the iterator, after advancing, still references an element + ## in the collection. False if the iterator, after advancing, is + ## one-past-the-final-element. + global record_iterator_next: function(it: opaque of Broker::RecordIterator): bool; + + ## Retrieve the data at an iterator's current position. + ## + ## it: an iterator. + ## + ## Returns: element in the collection that the iterator currently references. + global record_iterator_value: function(it: opaque of Broker::RecordIterator): Broker::Data; + +@endif +} + +@load base/bif/store.bif + +module Broker; + +@ifdef ( Broker::__enable ) + +function create_master(id: string, b: BackendType &default = MEMORY, + options: BackendOptions &default = BackendOptions()): opaque of Broker::Handle + { + return __create_master(id, b, options); + } + +function create_clone(id: string, b: BackendType &default = MEMORY, + options: BackendOptions &default = BackendOptions(), + resync: interval &default = 1sec): opaque of Broker::Handle + { + return __create_clone(id, b, options, resync); + } + +function create_frontend(id: string): opaque of Broker::Handle + { + return __create_frontend(id); + } + +function close_by_handle(h: opaque of Broker::Handle): bool + { + return __close_by_handle(h); + } + +function insert(h: opaque of Broker::Handle, k: Broker::Data, v: Broker::Data, + e: Broker::ExpiryTime &default = Broker::ExpiryTime()): bool + { + return __insert(h, k, v, e); + } + +function erase(h: opaque of Broker::Handle, k: Broker::Data): bool + { + return __erase(h, k); + } + +function clear(h: opaque of Broker::Handle): bool + { + return __clear(h); + } + +function increment(h: opaque of Broker::Handle, + k: Broker::Data, by: int &default = +1): bool + { + return __increment(h, k, by); + } + +function decrement(h: opaque of Broker::Handle, + k: Broker::Data, by: int &default = +1): bool + { + return __decrement(h, k, by); + } + +function add_to_set(h: opaque of Broker::Handle, + k: Broker::Data, element: Broker::Data): bool + { + return __add_to_set(h, k, element); + } + +function remove_from_set(h: opaque of Broker::Handle, + k: Broker::Data, element: Broker::Data): bool + { + return __remove_from_set(h, k, element); + } + +function push_left(h: opaque of Broker::Handle, k: Broker::Data, + items: Broker::DataVector): bool + { + return __push_left(h, k, items); + } + +function push_right(h: opaque of Broker::Handle, k: Broker::Data, + items: Broker::DataVector): bool + { + return __push_right(h, k, items); + } + +function pop_left(h: opaque of Broker::Handle, k: Broker::Data): QueryResult + { + return __pop_left(h, k); + } + +function pop_right(h: opaque of Broker::Handle, k: Broker::Data): QueryResult + { + return __pop_right(h, k); + } + +function lookup(h: opaque of Broker::Handle, k: Broker::Data): QueryResult + { + return __lookup(h, k); + } + +function exists(h: opaque of Broker::Handle, k: Broker::Data): QueryResult + { + return __exists(h, k); + } + +function keys(h: opaque of Broker::Handle): QueryResult + { + return __keys(h); + } + +function size(h: opaque of Broker::Handle): QueryResult + { + return __size(h); + } + +function data(d: any): Broker::Data + { + return __data(d); + } + +function data_type(d: Broker::Data): Broker::DataType + { + return __data_type(d); + } + +function refine_to_bool(d: Broker::Data): bool + { + return __refine_to_bool(d); + } + +function refine_to_int(d: Broker::Data): int + { + return __refine_to_int(d); + } + +function refine_to_count(d: Broker::Data): count + { + return __refine_to_count(d); + } + +function refine_to_double(d: Broker::Data): double + { + return __refine_to_double(d); + } + +function refine_to_string(d: Broker::Data): string + { + return __refine_to_string(d); + } + +function refine_to_addr(d: Broker::Data): addr + { + return __refine_to_addr(d); + } + +function refine_to_subnet(d: Broker::Data): subnet + { + return __refine_to_subnet(d); + } + +function refine_to_port(d: Broker::Data): port + { + return __refine_to_port(d); + } + +function refine_to_time(d: Broker::Data): time + { + return __refine_to_time(d); + } + +function refine_to_interval(d: Broker::Data): interval + { + return __refine_to_interval(d); + } + +function refine_to_enum_name(d: Broker::Data): string + { + return __refine_to_enum_name(d); + } + +function set_create(): Broker::Data + { + return __set_create(); + } + +function set_clear(s: Broker::Data): bool + { + return __set_clear(s); + } + +function set_size(s: Broker::Data): count + { + return __set_size(s); + } + +function set_contains(s: Broker::Data, key: Broker::Data): bool + { + return __set_contains(s, key); + } + +function set_insert(s: Broker::Data, key: Broker::Data): bool + { + return __set_insert(s, key); + } + +function set_remove(s: Broker::Data, key: Broker::Data): bool + { + return __set_remove(s, key); + } + +function set_iterator(s: Broker::Data): opaque of Broker::SetIterator + { + return __set_iterator(s); + } + +function set_iterator_last(it: opaque of Broker::SetIterator): bool + { + return __set_iterator_last(it); + } + +function set_iterator_next(it: opaque of Broker::SetIterator): bool + { + return __set_iterator_next(it); + } + +function set_iterator_value(it: opaque of Broker::SetIterator): Broker::Data + { + return __set_iterator_value(it); + } + +function table_create(): Broker::Data + { + return __table_create(); + } + +function table_clear(t: Broker::Data): bool + { + return __table_clear(t); + } + +function table_size(t: Broker::Data): count + { + return __table_size(t); + } + +function table_contains(t: Broker::Data, key: Broker::Data): bool + { + return __table_contains(t, key); + } + +function table_insert(t: Broker::Data, key: Broker::Data, val: Broker::Data): Broker::Data + { + return __table_insert(t, key, val); + } + +function table_remove(t: Broker::Data, key: Broker::Data): Broker::Data + { + return __table_remove(t, key); + } + +function table_lookup(t: Broker::Data, key: Broker::Data): Broker::Data + { + return __table_lookup(t, key); + } + +function table_iterator(t: Broker::Data): opaque of Broker::TableIterator + { + return __table_iterator(t); + } + +function table_iterator_last(it: opaque of Broker::TableIterator): bool + { + return __table_iterator_last(it); + } + +function table_iterator_next(it: opaque of Broker::TableIterator): bool + { + return __table_iterator_next(it); + } + +function table_iterator_value(it: opaque of Broker::TableIterator): Broker::TableItem + { + return __table_iterator_value(it); + } + +function vector_create(): Broker::Data + { + return __vector_create(); + } + +function vector_clear(v: Broker::Data): bool + { + return __vector_clear(v); + } + +function vector_size(v: Broker::Data): count + { + return __vector_size(v); + } + +function vector_insert(v: Broker::Data, d: Broker::Data, idx: count): bool + { + return __vector_insert(v, d, idx); + } + +function vector_replace(v: Broker::Data, d: Broker::Data, idx: count): Broker::Data + { + return __vector_replace(v, d, idx); + } + +function vector_remove(v: Broker::Data, idx: count): Broker::Data + { + return __vector_remove(v, idx); + } + +function vector_lookup(v: Broker::Data, idx: count): Broker::Data + { + return __vector_lookup(v, idx); + } + +function vector_iterator(v: Broker::Data): opaque of Broker::VectorIterator + { + return __vector_iterator(v); + } + +function vector_iterator_last(it: opaque of Broker::VectorIterator): bool + { + return __vector_iterator_last(it); + } + +function vector_iterator_next(it: opaque of Broker::VectorIterator): bool + { + return __vector_iterator_next(it); + } + +function vector_iterator_value(it: opaque of Broker::VectorIterator): Broker::Data + { + return __vector_iterator_value(it); + } + +function record_create(sz: count): Broker::Data + { + return __record_create(sz); + } + +function record_size(r: Broker::Data): count + { + return __record_size(r); + } + +function record_assign(r: Broker::Data, d: Broker::Data, idx: count): bool + { + return __record_assign(r, d, idx); + } + +function record_lookup(r: Broker::Data, idx: count): Broker::Data + { + return __record_lookup(r, idx); + } + +function record_iterator(r: Broker::Data): opaque of Broker::RecordIterator + { + return __record_iterator(r); + } + +function record_iterator_last(it: opaque of Broker::RecordIterator): bool + { + return __record_iterator_last(it); + } + +function record_iterator_next(it: opaque of Broker::RecordIterator): bool + { + return __record_iterator_next(it); + } + +function record_iterator_value(it: opaque of Broker::RecordIterator): Broker::Data + { + return __record_iterator_value(it); + } + +@endif diff --git a/scripts/base/frameworks/cluster/main.bro b/scripts/base/frameworks/cluster/main.bro index 3451cb4169..55fc084641 100644 --- a/scripts/base/frameworks/cluster/main.bro +++ b/scripts/base/frameworks/cluster/main.bro @@ -68,7 +68,7 @@ export { ## Events raised by TimeMachine instances and handled by workers. const tm2worker_events = /EMPTY/ &redef; - ## Events sent by the control host (i.e. BroControl) when dynamically + ## Events sent by the control host (i.e., BroControl) when dynamically ## connecting to a running instance to update settings or request data. const control_events = Control::controller_events &redef; diff --git a/scripts/base/frameworks/intel/main.bro b/scripts/base/frameworks/intel/main.bro index eba27ca56a..28e8a40baa 100644 --- a/scripts/base/frameworks/intel/main.bro +++ b/scripts/base/frameworks/intel/main.bro @@ -77,23 +77,34 @@ export { ## The type of data that the indicator represents. indicator_type: Type &log &optional; - ## If the indicator type was :bro:enum:`Intel::ADDR`, then this + ## If the indicator type was :bro:enum:`Intel::ADDR`, then this ## field will be present. host: addr &optional; ## Where the data was discovered. where: Where &log; - + ## The name of the node where the match was discovered. node: string &optional &log; - ## If the data was discovered within a connection, the + ## If the data was discovered within a connection, the ## connection record should go here to give context to the data. conn: connection &optional; + ## If the data was discovered within a connection, the + ## connection uid should go here to give context to the data. + ## If the *conn* field is provided, this will be automatically + ## filled out. + uid: string &optional; + ## If the data was discovered within a file, the file record ## should go here to provide context to the data. f: fa_file &optional; + + ## If the data was discovered within a file, the file uid should + ## go here to provide context to the data. If the *f* field is + ## provided, this will be automatically filled out. + fuid: string &optional; }; ## Record used for the logging framework representing a positive @@ -112,7 +123,8 @@ export { ## If a file was associated with this intelligence hit, ## this is the uid for the file. fuid: string &log &optional; - ## A mime type if the intelligence hit is related to a file. + + ## A mime type if the intelligence hit is related to a file. ## If the $f field is provided this will be automatically filled ## out. file_mime_type: string &log &optional; @@ -283,15 +295,14 @@ event Intel::match(s: Seen, items: set[Item]) &priority=5 if ( s?$f ) { + s$fuid = s$f$id; + if ( s$f?$conns && |s$f$conns| == 1 ) { for ( cid in s$f$conns ) s$conn = s$f$conns[cid]; } - if ( ! info?$fuid ) - info$fuid = s$f$id; - if ( ! info?$file_mime_type && s$f?$info && s$f$info?$mime_type ) info$file_mime_type = s$f$info$mime_type; @@ -299,12 +310,18 @@ event Intel::match(s: Seen, items: set[Item]) &priority=5 info$file_desc = Files::describe(s$f); } + if ( s?$fuid ) + info$fuid = s$fuid; + if ( s?$conn ) { - info$uid = s$conn$uid; + s$uid = s$conn$uid; info$id = s$conn$id; } + if ( s?$uid ) + info$uid = s$uid; + for ( item in items ) add info$sources[item$meta$source]; diff --git a/scripts/base/frameworks/netcontrol/main.bro b/scripts/base/frameworks/netcontrol/main.bro index 563188921d..65537ed9cf 100644 --- a/scripts/base/frameworks/netcontrol/main.bro +++ b/scripts/base/frameworks/netcontrol/main.bro @@ -23,20 +23,20 @@ export { # ### Generic functions and events. # ### - # Activates a plugin. - # - # p: The plugin to acticate. - # - # priority: The higher the priority, the earlier this plugin will be checked - # whether it supports an operation, relative to other plugins. + ## Activates a plugin. + ## + ## p: The plugin to acticate. + ## + ## priority: The higher the priority, the earlier this plugin will be checked + ## whether it supports an operation, relative to other plugins. global activate: function(p: PluginState, priority: int); - # Event that is used to initialize plugins. Place all plugin initialization - # related functionality in this event. + ## Event that is used to initialize plugins. Place all plugin initialization + ## related functionality in this event. global NetControl::init: event(); - # Event that is raised once all plugins activated in ``NetControl::init`` have finished - # their initialization. + ## Event that is raised once all plugins activated in ``NetControl::init`` + ## have finished their initialization. global NetControl::init_done: event(); # ### @@ -109,21 +109,24 @@ export { ## ## r: The rule to install. ## - ## Returns: If succesful, returns an ID string unique to the rule that can later - ## be used to refer to it. If unsuccessful, returns an empty string. The ID is also - ## assigned to ``r$id``. Note that "successful" means "a plugin knew how to handle - ## the rule", it doesn't necessarily mean that it was indeed successfully put in - ## place, because that might happen asynchronously and thus fail only later. + ## Returns: If succesful, returns an ID string unique to the rule that can + ## later be used to refer to it. If unsuccessful, returns an empty + ## string. The ID is also assigned to ``r$id``. Note that + ## "successful" means "a plugin knew how to handle the rule", it + ## doesn't necessarily mean that it was indeed successfully put in + ## place, because that might happen asynchronously and thus fail + ## only later. global add_rule: function(r: Rule) : string; ## Removes a rule. ## - ## id: The rule to remove, specified as the ID returned by :bro:id:`add_rule` . + ## id: The rule to remove, specified as the ID returned by :bro:id:`NetControl::add_rule`. ## - ## Returns: True if succesful, the relevant plugin indicated that it knew how - ## to handle the removal. Note that again "success" means the plugin accepted the - ## removal. They might still fail to put it into effect, as that might happen - ## asynchronously and thus go wrong at that point. + ## Returns: True if succesful, the relevant plugin indicated that it knew + ## how to handle the removal. Note that again "success" means the + ## plugin accepted the removal. They might still fail to put it + ## into effect, as that might happen asynchronously and thus go + ## wrong at that point. global remove_rule: function(id: string) : bool; ## Searches all rules affecting a certain IP address. @@ -156,7 +159,7 @@ export { ## r: The rule now removed. ## ## p: The state for the plugin that had the rule in place and now - ## removed it. + ## removed it. ## ## msg: An optional informational message by the plugin. global rule_removed: event(r: Rule, p: PluginState, msg: string &default=""); @@ -168,7 +171,7 @@ export { ## i: Additional flow information, if supported by the protocol. ## ## p: The state for the plugin that had the rule in place and now - ## removed it. + ## removed it. ## ## msg: An optional informational message by the plugin. global rule_timeout: event(r: Rule, i: FlowInfo, p: PluginState); diff --git a/scripts/base/frameworks/netcontrol/plugins/acld.bro b/scripts/base/frameworks/netcontrol/plugins/acld.bro index 76661bc857..a2f0fa2cc0 100644 --- a/scripts/base/frameworks/netcontrol/plugins/acld.bro +++ b/scripts/base/frameworks/netcontrol/plugins/acld.bro @@ -6,6 +6,8 @@ module NetControl; @load ../plugin @load base/frameworks/broker +@ifdef ( Broker::__enable ) + export { type AclRule : record { command: string; @@ -227,7 +229,7 @@ function acld_add_rule_fun(p: PluginState, r: Rule) : bool if ( ar$command == "" ) return F; - BrokerComm::event(p$acld_config$acld_topic, BrokerComm::event_args(acld_add_rule, p$acld_id, r, ar)); + Broker::send_event(p$acld_config$acld_topic, Broker::event_args(acld_add_rule, p$acld_id, r, ar)); return T; } @@ -242,18 +244,18 @@ function acld_remove_rule_fun(p: PluginState, r: Rule) : bool else return F; - BrokerComm::event(p$acld_config$acld_topic, BrokerComm::event_args(acld_remove_rule, p$acld_id, r, ar)); + Broker::send_event(p$acld_config$acld_topic, Broker::event_args(acld_remove_rule, p$acld_id, r, ar)); return T; } function acld_init(p: PluginState) { - BrokerComm::enable(); - BrokerComm::connect(cat(p$acld_config$acld_host), p$acld_config$acld_port, 1sec); - BrokerComm::subscribe_to_events(p$acld_config$acld_topic); + Broker::enable(); + Broker::connect(cat(p$acld_config$acld_host), p$acld_config$acld_port, 1sec); + Broker::subscribe_to_events(p$acld_config$acld_topic); } -event BrokerComm::outgoing_connection_established(peer_address: string, peer_port: port, peer_name: string) +event Broker::outgoing_connection_established(peer_address: string, peer_port: port, peer_name: string) { if ( [peer_port, peer_address] !in netcontrol_acld_peers ) # ok, this one was none of ours... @@ -292,3 +294,4 @@ function create_acld(config: AcldConfig) : PluginState return p; } +@endif diff --git a/scripts/base/frameworks/netcontrol/plugins/broker.bro b/scripts/base/frameworks/netcontrol/plugins/broker.bro index 619b6b607b..0687d70f82 100644 --- a/scripts/base/frameworks/netcontrol/plugins/broker.bro +++ b/scripts/base/frameworks/netcontrol/plugins/broker.bro @@ -8,6 +8,8 @@ module NetControl; @load ../plugin @load base/frameworks/broker +@ifdef ( Broker::__enable ) + export { ## Instantiates the broker plugin. global create_broker: function(host: addr, host_port: port, topic: string, can_expire: bool &default=F) : PluginState; @@ -96,24 +98,24 @@ function broker_name(p: PluginState) : string function broker_add_rule_fun(p: PluginState, r: Rule) : bool { - BrokerComm::event(p$broker_topic, BrokerComm::event_args(broker_add_rule, p$broker_id, r)); + Broker::send_event(p$broker_topic, Broker::event_args(broker_add_rule, p$broker_id, r)); return T; } function broker_remove_rule_fun(p: PluginState, r: Rule) : bool { - BrokerComm::event(p$broker_topic, BrokerComm::event_args(broker_remove_rule, p$broker_id, r)); + Broker::send_event(p$broker_topic, Broker::event_args(broker_remove_rule, p$broker_id, r)); return T; } function broker_init(p: PluginState) { - BrokerComm::enable(); - BrokerComm::connect(cat(p$broker_host), p$broker_port, 1sec); - BrokerComm::subscribe_to_events(p$broker_topic); + Broker::enable(); + Broker::connect(cat(p$broker_host), p$broker_port, 1sec); + Broker::subscribe_to_events(p$broker_topic); } -event BrokerComm::outgoing_connection_established(peer_address: string, peer_port: port, peer_name: string) +event Broker::outgoing_connection_established(peer_address: string, peer_port: port, peer_name: string) { if ( [peer_port, peer_address] !in netcontrol_broker_peers ) return; @@ -161,3 +163,5 @@ function create_broker(host: addr, host_port: port, topic: string, can_expire: b return p; } + +@endif diff --git a/scripts/base/frameworks/netcontrol/plugins/debug.bro b/scripts/base/frameworks/netcontrol/plugins/debug.bro index f421dc55e3..a26a151400 100644 --- a/scripts/base/frameworks/netcontrol/plugins/debug.bro +++ b/scripts/base/frameworks/netcontrol/plugins/debug.bro @@ -11,7 +11,7 @@ export { ## plugin simply logs the operations it receives. ## ## do_something: If true, the plugin will claim it supports all operations; if - ## false, it will indicate it doesn't support any. + ## false, it will indicate it doesn't support any. global create_debug: function(do_something: bool) : PluginState; } diff --git a/scripts/base/frameworks/netcontrol/types.bro b/scripts/base/frameworks/netcontrol/types.bro index 440d63d8bc..3147420c99 100644 --- a/scripts/base/frameworks/netcontrol/types.bro +++ b/scripts/base/frameworks/netcontrol/types.bro @@ -14,7 +14,7 @@ export { MAC, ##< Activity involving a MAC address. }; - ## Type of a :bro:id:`Flow` for defining a flow. + ## Type for defining a flow. type Flow: record { src_h: subnet &optional; ##< The source IP address/subnet. src_p: port &optional; ##< The source port number. @@ -27,10 +27,10 @@ export { ## Type defining the enity an :bro:id:`Rule` is operating on. type Entity: record { ty: EntityType; ##< Type of entity. - conn: conn_id &optional; ##< Used with :bro:id:`CONNECTION` . - flow: Flow &optional; ##< Used with :bro:id:`FLOW` . - ip: subnet &optional; ##< Used with bro:id:`ADDRESS`; can specifiy a CIDR subnet. - mac: string &optional; ##< Used with :bro:id:`MAC`. + conn: conn_id &optional; ##< Used with :bro:enum:`NetControl::CONNECTION`. + flow: Flow &optional; ##< Used with :bro:enum:`NetControl::FLOW`. + ip: subnet &optional; ##< Used with :bro:enum:`NetControl::ADDRESS` to specifiy a CIDR subnet. + mac: string &optional; ##< Used with :bro:enum:`NetControl::MAC`. }; ## Target of :bro:id:`Rule` action. @@ -68,7 +68,7 @@ export { WHITELIST, }; - ## Type of a :bro:id:`FlowMod` for defining a flow modification action. + ## Type for defining a flow modification action. type FlowMod: record { src_h: addr &optional; ##< The source IP address. src_p: count &optional; ##< The source port number. @@ -90,8 +90,8 @@ export { priority: int &default=default_priority; ##< Priority if multiple rules match an entity (larger value is higher priority). location: string &optional; ##< Optional string describing where/what installed the rule. - out_port: count &optional; ##< Argument for bro:id:`REDIRECT` rules. - mod: FlowMod &optional; ##< Argument for :bro:id:`MODIFY` rules. + out_port: count &optional; ##< Argument for :bro:enum:`NetControl::REDIRECT` rules. + mod: FlowMod &optional; ##< Argument for :bro:enum:`NetControl::MODIFY` rules. id: string &default=""; ##< Internally determined unique ID for this rule. Will be set when added. cid: count &default=0; ##< Internally determined unique numeric ID for this rule. Set when added. diff --git a/scripts/base/frameworks/notice/main.bro b/scripts/base/frameworks/notice/main.bro index 2418b499e5..a203f6a772 100644 --- a/scripts/base/frameworks/notice/main.bro +++ b/scripts/base/frameworks/notice/main.bro @@ -44,6 +44,7 @@ export { ACTION_ALARM, }; + ## Type that represents a set of actions. type ActionSet: set[Notice::Action]; ## The notice framework is able to do automatic notice suppression by @@ -52,6 +53,7 @@ export { ## suppression. const default_suppression_interval = 1hrs &redef; + ## The record type that is used for representing and logging notices. type Info: record { ## An absolute time indicating when the notice occurred, ## defaults to the current network time. diff --git a/scripts/base/frameworks/openflow/plugins/broker.bro b/scripts/base/frameworks/openflow/plugins/broker.bro index d6cf52a92c..a67b941e08 100644 --- a/scripts/base/frameworks/openflow/plugins/broker.bro +++ b/scripts/base/frameworks/openflow/plugins/broker.bro @@ -5,6 +5,8 @@ module OpenFlow; +@ifdef ( Broker::__enable ) + export { redef enum Plugin += { BROKER, @@ -47,26 +49,26 @@ function broker_describe(state: ControllerState): string function broker_flow_mod_fun(state: ControllerState, match: ofp_match, flow_mod: OpenFlow::ofp_flow_mod): bool { - BrokerComm::event(state$broker_topic, BrokerComm::event_args(broker_flow_mod, state$_name, state$broker_dpid, match, flow_mod)); + Broker::send_event(state$broker_topic, Broker::event_args(broker_flow_mod, state$_name, state$broker_dpid, match, flow_mod)); return T; } function broker_flow_clear_fun(state: OpenFlow::ControllerState): bool { - BrokerComm::event(state$broker_topic, BrokerComm::event_args(broker_flow_clear, state$_name, state$broker_dpid)); + Broker::send_event(state$broker_topic, Broker::event_args(broker_flow_clear, state$_name, state$broker_dpid)); return T; } function broker_init(state: OpenFlow::ControllerState) { - BrokerComm::enable(); - BrokerComm::connect(cat(state$broker_host), state$broker_port, 1sec); - BrokerComm::subscribe_to_events(state$broker_topic); # openflow success and failure events are directly sent back via the other plugin via broker. + Broker::enable(); + Broker::connect(cat(state$broker_host), state$broker_port, 1sec); + Broker::subscribe_to_events(state$broker_topic); # openflow success and failure events are directly sent back via the other plugin via broker. } -event BrokerComm::outgoing_connection_established(peer_address: string, peer_port: port, peer_name: string) +event Broker::outgoing_connection_established(peer_address: string, peer_port: port, peer_name: string) { if ( [peer_port, peer_address] !in broker_peers ) # ok, this one was none of ours... @@ -93,3 +95,4 @@ function broker_new(name: string, host: addr, host_port: port, topic: string, dp return c; } +@endif diff --git a/scripts/base/frameworks/packet-filter/netstats.bro b/scripts/base/frameworks/packet-filter/netstats.bro index b5ffe24f54..f1757d8d47 100644 --- a/scripts/base/frameworks/packet-filter/netstats.bro +++ b/scripts/base/frameworks/packet-filter/netstats.bro @@ -18,7 +18,7 @@ export { event net_stats_update(last_stat: NetStats) { - local ns = net_stats(); + local ns = get_net_stats(); local new_dropped = ns$pkts_dropped - last_stat$pkts_dropped; if ( new_dropped > 0 ) { @@ -38,5 +38,5 @@ event bro_init() # Since this currently only calculates packet drops, let's skip the stats # collection if reading traces. if ( ! reading_traces() ) - schedule stats_collection_interval { net_stats_update(net_stats()) }; + schedule stats_collection_interval { net_stats_update(get_net_stats()) }; } diff --git a/scripts/base/frameworks/sumstats/main.bro b/scripts/base/frameworks/sumstats/main.bro index 8dbdb61edd..edd80ede0f 100644 --- a/scripts/base/frameworks/sumstats/main.bro +++ b/scripts/base/frameworks/sumstats/main.bro @@ -5,7 +5,8 @@ module SumStats; export { - ## The various calculations are all defined as plugins. + ## Type to represent the calculations that are available. The calculations + ## are all defined as plugins. type Calculation: enum { PLACEHOLDER }; @@ -39,6 +40,7 @@ export { str: string &optional; }; + ## Represents a reducer. type Reducer: record { ## Observation stream identifier for the reducer ## to attach to. @@ -56,7 +58,7 @@ export { normalize_key: function(key: SumStats::Key): Key &optional; }; - ## Value calculated for an observation stream fed into a reducer. + ## Result calculated for an observation stream fed into a reducer. ## Most of the fields are added by plugins. type ResultVal: record { ## The time when the first observation was added to @@ -71,14 +73,15 @@ export { num: count &default=0; }; - ## Type to store results for multiple reducers. + ## Type to store a table of results for multiple reducers indexed by + ## observation stream identifier. type Result: table[string] of ResultVal; ## Type to store a table of sumstats results indexed by keys. type ResultTable: table[Key] of Result; - ## SumStats represent an aggregation of reducers along with - ## mechanisms to handle various situations like the epoch ending + ## Represents a SumStat, which consists of an aggregation of reducers along + ## with mechanisms to handle various situations like the epoch ending ## or thresholds being crossed. ## ## It's best to not access any global state outside @@ -101,21 +104,28 @@ export { ## The reducers for the SumStat. reducers: set[Reducer]; - ## Provide a function to calculate a value from the - ## :bro:see:`SumStats::Result` structure which will be used - ## for thresholding. - ## This is required if a *threshold* value is given. + ## A function that will be called once for each observation in order + ## to calculate a value from the :bro:see:`SumStats::Result` structure + ## which will be used for thresholding. + ## This function is required if a *threshold* value or + ## a *threshold_series* is given. threshold_val: function(key: SumStats::Key, result: SumStats::Result): double &optional; - ## The threshold value for calling the - ## *threshold_crossed* callback. + ## The threshold value for calling the *threshold_crossed* callback. + ## If you need more than one threshold value, then use + ## *threshold_series* instead. threshold: double &optional; - ## A series of thresholds for calling the - ## *threshold_crossed* callback. + ## A series of thresholds for calling the *threshold_crossed* + ## callback. These thresholds must be listed in ascending order, + ## because a threshold is not checked until the preceding one has + ## been crossed. threshold_series: vector of double &optional; ## A callback that is called when a threshold is crossed. + ## A threshold is crossed when the value returned from *threshold_val* + ## is greater than or equal to the threshold value, but only the first + ## time this happens within an epoch. threshold_crossed: function(key: SumStats::Key, result: SumStats::Result) &optional; ## A callback that receives each of the results at the @@ -130,6 +140,8 @@ export { }; ## Create a summary statistic. + ## + ## ss: The SumStat to create. global create: function(ss: SumStats::SumStat); ## Add data into an observation stream. This should be diff --git a/scripts/base/frameworks/sumstats/plugins/average.bro b/scripts/base/frameworks/sumstats/plugins/average.bro index 8f7f7b568f..160ca64d78 100644 --- a/scripts/base/frameworks/sumstats/plugins/average.bro +++ b/scripts/base/frameworks/sumstats/plugins/average.bro @@ -1,3 +1,5 @@ +##! Calculate the average. + @load ../main module SumStats; @@ -9,7 +11,7 @@ export { }; redef record ResultVal += { - ## For numeric data, this calculates the average of all values. + ## For numeric data, this is the average of all values. average: double &optional; }; } diff --git a/scripts/base/frameworks/sumstats/plugins/hll_unique.bro b/scripts/base/frameworks/sumstats/plugins/hll_unique.bro index 494cbf4667..43cafcff7f 100644 --- a/scripts/base/frameworks/sumstats/plugins/hll_unique.bro +++ b/scripts/base/frameworks/sumstats/plugins/hll_unique.bro @@ -1,3 +1,5 @@ +##! Calculate the number of unique values (using the HyperLogLog algorithm). + @load base/frameworks/sumstats module SumStats; diff --git a/scripts/base/frameworks/sumstats/plugins/last.bro b/scripts/base/frameworks/sumstats/plugins/last.bro index 430c2e375b..ca04114f61 100644 --- a/scripts/base/frameworks/sumstats/plugins/last.bro +++ b/scripts/base/frameworks/sumstats/plugins/last.bro @@ -1,3 +1,5 @@ +##! Keep the last X observations. + @load base/frameworks/sumstats @load base/utils/queue diff --git a/scripts/base/frameworks/sumstats/plugins/max.bro b/scripts/base/frameworks/sumstats/plugins/max.bro index d43ad9dc38..adcc6ae113 100644 --- a/scripts/base/frameworks/sumstats/plugins/max.bro +++ b/scripts/base/frameworks/sumstats/plugins/max.bro @@ -1,3 +1,5 @@ +##! Find the maximum value. + @load ../main module SumStats; @@ -9,7 +11,7 @@ export { }; redef record ResultVal += { - ## For numeric data, this tracks the maximum value given. + ## For numeric data, this tracks the maximum value. max: double &optional; }; } diff --git a/scripts/base/frameworks/sumstats/plugins/min.bro b/scripts/base/frameworks/sumstats/plugins/min.bro index 014755cf32..22cab1009c 100644 --- a/scripts/base/frameworks/sumstats/plugins/min.bro +++ b/scripts/base/frameworks/sumstats/plugins/min.bro @@ -1,3 +1,5 @@ +##! Find the minimum value. + @load ../main module SumStats; @@ -9,7 +11,7 @@ export { }; redef record ResultVal += { - ## For numeric data, this tracks the minimum value given. + ## For numeric data, this tracks the minimum value. min: double &optional; }; } diff --git a/scripts/base/frameworks/sumstats/plugins/sample.bro b/scripts/base/frameworks/sumstats/plugins/sample.bro index 809d696896..0200e85949 100644 --- a/scripts/base/frameworks/sumstats/plugins/sample.bro +++ b/scripts/base/frameworks/sumstats/plugins/sample.bro @@ -1,3 +1,5 @@ +##! Keep a random sample of values. + @load base/frameworks/sumstats/main module SumStats; @@ -10,7 +12,7 @@ export { }; redef record Reducer += { - ## A number of sample Observations to collect. + ## The number of sample Observations to collect. num_samples: count &default=0; }; diff --git a/scripts/base/frameworks/sumstats/plugins/std-dev.bro b/scripts/base/frameworks/sumstats/plugins/std-dev.bro index 2e5b95b212..bfb02c82cc 100644 --- a/scripts/base/frameworks/sumstats/plugins/std-dev.bro +++ b/scripts/base/frameworks/sumstats/plugins/std-dev.bro @@ -1,3 +1,5 @@ +##! Calculate the standard deviation. + @load ./variance @load ../main @@ -5,7 +7,7 @@ module SumStats; export { redef enum Calculation += { - ## Find the standard deviation of the values. + ## Calculate the standard deviation of the values. STD_DEV }; diff --git a/scripts/base/frameworks/sumstats/plugins/sum.bro b/scripts/base/frameworks/sumstats/plugins/sum.bro index 074b4b72f3..fb1d96bcd4 100644 --- a/scripts/base/frameworks/sumstats/plugins/sum.bro +++ b/scripts/base/frameworks/sumstats/plugins/sum.bro @@ -1,11 +1,13 @@ +##! Calculate the sum. + @load ../main module SumStats; export { redef enum Calculation += { - ## Sums the values given. For string values, - ## this will be the number of strings given. + ## Calculate the sum of the values. For string values, + ## this will be the number of strings. SUM }; diff --git a/scripts/base/frameworks/sumstats/plugins/topk.bro b/scripts/base/frameworks/sumstats/plugins/topk.bro index 0ef0f01393..e7107cb4fb 100644 --- a/scripts/base/frameworks/sumstats/plugins/topk.bro +++ b/scripts/base/frameworks/sumstats/plugins/topk.bro @@ -1,3 +1,5 @@ +##! Keep the top-k (i.e., most frequently occurring) observations. + @load base/frameworks/sumstats module SumStats; @@ -9,10 +11,13 @@ export { }; redef enum Calculation += { + ## Keep a top-k list of values. TOPK }; redef record ResultVal += { + ## A handle which can be passed to some built-in functions to get + ## the top-k results. topk: opaque of topk &optional; }; diff --git a/scripts/base/frameworks/sumstats/plugins/unique.bro b/scripts/base/frameworks/sumstats/plugins/unique.bro index abfbe3669d..5fcaa1dc3c 100644 --- a/scripts/base/frameworks/sumstats/plugins/unique.bro +++ b/scripts/base/frameworks/sumstats/plugins/unique.bro @@ -1,10 +1,12 @@ +##! Calculate the number of unique values. + @load ../main module SumStats; export { redef record Reducer += { - ## Maximum number of unique elements to store. + ## Maximum number of unique values to store. unique_max: count &optional; }; @@ -15,7 +17,7 @@ export { redef record ResultVal += { ## If cardinality is being tracked, the number of unique - ## items is tracked here. + ## values is tracked here. unique: count &default=0; }; } diff --git a/scripts/base/frameworks/sumstats/plugins/variance.bro b/scripts/base/frameworks/sumstats/plugins/variance.bro index 12d30cc4fe..989bf07eaf 100644 --- a/scripts/base/frameworks/sumstats/plugins/variance.bro +++ b/scripts/base/frameworks/sumstats/plugins/variance.bro @@ -1,3 +1,5 @@ +##! Calculate the variance. + @load ./average @load ../main @@ -5,12 +7,12 @@ module SumStats; export { redef enum Calculation += { - ## Find the variance of the values. + ## Calculate the variance of the values. VARIANCE }; redef record ResultVal += { - ## For numeric data, this calculates the variance. + ## For numeric data, this is the variance. variance: double &optional; }; } diff --git a/scripts/base/init-bare.bro b/scripts/base/init-bare.bro index 77db2ed761..c79fac5498 100644 --- a/scripts/base/init-bare.bro +++ b/scripts/base/init-bare.bro @@ -474,64 +474,127 @@ type NetStats: record { bytes_recvd: count &default=0; ##< Bytes received by Bro. }; -## Statistics about Bro's resource consumption. +type ConnStats: record { + total_conns: count; ##< + current_conns: count; ##< + current_conns_extern: count; ##< + sess_current_conns: count; ##< + + num_packets: count; + num_fragments: count; + max_fragments: count; + + num_tcp_conns: count; ##< Current number of TCP connections in memory. + max_tcp_conns: count; ##< Maximum number of concurrent TCP connections so far. + cumulative_tcp_conns: count; ##< Total number of TCP connections so far. + + num_udp_conns: count; ##< Current number of UDP flows in memory. + max_udp_conns: count; ##< Maximum number of concurrent UDP flows so far. + cumulative_udp_conns: count; ##< Total number of UDP flows so far. + + num_icmp_conns: count; ##< Current number of ICMP flows in memory. + max_icmp_conns: count; ##< Maximum number of concurrent ICMP flows so far. + cumulative_icmp_conns: count; ##< Total number of ICMP flows so far. + + killed_by_inactivity: count; +}; + +## Statistics about Bro's process. ## -## .. bro:see:: resource_usage +## .. bro:see:: get_proc_stats ## ## .. note:: All process-level values refer to Bro's main process only, not to ## the child process it spawns for doing communication. -type bro_resources: record { - version: string; ##< Bro version string. - debug: bool; ##< True if compiled with --enable-debug. - start_time: time; ##< Start time of process. - real_time: interval; ##< Elapsed real time since Bro started running. - user_time: interval; ##< User CPU seconds. - system_time: interval; ##< System CPU seconds. - mem: count; ##< Maximum memory consumed, in KB. - minor_faults: count; ##< Page faults not requiring actual I/O. - major_faults: count; ##< Page faults requiring actual I/O. - num_swap: count; ##< Times swapped out. - blocking_input: count; ##< Blocking input operations. - blocking_output: count; ##< Blocking output operations. - num_context: count; ##< Number of involuntary context switches. +type ProcStats: record { + debug: bool; ##< True if compiled with --enable-debug. + start_time: time; ##< Start time of process. + real_time: interval; ##< Elapsed real time since Bro started running. + user_time: interval; ##< User CPU seconds. + system_time: interval; ##< System CPU seconds. + mem: count; ##< Maximum memory consumed, in KB. + minor_faults: count; ##< Page faults not requiring actual I/O. + major_faults: count; ##< Page faults requiring actual I/O. + num_swap: count; ##< Times swapped out. + blocking_input: count; ##< Blocking input operations. + blocking_output: count; ##< Blocking output operations. + num_context: count; ##< Number of involuntary context switches. +}; - num_TCP_conns: count; ##< Current number of TCP connections in memory. - num_UDP_conns: count; ##< Current number of UDP flows in memory. - num_ICMP_conns: count; ##< Current number of ICMP flows in memory. - num_fragments: count; ##< Current number of fragments pending reassembly. - num_packets: count; ##< Total number of packets processed to date. - num_timers: count; ##< Current number of pending timers. - num_events_queued: count; ##< Total number of events queued so far. - num_events_dispatched: count; ##< Total number of events dispatched so far. - - max_TCP_conns: count; ##< Maximum number of concurrent TCP connections so far. - max_UDP_conns: count; ##< Maximum number of concurrent UDP connections so far. - max_ICMP_conns: count; ##< Maximum number of concurrent ICMP connections so far. - max_fragments: count; ##< Maximum number of concurrently buffered fragments so far. - max_timers: count; ##< Maximum number of concurrent timers pending so far. +type EventStats: record { + queued: count; ##< Total number of events queued so far. + dispatched: count; ##< Total number of events dispatched so far. }; ## Summary statistics of all regular expression matchers. ## +## .. bro:see:: get_reassembler_stats +type ReassemblerStats: record { + file_size: count; ##< Byte size of File reassembly tracking. + frag_size: count; ##< Byte size of Fragment reassembly tracking. + tcp_size: count; ##< Byte size of TCP reassembly tracking. + unknown_size: count; ##< Byte size of reassembly tracking for unknown purposes. +}; + +## Statistics of all regular expression matchers. +## ## .. bro:see:: get_matcher_stats -type matcher_stats: record { - matchers: count; ##< Number of distinct RE matchers. - dfa_states: count; ##< Number of DFA states across all matchers. - computed: count; ##< Number of computed DFA state transitions. - mem: count; ##< Number of bytes used by DFA states. - hits: count; ##< Number of cache hits. - misses: count; ##< Number of cache misses. - avg_nfa_states: count; ##< Average number of NFA states across all matchers. +type MatcherStats: record { + matchers: count; ##< Number of distinct RE matchers. + nfa_states: count; ##< Number of NFA states across all matchers. + dfa_states: count; ##< Number of DFA states across all matchers. + computed: count; ##< Number of computed DFA state transitions. + mem: count; ##< Number of bytes used by DFA states. + hits: count; ##< Number of cache hits. + misses: count; ##< Number of cache misses. +}; + +## Statistics of timers. +## +## .. bro:see:: get_timer_stats +type TimerStats: record { + current: count; ##< Current number of pending timers. + max: count; ##< Maximum number of concurrent timers pending so far. + cumulative: count; ##< Cumulative number of timers scheduled. +}; + +## Statistics of file analysis. +## +## .. bro:see:: get_file_analysis_stats +type FileAnalysisStats: record { + current: count; ##< Current number of files being analyzed. + max: count; ##< Maximum number of concurrent files so far. + cumulative: count; ##< Cumulative number of files analyzed. +}; + +## Statistics related to Bro's active use of DNS. These numbers are +## about Bro performing DNS queries on it's own, not traffic +## being seen. +## +## .. bro:see:: get_dns_stats +type DNSStats: record { + requests: count; ##< Number of DNS requests made + successful: count; ##< Number of successful DNS replies. + failed: count; ##< Number of DNS reply failures. + pending: count; ##< Current pending queries. + cached_hosts: count; ##< Number of cached hosts. + cached_addresses: count; ##< Number of cached addresses. }; ## Statistics about number of gaps in TCP connections. ## -## .. bro:see:: gap_report get_gap_summary -type gap_info: record { - ack_events: count; ##< How many ack events *could* have had gaps. - ack_bytes: count; ##< How many bytes those covered. - gap_events: count; ##< How many *did* have gaps. - gap_bytes: count; ##< How many bytes were missing in the gaps. +## .. bro:see:: get_gap_stats +type GapStats: record { + ack_events: count; ##< How many ack events *could* have had gaps. + ack_bytes: count; ##< How many bytes those covered. + gap_events: count; ##< How many *did* have gaps. + gap_bytes: count; ##< How many bytes were missing in the gaps. +}; + +## Statistics about threads. +## +## .. bro:see:: get_thread_stats +type ThreadStats: record { + num_threads: count; }; ## Deprecated. @@ -793,71 +856,6 @@ type entropy_test_result: record { serial_correlation: double; ##< Serial correlation coefficient. }; -# Prototypes of Bro built-in functions. -@load base/bif/strings.bif -@load base/bif/bro.bif -@load base/bif/reporter.bif - -## Deprecated. This is superseded by the new logging framework. -global log_file_name: function(tag: string): string &redef; - -## Deprecated. This is superseded by the new logging framework. -global open_log_file: function(tag: string): file &redef; - -## Specifies a directory for Bro to store its persistent state. All globals can -## be declared persistent via the :bro:attr:`&persistent` attribute. -const state_dir = ".state" &redef; - -## Length of the delays inserted when storing state incrementally. To avoid -## dropping packets when serializing larger volumes of persistent state to -## disk, Bro interleaves the operation with continued packet processing. -const state_write_delay = 0.01 secs &redef; - -global done_with_network = F; -event net_done(t: time) { done_with_network = T; } - -function log_file_name(tag: string): string - { - local suffix = getenv("BRO_LOG_SUFFIX") == "" ? "log" : getenv("BRO_LOG_SUFFIX"); - return fmt("%s.%s", tag, suffix); - } - -function open_log_file(tag: string): file - { - return open(log_file_name(tag)); - } - -## Internal function. -function add_interface(iold: string, inew: string): string - { - if ( iold == "" ) - return inew; - else - return fmt("%s %s", iold, inew); - } - -## Network interfaces to listen on. Use ``redef interfaces += "eth0"`` to -## extend. -global interfaces = "" &add_func = add_interface; - -## Internal function. -function add_signature_file(sold: string, snew: string): string - { - if ( sold == "" ) - return snew; - else - return cat(sold, " ", snew); - } - -## Signature files to read. Use ``redef signature_files += "foo.sig"`` to -## extend. Signature files added this way will be searched relative to -## ``BROPATH``. Using the ``@load-sigs`` directive instead is preferred -## since that can search paths relative to the current script. -global signature_files = "" &add_func = add_signature_file; - -## ``p0f`` fingerprint file to use. Will be searched relative to ``BROPATH``. -const passive_fingerprint_file = "base/misc/p0f.fp" &redef; - # TCP values for :bro:see:`endpoint` *state* field. # todo:: these should go into an enum to make them autodoc'able. const TCP_INACTIVE = 0; ##< Endpoint is still inactive. @@ -1768,6 +1766,71 @@ type gtp_delete_pdp_ctx_response_elements: record { ext: gtp_private_extension &optional; }; +# Prototypes of Bro built-in functions. +@load base/bif/strings.bif +@load base/bif/bro.bif +@load base/bif/reporter.bif + +## Deprecated. This is superseded by the new logging framework. +global log_file_name: function(tag: string): string &redef; + +## Deprecated. This is superseded by the new logging framework. +global open_log_file: function(tag: string): file &redef; + +## Specifies a directory for Bro to store its persistent state. All globals can +## be declared persistent via the :bro:attr:`&persistent` attribute. +const state_dir = ".state" &redef; + +## Length of the delays inserted when storing state incrementally. To avoid +## dropping packets when serializing larger volumes of persistent state to +## disk, Bro interleaves the operation with continued packet processing. +const state_write_delay = 0.01 secs &redef; + +global done_with_network = F; +event net_done(t: time) { done_with_network = T; } + +function log_file_name(tag: string): string + { + local suffix = getenv("BRO_LOG_SUFFIX") == "" ? "log" : getenv("BRO_LOG_SUFFIX"); + return fmt("%s.%s", tag, suffix); + } + +function open_log_file(tag: string): file + { + return open(log_file_name(tag)); + } + +## Internal function. +function add_interface(iold: string, inew: string): string + { + if ( iold == "" ) + return inew; + else + return fmt("%s %s", iold, inew); + } + +## Network interfaces to listen on. Use ``redef interfaces += "eth0"`` to +## extend. +global interfaces = "" &add_func = add_interface; + +## Internal function. +function add_signature_file(sold: string, snew: string): string + { + if ( sold == "" ) + return snew; + else + return cat(sold, " ", snew); + } + +## Signature files to read. Use ``redef signature_files += "foo.sig"`` to +## extend. Signature files added this way will be searched relative to +## ``BROPATH``. Using the ``@load-sigs`` directive instead is preferred +## since that can search paths relative to the current script. +global signature_files = "" &add_func = add_signature_file; + +## ``p0f`` fingerprint file to use. Will be searched relative to ``BROPATH``. +const passive_fingerprint_file = "base/misc/p0f.fp" &redef; + ## Definition of "secondary filters". A secondary filter is a BPF filter given ## as index in this table. For each such filter, the corresponding event is ## raised for all matching packets. @@ -3860,23 +3923,17 @@ global pkt_profile_file: file &redef; ## .. bro:see:: load_sample global load_sample_freq = 20 &redef; -## Rate at which to generate :bro:see:`gap_report` events assessing to what -## degree the measurement process appears to exhibit loss. -## -## .. bro:see:: gap_report -const gap_report_freq = 1.0 sec &redef; - ## Whether to attempt to automatically detect SYN/FIN/RST-filtered trace ## and not report missing segments for such connections. ## If this is enabled, then missing data at the end of connections may not ## be reported via :bro:see:`content_gap`. const detect_filtered_trace = F &redef; -## Whether we want :bro:see:`content_gap` and :bro:see:`gap_report` for partial +## Whether we want :bro:see:`content_gap` and :bro:see:`get_gap_summary` for partial ## connections. A connection is partial if it is missing a full handshake. Note ## that gap reports for partial connections might not be reliable. ## -## .. bro:see:: content_gap gap_report partial_connection +## .. bro:see:: content_gap get_gap_summary partial_connection const report_gaps_for_partial = F &redef; ## Flag to prevent Bro from exiting automatically when input is exhausted. diff --git a/scripts/base/init-default.bro b/scripts/base/init-default.bro index 3aeaea5e02..d0ee2238fa 100644 --- a/scripts/base/init-default.bro +++ b/scripts/base/init-default.bro @@ -37,10 +37,8 @@ @load base/frameworks/reporter @load base/frameworks/sumstats @load base/frameworks/tunnels -@ifdef ( BrokerComm::enable ) @load base/frameworks/openflow @load base/frameworks/netcontrol -@endif @load base/protocols/conn @load base/protocols/dce-rpc @@ -49,6 +47,7 @@ @load base/protocols/dns @load base/protocols/ftp @load base/protocols/http +@load base/protocols/imap @load base/protocols/irc @load base/protocols/krb @load base/protocols/modbus @@ -67,6 +66,7 @@ @load base/protocols/ssl @load base/protocols/syslog @load base/protocols/tunnels +@load base/protocols/xmpp @load base/files/pe @load base/files/hash diff --git a/scripts/base/misc/find-checksum-offloading.bro b/scripts/base/misc/find-checksum-offloading.bro index fae017fff1..334cf4a2db 100644 --- a/scripts/base/misc/find-checksum-offloading.bro +++ b/scripts/base/misc/find-checksum-offloading.bro @@ -26,7 +26,7 @@ event ChecksumOffloading::check() if ( done ) return; - local pkts_recvd = net_stats()$pkts_recvd; + local pkts_recvd = get_net_stats()$pkts_recvd; local bad_ip_checksum_pct = (pkts_recvd != 0) ? (bad_ip_checksums*1.0 / pkts_recvd*1.0) : 0; local bad_tcp_checksum_pct = (pkts_recvd != 0) ? (bad_tcp_checksums*1.0 / pkts_recvd*1.0) : 0; local bad_udp_checksum_pct = (pkts_recvd != 0) ? (bad_udp_checksums*1.0 / pkts_recvd*1.0) : 0; diff --git a/scripts/base/protocols/dns/consts.bro b/scripts/base/protocols/dns/consts.bro index 13af6c3e81..026588f777 100644 --- a/scripts/base/protocols/dns/consts.bro +++ b/scripts/base/protocols/dns/consts.bro @@ -26,6 +26,7 @@ export { [49] = "DHCID", [99] = "SPF", [100] = "DINFO", [101] = "UID", [102] = "GID", [103] = "UNSPEC", [249] = "TKEY", [250] = "TSIG", [251] = "IXFR", [252] = "AXFR", [253] = "MAILB", [254] = "MAILA", + [257] = "CAA", [32768] = "TA", [32769] = "DLV", [ANY] = "*", } &default = function(n: count): string { return fmt("query-%d", n); }; diff --git a/scripts/base/protocols/dns/main.bro b/scripts/base/protocols/dns/main.bro index 58a63293d0..05a44a0ba9 100644 --- a/scripts/base/protocols/dns/main.bro +++ b/scripts/base/protocols/dns/main.bro @@ -52,7 +52,7 @@ export { ## The Recursion Available bit in a response message indicates ## that the name server supports recursive queries. RA: bool &log &default=F; - ## A reserved field that is currently supposed to be zero in all + ## A reserved field that is usually zero in ## queries and responses. Z: count &log &default=0; ## The set of resource descriptions in the query answer. diff --git a/scripts/base/protocols/http/main.bro b/scripts/base/protocols/http/main.bro index e70d166f11..2988a1a646 100644 --- a/scripts/base/protocols/http/main.bro +++ b/scripts/base/protocols/http/main.bro @@ -21,6 +21,7 @@ export { ## not. const default_capture_password = F &redef; + ## The record type which contains the fields of the HTTP log. type Info: record { ## Timestamp for when the request happened. ts: time &log; diff --git a/scripts/base/protocols/imap/README b/scripts/base/protocols/imap/README new file mode 100644 index 0000000000..ba96748489 --- /dev/null +++ b/scripts/base/protocols/imap/README @@ -0,0 +1,5 @@ +Support for the Internet Message Access Protocol (IMAP). + +Note that currently the IMAP analyzer only supports analyzing IMAP sessions +until they do or do not switch to TLS using StartTLS. Hence, we do not get +mails from IMAP sessions, only X509 certificates. diff --git a/scripts/base/protocols/imap/__load__.bro b/scripts/base/protocols/imap/__load__.bro new file mode 100644 index 0000000000..aa3a41ef5e --- /dev/null +++ b/scripts/base/protocols/imap/__load__.bro @@ -0,0 +1,2 @@ +@load ./main + diff --git a/scripts/base/protocols/imap/main.bro b/scripts/base/protocols/imap/main.bro new file mode 100644 index 0000000000..9f0305c80c --- /dev/null +++ b/scripts/base/protocols/imap/main.bro @@ -0,0 +1,11 @@ + +module IMAP; + +const ports = { 143/tcp }; +redef likely_server_ports += { ports }; + +event bro_init() &priority=5 + { + Analyzer::register_for_ports(Analyzer::ANALYZER_IMAP, ports); + } + diff --git a/scripts/base/protocols/rfb/main.bro b/scripts/base/protocols/rfb/main.bro index 03e39a40f9..3bcb86890b 100644 --- a/scripts/base/protocols/rfb/main.bro +++ b/scripts/base/protocols/rfb/main.bro @@ -3,6 +3,7 @@ module RFB; export { redef enum Log::ID += { LOG }; + ## The record type which contains the fields of the RFB log. type Info: record { ## Timestamp for when the event happened. ts: time &log; diff --git a/scripts/base/protocols/sip/main.bro b/scripts/base/protocols/sip/main.bro index dc790ad560..f629049928 100644 --- a/scripts/base/protocols/sip/main.bro +++ b/scripts/base/protocols/sip/main.bro @@ -10,6 +10,7 @@ module SIP; export { redef enum Log::ID += { LOG }; + ## The record type which contains the fields of the SIP log. type Info: record { ## Timestamp for when the request happened. ts: time &log; diff --git a/scripts/base/protocols/smtp/main.bro b/scripts/base/protocols/smtp/main.bro index 6df9bddb54..766c0850bc 100644 --- a/scripts/base/protocols/smtp/main.bro +++ b/scripts/base/protocols/smtp/main.bro @@ -7,6 +7,7 @@ module SMTP; export { redef enum Log::ID += { LOG }; + ## The record type which contains the fields of the SMTP log. type Info: record { ## Time when the message was first seen. ts: time &log; diff --git a/scripts/base/protocols/socks/main.bro b/scripts/base/protocols/socks/main.bro index c63092f609..e22ed718c6 100644 --- a/scripts/base/protocols/socks/main.bro +++ b/scripts/base/protocols/socks/main.bro @@ -6,6 +6,7 @@ module SOCKS; export { redef enum Log::ID += { LOG }; + ## The record type which contains the fields of the SOCKS log. type Info: record { ## Time when the proxy connection was first detected. ts: time &log; diff --git a/scripts/base/protocols/ssh/main.bro b/scripts/base/protocols/ssh/main.bro index fad2da0b8e..d547e92e8f 100644 --- a/scripts/base/protocols/ssh/main.bro +++ b/scripts/base/protocols/ssh/main.bro @@ -8,6 +8,7 @@ export { ## The SSH protocol logging stream identifier. redef enum Log::ID += { LOG }; + ## The record type which contains the fields of the SSH log. type Info: record { ## Time when the SSH connection began. ts: time &log; diff --git a/scripts/base/protocols/ssl/main.bro b/scripts/base/protocols/ssl/main.bro index 8483f473f4..4c61df916a 100644 --- a/scripts/base/protocols/ssl/main.bro +++ b/scripts/base/protocols/ssl/main.bro @@ -8,6 +8,7 @@ module SSL; export { redef enum Log::ID += { LOG }; + ## The record type which contains the fields of the SSL log. type Info: record { ## Time when the SSL connection was first detected. ts: time &log; diff --git a/scripts/base/protocols/syslog/main.bro b/scripts/base/protocols/syslog/main.bro index 593c8ab9a2..6e74760225 100644 --- a/scripts/base/protocols/syslog/main.bro +++ b/scripts/base/protocols/syslog/main.bro @@ -7,7 +7,8 @@ module Syslog; export { redef enum Log::ID += { LOG }; - + + ## The record type which contains the fields of the syslog log. type Info: record { ## Timestamp when the syslog message was seen. ts: time &log; diff --git a/scripts/base/protocols/xmpp/README b/scripts/base/protocols/xmpp/README new file mode 100644 index 0000000000..3d2194ef3d --- /dev/null +++ b/scripts/base/protocols/xmpp/README @@ -0,0 +1,5 @@ +Support for the Extensible Messaging and Presence Protocol (XMPP). + +Note that currently the XMPP analyzer only supports analyzing XMPP sessions +until they do or do not switch to TLS using StartTLS. Hence, we do not get +actual chat information from XMPP sessions, only X509 certificates. diff --git a/scripts/base/protocols/xmpp/__load__.bro b/scripts/base/protocols/xmpp/__load__.bro new file mode 100644 index 0000000000..0f41578f8a --- /dev/null +++ b/scripts/base/protocols/xmpp/__load__.bro @@ -0,0 +1,3 @@ +@load ./main + +@load-sigs ./dpd.sig diff --git a/scripts/base/protocols/xmpp/dpd.sig b/scripts/base/protocols/xmpp/dpd.sig new file mode 100644 index 0000000000..50ae57a669 --- /dev/null +++ b/scripts/base/protocols/xmpp/dpd.sig @@ -0,0 +1,5 @@ +signature dpd_xmpp { + ip-proto == tcp + payload /^(<\?xml[^?>]*\?>)?[\n\r ]*]*xmlns='jabber:/ + enable "xmpp" +} diff --git a/scripts/base/protocols/xmpp/main.bro b/scripts/base/protocols/xmpp/main.bro new file mode 100644 index 0000000000..3d7a4cbc37 --- /dev/null +++ b/scripts/base/protocols/xmpp/main.bro @@ -0,0 +1,11 @@ + +module XMPP; + +const ports = { 5222/tcp, 5269/tcp }; +redef likely_server_ports += { ports }; + +event bro_init() &priority=5 + { + Analyzer::register_for_ports(Analyzer::ANALYZER_XMPP, ports); + } + diff --git a/scripts/policy/frameworks/control/controllee.bro b/scripts/policy/frameworks/control/controllee.bro index b4769764f4..9646d100ab 100644 --- a/scripts/policy/frameworks/control/controllee.bro +++ b/scripts/policy/frameworks/control/controllee.bro @@ -28,13 +28,9 @@ event Control::peer_status_request() local peer = Communication::nodes[p]; if ( ! peer$connected ) next; - - local res = resource_usage(); - status += fmt("%.6f peer=%s host=%s events_in=%s events_out=%s ops_in=%s ops_out=%s bytes_in=? bytes_out=?\n", - network_time(), - peer$peer$descr, peer$host, - res$num_events_queued, res$num_events_dispatched, - res$blocking_input, res$blocking_output); + + status += fmt("%.6f peer=%s host=%s\n", + network_time(), peer$peer$descr, peer$host); } event Control::peer_status_response(status); @@ -42,24 +38,24 @@ event Control::peer_status_request() event Control::net_stats_request() { - local ns = net_stats(); - local reply = fmt("%.6f recvd=%d dropped=%d link=%d\n", network_time(), + local ns = get_net_stats(); + local reply = fmt("%.6f recvd=%d dropped=%d link=%d\n", network_time(), ns$pkts_recvd, ns$pkts_dropped, ns$pkts_link); event Control::net_stats_response(reply); } - + event Control::configuration_update_request() { - # Generate the alias event. + # Generate the alias event. event Control::configuration_update(); - + # Don't need to do anything in particular here, it's just indicating that # the configuration is going to be updated. This event could be handled - # by other scripts if they need to do some ancilliary processing if + # by other scripts if they need to do some ancilliary processing if # redef-able consts are modified at runtime. event Control::configuration_update_response(); } - + event Control::shutdown_request() { # Send the acknowledgement event. diff --git a/scripts/policy/frameworks/files/entropy-test-all-files.bro b/scripts/policy/frameworks/files/entropy-test-all-files.bro index fd02b9ecaa..9c704211f8 100644 --- a/scripts/policy/frameworks/files/entropy-test-all-files.bro +++ b/scripts/policy/frameworks/files/entropy-test-all-files.bro @@ -17,4 +17,4 @@ event file_new(f: fa_file) event file_entropy(f: fa_file, ent: entropy_test_result) { f$info$entropy = ent$entropy; - } \ No newline at end of file + } diff --git a/scripts/policy/frameworks/intel/seen/ssl.bro b/scripts/policy/frameworks/intel/seen/ssl.bro index 7bfbef4e9b..89aebc1891 100644 --- a/scripts/policy/frameworks/intel/seen/ssl.bro +++ b/scripts/policy/frameworks/intel/seen/ssl.bro @@ -20,6 +20,7 @@ event ssl_established(c: connection) if ( c$ssl$cert_chain[0]$x509?$certificate && c$ssl$cert_chain[0]$x509$certificate?$cn ) Intel::seen([$indicator=c$ssl$cert_chain[0]$x509$certificate$cn, $indicator_type=Intel::DOMAIN, + $fuid=c$ssl$cert_chain_fuids[0], $conn=c, $where=X509::IN_CERT]); } diff --git a/scripts/policy/frameworks/intel/seen/x509.bro b/scripts/policy/frameworks/intel/seen/x509.bro index 3a2859b6d5..9dcbc3edb9 100644 --- a/scripts/policy/frameworks/intel/seen/x509.bro +++ b/scripts/policy/frameworks/intel/seen/x509.bro @@ -26,3 +26,14 @@ event x509_certificate(f: fa_file, cert_ref: opaque of x509, cert: X509::Certifi $where=X509::IN_CERT]); } } + +event file_hash(f: fa_file, kind: string, hash: string) + { + if ( ! f?$info || ! f$info?$x509 || kind != "sha1" ) + return; + + Intel::seen([$indicator=hash, + $indicator_type=Intel::CERT_HASH, + $f=f, + $where=X509::IN_CERT]); + } diff --git a/scripts/policy/misc/capture-loss.bro b/scripts/policy/misc/capture-loss.bro index 28f468a1c8..648e3d6717 100644 --- a/scripts/policy/misc/capture-loss.bro +++ b/scripts/policy/misc/capture-loss.bro @@ -56,7 +56,7 @@ event CaptureLoss::take_measurement(last_ts: time, last_acks: count, last_gaps: } local now = network_time(); - local g = get_gap_summary(); + local g = get_gap_stats(); local acks = g$ack_events - last_acks; local gaps = g$gap_events - last_gaps; local pct_lost = (acks == 0) ? 0.0 : (100 * (1.0 * gaps) / (1.0 * acks)); diff --git a/scripts/policy/misc/stats.bro b/scripts/policy/misc/stats.bro index 215a3bb9de..4dee0d4128 100644 --- a/scripts/policy/misc/stats.bro +++ b/scripts/policy/misc/stats.bro @@ -1,6 +1,4 @@ -##! Log memory/packet/lag statistics. Differs from -##! :doc:`/scripts/policy/misc/profiling.bro` in that this -##! is lighter-weight (much less info, and less load to generate). +##! Log memory/packet/lag statistics. @load base/frameworks/notice @@ -10,7 +8,7 @@ export { redef enum Log::ID += { LOG }; ## How often stats are reported. - const stats_report_interval = 1min &redef; + const report_interval = 5min &redef; type Info: record { ## Timestamp for the measurement. @@ -21,27 +19,63 @@ export { mem: count &log; ## Number of packets processed since the last stats interval. pkts_proc: count &log; - ## Number of events processed since the last stats interval. - events_proc: count &log; - ## Number of events that have been queued since the last stats - ## interval. - events_queued: count &log; - - ## Lag between the wall clock and packet timestamps if reading - ## live traffic. - lag: interval &log &optional; - ## Number of packets received since the last stats interval if + ## Number of bytes received since the last stats interval if ## reading live traffic. - pkts_recv: count &log &optional; + bytes_recv: count &log; + ## Number of packets dropped since the last stats interval if ## reading live traffic. pkts_dropped: count &log &optional; ## Number of packets seen on the link since the last stats ## interval if reading live traffic. pkts_link: count &log &optional; - ## Number of bytes received since the last stats interval if - ## reading live traffic. - bytes_recv: count &log &optional; + ## Lag between the wall clock and packet timestamps if reading + ## live traffic. + pkt_lag: interval &log &optional; + + ## Number of events processed since the last stats interval. + events_proc: count &log; + ## Number of events that have been queued since the last stats + ## interval. + events_queued: count &log; + + ## TCP connections currently in memory. + active_tcp_conns: count &log; + ## UDP connections currently in memory. + active_udp_conns: count &log; + ## ICMP connections currently in memory. + active_icmp_conns: count &log; + + ## TCP connections seen since last stats interval. + tcp_conns: count &log; + ## UDP connections seen since last stats interval. + udp_conns: count &log; + ## ICMP connections seen since last stats interval. + icmp_conns: count &log; + + ## Number of timers scheduled since last stats interval. + timers: count &log; + ## Current number of scheduled timers. + active_timers: count &log; + + ## Number of files seen since last stats interval. + files: count &log; + ## Current number of files actively being seen. + active_files: count &log; + + ## Number of DNS requests seen since last stats interval. + dns_requests: count &log; + ## Current number of DNS requests awaiting a reply. + active_dns_requests: count &log; + + ## Current size of TCP data in reassembly. + reassem_tcp_size: count &log; + ## Current size of File data in reassembly. + reassem_file_size: count &log; + ## Current size of packet fragment data in reassembly. + reassem_frag_size: count &log; + ## Current size of unkown data in reassembly (this is only PIA buffer right now). + reassem_unknown_size: count &log; }; ## Event to catch stats as they are written to the logging stream. @@ -53,38 +87,69 @@ event bro_init() &priority=5 Log::create_stream(Stats::LOG, [$columns=Info, $ev=log_stats, $path="stats"]); } -event check_stats(last_ts: time, last_ns: NetStats, last_res: bro_resources) +event check_stats(then: time, last_ns: NetStats, last_cs: ConnStats, last_ps: ProcStats, last_es: EventStats, last_rs: ReassemblerStats, last_ts: TimerStats, last_fs: FileAnalysisStats, last_ds: DNSStats) { - local now = current_time(); - local ns = net_stats(); - local res = resource_usage(); + local nettime = network_time(); + local ns = get_net_stats(); + local cs = get_conn_stats(); + local ps = get_proc_stats(); + local es = get_event_stats(); + local rs = get_reassembler_stats(); + local ts = get_timer_stats(); + local fs = get_file_analysis_stats(); + local ds = get_dns_stats(); if ( bro_is_terminating() ) # No more stats will be written or scheduled when Bro is # shutting down. return; - local info: Info = [$ts=now, $peer=peer_description, $mem=res$mem/1000000, - $pkts_proc=res$num_packets - last_res$num_packets, - $events_proc=res$num_events_dispatched - last_res$num_events_dispatched, - $events_queued=res$num_events_queued - last_res$num_events_queued]; + local info: Info = [$ts=nettime, + $peer=peer_description, + $mem=ps$mem/1048576, + $pkts_proc=ns$pkts_recvd - last_ns$pkts_recvd, + $bytes_recv = ns$bytes_recvd - last_ns$bytes_recvd, + + $active_tcp_conns=cs$num_tcp_conns, + $tcp_conns=cs$cumulative_tcp_conns - last_cs$cumulative_tcp_conns, + $active_udp_conns=cs$num_udp_conns, + $udp_conns=cs$cumulative_udp_conns - last_cs$cumulative_udp_conns, + $active_icmp_conns=cs$num_icmp_conns, + $icmp_conns=cs$cumulative_icmp_conns - last_cs$cumulative_icmp_conns, + + $reassem_tcp_size=rs$tcp_size, + $reassem_file_size=rs$file_size, + $reassem_frag_size=rs$frag_size, + $reassem_unknown_size=rs$unknown_size, + + $events_proc=es$dispatched - last_es$dispatched, + $events_queued=es$queued - last_es$queued, + + $timers=ts$cumulative - last_ts$cumulative, + $active_timers=ts$current, + + $files=fs$cumulative - last_fs$cumulative, + $active_files=fs$current, + + $dns_requests=ds$requests - last_ds$requests, + $active_dns_requests=ds$pending + ]; + + # Someone's going to have to explain what this is and add a field to the Info record. + # info$util = 100.0*((ps$user_time + ps$system_time) - (last_ps$user_time + last_ps$system_time))/(now-then); if ( reading_live_traffic() ) { - info$lag = now - network_time(); - # Someone's going to have to explain what this is and add a field to the Info record. - # info$util = 100.0*((res$user_time + res$system_time) - (last_res$user_time + last_res$system_time))/(now-last_ts); - info$pkts_recv = ns$pkts_recvd - last_ns$pkts_recvd; + info$pkt_lag = current_time() - nettime; info$pkts_dropped = ns$pkts_dropped - last_ns$pkts_dropped; info$pkts_link = ns$pkts_link - last_ns$pkts_link; - info$bytes_recv = ns$bytes_recvd - last_ns$bytes_recvd; } Log::write(Stats::LOG, info); - schedule stats_report_interval { check_stats(now, ns, res) }; + schedule report_interval { check_stats(nettime, ns, cs, ps, es, rs, ts, fs, ds) }; } event bro_init() { - schedule stats_report_interval { check_stats(current_time(), net_stats(), resource_usage()) }; + schedule report_interval { check_stats(network_time(), get_net_stats(), get_conn_stats(), get_proc_stats(), get_event_stats(), get_reassembler_stats(), get_timer_stats(), get_file_analysis_stats(), get_dns_stats()) }; } diff --git a/src/CMakeLists.txt b/src/CMakeLists.txt index 9a807b3182..7b521125e4 100644 --- a/src/CMakeLists.txt +++ b/src/CMakeLists.txt @@ -118,6 +118,7 @@ include(BifCl) set(BIF_SRCS bro.bif + stats.bif event.bif const.bif types.bif diff --git a/src/Conn.cc b/src/Conn.cc index 3f6757d89c..1082230869 100644 --- a/src/Conn.cc +++ b/src/Conn.cc @@ -108,9 +108,9 @@ bool ConnectionTimer::DoUnserialize(UnserialInfo* info) return true; } -unsigned int Connection::total_connections = 0; -unsigned int Connection::current_connections = 0; -unsigned int Connection::external_connections = 0; +uint64 Connection::total_connections = 0; +uint64 Connection::current_connections = 0; +uint64 Connection::external_connections = 0; IMPLEMENT_SERIAL(Connection, SER_CONNECTION); diff --git a/src/Conn.h b/src/Conn.h index 11dbb11abe..bd12ddd041 100644 --- a/src/Conn.h +++ b/src/Conn.h @@ -220,11 +220,11 @@ public: unsigned int MemoryAllocation() const; unsigned int MemoryAllocationConnVal() const; - static unsigned int TotalConnections() + static uint64 TotalConnections() { return total_connections; } - static unsigned int CurrentConnections() + static uint64 CurrentConnections() { return current_connections; } - static unsigned int CurrentExternalConnections() + static uint64 CurrentExternalConnections() { return external_connections; } // Returns true if the history was already seen, false otherwise. @@ -315,9 +315,9 @@ protected: unsigned int saw_first_orig_packet:1, saw_first_resp_packet:1; // Count number of connections. - static unsigned int total_connections; - static unsigned int current_connections; - static unsigned int external_connections; + static uint64 total_connections; + static uint64 current_connections; + static uint64 external_connections; string history; uint32 hist_seen; diff --git a/src/DFA.cc b/src/DFA.cc index e7b2279ed5..5885a9bf3b 100644 --- a/src/DFA.cc +++ b/src/DFA.cc @@ -346,6 +346,7 @@ DFA_State* DFA_State_Cache::Lookup(const NFA_state_list& nfas, ++misses; return 0; } + ++hits; delete *hash; *hash = 0; @@ -433,19 +434,6 @@ void DFA_Machine::Dump(FILE* f) start_state->ClearMarks(); } -void DFA_Machine::DumpStats(FILE* f) - { - DFA_State_Cache::Stats stats; - dfa_state_cache->GetStats(&stats); - - fprintf(f, "Computed dfa_states = %d; Classes = %d; Computed trans. = %d; Uncomputed trans. = %d\n", - stats.dfa_states, EC()->NumClasses(), - stats.computed, stats.uncomputed); - - fprintf(f, "DFA cache hits = %d; misses = %d\n", - stats.hits, stats.misses); - } - unsigned int DFA_Machine::MemoryAllocation() const { DFA_State_Cache::Stats s; diff --git a/src/DFA.h b/src/DFA.h index 00cfdc3d39..a63beca9ac 100644 --- a/src/DFA.h +++ b/src/DFA.h @@ -89,10 +89,9 @@ public: int NumEntries() const { return states.Length(); } struct Stats { - unsigned int dfa_states; - - // Sum over all NFA states per DFA state. + // Sum of all NFA states unsigned int nfa_states; + unsigned int dfa_states; unsigned int computed; unsigned int uncomputed; unsigned int mem; @@ -132,7 +131,6 @@ public: void Describe(ODesc* d) const; void Dump(FILE* f); - void DumpStats(FILE* f); unsigned int MemoryAllocation() const; diff --git a/src/Dict.cc b/src/Dict.cc index 1d32eccde3..9e68d64089 100644 --- a/src/Dict.cc +++ b/src/Dict.cc @@ -66,6 +66,7 @@ Dictionary::Dictionary(dict_order ordering, int initial_size) delete_func = 0; tbl_next_ind = 0; + cumulative_entries = 0; num_buckets2 = num_entries2 = max_num_entries2 = thresh_entries2 = 0; den_thresh2 = 0; } @@ -444,6 +445,7 @@ void* Dictionary::Insert(DictEntry* new_entry, int copy_key) // on lists than prepending. chain->append(new_entry); + ++cumulative_entries; if ( *max_num_entries_ptr < ++*num_entries_ptr ) *max_num_entries_ptr = *num_entries_ptr; diff --git a/src/Dict.h b/src/Dict.h index 3a2239ef54..2def5ea28f 100644 --- a/src/Dict.h +++ b/src/Dict.h @@ -71,6 +71,12 @@ public: max_num_entries + max_num_entries2 : max_num_entries; } + // Total number of entries ever. + uint64 NumCumulativeInserts() const + { + return cumulative_entries; + } + // True if the dictionary is ordered, false otherwise. int IsOrdered() const { return order != 0; } @@ -166,6 +172,7 @@ private: int num_buckets; int num_entries; int max_num_entries; + uint64 cumulative_entries; double den_thresh; int thresh_entries; diff --git a/src/Event.cc b/src/Event.cc index 89e745361f..5d54752a5a 100644 --- a/src/Event.cc +++ b/src/Event.cc @@ -10,8 +10,8 @@ EventMgr mgr; -int num_events_queued = 0; -int num_events_dispatched = 0; +uint64 num_events_queued = 0; +uint64 num_events_dispatched = 0; Event::Event(EventHandlerPtr arg_handler, val_list* arg_args, SourceID arg_src, analyzer::ID arg_aid, TimerMgr* arg_mgr, diff --git a/src/Event.h b/src/Event.h index 6f9c9d10c3..0d004d526c 100644 --- a/src/Event.h +++ b/src/Event.h @@ -72,8 +72,8 @@ protected: Event* next_event; }; -extern int num_events_queued; -extern int num_events_dispatched; +extern uint64 num_events_queued; +extern uint64 num_events_dispatched; class EventMgr : public BroObj { public: diff --git a/src/Frag.cc b/src/Frag.cc index 6a8b901a73..842059e218 100644 --- a/src/Frag.cc +++ b/src/Frag.cc @@ -28,7 +28,7 @@ void FragTimer::Dispatch(double t, int /* is_expire */) FragReassembler::FragReassembler(NetSessions* arg_s, const IP_Hdr* ip, const u_char* pkt, HashKey* k, double t) - : Reassembler(0) + : Reassembler(0, REASSEM_FRAG) { s = arg_s; key = k; diff --git a/src/Func.cc b/src/Func.cc index e1eadb8c9f..ccb2570f70 100644 --- a/src/Func.cc +++ b/src/Func.cc @@ -628,10 +628,12 @@ void builtin_error(const char* msg, BroObj* arg) } #include "bro.bif.func_h" +#include "stats.bif.func_h" #include "reporter.bif.func_h" #include "strings.bif.func_h" #include "bro.bif.func_def" +#include "stats.bif.func_def" #include "reporter.bif.func_def" #include "strings.bif.func_def" @@ -640,13 +642,22 @@ void builtin_error(const char* msg, BroObj* arg) void init_builtin_funcs() { - bro_resources = internal_type("bro_resources")->AsRecordType(); - net_stats = internal_type("NetStats")->AsRecordType(); - matcher_stats = internal_type("matcher_stats")->AsRecordType(); + ProcStats = internal_type("ProcStats")->AsRecordType(); + NetStats = internal_type("NetStats")->AsRecordType(); + MatcherStats = internal_type("MatcherStats")->AsRecordType(); + ConnStats = internal_type("ConnStats")->AsRecordType(); + ReassemblerStats = internal_type("ReassemblerStats")->AsRecordType(); + DNSStats = internal_type("DNSStats")->AsRecordType(); + GapStats = internal_type("GapStats")->AsRecordType(); + EventStats = internal_type("EventStats")->AsRecordType(); + TimerStats = internal_type("TimerStats")->AsRecordType(); + FileAnalysisStats = internal_type("FileAnalysisStats")->AsRecordType(); + ThreadStats = internal_type("ThreadStats")->AsRecordType(); + var_sizes = internal_type("var_sizes")->AsTableType(); - gap_info = internal_type("gap_info")->AsRecordType(); #include "bro.bif.func_init" +#include "stats.bif.func_init" #include "reporter.bif.func_init" #include "strings.bif.func_init" diff --git a/src/IP.cc b/src/IP.cc index 3a19f02d23..ebe778e3d7 100644 --- a/src/IP.cc +++ b/src/IP.cc @@ -1,5 +1,9 @@ // See the file "COPYING" in the main distribution directory for copyright. +#include +#include +#include + #include "IP.h" #include "Type.h" #include "Val.h" @@ -403,6 +407,17 @@ RecordVal* IP_Hdr::BuildPktHdrVal(RecordVal* pkt_hdr, int sindex) const break; } + case IPPROTO_ICMPV6: + { + const struct icmp6_hdr* icmpp = (const struct icmp6_hdr*) data; + RecordVal* icmp_hdr = new RecordVal(icmp_hdr_type); + + icmp_hdr->Assign(0, new Val(icmpp->icmp6_type, TYPE_COUNT)); + + pkt_hdr->Assign(sindex + 4, icmp_hdr); + break; + } + default: { // This is not a protocol we understand. diff --git a/src/NFA.cc b/src/NFA.cc index def04d79a1..4d18f75226 100644 --- a/src/NFA.cc +++ b/src/NFA.cc @@ -285,11 +285,6 @@ void NFA_Machine::Dump(FILE* f) first_state->ClearMarks(); } -void NFA_Machine::DumpStats(FILE* f) - { - fprintf(f, "highest NFA state ID is %d\n", nfa_state_id); - } - NFA_Machine* make_alternate(NFA_Machine* m1, NFA_Machine* m2) { if ( ! m1 ) diff --git a/src/NFA.h b/src/NFA.h index 9877b8787c..88ce3429c9 100644 --- a/src/NFA.h +++ b/src/NFA.h @@ -105,7 +105,6 @@ public: void Describe(ODesc* d) const; void Dump(FILE* f); - void DumpStats(FILE* f); unsigned int MemoryAllocation() const { return padded_sizeof(*this) + first_state->TotalMemoryAllocation(); } diff --git a/src/NetVar.cc b/src/NetVar.cc index 8ebed20554..75613364e2 100644 --- a/src/NetVar.cc +++ b/src/NetVar.cc @@ -15,6 +15,8 @@ RecordType* icmp_conn; RecordType* icmp_context; RecordType* SYN_packet; RecordType* pcap_packet; +RecordType* raw_pkt_hdr_type; +RecordType* l2_hdr_type; RecordType* signature_state; EnumType* transport_proto; TableType* string_set; @@ -191,7 +193,6 @@ Val* pkt_profile_file; int load_sample_freq; double gap_report_freq; -RecordType* gap_info; int packet_filter_default; @@ -318,6 +319,8 @@ void init_net_var() signature_state = internal_type("signature_state")->AsRecordType(); SYN_packet = internal_type("SYN_packet")->AsRecordType(); pcap_packet = internal_type("pcap_packet")->AsRecordType(); + raw_pkt_hdr_type = internal_type("raw_pkt_hdr")->AsRecordType(); + l2_hdr_type = internal_type("l2_hdr")->AsRecordType(); transport_proto = internal_type("transport_proto")->AsEnumType(); string_set = internal_type("string_set")->AsTableType(); string_array = internal_type("string_array")->AsTableType(); diff --git a/src/NetVar.h b/src/NetVar.h index 1c021e1fb3..2b8ebd69c2 100644 --- a/src/NetVar.h +++ b/src/NetVar.h @@ -19,6 +19,8 @@ extern RecordType* icmp_context; extern RecordType* signature_state; extern RecordType* SYN_packet; extern RecordType* pcap_packet; +extern RecordType* raw_pkt_hdr_type; +extern RecordType* l2_hdr_type; extern EnumType* transport_proto; extern TableType* string_set; extern TableType* string_array; @@ -194,9 +196,6 @@ extern Val* pkt_profile_file; extern int load_sample_freq; -extern double gap_report_freq; -extern RecordType* gap_info; - extern int packet_filter_default; extern int sig_max_group_size; diff --git a/src/PriorityQueue.cc b/src/PriorityQueue.cc index 75b731142e..5fe0cbef81 100644 --- a/src/PriorityQueue.cc +++ b/src/PriorityQueue.cc @@ -13,7 +13,7 @@ PriorityQueue::PriorityQueue(int initial_size) { max_heap_size = initial_size; heap = new PQ_Element*[max_heap_size]; - peak_heap_size = heap_size = 0; + peak_heap_size = heap_size = cumulative_num = 0; } PriorityQueue::~PriorityQueue() @@ -62,6 +62,8 @@ int PriorityQueue::Add(PQ_Element* e) BubbleUp(heap_size); + ++cumulative_num; + if ( ++heap_size > peak_heap_size ) peak_heap_size = heap_size; diff --git a/src/PriorityQueue.h b/src/PriorityQueue.h index 87e10aa7ac..bb1caad592 100644 --- a/src/PriorityQueue.h +++ b/src/PriorityQueue.h @@ -4,6 +4,7 @@ #define __PriorityQueue__ #include +#include "util.h" class PriorityQueue; @@ -53,6 +54,7 @@ public: int Size() const { return heap_size; } int PeakSize() const { return peak_heap_size; } + uint64 CumulativeNum() const { return cumulative_num; } protected: int Resize(int new_size); @@ -92,6 +94,7 @@ protected: int heap_size; int peak_heap_size; int max_heap_size; + uint64 cumulative_num; }; #endif diff --git a/src/Reassem.cc b/src/Reassem.cc index 54f27bd895..14d894be4f 100644 --- a/src/Reassem.cc +++ b/src/Reassem.cc @@ -1,6 +1,7 @@ // See the file "COPYING" in the main distribution directory for copyright. #include +#include #include "bro-config.h" @@ -10,7 +11,8 @@ static const bool DEBUG_reassem = false; DataBlock::DataBlock(const u_char* data, uint64 size, uint64 arg_seq, - DataBlock* arg_prev, DataBlock* arg_next) + DataBlock* arg_prev, DataBlock* arg_next, + ReassemblerType reassem_type) { seq = arg_seq; upper = seq + size; @@ -26,17 +28,21 @@ DataBlock::DataBlock(const u_char* data, uint64 size, uint64 arg_seq, if ( next ) next->prev = this; + rtype = reassem_type; + Reassembler::sizes[rtype] += pad_size(size) + padded_sizeof(DataBlock); Reassembler::total_size += pad_size(size) + padded_sizeof(DataBlock); } uint64 Reassembler::total_size = 0; +uint64 Reassembler::sizes[REASSEM_NUM]; -Reassembler::Reassembler(uint64 init_seq) +Reassembler::Reassembler(uint64 init_seq, ReassemblerType reassem_type) { blocks = last_block = 0; old_blocks = last_old_block = 0; total_old_blocks = max_old_blocks = 0; trim_seq = last_reassem_seq = init_seq; + rtype = reassem_type; } Reassembler::~Reassembler() @@ -110,7 +116,7 @@ void Reassembler::NewBlock(double t, uint64 seq, uint64 len, const u_char* data) if ( ! blocks ) blocks = last_block = start_block = - new DataBlock(data, len, seq, 0, 0); + new DataBlock(data, len, seq, 0, 0, rtype); else start_block = AddAndCheck(blocks, seq, upper_seq, data); @@ -275,7 +281,7 @@ DataBlock* Reassembler::AddAndCheck(DataBlock* b, uint64 seq, uint64 upper, if ( last_block && seq == last_block->upper ) { last_block = new DataBlock(data, upper - seq, seq, - last_block, 0); + last_block, 0, rtype); return last_block; } @@ -288,7 +294,7 @@ DataBlock* Reassembler::AddAndCheck(DataBlock* b, uint64 seq, uint64 upper, { // b is the last block, and it comes completely before // the new block. - last_block = new DataBlock(data, upper - seq, seq, b, 0); + last_block = new DataBlock(data, upper - seq, seq, b, 0, rtype); return last_block; } @@ -297,7 +303,7 @@ DataBlock* Reassembler::AddAndCheck(DataBlock* b, uint64 seq, uint64 upper, if ( upper <= b->seq ) { // The new block comes completely before b. - new_b = new DataBlock(data, upper - seq, seq, b->prev, b); + new_b = new DataBlock(data, upper - seq, seq, b->prev, b, rtype); if ( b == blocks ) blocks = new_b; return new_b; @@ -308,7 +314,7 @@ DataBlock* Reassembler::AddAndCheck(DataBlock* b, uint64 seq, uint64 upper, { // The new block has a prefix that comes before b. uint64 prefix_len = b->seq - seq; - new_b = new DataBlock(data, prefix_len, seq, b->prev, b); + new_b = new DataBlock(data, prefix_len, seq, b->prev, b, rtype); if ( b == blocks ) blocks = new_b; @@ -342,6 +348,11 @@ DataBlock* Reassembler::AddAndCheck(DataBlock* b, uint64 seq, uint64 upper, return new_b; } +uint64 Reassembler::MemoryAllocation(ReassemblerType rtype) + { + return Reassembler::sizes[rtype]; + } + bool Reassembler::Serialize(SerialInfo* info) const { return SerialObj::Serialize(info); diff --git a/src/Reassem.h b/src/Reassem.h index e55c809990..1672a4f9dd 100644 --- a/src/Reassem.h +++ b/src/Reassem.h @@ -6,10 +6,23 @@ #include "Obj.h" #include "IPAddr.h" +// Whenever subclassing the Reassembler class +// you should add to this for known subclasses. +enum ReassemblerType { + REASSEM_UNKNOWN, + REASSEM_TCP, + REASSEM_FRAG, + REASSEM_FILE, + + // Terminal value. Add new above. + REASSEM_NUM, +}; + class DataBlock { public: DataBlock(const u_char* data, uint64 size, uint64 seq, - DataBlock* prev, DataBlock* next); + DataBlock* prev, DataBlock* next, + ReassemblerType reassem_type = REASSEM_UNKNOWN); ~DataBlock(); @@ -19,13 +32,12 @@ public: DataBlock* prev; // previous block with lower seq # uint64 seq, upper; u_char* block; + ReassemblerType rtype; }; - - class Reassembler : public BroObj { public: - Reassembler(uint64 init_seq); + Reassembler(uint64 init_seq, ReassemblerType reassem_type = REASSEM_UNKNOWN); virtual ~Reassembler(); void NewBlock(double t, uint64 seq, uint64 len, const u_char* data); @@ -51,6 +63,9 @@ public: // Sum over all data buffered in some reassembler. static uint64 TotalMemoryAllocation() { return total_size; } + // Data buffered by type of reassembler. + static uint64 MemoryAllocation(ReassemblerType rtype); + void SetMaxOldBlocks(uint32 count) { max_old_blocks = count; } protected: @@ -82,12 +97,16 @@ protected: uint32 max_old_blocks; uint32 total_old_blocks; + ReassemblerType rtype; + static uint64 total_size; + static uint64 sizes[REASSEM_NUM]; }; inline DataBlock::~DataBlock() { Reassembler::total_size -= pad_size(upper - seq) + padded_sizeof(DataBlock); + Reassembler::sizes[rtype] -= pad_size(upper - seq) + padded_sizeof(DataBlock); delete [] block; } diff --git a/src/RuleCondition.cc b/src/RuleCondition.cc index 68eb13121f..40ef5f0ad1 100644 --- a/src/RuleCondition.cc +++ b/src/RuleCondition.cc @@ -111,7 +111,7 @@ bool RuleConditionPayloadSize::DoMatch(Rule* rule, RuleEndpointState* state, return payload_size >= val; default: - reporter->InternalError("unknown comparision type"); + reporter->InternalError("unknown comparison type"); } // Should not be reached diff --git a/src/RuleMatcher.cc b/src/RuleMatcher.cc index f40a5c4349..c88bb77a4f 100644 --- a/src/RuleMatcher.cc +++ b/src/RuleMatcher.cc @@ -21,7 +21,7 @@ // it may fail to match. Work-around: Insert an always // matching "payload" pattern (not done in snort2bro yet) // - tcp-state always evaluates to true -// (implemented but deactivated for comparision to Snort) +// (implemented but deactivated for comparison to Snort) uint32 RuleHdrTest::idcounter = 0; @@ -1174,7 +1174,7 @@ void RuleMatcher::GetStats(Stats* stats, RuleHdrTest* hdr_test) stats->mem = 0; stats->hits = 0; stats->misses = 0; - stats->avg_nfa_states = 0; + stats->nfa_states = 0; hdr_test = root; } @@ -1195,15 +1195,10 @@ void RuleMatcher::GetStats(Stats* stats, RuleHdrTest* hdr_test) stats->mem += cstats.mem; stats->hits += cstats.hits; stats->misses += cstats.misses; - stats->avg_nfa_states += cstats.nfa_states; + stats->nfa_states += cstats.nfa_states; } } - if ( stats->dfa_states ) - stats->avg_nfa_states /= stats->dfa_states; - else - stats->avg_nfa_states = 0; - for ( RuleHdrTest* h = hdr_test->child; h; h = h->sibling ) GetStats(stats, h); } diff --git a/src/RuleMatcher.h b/src/RuleMatcher.h index 6ffc971db1..b16a1556f9 100644 --- a/src/RuleMatcher.h +++ b/src/RuleMatcher.h @@ -297,6 +297,9 @@ public: struct Stats { unsigned int matchers; // # distinct RE matchers + // NFA states across all matchers. + unsigned int nfa_states; + // # DFA states across all matchers unsigned int dfa_states; unsigned int computed; // # computed DFA state transitions @@ -305,9 +308,6 @@ public: // # cache hits (sampled, multiply by MOVE_TO_FRONT_SAMPLE_SIZE) unsigned int hits; unsigned int misses; // # cache misses - - // Average # NFA states per DFA state. - unsigned int avg_nfa_states; }; Val* BuildRuleStateValue(const Rule* rule, diff --git a/src/Serializer.cc b/src/Serializer.cc index 49e57c0216..5c1ae6077c 100644 --- a/src/Serializer.cc +++ b/src/Serializer.cc @@ -437,7 +437,7 @@ bool Serializer::UnserializeCall(UnserialInfo* info) bool Serializer::UnserializeStateAccess(UnserialInfo* info) { - SetErrorDescr("unserializing state acess"); + SetErrorDescr("unserializing state access"); StateAccess* s = StateAccess::Unserialize(info); diff --git a/src/Sessions.cc b/src/Sessions.cc index b8bfe82b34..aae6712ef2 100644 --- a/src/Sessions.cc +++ b/src/Sessions.cc @@ -1156,19 +1156,18 @@ void NetSessions::Drain() void NetSessions::GetStats(SessionStats& s) const { s.num_TCP_conns = tcp_conns.Length(); + s.cumulative_TCP_conns = tcp_conns.NumCumulativeInserts(); s.num_UDP_conns = udp_conns.Length(); + s.cumulative_UDP_conns = udp_conns.NumCumulativeInserts(); s.num_ICMP_conns = icmp_conns.Length(); + s.cumulative_ICMP_conns = icmp_conns.NumCumulativeInserts(); s.num_fragments = fragments.Length(); s.num_packets = num_packets_processed; - s.num_timers = timer_mgr->Size(); - s.num_events_queued = num_events_queued; - s.num_events_dispatched = num_events_dispatched; s.max_TCP_conns = tcp_conns.MaxLength(); s.max_UDP_conns = udp_conns.MaxLength(); s.max_ICMP_conns = icmp_conns.MaxLength(); s.max_fragments = fragments.MaxLength(); - s.max_timers = timer_mgr->PeakSize(); } Connection* NetSessions::NewConn(HashKey* k, double t, const ConnID* id, diff --git a/src/Sessions.h b/src/Sessions.h index 2aca292789..8da658633c 100644 --- a/src/Sessions.h +++ b/src/Sessions.h @@ -32,19 +32,20 @@ namespace analyzer { namespace arp { class ARP_Analyzer; } } struct SessionStats { int num_TCP_conns; - int num_UDP_conns; - int num_ICMP_conns; - int num_fragments; - int num_packets; - int num_timers; - int num_events_queued; - int num_events_dispatched; - int max_TCP_conns; + uint64 cumulative_TCP_conns; + + int num_UDP_conns; int max_UDP_conns; + uint64 cumulative_UDP_conns; + + int num_ICMP_conns; int max_ICMP_conns; + uint64 cumulative_ICMP_conns; + + int num_fragments; int max_fragments; - int max_timers; + uint64 num_packets; }; // Drains and deletes a timer manager if it hasn't seen any advances @@ -242,7 +243,7 @@ protected: OSFingerprint* SYN_OS_Fingerprinter; int build_backdoor_analyzer; int dump_this_packet; // if true, current packet should be recorded - int num_packets_processed; + uint64 num_packets_processed; PacketProfiler* pkt_profiler; // We may use independent timer managers for different sets of related diff --git a/src/StateAccess.cc b/src/StateAccess.cc index aa4a1f36d2..6e73c8cf61 100644 --- a/src/StateAccess.cc +++ b/src/StateAccess.cc @@ -150,7 +150,7 @@ bool StateAccess::CheckOld(const char* op, ID* id, Val* index, if ( should && is ) { - // There's no general comparision for non-atomic vals currently. + // There's no general comparison for non-atomic vals currently. if ( ! (is_atomic_val(is) && is_atomic_val(should)) ) return true; diff --git a/src/Stats.cc b/src/Stats.cc index eb5ac67e26..d1f447c05c 100644 --- a/src/Stats.cc +++ b/src/Stats.cc @@ -14,7 +14,7 @@ #include "broker/Manager.h" #endif -int killed_by_inactivity = 0; +uint64 killed_by_inactivity = 0; uint64 tot_ack_events = 0; uint64 tot_ack_bytes = 0; @@ -82,7 +82,7 @@ void ProfileLogger::Log() struct timeval tv_utime = r.ru_utime; struct timeval tv_stime = r.ru_stime; - unsigned int total, malloced; + uint64 total, malloced; get_memory_usage(&total, &malloced); static unsigned int first_total = 0; @@ -110,7 +110,7 @@ void ProfileLogger::Log() file->Write(fmt("\n%.06f ------------------------\n", network_time)); } - file->Write(fmt("%.06f Memory: total=%dK total_adj=%dK malloced: %dK\n", + file->Write(fmt("%.06f Memory: total=%" PRId64 "K total_adj=%" PRId64 "K malloced: %" PRId64 "K\n", network_time, total / 1024, (total - first_total) / 1024, malloced / 1024)); @@ -120,7 +120,7 @@ void ProfileLogger::Log() int conn_mem_use = expensive ? sessions->ConnectionMemoryUsage() : 0; - file->Write(fmt("%.06f Conns: total=%d current=%d/%d ext=%d mem=%dK avg=%.1f table=%dK connvals=%dK\n", + file->Write(fmt("%.06f Conns: total=%" PRIu64 " current=%" PRIu64 "/%" PRIi32 " ext=%" PRIu64 " mem=%" PRIi32 "K avg=%.1f table=%" PRIu32 "K connvals=%" PRIu32 "K\n", network_time, Connection::TotalConnections(), Connection::CurrentConnections(), @@ -161,10 +161,10 @@ void ProfileLogger::Log() )); */ - file->Write(fmt("%.06f Connections expired due to inactivity: %d\n", + file->Write(fmt("%.06f Connections expired due to inactivity: %" PRIu64 "\n", network_time, killed_by_inactivity)); - file->Write(fmt("%.06f Total reassembler data: %" PRIu64"K\n", network_time, + file->Write(fmt("%.06f Total reassembler data: %" PRIu64 "K\n", network_time, Reassembler::TotalMemoryAllocation() / 1024)); // Signature engine. @@ -173,9 +173,9 @@ void ProfileLogger::Log() RuleMatcher::Stats stats; rule_matcher->GetStats(&stats); - file->Write(fmt("%06f RuleMatcher: matchers=%d dfa_states=%d ncomputed=%d " - "mem=%dK avg_nfa_states=%d\n", network_time, stats.matchers, - stats.dfa_states, stats.computed, stats.mem / 1024, stats.avg_nfa_states)); + file->Write(fmt("%06f RuleMatcher: matchers=%d nfa_states=%d dfa_states=%d " + "ncomputed=%d mem=%dK\n", network_time, stats.matchers, + stats.nfa_states, stats.dfa_states, stats.computed, stats.mem / 1024)); } file->Write(fmt("%.06f Timers: current=%d max=%d mem=%dK lag=%.2fs\n", @@ -469,10 +469,10 @@ void PacketProfiler::ProfilePkt(double t, unsigned int bytes) double curr_Rtime = ptimestamp.tv_sec + ptimestamp.tv_usec / 1e6; - unsigned int curr_mem; + uint64 curr_mem; get_memory_usage(&curr_mem, 0); - file->Write(fmt("%.06f %.03f %d %d %.03f %.03f %.03f %d\n", + file->Write(fmt("%.06f %.03f %" PRIu64 " %" PRIu64 " %.03f %.03f %.03f %" PRIu64 "\n", t, time-last_timestamp, pkt_cnt, byte_cnt, curr_Rtime - last_Rtime, curr_Utime - last_Utime, diff --git a/src/Stats.h b/src/Stats.h index 1bcc2e18dc..7fbec8cab6 100644 --- a/src/Stats.h +++ b/src/Stats.h @@ -102,7 +102,7 @@ extern ProfileLogger* segment_logger; extern SampleLogger* sample_logger; // Connection statistics. -extern int killed_by_inactivity; +extern uint64 killed_by_inactivity; // Content gap statistics. extern uint64 tot_ack_events; @@ -127,9 +127,9 @@ protected: double update_freq; double last_Utime, last_Stime, last_Rtime; double last_timestamp, time; - unsigned int last_mem; - unsigned int pkt_cnt; - unsigned int byte_cnt; + uint64 last_mem; + uint64 pkt_cnt; + uint64 byte_cnt; }; #endif diff --git a/src/Timer.h b/src/Timer.h index 615c8bf69a..e095421c30 100644 --- a/src/Timer.h +++ b/src/Timer.h @@ -109,11 +109,12 @@ public: virtual int Size() const = 0; virtual int PeakSize() const = 0; + virtual uint64 CumulativeNum() const = 0; double LastTimestamp() const { return last_timestamp; } // Returns time of last advance in global network time. double LastAdvance() const { return last_advance; } - + static unsigned int* CurrentTimers() { return current_timers; } protected: @@ -148,6 +149,7 @@ public: int Size() const { return q->Size(); } int PeakSize() const { return q->PeakSize(); } + uint64 CumulativeNum() const { return q->CumulativeNum(); } unsigned int MemoryUsage() const; protected: @@ -170,6 +172,7 @@ public: int Size() const { return cq_size(cq); } int PeakSize() const { return cq_max_size(cq); } + uint64 CumulativeNum() const { return cq_cumulative_num(cq); } unsigned int MemoryUsage() const; protected: diff --git a/src/analyzer/Analyzer.cc b/src/analyzer/Analyzer.cc index b4048af467..5cf3fcb58d 100644 --- a/src/analyzer/Analyzer.cc +++ b/src/analyzer/Analyzer.cc @@ -395,7 +395,7 @@ bool Analyzer::AddChildAnalyzer(Analyzer* analyzer, bool init) // the list. analyzer->parent = this; - children.push_back(analyzer); + new_children.push_back(analyzer); if ( init ) analyzer->Init(); @@ -474,6 +474,13 @@ Analyzer* Analyzer::FindChild(ID arg_id) return child; } + LOOP_OVER_GIVEN_CHILDREN(i, new_children) + { + Analyzer* child = (*i)->FindChild(arg_id); + if ( child ) + return child; + } + return 0; } @@ -489,6 +496,13 @@ Analyzer* Analyzer::FindChild(Tag arg_tag) return child; } + LOOP_OVER_GIVEN_CHILDREN(i, new_children) + { + Analyzer* child = (*i)->FindChild(arg_tag); + if ( child ) + return child; + } + return 0; } diff --git a/src/analyzer/Analyzer.h b/src/analyzer/Analyzer.h index 83157aadde..df77a990ce 100644 --- a/src/analyzer/Analyzer.h +++ b/src/analyzer/Analyzer.h @@ -427,6 +427,10 @@ public: /** * Returns a list of all direct child analyzers. + * + * Note that this does not include the list of analyzers that are + * currently queued up to be added. If you just added an analyzer, + * it will not immediately be in this list. */ const analyzer_list& GetChildren() { return children; } diff --git a/src/analyzer/Manager.cc b/src/analyzer/Manager.cc index 67aa6a0d33..6082f433da 100644 --- a/src/analyzer/Manager.cc +++ b/src/analyzer/Manager.cc @@ -361,7 +361,6 @@ bool Manager::BuildInitialAnalyzerTree(Connection* conn) icmp::ICMP_Analyzer* icmp = 0; TransportLayerAnalyzer* root = 0; pia::PIA* pia = 0; - bool analyzed = false; bool check_port = false; switch ( conn->ConnTransport() ) { @@ -383,7 +382,6 @@ bool Manager::BuildInitialAnalyzerTree(Connection* conn) case TRANSPORT_ICMP: { root = icmp = new icmp::ICMP_Analyzer(conn); DBG_ANALYZER(conn, "activated ICMP analyzer"); - analyzed = true; break; } @@ -495,16 +493,10 @@ bool Manager::BuildInitialAnalyzerTree(Connection* conn) if ( pia ) root->AddChildAnalyzer(pia->AsAnalyzer()); - if ( root->GetChildren().size() ) - analyzed = true; - conn->SetRootAnalyzer(root, pia); root->Init(); root->InitChildren(); - if ( ! analyzed ) - conn->SetLifetime(non_analyzed_lifetime); - PLUGIN_HOOK_VOID(HOOK_SETUP_ANALYZER_TREE, HookSetupAnalyzerTree(conn)); return true; diff --git a/src/analyzer/protocol/CMakeLists.txt b/src/analyzer/protocol/CMakeLists.txt index 0542a905f4..ff34d243e8 100644 --- a/src/analyzer/protocol/CMakeLists.txt +++ b/src/analyzer/protocol/CMakeLists.txt @@ -17,6 +17,7 @@ add_subdirectory(gtpv1) add_subdirectory(http) add_subdirectory(icmp) add_subdirectory(ident) +add_subdirectory(imap) add_subdirectory(interconn) add_subdirectory(irc) add_subdirectory(krb) @@ -46,4 +47,5 @@ add_subdirectory(syslog) add_subdirectory(tcp) add_subdirectory(teredo) add_subdirectory(udp) +add_subdirectory(xmpp) add_subdirectory(zip) diff --git a/src/analyzer/protocol/arp/ARP.cc b/src/analyzer/protocol/arp/ARP.cc index 5cbb25451b..b9af26ecfa 100644 --- a/src/analyzer/protocol/arp/ARP.cc +++ b/src/analyzer/protocol/arp/ARP.cc @@ -10,9 +10,6 @@ using namespace analyzer::arp; ARP_Analyzer::ARP_Analyzer() { - bad_arp = internal_handler("bad_arp"); - arp_request = internal_handler("arp_request"); - arp_reply = internal_handler("arp_reply"); } ARP_Analyzer::~ARP_Analyzer() diff --git a/src/analyzer/protocol/arp/ARP.h b/src/analyzer/protocol/arp/ARP.h index c4deddee03..1bdd382714 100644 --- a/src/analyzer/protocol/arp/ARP.h +++ b/src/analyzer/protocol/arp/ARP.h @@ -50,10 +50,6 @@ protected: StringVal* EthAddrToStr(const u_char* addr); void BadARP(const struct arp_pkthdr* hdr, const char* string); void Corrupted(const char* string); - - EventHandlerPtr arp_corrupted_packet; - EventHandlerPtr arp_request; - EventHandlerPtr arp_reply; }; } } // namespace analyzer::* diff --git a/src/analyzer/protocol/dns/DNS.cc b/src/analyzer/protocol/dns/DNS.cc index b449589e6c..1fc94a80ba 100644 --- a/src/analyzer/protocol/dns/DNS.cc +++ b/src/analyzer/protocol/dns/DNS.cc @@ -282,6 +282,10 @@ int DNS_Interpreter::ParseAnswer(DNS_MsgInfo* msg, status = ParseRR_TXT(msg, data, len, rdlength, msg_start); break; + case TYPE_CAA: + status = ParseRR_CAA(msg, data, len, rdlength, msg_start); + break; + case TYPE_NBS: status = ParseRR_NBS(msg, data, len, rdlength, msg_start); break; @@ -904,6 +908,51 @@ int DNS_Interpreter::ParseRR_TXT(DNS_MsgInfo* msg, return rdlength == 0; } +int DNS_Interpreter::ParseRR_CAA(DNS_MsgInfo* msg, + const u_char*& data, int& len, int rdlength, + const u_char* msg_start) + { + if ( ! dns_CAA_reply || msg->skip_event ) + { + data += rdlength; + len -= rdlength; + return 1; + } + + unsigned int flags = ExtractShort(data, len); + unsigned int tagLen = flags & 0xff; + flags = flags >> 8; + rdlength -= 2; + if ( (int) tagLen >= rdlength ) + { + analyzer->Weird("DNS_CAA_char_str_past_rdlen"); + return 0; + } + BroString* tag = new BroString(data, tagLen, 1); + len -= tagLen; + data += tagLen; + rdlength -= tagLen; + BroString* value = new BroString(data, rdlength, 0); + + len -= value->Len(); + data += value->Len(); + rdlength -= value->Len(); + + val_list* vl = new val_list; + + vl->append(analyzer->BuildConnVal()); + vl->append(msg->BuildHdrVal()); + vl->append(msg->BuildAnswerVal()); + vl->append(new Val(flags, TYPE_COUNT)); + vl->append(new StringVal(tag)); + vl->append(new StringVal(value)); + + analyzer->ConnectionEvent(dns_CAA_reply, vl); + + return rdlength == 0; + } + + void DNS_Interpreter::SendReplyOrRejectEvent(DNS_MsgInfo* msg, EventHandlerPtr event, const u_char*& data, int& len, diff --git a/src/analyzer/protocol/dns/DNS.h b/src/analyzer/protocol/dns/DNS.h index 59f51812ca..87618cd18e 100644 --- a/src/analyzer/protocol/dns/DNS.h +++ b/src/analyzer/protocol/dns/DNS.h @@ -56,6 +56,7 @@ typedef enum { TYPE_EDNS = 41, ///< OPT pseudo-RR (RFC 2671) TYPE_TKEY = 249, ///< Transaction Key (RFC 2930) TYPE_TSIG = 250, ///< Transaction Signature (RFC 2845) + TYPE_CAA = 257, ///< Certification Authority Authorization (RFC 6844) // The following are only valid in queries. TYPE_AXFR = 252, @@ -132,7 +133,7 @@ public: StringVal* query_name; RR_Type atype; int aclass; ///< normally = 1, inet - int ttl; + uint32 ttl; DNS_AnswerType answer_type; int skip_event; ///< if true, don't generate corresponding events @@ -211,6 +212,9 @@ protected: int ParseRR_TXT(DNS_MsgInfo* msg, const u_char*& data, int& len, int rdlength, const u_char* msg_start); + int ParseRR_CAA(DNS_MsgInfo* msg, + const u_char*& data, int& len, int rdlength, + const u_char* msg_start); int ParseRR_TSIG(DNS_MsgInfo* msg, const u_char*& data, int& len, int rdlength, const u_char* msg_start); diff --git a/src/analyzer/protocol/dns/events.bif b/src/analyzer/protocol/dns/events.bif index 9350939a2e..ae796c8e4c 100644 --- a/src/analyzer/protocol/dns/events.bif +++ b/src/analyzer/protocol/dns/events.bif @@ -378,6 +378,25 @@ event dns_MX_reply%(c: connection, msg: dns_msg, ans: dns_answer, name: string, ## dns_skip_addl dns_skip_all_addl dns_skip_all_auth dns_skip_auth event dns_TXT_reply%(c: connection, msg: dns_msg, ans: dns_answer, strs: string_vec%); +## Generated for DNS replies of type *CAA* (Certification Authority Authorization). +## For replies with multiple answers, an individual event of the corresponding type +## is raised for each. +## See `RFC 6844 `__ for more details. +## +## c: The connection, which may be UDP or TCP depending on the type of the +## transport-layer session being analyzed. +## +## msg: The parsed DNS message header. +## +## ans: The type-independent part of the parsed answer record. +## +## flags: The flags byte of the CAA reply. +## +## tag: The property identifier of the CAA reply. +## +## value: The property value of the CAA reply. +event dns_CAA_reply%(c: connection, msg: dns_msg, ans: dns_answer, flags: count, tag: string, value: string%); + ## Generated for DNS replies of type *SRV*. For replies with multiple answers, ## an individual event of the corresponding type is raised for each. ## diff --git a/src/analyzer/protocol/imap/CMakeLists.txt b/src/analyzer/protocol/imap/CMakeLists.txt new file mode 100644 index 0000000000..921dde2444 --- /dev/null +++ b/src/analyzer/protocol/imap/CMakeLists.txt @@ -0,0 +1,12 @@ + +include(BroPlugin) + +include_directories(BEFORE ${CMAKE_CURRENT_SOURCE_DIR} ${CMAKE_CURRENT_BINARY_DIR}) + +bro_plugin_begin(Bro IMAP) +bro_plugin_cc(Plugin.cc) +bro_plugin_cc(IMAP.cc) +bro_plugin_bif(events.bif) +bro_plugin_pac(imap.pac imap-analyzer.pac imap-protocol.pac) +bro_plugin_end() + diff --git a/src/analyzer/protocol/imap/IMAP.cc b/src/analyzer/protocol/imap/IMAP.cc new file mode 100644 index 0000000000..ea09a66717 --- /dev/null +++ b/src/analyzer/protocol/imap/IMAP.cc @@ -0,0 +1,85 @@ +// See the file "COPYING" in the main distribution directory for copyright. + +#include "IMAP.h" +#include "analyzer/protocol/tcp/TCP_Reassembler.h" +#include "analyzer/Manager.h" + +using namespace analyzer::imap; + +IMAP_Analyzer::IMAP_Analyzer(Connection* conn) + : tcp::TCP_ApplicationAnalyzer("IMAP", conn) + { + interp = new binpac::IMAP::IMAP_Conn(this); + had_gap = false; + tls_active = false; + } + +IMAP_Analyzer::~IMAP_Analyzer() + { + delete interp; + } + +void IMAP_Analyzer::Done() + { + tcp::TCP_ApplicationAnalyzer::Done(); + + interp->FlowEOF(true); + interp->FlowEOF(false); + } + +void IMAP_Analyzer::EndpointEOF(bool is_orig) + { + tcp::TCP_ApplicationAnalyzer::EndpointEOF(is_orig); + interp->FlowEOF(is_orig); + } + +void IMAP_Analyzer::DeliverStream(int len, const u_char* data, bool orig) + { + tcp::TCP_ApplicationAnalyzer::DeliverStream(len, data, orig); + + if ( tls_active ) + { + // If TLS has been initiated, forward to child and abort further + // processing + ForwardStream(len, data, orig); + return; + } + + assert(TCP()); + if ( TCP()->IsPartial() ) + return; + + if ( had_gap ) + // If only one side had a content gap, we could still try to + // deliver data to the other side if the script layer can + // handle this. + return; + + try + { + interp->NewData(orig, data, data + len); + } + catch ( const binpac::Exception& e ) + { + ProtocolViolation(fmt("Binpac exception: %s", e.c_msg())); + } + } + +void IMAP_Analyzer::Undelivered(uint64 seq, int len, bool orig) + { + tcp::TCP_ApplicationAnalyzer::Undelivered(seq, len, orig); + had_gap = true; + interp->NewGap(orig, len); + } + +void IMAP_Analyzer::StartTLS() + { + // StartTLS was called. This means we saw a client starttls followed + // by a server proceed. From here on, everything should be a binary + // TLS datastream. + tls_active = true; + + Analyzer* ssl = analyzer_mgr->InstantiateAnalyzer("SSL", Conn()); + if ( ssl ) + AddChildAnalyzer(ssl); + } diff --git a/src/analyzer/protocol/imap/IMAP.h b/src/analyzer/protocol/imap/IMAP.h new file mode 100644 index 0000000000..e71770d360 --- /dev/null +++ b/src/analyzer/protocol/imap/IMAP.h @@ -0,0 +1,40 @@ +// See the file "COPYING" in the main distribution directory for copyright. + +#ifndef ANALYZER_PROTOCOL_IMAP_IMAP_H +#define ANALYZER_PROTOCOL_IMAP_IMAP_H + +// for std::transform +#include +#include "analyzer/protocol/tcp/TCP.h" + +#include "imap_pac.h" + +namespace analyzer { namespace imap { + +class IMAP_Analyzer : public tcp::TCP_ApplicationAnalyzer { +public: + IMAP_Analyzer(Connection* conn); + virtual ~IMAP_Analyzer(); + + virtual void Done(); + virtual void DeliverStream(int len, const u_char* data, bool orig); + virtual void Undelivered(uint64 seq, int len, bool orig); + + // Overriden from tcp::TCP_ApplicationAnalyzer. + virtual void EndpointEOF(bool is_orig); + + void StartTLS(); + + static analyzer::Analyzer* Instantiate(Connection* conn) + { return new IMAP_Analyzer(conn); } + +protected: + binpac::IMAP::IMAP_Conn* interp; + bool had_gap; + + bool tls_active; +}; + +} } // namespace analyzer::* + +#endif /* ANALYZER_PROTOCOL_IMAP_IMAP_H */ diff --git a/src/analyzer/protocol/imap/Plugin.cc b/src/analyzer/protocol/imap/Plugin.cc new file mode 100644 index 0000000000..63358f1aeb --- /dev/null +++ b/src/analyzer/protocol/imap/Plugin.cc @@ -0,0 +1,22 @@ +// See the file in the main distribution directory for copyright. +#include "plugin/Plugin.h" +#include "IMAP.h" + +namespace plugin { +namespace Bro_IMAP { + +class Plugin : public plugin::Plugin { +public: + plugin::Configuration Configure() + { + AddComponent(new ::analyzer::Component("IMAP", ::analyzer::imap::IMAP_Analyzer::Instantiate)); + + plugin::Configuration config; + config.name = "Bro::IMAP"; + config.description = "IMAP analyzer (StartTLS only)"; + return config; + } +} plugin; + +} +} diff --git a/src/analyzer/protocol/imap/events.bif b/src/analyzer/protocol/imap/events.bif new file mode 100644 index 0000000000..8d70dda26f --- /dev/null +++ b/src/analyzer/protocol/imap/events.bif @@ -0,0 +1,13 @@ +## Generated when a server sends a capability list to the client, +## after being queried using the CAPABILITY command. +## +## c: The connection. +## +## capabilities: The list of IMAP capabilities as sent by the server. +event imap_capabilities%(c: connection, capabilities: string_vec%); + +## Generated when a IMAP connection goes encrypted after a successful +## StartTLS exchange between the client and the server. +## +## c: The connection. +event imap_starttls%(c: connection%); diff --git a/src/analyzer/protocol/imap/imap-analyzer.pac b/src/analyzer/protocol/imap/imap-analyzer.pac new file mode 100644 index 0000000000..353aadb7ce --- /dev/null +++ b/src/analyzer/protocol/imap/imap-analyzer.pac @@ -0,0 +1,76 @@ +refine connection IMAP_Conn += { + + %member{ + string client_starttls_id; + %} + + %init{ + %} + + function proc_imap_token(is_orig: bool, tag: bytestring, command: bytestring): bool + %{ + string commands = std_str(command); + std::transform(commands.begin(), commands.end(), commands.begin(), ::tolower); + + string tags = std_str(tag); + + //printf("imap %s %s\n", commands.c_str(), tags.c_str()); + + if ( !is_orig && tags == "*" && commands == "ok" ) + bro_analyzer()->ProtocolConfirmation(); + + if ( is_orig && ( command == "capability" || commands == "starttls" ) ) + bro_analyzer()->ProtocolConfirmation(); + + if ( command == "authenticate" || command == "login" || command == "examine" || command == "create" || command == "list" || command == "fetch" ) + { + bro_analyzer()->ProtocolConfirmation(); + // Handshake has passed the phase where we should see StartTLS. Simply skip from hereon... + bro_analyzer()->SetSkip(true); + return true; + } + + if ( is_orig && commands == "starttls" ) + { + if ( !client_starttls_id.empty() ) + reporter->Weird(bro_analyzer()->Conn(), "IMAP: client sent duplicate StartTLS"); + + client_starttls_id = tags; + } + + if ( !is_orig && !client_starttls_id.empty() && tags == client_starttls_id ) + { + if ( commands == "ok" ) + { + bro_analyzer()->StartTLS(); + BifEvent::generate_imap_starttls(bro_analyzer(), bro_analyzer()->Conn()); + } + else + reporter->Weird(bro_analyzer()->Conn(), "IMAP: server refused StartTLS"); + } + + return true; + %} + + function proc_server_capability(capabilities: Capability[]): bool + %{ + VectorVal* capv = new VectorVal(internal_type("string_vec")->AsVectorType()); + for ( unsigned int i = 0; i< capabilities->size(); i++ ) + { + const bytestring& capability = (*capabilities)[i]->cap(); + capv->Assign(i, new StringVal(capability.length(), (const char*)capability.data())); + } + + BifEvent::generate_imap_capabilities(bro_analyzer(), bro_analyzer()->Conn(), capv); + return true; + %} + +}; + +refine typeattr ImapToken += &let { + proc: bool = $context.connection.proc_imap_token(is_orig, tag, command); +}; + +refine typeattr ServerCapability += &let { + proc: bool = $context.connection.proc_server_capability(capabilities); +}; diff --git a/src/analyzer/protocol/imap/imap-protocol.pac b/src/analyzer/protocol/imap/imap-protocol.pac new file mode 100644 index 0000000000..b1964b2bb8 --- /dev/null +++ b/src/analyzer/protocol/imap/imap-protocol.pac @@ -0,0 +1,70 @@ +# commands that we support parsing. The numbers do not really mean anything +# in this case +enum ImapCommand { + CMD_CAPABILITY, + CMD_UNKNOWN +} + +type TAG = RE/[[:alnum:][:punct:]]+/; +type CONTENT = RE/[^\r\n]*/; +type SPACING = RE/[ ]+/; +type OPTIONALSPACING = RE/[ ]*/; +type NEWLINE = RE/[\r\n]+/; +type OPTIONALNEWLINE = RE/[\r\n]*/; + +type IMAP_PDU(is_orig: bool) = ImapToken(is_orig)[] &until($input.length() == 0); + +type ImapToken(is_orig: bool) = record { + tag : TAG; + : SPACING; + command: TAG; + : OPTIONALSPACING; + client_or_server: case is_orig of { + true -> client: UnknownCommand(this) ; + false -> server: ServerContentText(this); + } &requires(pcommand) ; +} &let { + pcommand: int = $context.connection.determine_command(is_orig, tag, command); +}; + +type ServerContentText(rec: ImapToken) = case rec.pcommand of { + CMD_CAPABILITY -> capability: ServerCapability(rec); + default -> unknown: UnknownCommand(rec); +}; + +type Capability = record { + cap: TAG; + : OPTIONALSPACING; + nl: OPTIONALNEWLINE; +}; + +type ServerCapability(rec: ImapToken) = record { + capabilities: Capability[] &until($context.connection.strlen($element.nl) > 0); +}; + +type UnknownCommand(rec: ImapToken) = record { + tagcontent: CONTENT; + : NEWLINE; +}; + +refine connection IMAP_Conn += { + + function determine_command(is_orig: bool, tag: bytestring, command: bytestring): int + %{ + string cmdstr = std_str(command); + std::transform(cmdstr.begin(), cmdstr.end(), cmdstr.begin(), ::tolower); + string tagstr = std_str(tag); + + if ( !is_orig && cmdstr == "capability" && tag == "*" ) { + return CMD_CAPABILITY; + } + + return CMD_UNKNOWN; + %} + + function strlen(str: bytestring): int + %{ + return str.length(); + %} + +}; diff --git a/src/analyzer/protocol/imap/imap.pac b/src/analyzer/protocol/imap/imap.pac new file mode 100644 index 0000000000..f5c7559294 --- /dev/null +++ b/src/analyzer/protocol/imap/imap.pac @@ -0,0 +1,37 @@ +# binpac file for the IMAP analyzer. +# Note that we currently do not even try to parse the protocol +# completely -- this is only supposed to be able to parse imap +# till StartTLS does (or does not) kick in. + +%include binpac.pac +%include bro.pac + +%extern{ +#include "events.bif.h" + +namespace analyzer { namespace imap { class IMAP_Analyzer; } } +namespace binpac { namespace IMAP { class IMAP_Conn; } } +typedef analyzer::imap::IMAP_Analyzer* IMAPAnalyzer; + +#include "IMAP.h" +%} + +extern type IMAPAnalyzer; + +analyzer IMAP withcontext { + connection: IMAP_Conn; + flow: IMAP_Flow; +}; + +connection IMAP_Conn(bro_analyzer: IMAPAnalyzer) { + upflow = IMAP_Flow(true); + downflow = IMAP_Flow(false); +}; + +%include imap-protocol.pac + +flow IMAP_Flow(is_orig: bool) { + datagram = IMAP_PDU(is_orig) withcontext(connection, this); +}; + +%include imap-analyzer.pac diff --git a/src/analyzer/protocol/smtp/SMTP.cc b/src/analyzer/protocol/smtp/SMTP.cc index efc55ecc74..8296f83cb3 100644 --- a/src/analyzer/protocol/smtp/SMTP.cc +++ b/src/analyzer/protocol/smtp/SMTP.cc @@ -756,6 +756,7 @@ void SMTP_Analyzer::UpdateState(const int cmd_code, const int reply_code, bool o break; case SMTP_CMD_STARTTLS: + case SMTP_CMD_X_ANONYMOUSTLS: if ( st != SMTP_READY ) UnexpectedCommand(cmd_code, reply_code); @@ -818,6 +819,10 @@ int SMTP_Analyzer::ParseCmd(int cmd_len, const char* cmd) if ( ! cmd ) return -1; + // special case because we cannot define our usual macros with "-" + if ( strncmp(cmd, "X-ANONYMOUSTLS", cmd_len) == 0 ) + return SMTP_CMD_X_ANONYMOUSTLS; + for ( int code = SMTP_CMD_EHLO; code < SMTP_CMD_LAST; ++code ) if ( ! strncasecmp(cmd, smtp_cmd_word[code - SMTP_CMD_EHLO], cmd_len) ) return code; diff --git a/src/analyzer/protocol/smtp/SMTP.h b/src/analyzer/protocol/smtp/SMTP.h index e8010d9aef..b4396f28f7 100644 --- a/src/analyzer/protocol/smtp/SMTP.h +++ b/src/analyzer/protocol/smtp/SMTP.h @@ -30,7 +30,7 @@ typedef enum { SMTP_IN_DATA, // 6: after DATA SMTP_AFTER_DATA, // 7: after . and before reply SMTP_IN_AUTH, // 8: after AUTH and 334 - SMTP_IN_TLS, // 9: after STARTTLS and 220 + SMTP_IN_TLS, // 9: after STARTTLS/X-ANONYMOUSTLS and 220 SMTP_QUIT, // 10: after QUIT SMTP_AFTER_GAP, // 11: after a gap is detected SMTP_GAP_RECOVERY, // 12: after the first reply after a gap diff --git a/src/analyzer/protocol/smtp/SMTP_cmd.def b/src/analyzer/protocol/smtp/SMTP_cmd.def index 545136048d..72ef292d17 100644 --- a/src/analyzer/protocol/smtp/SMTP_cmd.def +++ b/src/analyzer/protocol/smtp/SMTP_cmd.def @@ -11,6 +11,8 @@ SMTP_CMD_DEF(VRFY) SMTP_CMD_DEF(EXPN) SMTP_CMD_DEF(HELP) SMTP_CMD_DEF(NOOP) +SMTP_CMD_DEF(STARTTLS) // RFC 2487 +SMTP_CMD_DEF(X_ANONYMOUSTLS) // The following two commands never explicitly appear in user input. SMTP_CMD_DEF(CONN_ESTABLISHMENT) // not an explicit SMTP command @@ -20,15 +22,14 @@ SMTP_CMD_DEF(END_OF_DATA) // not an explicit SMTP command // become deprecated (RFC 2821). // Client SHOULD NOT use SEND/SOML/SAML -SMTP_CMD_DEF(SEND) +SMTP_CMD_DEF(SEND) SMTP_CMD_DEF(SOML) SMTP_CMD_DEF(SAML) // System SHOULD NOT support TURN in absence of authentication. -SMTP_CMD_DEF(TURN) +SMTP_CMD_DEF(TURN) // SMTP extensions not supported yet. -SMTP_CMD_DEF(STARTTLS) // RFC 2487 SMTP_CMD_DEF(BDAT) // RFC 3030 SMTP_CMD_DEF(ETRN) // RFC 1985 SMTP_CMD_DEF(AUTH) // RFC 2554 diff --git a/src/analyzer/protocol/smtp/events.bif b/src/analyzer/protocol/smtp/events.bif index cffe3ba202..898e98e0d1 100644 --- a/src/analyzer/protocol/smtp/events.bif +++ b/src/analyzer/protocol/smtp/events.bif @@ -99,8 +99,8 @@ event smtp_data%(c: connection, is_orig: bool, data: string%); ## .. bro:see:: smtp_data smtp_request smtp_reply event smtp_unexpected%(c: connection, is_orig: bool, msg: string, detail: string%); -## Generated if a connection switched to using TLS using STARTTLS. After this -## event no more SMTP events will be raised for the connection. See the SSL +## Generated if a connection switched to using TLS using STARTTLS or X-ANONYMOUSTLS. +## After this event no more SMTP events will be raised for the connection. See the SSL ## analyzer for related SSL events, which will now be generated. ## ## c: The connection. diff --git a/src/analyzer/protocol/ssh/events.bif b/src/analyzer/protocol/ssh/events.bif index 57b736ac85..2c8079d9b7 100644 --- a/src/analyzer/protocol/ssh/events.bif +++ b/src/analyzer/protocol/ssh/events.bif @@ -120,7 +120,7 @@ event ssh1_server_host_key%(c: connection, p: string, e: string%); ## This event is generated when an :abbr:`SSH (Secure Shell)` ## encrypted packet is seen. This event is not handled by default, but ## is provided for heuristic analysis scripts. Note that you have to set -## :bro:id:`SSH::skip_processing_after_detection` to false to use this +## :bro:id:`SSH::disable_analyzer_after_detection` to false to use this ## event. This carries a performance penalty. ## ## c: The connection over which the :abbr:`SSH (Secure Shell)` diff --git a/src/analyzer/protocol/ssl/DTLS.cc b/src/analyzer/protocol/ssl/DTLS.cc index c90e414031..5301e962d4 100644 --- a/src/analyzer/protocol/ssl/DTLS.cc +++ b/src/analyzer/protocol/ssl/DTLS.cc @@ -35,6 +35,11 @@ void DTLS_Analyzer::Done() void DTLS_Analyzer::DeliverPacket(int len, const u_char* data, bool orig, uint64 seq, const IP_Hdr* ip, int caplen) { Analyzer::DeliverPacket(len, data, orig, seq, ip, caplen); + + // In this case the packet is a STUN packet. Skip it without complaining. + if ( len > 20 && data[4] == 0x21 && data[5] == 0x12 && data[6] == 0xa4 && data[7] == 0x42 ) + return; + interp->NewData(orig, data, data + len); } diff --git a/src/analyzer/protocol/ssl/tls-handshake-protocol.pac b/src/analyzer/protocol/ssl/tls-handshake-protocol.pac index b24352d099..3b65e63ee7 100644 --- a/src/analyzer/protocol/ssl/tls-handshake-protocol.pac +++ b/src/analyzer/protocol/ssl/tls-handshake-protocol.pac @@ -75,7 +75,7 @@ type ClientHello(rec: HandshakeRecord) = record { session_len : uint8; session_id : uint8[session_len]; dtls_cookie: case client_version of { - DTLSv10 -> cookie: ClientHelloCookie(rec); + DTLSv10, DTLSv12 -> cookie: ClientHelloCookie(rec); default -> nothing: bytestring &length=0; }; csuit_len : uint16 &check(csuit_len > 1 && csuit_len % 2 == 0); diff --git a/src/analyzer/protocol/tcp/TCP.cc b/src/analyzer/protocol/tcp/TCP.cc index 8b3876c7ce..56c01fa358 100644 --- a/src/analyzer/protocol/tcp/TCP.cc +++ b/src/analyzer/protocol/tcp/TCP.cc @@ -408,11 +408,6 @@ void TCP_Analyzer::EnableReassembly() TCP_Reassembler::Forward, orig), new TCP_Reassembler(this, this, TCP_Reassembler::Forward, resp)); - - reassembling = 1; - - if ( new_connection_contents ) - Event(new_connection_contents); } void TCP_Analyzer::SetReassembler(TCP_Reassembler* rorig, @@ -423,10 +418,10 @@ void TCP_Analyzer::SetReassembler(TCP_Reassembler* rorig, resp->AddReassembler(rresp); rresp->SetDstAnalyzer(this); - reassembling = 1; - - if ( new_connection_contents ) + if ( new_connection_contents && reassembling == 0 ) Event(new_connection_contents); + + reassembling = 1; } const struct tcphdr* TCP_Analyzer::ExtractTCP_Header(const u_char*& data, diff --git a/src/analyzer/protocol/tcp/TCP_Reassembler.cc b/src/analyzer/protocol/tcp/TCP_Reassembler.cc index 5b88d2dafb..0095947071 100644 --- a/src/analyzer/protocol/tcp/TCP_Reassembler.cc +++ b/src/analyzer/protocol/tcp/TCP_Reassembler.cc @@ -5,9 +5,6 @@ #include "analyzer/protocol/tcp/TCP.h" #include "TCP_Endpoint.h" -// Only needed for gap_report events. -#include "Event.h" - #include "events.bif.h" using namespace analyzer::tcp; @@ -18,17 +15,11 @@ const bool DEBUG_tcp_contents = false; const bool DEBUG_tcp_connection_close = false; const bool DEBUG_tcp_match_undelivered = false; -static double last_gap_report = 0.0; -static uint64 last_ack_events = 0; -static uint64 last_ack_bytes = 0; -static uint64 last_gap_events = 0; -static uint64 last_gap_bytes = 0; - TCP_Reassembler::TCP_Reassembler(analyzer::Analyzer* arg_dst_analyzer, TCP_Analyzer* arg_tcp_analyzer, TCP_Reassembler::Type arg_type, TCP_Endpoint* arg_endp) - : Reassembler(1) + : Reassembler(1, REASSEM_TCP) { dst_analyzer = arg_dst_analyzer; tcp_analyzer = arg_tcp_analyzer; @@ -45,7 +36,7 @@ TCP_Reassembler::TCP_Reassembler(analyzer::Analyzer* arg_dst_analyzer, if ( tcp_max_old_segments ) SetMaxOldBlocks(tcp_max_old_segments); - if ( tcp_contents ) + if ( ::tcp_contents ) { // Val dst_port_val(ntohs(Conn()->RespPort()), TYPE_PORT); PortVal dst_port_val(ntohs(tcp_analyzer->Conn()->RespPort()), @@ -387,7 +378,6 @@ void TCP_Reassembler::BlockInserted(DataBlock* start_block) { // New stuff. uint64 len = b->Size(); uint64 seq = last_reassem_seq; - last_reassem_seq += len; if ( record_contents_file ) @@ -548,35 +538,6 @@ void TCP_Reassembler::AckReceived(uint64 seq) tot_gap_bytes += num_missing; tcp_analyzer->Event(ack_above_hole); } - - double dt = network_time - last_gap_report; - - if ( gap_report && gap_report_freq > 0.0 && - dt >= gap_report_freq ) - { - uint64 devents = tot_ack_events - last_ack_events; - uint64 dbytes = tot_ack_bytes - last_ack_bytes; - uint64 dgaps = tot_gap_events - last_gap_events; - uint64 dgap_bytes = tot_gap_bytes - last_gap_bytes; - - RecordVal* r = new RecordVal(gap_info); - r->Assign(0, new Val(devents, TYPE_COUNT)); - r->Assign(1, new Val(dbytes, TYPE_COUNT)); - r->Assign(2, new Val(dgaps, TYPE_COUNT)); - r->Assign(3, new Val(dgap_bytes, TYPE_COUNT)); - - val_list* vl = new val_list; - vl->append(new IntervalVal(dt, Seconds)); - vl->append(r); - - mgr.QueueEvent(gap_report, vl); - - last_gap_report = network_time; - last_ack_events = tot_ack_events; - last_ack_bytes = tot_ack_bytes; - last_gap_events = tot_gap_events; - last_gap_bytes = tot_gap_bytes; - } } // Check EOF here because t_reassem->LastReassemSeq() may have diff --git a/src/analyzer/protocol/tcp/functions.bif b/src/analyzer/protocol/tcp/functions.bif index 9fca05329a..75353180c6 100644 --- a/src/analyzer/protocol/tcp/functions.bif +++ b/src/analyzer/protocol/tcp/functions.bif @@ -63,26 +63,6 @@ function get_resp_seq%(cid: conn_id%): count } %} -## Returns statistics about TCP gaps. -## -## Returns: A record with TCP gap statistics. -## -## .. bro:see:: do_profiling -## net_stats -## resource_usage -## dump_rule_stats -## get_matcher_stats -function get_gap_summary%(%): gap_info - %{ - RecordVal* r = new RecordVal(gap_info); - r->Assign(0, new Val(tot_ack_events, TYPE_COUNT)); - r->Assign(1, new Val(tot_ack_bytes, TYPE_COUNT)); - r->Assign(2, new Val(tot_gap_events, TYPE_COUNT)); - r->Assign(3, new Val(tot_gap_bytes, TYPE_COUNT)); - - return r; - %} - ## Associates a file handle with a connection for writing TCP byte stream ## contents. ## diff --git a/src/analyzer/protocol/xmpp/CMakeLists.txt b/src/analyzer/protocol/xmpp/CMakeLists.txt new file mode 100644 index 0000000000..ec5bb84837 --- /dev/null +++ b/src/analyzer/protocol/xmpp/CMakeLists.txt @@ -0,0 +1,12 @@ + +include(BroPlugin) + +include_directories(BEFORE ${CMAKE_CURRENT_SOURCE_DIR} ${CMAKE_CURRENT_BINARY_DIR}) + +bro_plugin_begin(Bro XMPP) +bro_plugin_cc(Plugin.cc) +bro_plugin_cc(XMPP.cc) +bro_plugin_bif(events.bif) +bro_plugin_pac(xmpp.pac xmpp-analyzer.pac xmpp-protocol.pac) +bro_plugin_end() + diff --git a/src/analyzer/protocol/xmpp/Plugin.cc b/src/analyzer/protocol/xmpp/Plugin.cc new file mode 100644 index 0000000000..d3bfcc5b10 --- /dev/null +++ b/src/analyzer/protocol/xmpp/Plugin.cc @@ -0,0 +1,23 @@ +// See the file in the main distribution directory for copyright. +#include "plugin/Plugin.h" + +#include "XMPP.h" + +namespace plugin { +namespace Bro_XMPP { + +class Plugin : public plugin::Plugin { +public: + plugin::Configuration Configure() + { + AddComponent(new ::analyzer::Component("XMPP", ::analyzer::xmpp::XMPP_Analyzer::Instantiate)); + + plugin::Configuration config; + config.name = "Bro::XMPP"; + config.description = "XMPP analyzer (StartTLS only)"; + return config; + } +} plugin; + +} +} diff --git a/src/analyzer/protocol/xmpp/XMPP.cc b/src/analyzer/protocol/xmpp/XMPP.cc new file mode 100644 index 0000000000..72229aeaba --- /dev/null +++ b/src/analyzer/protocol/xmpp/XMPP.cc @@ -0,0 +1,85 @@ +// See the file "COPYING" in the main distribution directory for copyright. + +#include "XMPP.h" +#include "analyzer/protocol/tcp/TCP_Reassembler.h" +#include "analyzer/Manager.h" + +using namespace analyzer::xmpp; + +XMPP_Analyzer::XMPP_Analyzer(Connection* conn) + : tcp::TCP_ApplicationAnalyzer("XMPP", conn) + { + interp = unique_ptr(new binpac::XMPP::XMPP_Conn(this)); + had_gap = false; + tls_active = false; + } + +XMPP_Analyzer::~XMPP_Analyzer() + { + } + +void XMPP_Analyzer::Done() + { + tcp::TCP_ApplicationAnalyzer::Done(); + + interp->FlowEOF(true); + interp->FlowEOF(false); + } + +void XMPP_Analyzer::EndpointEOF(bool is_orig) + { + tcp::TCP_ApplicationAnalyzer::EndpointEOF(is_orig); + interp->FlowEOF(is_orig); + } + +void XMPP_Analyzer::DeliverStream(int len, const u_char* data, bool orig) + { + tcp::TCP_ApplicationAnalyzer::DeliverStream(len, data, orig); + + if ( tls_active ) + { + // If TLS has been initiated, forward to child and abort further + // processing + ForwardStream(len, data, orig); + return; + } + + assert(TCP()); + if ( TCP()->IsPartial() ) + return; + + if ( had_gap ) + // If only one side had a content gap, we could still try to + // deliver data to the other side if the script layer can + // handle this. + return; + + try + { + interp->NewData(orig, data, data + len); + } + catch ( const binpac::Exception& e ) + { + ProtocolViolation(fmt("Binpac exception: %s", e.c_msg())); + } + } + +void XMPP_Analyzer::Undelivered(uint64 seq, int len, bool orig) + { + tcp::TCP_ApplicationAnalyzer::Undelivered(seq, len, orig); + had_gap = true; + interp->NewGap(orig, len); + } + +void XMPP_Analyzer::StartTLS() + { + // StartTLS was called. This means we saw a client starttls followed + // by a server proceed. From here on, everything should be a binary + // TLS datastream. + + tls_active = true; + + Analyzer* ssl = analyzer_mgr->InstantiateAnalyzer("SSL", Conn()); + if ( ssl ) + AddChildAnalyzer(ssl); + } diff --git a/src/analyzer/protocol/xmpp/XMPP.h b/src/analyzer/protocol/xmpp/XMPP.h new file mode 100644 index 0000000000..202403748a --- /dev/null +++ b/src/analyzer/protocol/xmpp/XMPP.h @@ -0,0 +1,38 @@ +// See the file "COPYING" in the main distribution directory for copyright. + +#ifndef ANALYZER_PROTOCOL_XMPP_XMPP_H +#define ANALYZER_PROTOCOL_XMPP_XMPP_H + +#include "analyzer/protocol/tcp/TCP.h" + +#include "xmpp_pac.h" + +namespace analyzer { namespace xmpp { + +class XMPP_Analyzer : public tcp::TCP_ApplicationAnalyzer { +public: + XMPP_Analyzer(Connection* conn); + virtual ~XMPP_Analyzer(); + + void Done() override; + void DeliverStream(int len, const u_char* data, bool orig) override; + void Undelivered(uint64 seq, int len, bool orig) override; + + // Overriden from tcp::TCP_ApplicationAnalyzer. + void EndpointEOF(bool is_orig) override; + + void StartTLS(); + + static analyzer::Analyzer* Instantiate(Connection* conn) + { return new XMPP_Analyzer(conn); } + +protected: + std::unique_ptr interp; + bool had_gap; + + bool tls_active; +}; + +} } // namespace analyzer::* + +#endif /* ANALYZER_PROTOCOL_XMPP_XMPP_H */ diff --git a/src/analyzer/protocol/xmpp/events.bif b/src/analyzer/protocol/xmpp/events.bif new file mode 100644 index 0000000000..ee36bd5333 --- /dev/null +++ b/src/analyzer/protocol/xmpp/events.bif @@ -0,0 +1,5 @@ +## Generated when a XMPP connection goes encrypted after a successful +## StartTLS exchange between the client and the server. +## +## c: The connection. +event xmpp_starttls%(c: connection%); diff --git a/src/analyzer/protocol/xmpp/xmpp-analyzer.pac b/src/analyzer/protocol/xmpp/xmpp-analyzer.pac new file mode 100644 index 0000000000..3240b57bb3 --- /dev/null +++ b/src/analyzer/protocol/xmpp/xmpp-analyzer.pac @@ -0,0 +1,45 @@ +refine connection XMPP_Conn += { + + %member{ + bool client_starttls; + %} + + %init{ + client_starttls = false; + %} + + function proc_xmpp_token(is_orig: bool, name: bytestring, rest: bytestring): bool + %{ + string token = std_str(name); + + if ( is_orig && token == "stream:stream" ) + // Yup, looks like xmpp... + bro_analyzer()->ProtocolConfirmation(); + + if ( token == "success" || token == "message" || token == "db:result" + || token == "db:verify" || token == "presence" ) + // Handshake has passed the phase where we should see StartTLS. Simply skip from hereon... + bro_analyzer()->SetSkip(true); + + if ( is_orig && token == "starttls" ) + client_starttls = true; + + if ( !is_orig && token == "proceed" && client_starttls ) + { + bro_analyzer()->StartTLS(); + BifEvent::generate_xmpp_starttls(bro_analyzer(), bro_analyzer()->Conn()); + } + else if ( !is_orig && token == "proceed" ) + reporter->Weird(bro_analyzer()->Conn(), "XMPP: proceed without starttls"); + + //printf("Processed: %d %s %s \n", is_orig, c_str(name), c_str(rest)); + + return true; + %} + +}; + +refine typeattr XMPP_TOKEN += &let { + proc: bool = $context.connection.proc_xmpp_token(is_orig, name, rest); +}; + diff --git a/src/analyzer/protocol/xmpp/xmpp-protocol.pac b/src/analyzer/protocol/xmpp/xmpp-protocol.pac new file mode 100644 index 0000000000..9b21679c30 --- /dev/null +++ b/src/analyzer/protocol/xmpp/xmpp-protocol.pac @@ -0,0 +1,18 @@ +type XML_START = RE//; +type XML_NAME = RE/\/?[?:[:alnum:]]+/; +type XML_REST = RE/[^<>]*/; +type SPACING = RE/[ \r\n]*/; +type CONTENT = RE/[^<>]*/; + +type XMPP_PDU(is_orig: bool) = XMPP_TOKEN(is_orig)[] &until($input.length() == 0); + +type XMPP_TOKEN(is_orig: bool) = record { + : SPACING; + : XML_START; + name: XML_NAME; + rest: XML_REST; + : XML_END; + tagcontent: CONTENT; +}; + diff --git a/src/analyzer/protocol/xmpp/xmpp.pac b/src/analyzer/protocol/xmpp/xmpp.pac new file mode 100644 index 0000000000..e6b5f4bba0 --- /dev/null +++ b/src/analyzer/protocol/xmpp/xmpp.pac @@ -0,0 +1,38 @@ +# binpac file for the XMPP analyzer. +# Note that we currently do not even try to parse the protocol +# completely -- this is only supposed to be able to parse xmpp +# till StartTLS does (or does not) kick in. + +%include binpac.pac +%include bro.pac + + +%extern{ +#include "events.bif.h" + +namespace analyzer { namespace xmpp { class XMPP_Analyzer; } } +namespace binpac { namespace XMPP { class XMPP_Conn; } } +typedef analyzer::xmpp::XMPP_Analyzer* XMPPAnalyzer; + +#include "XMPP.h" +%} + +extern type XMPPAnalyzer; + +analyzer XMPP withcontext { + connection: XMPP_Conn; + flow: XMPP_Flow; +}; + +connection XMPP_Conn(bro_analyzer: XMPPAnalyzer) { + upflow = XMPP_Flow(true); + downflow = XMPP_Flow(false); +}; + +%include xmpp-protocol.pac + +flow XMPP_Flow(is_orig: bool) { + datagram = XMPP_PDU(is_orig) withcontext(connection, this); +}; + +%include xmpp-analyzer.pac diff --git a/src/bro.bif b/src/bro.bif index 2c55c2bc95..ee3add586d 100644 --- a/src/bro.bif +++ b/src/bro.bif @@ -26,15 +26,8 @@ using namespace std; -RecordType* net_stats; -RecordType* bro_resources; -RecordType* matcher_stats; TableType* var_sizes; -// This one is extern, since it's used beyond just built-ins, -// and hence it's declared in NetVar.{h,cc}. -extern RecordType* gap_info; - static iosource::PktDumper* addl_pkt_dumper = 0; bro_int_t parse_int(const char*& fmt) @@ -145,12 +138,17 @@ static void do_fmt(const char*& fmt, Val* v, ODesc* d) } time_t time = time_t(v->InternalDouble()); + struct tm t; + int is_time_fmt = *fmt == 'T'; + if ( ! localtime_r(&time, &t) ) + s.AddSP(""); + if ( ! strftime(out_buf, sizeof(out_buf), is_time_fmt ? "%Y-%m-%d-%H:%M" : "%Y-%m-%d-%H:%M:%S", - localtime(&time)) ) + &t) ) s.AddSP(""); else @@ -1480,8 +1478,6 @@ function cat_sep%(sep: string, def: string, ...%): string ## ## - ``.``: Precision of floating point specifiers ``[efg]`` (< 128) ## -## - ``A``: Escape only NUL bytes (each one replaced with ``\0``) in a string -## ## - ``[DTdxsefg]``: Format specifier ## ## - ``[DT]``: ISO timestamp with microsecond precision @@ -1727,156 +1723,6 @@ function reading_traces%(%): bool return new Val(reading_traces, TYPE_BOOL); %} -## Returns packet capture statistics. Statistics include the number of -## packets *(i)* received by Bro, *(ii)* dropped, and *(iii)* seen on the -## link (not always available). -## -## Returns: A record of packet statistics. -## -## .. bro:see:: do_profiling -## resource_usage -## get_matcher_stats -## dump_rule_stats -## get_gap_summary -function net_stats%(%): NetStats - %{ - unsigned int recv = 0; - unsigned int drop = 0; - unsigned int link = 0; - unsigned int bytes_recv = 0; - - const iosource::Manager::PktSrcList& pkt_srcs(iosource_mgr->GetPktSrcs()); - - for ( iosource::Manager::PktSrcList::const_iterator i = pkt_srcs.begin(); - i != pkt_srcs.end(); i++ ) - { - iosource::PktSrc* ps = *i; - - struct iosource::PktSrc::Stats stat; - ps->Statistics(&stat); - recv += stat.received; - drop += stat.dropped; - link += stat.link; - bytes_recv += stat.bytes_received; - } - - RecordVal* ns = new RecordVal(net_stats); - ns->Assign(0, new Val(recv, TYPE_COUNT)); - ns->Assign(1, new Val(drop, TYPE_COUNT)); - ns->Assign(2, new Val(link, TYPE_COUNT)); - ns->Assign(3, new Val(bytes_recv, TYPE_COUNT)); - - return ns; - %} - -## Returns Bro process statistics. Statistics include real/user/sys CPU time, -## memory usage, page faults, number of TCP/UDP/ICMP connections, timers, -## and events queued/dispatched. -## -## Returns: A record with resource usage statistics. -## -## .. bro:see:: do_profiling -## net_stats -## get_matcher_stats -## dump_rule_stats -## get_gap_summary -function resource_usage%(%): bro_resources - %{ - struct rusage r; - - if ( getrusage(RUSAGE_SELF, &r) < 0 ) - reporter->InternalError("getrusage() failed in bro_resource_usage()"); - - double elapsed_time = current_time() - bro_start_time; - - double user_time = - double(r.ru_utime.tv_sec) + double(r.ru_utime.tv_usec) / 1e6; - double system_time = - double(r.ru_stime.tv_sec) + double(r.ru_stime.tv_usec) / 1e6; - - RecordVal* res = new RecordVal(bro_resources); - int n = 0; - - res->Assign(n++, new StringVal(bro_version())); - -#ifdef DEBUG - res->Assign(n++, new Val(1, TYPE_COUNT)); -#else - res->Assign(n++, new Val(0, TYPE_COUNT)); -#endif - - res->Assign(n++, new Val(bro_start_time, TYPE_TIME)); - - res->Assign(n++, new IntervalVal(elapsed_time, Seconds)); - res->Assign(n++, new IntervalVal(user_time, Seconds)); - res->Assign(n++, new IntervalVal(system_time, Seconds)); - - unsigned int total_mem; - get_memory_usage(&total_mem, 0); - res->Assign(n++, new Val(unsigned(total_mem), TYPE_COUNT)); - - res->Assign(n++, new Val(unsigned(r.ru_minflt), TYPE_COUNT)); - res->Assign(n++, new Val(unsigned(r.ru_majflt), TYPE_COUNT)); - res->Assign(n++, new Val(unsigned(r.ru_nswap), TYPE_COUNT)); - res->Assign(n++, new Val(unsigned(r.ru_inblock), TYPE_COUNT)); - res->Assign(n++, new Val(unsigned(r.ru_oublock), TYPE_COUNT)); - res->Assign(n++, new Val(unsigned(r.ru_nivcsw), TYPE_COUNT)); - - SessionStats s; - if ( sessions ) - sessions->GetStats(s); - -#define ADD_STAT(x) \ - res->Assign(n++, new Val(unsigned(sessions ? x : 0), TYPE_COUNT)); - - ADD_STAT(s.num_TCP_conns); - ADD_STAT(s.num_UDP_conns); - ADD_STAT(s.num_ICMP_conns); - ADD_STAT(s.num_fragments); - ADD_STAT(s.num_packets); - ADD_STAT(s.num_timers); - ADD_STAT(s.num_events_queued); - ADD_STAT(s.num_events_dispatched); - ADD_STAT(s.max_TCP_conns); - ADD_STAT(s.max_UDP_conns); - ADD_STAT(s.max_ICMP_conns); - ADD_STAT(s.max_fragments); - ADD_STAT(s.max_timers); - - return res; - %} - -## Returns statistics about the regular expression engine. Statistics include -## the number of distinct matchers, DFA states, DFA state transitions, memory -## usage of DFA states, cache hits/misses, and average number of NFA states -## across all matchers. -## -## Returns: A record with matcher statistics. -## -## .. bro:see:: do_profiling -## net_stats -## resource_usage -## dump_rule_stats -## get_gap_summary -function get_matcher_stats%(%): matcher_stats - %{ - RuleMatcher::Stats s; - memset(&s, 0, sizeof(s)); - - if ( rule_matcher ) - rule_matcher->GetStats(&s); - - RecordVal* r = new RecordVal(matcher_stats); - r->Assign(0, new Val(s.matchers, TYPE_COUNT)); - r->Assign(1, new Val(s.dfa_states, TYPE_COUNT)); - r->Assign(2, new Val(s.computed, TYPE_COUNT)); - r->Assign(3, new Val(s.mem, TYPE_COUNT)); - r->Assign(4, new Val(s.hits, TYPE_COUNT)); - r->Assign(5, new Val(s.misses, TYPE_COUNT)); - r->Assign(6, new Val(s.avg_nfa_states, TYPE_COUNT)); - - return r; - %} ## Generates a table of the size of all global variables. The table index is ## the variable name and the value is the variable size in bytes. @@ -2014,11 +1860,17 @@ function record_fields%(rec: any%): record_field_table ## timers, and script-level state. The script variable :bro:id:`profiling_file` ## holds the name of the file. ## -## .. bro:see:: net_stats -## resource_usage +## .. bro:see:: get_conn_stats +## get_dns_stats +## get_event_stats +## get_file_analysis_stats +## get_gap_stats ## get_matcher_stats -## dump_rule_stats -## get_gap_summary +## get_net_stats +## get_proc_stats +## get_reassembler_stats +## get_thread_stats +## get_timer_stats function do_profiling%(%) : any %{ if ( profiling_logger ) @@ -2080,13 +1932,7 @@ function is_local_interface%(ip: addr%) : bool ## ## Returns: True (unconditionally). ## -## .. bro:see:: do_profiling -## resource_usage -## get_matcher_stats -## net_stats -## get_gap_summary -## -## .. todo:: The return value should be changed to any or check appropriately. +## .. bro:see:: get_matcher_stats function dump_rule_stats%(f: file%): bool %{ if ( rule_matcher ) @@ -2467,7 +2313,7 @@ function to_subnet%(sn: string%): subnet ## ## Returns: The *a* address as a :bro:type:`subnet`. ## -## .. bro:see:: to_subset +## .. bro:see:: to_subnet function addr_to_subnet%(a: addr%): subnet %{ int width = (a->AsAddr().GetFamily() == IPv4 ? 32 : 128); @@ -2481,7 +2327,7 @@ function addr_to_subnet%(a: addr%): subnet ## ## Returns: The *s* subnet as a :bro:type:`addr`. ## -## .. bro:see:: to_subset +## .. bro:see:: to_subnet function subnet_to_addr%(sn: subnet%): addr %{ return new AddrVal(sn->Prefix()); @@ -2493,7 +2339,7 @@ function subnet_to_addr%(sn: subnet%): addr ## ## Returns: The width of the subnet. ## -## .. bro:see:: to_subset +## .. bro:see:: to_subnet function subnet_width%(sn: subnet%): count %{ return new Val(sn->Width(), TYPE_COUNT); @@ -3142,9 +2988,11 @@ function strftime%(fmt: string, d: time%) : string %{ static char buffer[128]; - time_t t = time_t(d); + time_t timeval = time_t(d); + struct tm t; - if ( strftime(buffer, 128, fmt->CheckString(), localtime(&t)) == 0 ) + if ( ! localtime_r(&timeval, &t) || + ! strftime(buffer, 128, fmt->CheckString(), &t) ) return new StringVal(""); return new StringVal(buffer); @@ -3162,9 +3010,10 @@ function strftime%(fmt: string, d: time%) : string function strptime%(fmt: string, d: string%) : time %{ const time_t timeval = time_t(); - struct tm t = *localtime(&timeval); + struct tm t; - if ( strptime(d->CheckString(), fmt->CheckString(), &t) == NULL ) + if ( ! localtime_r(&timeval, &t) || + ! strptime(d->CheckString(), fmt->CheckString(), &t) ) { reporter->Warning("strptime conversion failed: fmt:%s d:%s", fmt->CheckString(), d->CheckString()); return new Val(0.0, TYPE_TIME); @@ -3458,6 +3307,26 @@ function get_current_packet%(%) : pcap_packet return pkt; %} +## Function to get the raw headers of the currently processed packet. +## +## Returns: The :bro:type:`raw_pkt_hdr` record containing the Layer 2, 3 and +## 4 headers of the currently processed packet. +## +## .. bro:see:: raw_pkt_hdr get_current_packet +function get_current_packet_header%(%) : raw_pkt_hdr + %{ + const Packet* p; + + if ( current_pktsrc && + current_pktsrc->GetCurrentPacket(&p) ) + { + return p->BuildPktHdrVal(); + } + + RecordVal* hdr = new RecordVal(raw_pkt_hdr_type); + return hdr; + %} + ## Writes a given packet to a file. ## ## pkt: The PCAP packet. diff --git a/src/broker/Data.cc b/src/broker/Data.cc index 8f66427bb5..bc4197a974 100644 --- a/src/broker/Data.cc +++ b/src/broker/Data.cc @@ -318,25 +318,27 @@ struct val_converter { auto rt = type->AsRecordType(); auto rval = new RecordVal(rt); + auto idx = 0u; for ( auto i = 0u; i < static_cast(rt->NumFields()); ++i ) { if ( require_log_attr && ! rt->FieldDecl(i)->FindAttr(ATTR_LOG) ) continue; - if ( i >= a.fields.size() ) + if ( idx >= a.fields.size() ) { Unref(rval); return nullptr; } - if ( ! a.fields[i] ) + if ( ! a.fields[idx] ) { rval->Assign(i, nullptr); + ++idx; continue; } - auto item_val = bro_broker::data_to_val(move(*a.fields[i]), + auto item_val = bro_broker::data_to_val(move(*a.fields[idx]), rt->FieldType(i)); if ( ! item_val ) @@ -346,6 +348,7 @@ struct val_converter { } rval->Assign(i, item_val); + ++idx; } return rval; @@ -539,7 +542,7 @@ broker::util::optional bro_broker::val_to_data(Val* v) return {rval}; } default: - reporter->Error("unsupported BrokerComm::Data type: %s", + reporter->Error("unsupported Broker::Data type: %s", type_name(v->Type()->Tag())); break; } @@ -549,7 +552,7 @@ broker::util::optional bro_broker::val_to_data(Val* v) RecordVal* bro_broker::make_data_val(Val* v) { - auto rval = new RecordVal(BifType::Record::BrokerComm::Data); + auto rval = new RecordVal(BifType::Record::Broker::Data); auto data = val_to_data(v); if ( data ) @@ -560,7 +563,7 @@ RecordVal* bro_broker::make_data_val(Val* v) RecordVal* bro_broker::make_data_val(broker::data d) { - auto rval = new RecordVal(BifType::Record::BrokerComm::Data); + auto rval = new RecordVal(BifType::Record::Broker::Data); rval->Assign(0, new DataVal(move(d))); return rval; } @@ -570,92 +573,92 @@ struct data_type_getter { result_type operator()(bool a) { - return new EnumVal(BifEnum::BrokerComm::BOOL, - BifType::Enum::BrokerComm::DataType); + return new EnumVal(BifEnum::Broker::BOOL, + BifType::Enum::Broker::DataType); } result_type operator()(uint64_t a) { - return new EnumVal(BifEnum::BrokerComm::COUNT, - BifType::Enum::BrokerComm::DataType); + return new EnumVal(BifEnum::Broker::COUNT, + BifType::Enum::Broker::DataType); } result_type operator()(int64_t a) { - return new EnumVal(BifEnum::BrokerComm::INT, - BifType::Enum::BrokerComm::DataType); + return new EnumVal(BifEnum::Broker::INT, + BifType::Enum::Broker::DataType); } result_type operator()(double a) { - return new EnumVal(BifEnum::BrokerComm::DOUBLE, - BifType::Enum::BrokerComm::DataType); + return new EnumVal(BifEnum::Broker::DOUBLE, + BifType::Enum::Broker::DataType); } result_type operator()(const std::string& a) { - return new EnumVal(BifEnum::BrokerComm::STRING, - BifType::Enum::BrokerComm::DataType); + return new EnumVal(BifEnum::Broker::STRING, + BifType::Enum::Broker::DataType); } result_type operator()(const broker::address& a) { - return new EnumVal(BifEnum::BrokerComm::ADDR, - BifType::Enum::BrokerComm::DataType); + return new EnumVal(BifEnum::Broker::ADDR, + BifType::Enum::Broker::DataType); } result_type operator()(const broker::subnet& a) { - return new EnumVal(BifEnum::BrokerComm::SUBNET, - BifType::Enum::BrokerComm::DataType); + return new EnumVal(BifEnum::Broker::SUBNET, + BifType::Enum::Broker::DataType); } result_type operator()(const broker::port& a) { - return new EnumVal(BifEnum::BrokerComm::PORT, - BifType::Enum::BrokerComm::DataType); + return new EnumVal(BifEnum::Broker::PORT, + BifType::Enum::Broker::DataType); } result_type operator()(const broker::time_point& a) { - return new EnumVal(BifEnum::BrokerComm::TIME, - BifType::Enum::BrokerComm::DataType); + return new EnumVal(BifEnum::Broker::TIME, + BifType::Enum::Broker::DataType); } result_type operator()(const broker::time_duration& a) { - return new EnumVal(BifEnum::BrokerComm::INTERVAL, - BifType::Enum::BrokerComm::DataType); + return new EnumVal(BifEnum::Broker::INTERVAL, + BifType::Enum::Broker::DataType); } result_type operator()(const broker::enum_value& a) { - return new EnumVal(BifEnum::BrokerComm::ENUM, - BifType::Enum::BrokerComm::DataType); + return new EnumVal(BifEnum::Broker::ENUM, + BifType::Enum::Broker::DataType); } result_type operator()(const broker::set& a) { - return new EnumVal(BifEnum::BrokerComm::SET, - BifType::Enum::BrokerComm::DataType); + return new EnumVal(BifEnum::Broker::SET, + BifType::Enum::Broker::DataType); } result_type operator()(const broker::table& a) { - return new EnumVal(BifEnum::BrokerComm::TABLE, - BifType::Enum::BrokerComm::DataType); + return new EnumVal(BifEnum::Broker::TABLE, + BifType::Enum::Broker::DataType); } result_type operator()(const broker::vector& a) { - return new EnumVal(BifEnum::BrokerComm::VECTOR, - BifType::Enum::BrokerComm::DataType); + return new EnumVal(BifEnum::Broker::VECTOR, + BifType::Enum::Broker::DataType); } result_type operator()(const broker::record& a) { - return new EnumVal(BifEnum::BrokerComm::RECORD, - BifType::Enum::BrokerComm::DataType); + return new EnumVal(BifEnum::Broker::RECORD, + BifType::Enum::Broker::DataType); } }; @@ -670,7 +673,7 @@ broker::data& bro_broker::opaque_field_to_data(RecordVal* v, Frame* f) if ( ! d ) reporter->RuntimeError(f->GetCall()->GetLocationInfo(), - "BrokerComm::Data's opaque field is not set"); + "Broker::Data's opaque field is not set"); return static_cast(d)->data; } diff --git a/src/broker/Data.h b/src/broker/Data.h index 84495056be..0045ad58ad 100644 --- a/src/broker/Data.h +++ b/src/broker/Data.h @@ -21,25 +21,25 @@ extern OpaqueType* opaque_of_record_iterator; TransportProto to_bro_port_proto(broker::port::protocol tp); /** - * Create a BrokerComm::Data value from a Bro value. + * Create a Broker::Data value from a Bro value. * @param v the Bro value to convert to a Broker data value. - * @return a BrokerComm::Data value, where the optional field is set if the conversion + * @return a Broker::Data value, where the optional field is set if the conversion * was possible, else it is unset. */ RecordVal* make_data_val(Val* v); /** - * Create a BrokerComm::Data value from a Broker data value. + * Create a Broker::Data value from a Broker data value. * @param d the Broker value to wrap in an opaque type. - * @return a BrokerComm::Data value that wraps the Broker value. + * @return a Broker::Data value that wraps the Broker value. */ RecordVal* make_data_val(broker::data d); /** - * Get the type of Broker data that BrokerComm::Data wraps. - * @param v a BrokerComm::Data value. + * Get the type of Broker data that Broker::Data wraps. + * @param v a Broker::Data value. * @param frame used to get location info upon error. - * @return a BrokerComm::DataType value. + * @return a Broker::DataType value. */ EnumVal* get_data_type(RecordVal* v, Frame* frame); @@ -141,8 +141,8 @@ struct type_name_getter { }; /** - * Retrieve Broker data value associated with a BrokerComm::Data Bro value. - * @param v a BrokerComm::Data value. + * Retrieve Broker data value associated with a Broker::Data Bro value. + * @param v a Broker::Data value. * @param f used to get location information on error. * @return a reference to the wrapped Broker data value. A runtime interpreter * exception is thrown if the the optional opaque value of \a v is not set. @@ -183,9 +183,9 @@ inline T& require_data_type(RecordVal* v, TypeTag tag, Frame* f) } /** - * Convert a BrokerComm::Data Bro value to a Bro value of a given type. + * Convert a Broker::Data Bro value to a Bro value of a given type. * @tparam a type that a Broker data variant may contain. - * @param v a BrokerComm::Data value. + * @param v a Broker::Data value. * @param tag a Bro type to convert to. * @param f used to get location information on error. * A runtime interpret exception is thrown if trying to access a type which @@ -243,7 +243,7 @@ public: RecordIterator(RecordVal* v, TypeTag tag, Frame* f) : OpaqueVal(bro_broker::opaque_of_record_iterator), - dat(require_data_type(v, TYPE_VECTOR, f)), + dat(require_data_type(v, TYPE_RECORD, f)), it(dat.fields.begin()) {} diff --git a/src/broker/Manager.cc b/src/broker/Manager.cc index 06ece6d6c1..334b7f84f5 100644 --- a/src/broker/Manager.cc +++ b/src/broker/Manager.cc @@ -77,20 +77,20 @@ bool bro_broker::Manager::Enable(Val* broker_endpoint_flags) if ( endpoint != nullptr ) return true; - auto send_flags_type = internal_type("BrokerComm::SendFlags")->AsRecordType(); + auto send_flags_type = internal_type("Broker::SendFlags")->AsRecordType(); send_flags_self_idx = require_field(send_flags_type, "self"); send_flags_peers_idx = require_field(send_flags_type, "peers"); send_flags_unsolicited_idx = require_field(send_flags_type, "unsolicited"); log_id_type = internal_type("Log::ID")->AsEnumType(); - bro_broker::opaque_of_data_type = new OpaqueType("BrokerComm::Data"); - bro_broker::opaque_of_set_iterator = new OpaqueType("BrokerComm::SetIterator"); - bro_broker::opaque_of_table_iterator = new OpaqueType("BrokerComm::TableIterator"); - bro_broker::opaque_of_vector_iterator = new OpaqueType("BrokerComm::VectorIterator"); - bro_broker::opaque_of_record_iterator = new OpaqueType("BrokerComm::RecordIterator"); - bro_broker::opaque_of_store_handle = new OpaqueType("BrokerStore::Handle"); - vector_of_data_type = new VectorType(internal_type("BrokerComm::Data")->Ref()); + bro_broker::opaque_of_data_type = new OpaqueType("Broker::Data"); + bro_broker::opaque_of_set_iterator = new OpaqueType("Broker::SetIterator"); + bro_broker::opaque_of_table_iterator = new OpaqueType("Broker::TableIterator"); + bro_broker::opaque_of_vector_iterator = new OpaqueType("Broker::VectorIterator"); + bro_broker::opaque_of_record_iterator = new OpaqueType("Broker::RecordIterator"); + bro_broker::opaque_of_store_handle = new OpaqueType("Broker::Handle"); + vector_of_data_type = new VectorType(internal_type("Broker::Data")->Ref()); auto res = broker::init(); @@ -110,7 +110,7 @@ bool bro_broker::Manager::Enable(Val* broker_endpoint_flags) } const char* name; - auto name_from_script = internal_val("BrokerComm::endpoint_name")->AsString(); + auto name_from_script = internal_val("Broker::endpoint_name")->AsString(); if ( name_from_script->Len() ) name = name_from_script->CheckString(); @@ -290,7 +290,7 @@ bool bro_broker::Manager::AutoEvent(string topic, Val* event, Val* flags) if ( event->Type()->Tag() != TYPE_FUNC ) { - reporter->Error("BrokerComm::auto_event must operate on an event"); + reporter->Error("Broker::auto_event must operate on an event"); return false; } @@ -298,7 +298,7 @@ bool bro_broker::Manager::AutoEvent(string topic, Val* event, Val* flags) if ( event_val->Flavor() != FUNC_FLAVOR_EVENT ) { - reporter->Error("BrokerComm::auto_event must operate on an event"); + reporter->Error("Broker::auto_event must operate on an event"); return false; } @@ -306,7 +306,7 @@ bool bro_broker::Manager::AutoEvent(string topic, Val* event, Val* flags) if ( ! handler ) { - reporter->Error("BrokerComm::auto_event failed to lookup event '%s'", + reporter->Error("Broker::auto_event failed to lookup event '%s'", event_val->Name()); return false; } @@ -322,7 +322,7 @@ bool bro_broker::Manager::AutoEventStop(const string& topic, Val* event) if ( event->Type()->Tag() != TYPE_FUNC ) { - reporter->Error("BrokerComm::auto_event_stop must operate on an event"); + reporter->Error("Broker::auto_event_stop must operate on an event"); return false; } @@ -330,7 +330,7 @@ bool bro_broker::Manager::AutoEventStop(const string& topic, Val* event) if ( event_val->Flavor() != FUNC_FLAVOR_EVENT ) { - reporter->Error("BrokerComm::auto_event_stop must operate on an event"); + reporter->Error("Broker::auto_event_stop must operate on an event"); return false; } @@ -338,7 +338,7 @@ bool bro_broker::Manager::AutoEventStop(const string& topic, Val* event) if ( ! handler ) { - reporter->Error("BrokerComm::auto_event_stop failed to lookup event '%s'", + reporter->Error("Broker::auto_event_stop failed to lookup event '%s'", event_val->Name()); return false; } @@ -353,7 +353,7 @@ RecordVal* bro_broker::Manager::MakeEventArgs(val_list* args) if ( ! Enabled() ) return nullptr; - auto rval = new RecordVal(BifType::Record::BrokerComm::EventArgs); + auto rval = new RecordVal(BifType::Record::Broker::EventArgs); auto arg_vec = new VectorVal(vector_of_data_type); rval->Assign(1, arg_vec); Func* func = 0; @@ -368,7 +368,7 @@ RecordVal* bro_broker::Manager::MakeEventArgs(val_list* args) if ( arg_val->Type()->Tag() != TYPE_FUNC ) { - reporter->Error("1st param of BrokerComm::event_args must be event"); + reporter->Error("1st param of Broker::event_args must be event"); return rval; } @@ -376,7 +376,7 @@ RecordVal* bro_broker::Manager::MakeEventArgs(val_list* args) if ( func->Flavor() != FUNC_FLAVOR_EVENT ) { - reporter->Error("1st param of BrokerComm::event_args must be event"); + reporter->Error("1st param of Broker::event_args must be event"); return rval; } @@ -384,7 +384,7 @@ RecordVal* bro_broker::Manager::MakeEventArgs(val_list* args) if ( num_args != args->length() - 1 ) { - reporter->Error("bad # of BrokerComm::event_args: got %d, expect %d", + reporter->Error("bad # of Broker::event_args: got %d, expect %d", args->length(), num_args + 1); return rval; } @@ -398,7 +398,7 @@ RecordVal* bro_broker::Manager::MakeEventArgs(val_list* args) if ( ! same_type((*args)[i]->Type(), expected_type) ) { rval->Assign(0, 0); - reporter->Error("BrokerComm::event_args param %d type mismatch", i); + reporter->Error("Broker::event_args param %d type mismatch", i); return rval; } @@ -408,7 +408,7 @@ RecordVal* bro_broker::Manager::MakeEventArgs(val_list* args) { Unref(data_val); rval->Assign(0, 0); - reporter->Error("BrokerComm::event_args unsupported event/params"); + reporter->Error("Broker::event_args unsupported event/params"); return rval; } @@ -584,7 +584,7 @@ struct response_converter { case broker::store::query::tag::lookup: // A boolean result means the key doesn't exist (if it did, then // the result would contain the broker::data value, not a bool). - return new RecordVal(BifType::Record::BrokerComm::Data); + return new RecordVal(BifType::Record::Broker::Data); default: return bro_broker::make_data_val(broker::data{d}); } @@ -639,36 +639,36 @@ void bro_broker::Manager::Process() { switch ( u.status ) { case broker::outgoing_connection_status::tag::established: - if ( BrokerComm::outgoing_connection_established ) + if ( Broker::outgoing_connection_established ) { val_list* vl = new val_list; vl->append(new StringVal(u.relation.remote_tuple().first)); vl->append(new PortVal(u.relation.remote_tuple().second, TRANSPORT_TCP)); vl->append(new StringVal(u.peer_name)); - mgr.QueueEvent(BrokerComm::outgoing_connection_established, vl); + mgr.QueueEvent(Broker::outgoing_connection_established, vl); } break; case broker::outgoing_connection_status::tag::disconnected: - if ( BrokerComm::outgoing_connection_broken ) + if ( Broker::outgoing_connection_broken ) { val_list* vl = new val_list; vl->append(new StringVal(u.relation.remote_tuple().first)); vl->append(new PortVal(u.relation.remote_tuple().second, TRANSPORT_TCP)); - mgr.QueueEvent(BrokerComm::outgoing_connection_broken, vl); + mgr.QueueEvent(Broker::outgoing_connection_broken, vl); } break; case broker::outgoing_connection_status::tag::incompatible: - if ( BrokerComm::outgoing_connection_incompatible ) + if ( Broker::outgoing_connection_incompatible ) { val_list* vl = new val_list; vl->append(new StringVal(u.relation.remote_tuple().first)); vl->append(new PortVal(u.relation.remote_tuple().second, TRANSPORT_TCP)); - mgr.QueueEvent(BrokerComm::outgoing_connection_incompatible, vl); + mgr.QueueEvent(Broker::outgoing_connection_incompatible, vl); } break; @@ -684,20 +684,20 @@ void bro_broker::Manager::Process() { switch ( u.status ) { case broker::incoming_connection_status::tag::established: - if ( BrokerComm::incoming_connection_established ) + if ( Broker::incoming_connection_established ) { val_list* vl = new val_list; vl->append(new StringVal(u.peer_name)); - mgr.QueueEvent(BrokerComm::incoming_connection_established, vl); + mgr.QueueEvent(Broker::incoming_connection_established, vl); } break; case broker::incoming_connection_status::tag::disconnected: - if ( BrokerComm::incoming_connection_broken ) + if ( Broker::incoming_connection_broken ) { val_list* vl = new val_list; vl->append(new StringVal(u.peer_name)); - mgr.QueueEvent(BrokerComm::incoming_connection_broken, vl); + mgr.QueueEvent(Broker::incoming_connection_broken, vl); } break; @@ -718,7 +718,7 @@ void bro_broker::Manager::Process() ps.second.received += print_messages.size(); - if ( ! BrokerComm::print_handler ) + if ( ! Broker::print_handler ) continue; for ( auto& pm : print_messages ) @@ -741,7 +741,7 @@ void bro_broker::Manager::Process() val_list* vl = new val_list; vl->append(new StringVal(move(*msg))); - mgr.QueueEvent(BrokerComm::print_handler, vl); + mgr.QueueEvent(Broker::print_handler, vl); } } diff --git a/src/broker/Manager.h b/src/broker/Manager.h index 9e1ac7a70b..9fb7b9e328 100644 --- a/src/broker/Manager.h +++ b/src/broker/Manager.h @@ -63,7 +63,7 @@ public: /** * Enable use of communication. * @param flags used to tune the local Broker endpoint's behavior. - * See the BrokerComm::EndpointFlags record type. + * See the Broker::EndpointFlags record type. * @return true if communication is successfully initialized. */ bool Enable(Val* flags); @@ -122,7 +122,7 @@ public: * of this topic name. * @param msg the string to send to peers. * @param flags tune the behavior of how the message is send. - * See the BrokerComm::SendFlags record type. + * See the Broker::SendFlags record type. * @return true if the message is sent successfully. */ bool Print(std::string topic, std::string msg, Val* flags); @@ -135,7 +135,7 @@ public: * @param msg the event to send to peers, which is the name of the event * as a string followed by all of its arguments. * @param flags tune the behavior of how the message is send. - * See the BrokerComm::SendFlags record type. + * See the Broker::SendFlags record type. * @return true if the message is sent successfully. */ bool Event(std::string topic, broker::message msg, int flags); @@ -146,9 +146,9 @@ public: * Peers advertise interest by registering a subscription to some prefix * of this topic name. * @param args the event and its arguments to send to peers. See the - * BrokerComm::EventArgs record type. + * Broker::EventArgs record type. * @param flags tune the behavior of how the message is send. - * See the BrokerComm::SendFlags record type. + * See the Broker::SendFlags record type. * @return true if the message is sent successfully. */ bool Event(std::string topic, RecordVal* args, Val* flags); @@ -160,7 +160,7 @@ public: * @param columns the data which comprises the log entry. * @param info the record type corresponding to the log's columns. * @param flags tune the behavior of how the message is send. - * See the BrokerComm::SendFlags record type. + * See the Broker::SendFlags record type. * @return true if the message is sent successfully. */ bool Log(EnumVal* stream_id, RecordVal* columns, RecordType* info, @@ -174,7 +174,7 @@ public: * of this topic name. * @param event a Bro event value. * @param flags tune the behavior of how the message is send. - * See the BrokerComm::SendFlags record type. + * See the Broker::SendFlags record type. * @return true if automatic event sending is now enabled. */ bool AutoEvent(std::string topic, Val* event, Val* flags); @@ -320,7 +320,7 @@ public: Stats ConsumeStatistics(); /** - * Convert BrokerComm::SendFlags to int flags for use with broker::send(). + * Convert Broker::SendFlags to int flags for use with broker::send(). */ static int send_flags_to_int(Val* flags); @@ -335,7 +335,7 @@ private: void Process() override; const char* Tag() override - { return "BrokerComm::Manager"; } + { return "Broker::Manager"; } broker::endpoint& Endpoint() { return *endpoint; } diff --git a/src/broker/Store.cc b/src/broker/Store.cc index f9effa6d9e..97954bb328 100644 --- a/src/broker/Store.cc +++ b/src/broker/Store.cc @@ -14,12 +14,12 @@ OpaqueType* bro_broker::opaque_of_store_handle; bro_broker::StoreHandleVal::StoreHandleVal(broker::store::identifier id, bro_broker::StoreType arg_type, - broker::util::optional arg_back, + broker::util::optional arg_back, RecordVal* backend_options, std::chrono::duration resync) : OpaqueVal(opaque_of_store_handle), store(), store_type(arg_type), backend_type(arg_back) { - using BifEnum::BrokerStore::BackendType; + using BifEnum::Broker::BackendType; std::unique_ptr backend; if ( backend_type ) @@ -91,7 +91,7 @@ bro_broker::StoreHandleVal::StoreHandleVal(broker::store::identifier id, void bro_broker::StoreHandleVal::ValDescribe(ODesc* d) const { - using BifEnum::BrokerStore::BackendType; + using BifEnum::Broker::BackendType; d->Add("broker::store::"); switch ( store_type ) { diff --git a/src/broker/Store.h b/src/broker/Store.h index 5823e0c3f8..4b673e70dc 100644 --- a/src/broker/Store.h +++ b/src/broker/Store.h @@ -25,9 +25,9 @@ enum StoreType { }; /** - * Create a BrokerStore::QueryStatus value. + * Create a Broker::QueryStatus value. * @param success whether the query status should be set to success or failure. - * @return a BrokerStore::QueryStatus value. + * @return a Broker::QueryStatus value. */ inline EnumVal* query_status(bool success) { @@ -37,34 +37,34 @@ inline EnumVal* query_status(bool success) if ( ! store_query_status ) { - store_query_status = internal_type("BrokerStore::QueryStatus")->AsEnumType(); - success_val = store_query_status->Lookup("BrokerStore", "SUCCESS"); - failure_val = store_query_status->Lookup("BrokerStore", "FAILURE"); + store_query_status = internal_type("Broker::QueryStatus")->AsEnumType(); + success_val = store_query_status->Lookup("Broker", "SUCCESS"); + failure_val = store_query_status->Lookup("Broker", "FAILURE"); } return new EnumVal(success ? success_val : failure_val, store_query_status); } /** - * @return a BrokerStore::QueryResult value that has a BrokerStore::QueryStatus indicating + * @return a Broker::QueryResult value that has a Broker::QueryStatus indicating * a failure. */ inline RecordVal* query_result() { - auto rval = new RecordVal(BifType::Record::BrokerStore::QueryResult); + auto rval = new RecordVal(BifType::Record::Broker::QueryResult); rval->Assign(0, query_status(false)); - rval->Assign(1, new RecordVal(BifType::Record::BrokerComm::Data)); + rval->Assign(1, new RecordVal(BifType::Record::Broker::Data)); return rval; } /** * @param data the result of the query. - * @return a BrokerStore::QueryResult value that has a BrokerStore::QueryStatus indicating + * @return a Broker::QueryResult value that has a Broker::QueryStatus indicating * a success. */ inline RecordVal* query_result(RecordVal* data) { - auto rval = new RecordVal(BifType::Record::BrokerStore::QueryResult); + auto rval = new RecordVal(BifType::Record::Broker::QueryResult); rval->Assign(0, query_status(true)); rval->Assign(1, data); return rval; @@ -130,7 +130,7 @@ public: StoreHandleVal(broker::store::identifier id, bro_broker::StoreType arg_type, - broker::util::optional arg_back, + broker::util::optional arg_back, RecordVal* backend_options, std::chrono::duration resync = std::chrono::seconds(1)); @@ -140,7 +140,7 @@ public: broker::store::frontend* store; bro_broker::StoreType store_type; - broker::util::optional backend_type; + broker::util::optional backend_type; protected: diff --git a/src/broker/comm.bif b/src/broker/comm.bif index f8dd546965..3bc8fa7dff 100644 --- a/src/broker/comm.bif +++ b/src/broker/comm.bif @@ -5,139 +5,102 @@ #include "broker/Manager.h" %%} -module BrokerComm; +module Broker; -type BrokerComm::EndpointFlags: record; +type Broker::EndpointFlags: record; -## Enable use of communication. -## -## flags: used to tune the local Broker endpoint behavior. -## -## Returns: true if communication is successfully initialized. -function BrokerComm::enable%(flags: EndpointFlags &default = EndpointFlags()%): bool +function Broker::__enable%(flags: EndpointFlags%): bool %{ return new Val(broker_mgr->Enable(flags), TYPE_BOOL); %} -## Changes endpoint flags originally supplied to :bro:see:`BrokerComm::enable`. -## -## flags: the new endpoint behavior flags to use. -## -## Returns: true if flags were changed. -function BrokerComm::set_endpoint_flags%(flags: EndpointFlags &default = EndpointFlags()%): bool +function Broker::__set_endpoint_flags%(flags: EndpointFlags%): bool %{ return new Val(broker_mgr->SetEndpointFlags(flags), TYPE_BOOL); %} -## Allow sending messages to peers if associated with the given topic. -## This has no effect if auto publication behavior is enabled via the flags -## supplied to :bro:see:`BrokerComm::enable` or :bro:see:`BrokerComm::set_endpoint_flags`. -## -## topic: a topic to allow messages to be published under. -## -## Returns: true if successful. -function BrokerComm::publish_topic%(topic: string%): bool +function Broker::__publish_topic%(topic: string%): bool %{ return new Val(broker_mgr->PublishTopic(topic->CheckString()), TYPE_BOOL); %} -## Disallow sending messages to peers if associated with the given topic. -## This has no effect if auto publication behavior is enabled via the flags -## supplied to :bro:see:`BrokerComm::enable` or :bro:see:`BrokerComm::set_endpoint_flags`. -## -## topic: a topic to disallow messages to be published under. -## -## Returns: true if successful. -function BrokerComm::unpublish_topic%(topic: string%): bool +function Broker::__unpublish_topic%(topic: string%): bool %{ return new Val(broker_mgr->UnpublishTopic(topic->CheckString()), TYPE_BOOL); %} ## Allow advertising interest in the given topic to peers. ## This has no effect if auto advertise behavior is enabled via the flags -## supplied to :bro:see:`BrokerComm::enable` or :bro:see:`BrokerComm::set_endpoint_flags`. +## supplied to :bro:see:`Broker::enable` or :bro:see:`Broker::set_endpoint_flags`. ## ## topic: a topic to allow advertising interest/subscription to peers. ## ## Returns: true if successful. -function BrokerComm::advertise_topic%(topic: string%): bool +function Broker::advertise_topic%(topic: string%): bool %{ return new Val(broker_mgr->AdvertiseTopic(topic->CheckString()), TYPE_BOOL); %} ## Disallow advertising interest in the given topic to peers. ## This has no effect if auto advertise behavior is enabled via the flags -## supplied to :bro:see:`BrokerComm::enable` or :bro:see:`BrokerComm::set_endpoint_flags`. +## supplied to :bro:see:`Broker::enable` or :bro:see:`Broker::set_endpoint_flags`. ## ## topic: a topic to disallow advertising interest/subscription to peers. ## ## Returns: true if successful. -function BrokerComm::unadvertise_topic%(topic: string%): bool +function Broker::unadvertise_topic%(topic: string%): bool %{ return new Val(broker_mgr->UnadvertiseTopic(topic->CheckString()), TYPE_BOOL); %} ## Generated when a connection has been established due to a previous call -## to :bro:see:`BrokerComm::connect`. +## to :bro:see:`Broker::connect`. ## ## peer_address: the address used to connect to the peer. ## ## peer_port: the port used to connect to the peer. ## ## peer_name: the name by which the peer identified itself. -event BrokerComm::outgoing_connection_established%(peer_address: string, +event Broker::outgoing_connection_established%(peer_address: string, peer_port: port, peer_name: string%); ## Generated when a previously established connection becomes broken. ## Reconnection will automatically be attempted at a frequency given -## by the original call to :bro:see:`BrokerComm::connect`. +## by the original call to :bro:see:`Broker::connect`. ## ## peer_address: the address used to connect to the peer. ## ## peer_port: the port used to connect to the peer. ## -## .. bro:see:: BrokerComm::outgoing_connection_established -event BrokerComm::outgoing_connection_broken%(peer_address: string, +## .. bro:see:: Broker::outgoing_connection_established +event Broker::outgoing_connection_broken%(peer_address: string, peer_port: port%); -## Generated when a connection via :bro:see:`BrokerComm::connect` has failed +## Generated when a connection via :bro:see:`Broker::connect` has failed ## because the remote side is incompatible. ## ## peer_address: the address used to connect to the peer. ## ## peer_port: the port used to connect to the peer. -event BrokerComm::outgoing_connection_incompatible%(peer_address: string, +event Broker::outgoing_connection_incompatible%(peer_address: string, peer_port: port%); ## Generated when a peer has established a connection with this process -## as a result of previously performing a :bro:see:`BrokerComm::listen`. +## as a result of previously performing a :bro:see:`Broker::listen`. ## ## peer_name: the name by which the peer identified itself. -event BrokerComm::incoming_connection_established%(peer_name: string%); +event Broker::incoming_connection_established%(peer_name: string%); ## Generated when a peer that previously established a connection with this ## process becomes disconnected. ## ## peer_name: the name by which the peer identified itself. ## -## .. bro:see:: BrokerComm::incoming_connection_established -event BrokerComm::incoming_connection_broken%(peer_name: string%); +## .. bro:see:: Broker::incoming_connection_established +event Broker::incoming_connection_broken%(peer_name: string%); -## Listen for remote connections. -## -## p: the TCP port to listen on. -## -## a: an address string on which to accept connections, e.g. -## "127.0.0.1". An empty string refers to @p INADDR_ANY. -## -## reuse: equivalent to behavior of SO_REUSEADDR. -## -## Returns: true if the local endpoint is now listening for connections. -## -## .. bro:see:: BrokerComm::incoming_connection_established -function BrokerComm::listen%(p: port, a: string &default = "", - reuse: bool &default = T%): bool +function Broker::__listen%(p: port, a: string, reuse: bool%): bool %{ if ( ! p->IsTCP() ) { @@ -150,22 +113,7 @@ function BrokerComm::listen%(p: port, a: string &default = "", return new Val(rval, TYPE_BOOL); %} -## Initiate a remote connection. -## -## a: an address to connect to, e.g. "localhost" or "127.0.0.1". -## -## p: the TCP port on which the remote side is listening. -## -## retry: an interval at which to retry establishing the -## connection with the remote peer if it cannot be made initially, or -## if it ever becomes disconnected. -## -## Returns: true if it's possible to try connecting with the peer and -## it's a new peer. The actual connection may not be established -## until a later point in time. -## -## .. bro:see:: BrokerComm::outgoing_connection_established -function BrokerComm::connect%(a: string, p: port, retry: interval%): bool +function Broker::__connect%(a: string, p: port, retry: interval%): bool %{ if ( ! p->IsTCP() ) { @@ -178,15 +126,7 @@ function BrokerComm::connect%(a: string, p: port, retry: interval%): bool return new Val(rval, TYPE_BOOL); %} -## Remove a remote connection. -## -## a: the address used in previous successful call to :bro:see:`BrokerComm::connect`. -## -## p: the port used in previous successful call to :bro:see:`BrokerComm::connect`. -## -## Returns: true if the arguments match a previously successful call to -## :bro:see:`BrokerComm::connect`. -function BrokerComm::disconnect%(a: string, p: port%): bool +function Broker::__disconnect%(a: string, p: port%): bool %{ if ( ! p->IsTCP() ) { diff --git a/src/broker/data.bif b/src/broker/data.bif index 9ea1ca1e86..d526d0a779 100644 --- a/src/broker/data.bif +++ b/src/broker/data.bif @@ -5,9 +5,9 @@ #include "broker/Data.h" %%} -module BrokerComm; +module Broker; -## Enumerates the possible types that :bro:see:`BrokerComm::Data` may be in +## Enumerates the possible types that :bro:see:`Broker::Data` may be in ## terms of Bro data types. enum DataType %{ BOOL, @@ -27,97 +27,48 @@ enum DataType %{ RECORD, %} -type BrokerComm::Data: record; +type Broker::Data: record; -type BrokerComm::TableItem: record; +type Broker::TableItem: record; -## Convert any Bro value to communication data. -## -## d: any Bro value to attempt to convert (not all types are supported). -## -## Returns: the converted communication data. The returned record's optional -## field will not be set if the conversion was not possible (this can -## happen if the Bro data type does not support being converted to -## communication data). -function BrokerComm::data%(d: any%): BrokerComm::Data +function Broker::__data%(d: any%): Broker::Data %{ return bro_broker::make_data_val(d); %} -## Retrieve the type of data associated with communication data. -## -## d: the communication data. -## -## Returns: the data type associated with the communication data. -function BrokerComm::data_type%(d: BrokerComm::Data%): BrokerComm::DataType +function Broker::__data_type%(d: Broker::Data%): Broker::DataType %{ return bro_broker::get_data_type(d->AsRecordVal(), frame); %} -## Convert communication data with a type of :bro:see:`BrokerComm::BOOL` to -## an actual Bro value. -## -## d: the communication data to convert. -## -## Returns: the value retrieved from the communication data. -function BrokerComm::refine_to_bool%(d: BrokerComm::Data%): bool +function Broker::__refine_to_bool%(d: Broker::Data%): bool %{ return bro_broker::refine(d->AsRecordVal(), TYPE_BOOL, frame); %} -## Convert communication data with a type of :bro:see:`BrokerComm::INT` to -## an actual Bro value. -## -## d: the communication data to convert. -## -## Returns: the value retrieved from the communication data. -function BrokerComm::refine_to_int%(d: BrokerComm::Data%): int +function Broker::__refine_to_int%(d: Broker::Data%): int %{ return bro_broker::refine(d->AsRecordVal(), TYPE_INT, frame); %} -## Convert communication data with a type of :bro:see:`BrokerComm::COUNT` to -## an actual Bro value. -## -## d: the communication data to convert. -## -## Returns: the value retrieved from the communication data. -function BrokerComm::refine_to_count%(d: BrokerComm::Data%): count +function Broker::__refine_to_count%(d: Broker::Data%): count %{ return bro_broker::refine(d->AsRecordVal(), TYPE_COUNT, frame); %} -## Convert communication data with a type of :bro:see:`BrokerComm::DOUBLE` to -## an actual Bro value. -## -## d: the communication data to convert. -## -## Returns: the value retrieved from the communication data. -function BrokerComm::refine_to_double%(d: BrokerComm::Data%): double +function Broker::__refine_to_double%(d: Broker::Data%): double %{ return bro_broker::refine(d->AsRecordVal(), TYPE_DOUBLE, frame); %} -## Convert communication data with a type of :bro:see:`BrokerComm::STRING` to -## an actual Bro value. -## -## d: the communication data to convert. -## -## Returns: the value retrieved from the communication data. -function BrokerComm::refine_to_string%(d: BrokerComm::Data%): string +function Broker::__refine_to_string%(d: Broker::Data%): string %{ return new StringVal(bro_broker::require_data_type(d->AsRecordVal(), TYPE_STRING, frame)); %} -## Convert communication data with a type of :bro:see:`BrokerComm::ADDR` to -## an actual Bro value. -## -## d: the communication data to convert. -## -## Returns: the value retrieved from the communication data. -function BrokerComm::refine_to_addr%(d: BrokerComm::Data%): addr +function Broker::__refine_to_addr%(d: Broker::Data%): addr %{ auto& a = bro_broker::require_data_type(d->AsRecordVal(), TYPE_ADDR, frame); @@ -125,13 +76,7 @@ function BrokerComm::refine_to_addr%(d: BrokerComm::Data%): addr return new AddrVal(IPAddr(*bits)); %} -## Convert communication data with a type of :bro:see:`BrokerComm::SUBNET` to -## an actual Bro value. -## -## d: the communication data to convert. -## -## Returns: the value retrieved from the communication data. -function BrokerComm::refine_to_subnet%(d: BrokerComm::Data%): subnet +function Broker::__refine_to_subnet%(d: Broker::Data%): subnet %{ auto& a = bro_broker::require_data_type(d->AsRecordVal(), TYPE_SUBNET, frame); @@ -139,71 +84,40 @@ function BrokerComm::refine_to_subnet%(d: BrokerComm::Data%): subnet return new SubNetVal(IPPrefix(IPAddr(*bits), a.length())); %} -## Convert communication data with a type of :bro:see:`BrokerComm::PORT` to -## an actual Bro value. -## -## d: the communication data to convert. -## -## Returns: the value retrieved from the communication data. -function BrokerComm::refine_to_port%(d: BrokerComm::Data%): port +function Broker::__refine_to_port%(d: Broker::Data%): port %{ auto& a = bro_broker::require_data_type(d->AsRecordVal(), - TYPE_SUBNET, frame); + TYPE_PORT, frame); return new PortVal(a.number(), bro_broker::to_bro_port_proto(a.type())); %} -## Convert communication data with a type of :bro:see:`BrokerComm::TIME` to -## an actual Bro value. -## -## d: the communication data to convert. -## -## Returns: the value retrieved from the communication data. -function BrokerComm::refine_to_time%(d: BrokerComm::Data%): time +function Broker::__refine_to_time%(d: Broker::Data%): time %{ auto v = bro_broker::require_data_type(d->AsRecordVal(), TYPE_TIME, frame).value; return new Val(v, TYPE_TIME); %} -## Convert communication data with a type of :bro:see:`BrokerComm::INTERVAL` to -## an actual Bro value. -## -## d: the communication data to convert. -## -## Returns: the value retrieved from the communication data. -function BrokerComm::refine_to_interval%(d: BrokerComm::Data%): interval +function Broker::__refine_to_interval%(d: Broker::Data%): interval %{ auto v = bro_broker::require_data_type(d->AsRecordVal(), - TYPE_TIME, frame).value; + TYPE_INTERVAL, frame).value; return new Val(v, TYPE_INTERVAL); %} -## Convert communication data with a type of :bro:see:`BrokerComm::ENUM` to -## the name of the enum value. :bro:see:`lookup_ID` may be used to convert -## the name to the actual enum value. -## -## d: the communication data to convert. -## -## Returns: the enum name retrieved from the communication data. -function BrokerComm::refine_to_enum_name%(d: BrokerComm::Data%): string +function Broker::__refine_to_enum_name%(d: Broker::Data%): string %{ auto& v = bro_broker::require_data_type(d->AsRecordVal(), TYPE_ENUM, frame).name; return new StringVal(v); %} -## Create communication data of type "set". -function BrokerComm::set_create%(%): BrokerComm::Data +function Broker::__set_create%(%): Broker::Data %{ return bro_broker::make_data_val(broker::set()); %} -## Remove all elements within a set. -## -## s: the set to clear. -## -## Returns: always true. -function BrokerComm::set_clear%(s: BrokerComm::Data%): bool +function Broker::__set_clear%(s: Broker::Data%): bool %{ auto& v = bro_broker::require_data_type(s->AsRecordVal(), TYPE_TABLE, frame); @@ -211,26 +125,14 @@ function BrokerComm::set_clear%(s: BrokerComm::Data%): bool return new Val(true, TYPE_BOOL); %} -## Get the number of elements within a set. -## -## s: the set to query. -## -## Returns: the number of elements in the set. -function BrokerComm::set_size%(s: BrokerComm::Data%): count +function Broker::__set_size%(s: Broker::Data%): count %{ auto& v = bro_broker::require_data_type(s->AsRecordVal(), TYPE_TABLE, frame); return new Val(static_cast(v.size()), TYPE_COUNT); %} -## Check if a set contains a particular element. -## -## s: the set to query. -## -## key: the element to check for existence. -## -## Returns: true if the key exists in the set. -function BrokerComm::set_contains%(s: BrokerComm::Data, key: BrokerComm::Data%): bool +function Broker::__set_contains%(s: Broker::Data, key: Broker::Data%): bool %{ auto& v = bro_broker::require_data_type(s->AsRecordVal(), TYPE_TABLE, frame); @@ -238,14 +140,7 @@ function BrokerComm::set_contains%(s: BrokerComm::Data, key: BrokerComm::Data%): return new Val(v.find(k) != v.end(), TYPE_BOOL); %} -## Insert an element into a set. -## -## s: the set to modify. -## -## key: the element to insert. -## -## Returns: true if the key was inserted, or false if it already existed. -function BrokerComm::set_insert%(s: BrokerComm::Data, key: BrokerComm::Data%): bool +function Broker::__set_insert%(s: Broker::Data, key: Broker::Data%): bool %{ auto& v = bro_broker::require_data_type(s->AsRecordVal(), TYPE_TABLE, frame); @@ -253,14 +148,7 @@ function BrokerComm::set_insert%(s: BrokerComm::Data, key: BrokerComm::Data%): b return new Val(v.insert(k).second, TYPE_BOOL); %} -## Remove an element from a set. -## -## s: the set to modify. -## -## key: the element to remove. -## -## Returns: true if the element existed in the set and is now removed. -function BrokerComm::set_remove%(s: BrokerComm::Data, key: BrokerComm::Data%): bool +function Broker::__set_remove%(s: Broker::Data, key: Broker::Data%): bool %{ auto& v = bro_broker::require_data_type(s->AsRecordVal(), TYPE_TABLE, frame); @@ -268,37 +156,18 @@ function BrokerComm::set_remove%(s: BrokerComm::Data, key: BrokerComm::Data%): b return new Val(v.erase(k) > 0, TYPE_BOOL); %} -## Create an iterator for a set. Note that this makes a copy of the set -## internally to ensure the iterator is always valid. -## -## s: the set to iterate over. -## -## Returns: an iterator. -function BrokerComm::set_iterator%(s: BrokerComm::Data%): opaque of BrokerComm::SetIterator +function Broker::__set_iterator%(s: Broker::Data%): opaque of Broker::SetIterator %{ return new bro_broker::SetIterator(s->AsRecordVal(), TYPE_TABLE, frame); %} -## Check if there are no more elements to iterate over. -## -## it: an iterator. -## -## Returns: true if there are no more elements to iterator over, i.e. -## the iterator is one-past-the-final-element. -function BrokerComm::set_iterator_last%(it: opaque of BrokerComm::SetIterator%): bool +function Broker::__set_iterator_last%(it: opaque of Broker::SetIterator%): bool %{ auto set_it = static_cast(it); return new Val(set_it->it == set_it->dat.end(), TYPE_BOOL); %} -## Advance an iterator. -## -## it: an iterator. -## -## Returns: true if the iterator, after advancing, still references an element -## in the collection. False if the iterator, after advancing, is -## one-past-the-final-element. -function BrokerComm::set_iterator_next%(it: opaque of BrokerComm::SetIterator%): bool +function Broker::__set_iterator_next%(it: opaque of Broker::SetIterator%): bool %{ auto set_it = static_cast(it); @@ -309,15 +178,10 @@ function BrokerComm::set_iterator_next%(it: opaque of BrokerComm::SetIterator%): return new Val(set_it->it != set_it->dat.end(), TYPE_BOOL); %} -## Retrieve the data at an iterator's current position. -## -## it: an iterator. -## -## Returns: element in the collection that the iterator currently references. -function BrokerComm::set_iterator_value%(it: opaque of BrokerComm::SetIterator%): BrokerComm::Data +function Broker::__set_iterator_value%(it: opaque of Broker::SetIterator%): Broker::Data %{ auto set_it = static_cast(it); - auto rval = new RecordVal(BifType::Record::BrokerComm::Data); + auto rval = new RecordVal(BifType::Record::Broker::Data); if ( set_it->it == set_it->dat.end() ) { @@ -331,18 +195,12 @@ function BrokerComm::set_iterator_value%(it: opaque of BrokerComm::SetIterator%) return rval; %} -## Create communication data of type "table". -function BrokerComm::table_create%(%): BrokerComm::Data +function Broker::__table_create%(%): Broker::Data %{ return bro_broker::make_data_val(broker::table()); %} -## Remove all elements within a table. -## -## t: the table to clear. -## -## Returns: always true. -function BrokerComm::table_clear%(t: BrokerComm::Data%): bool +function Broker::__table_clear%(t: Broker::Data%): bool %{ auto& v = bro_broker::require_data_type(t->AsRecordVal(), TYPE_TABLE, frame); @@ -350,26 +208,14 @@ function BrokerComm::table_clear%(t: BrokerComm::Data%): bool return new Val(true, TYPE_BOOL); %} -## Get the number of elements within a table. -## -## t: the table to query. -## -## Returns: the number of elements in the table. -function BrokerComm::table_size%(t: BrokerComm::Data%): count +function Broker::__table_size%(t: Broker::Data%): count %{ auto& v = bro_broker::require_data_type(t->AsRecordVal(), TYPE_TABLE, frame); return new Val(static_cast(v.size()), TYPE_COUNT); %} -## Check if a table contains a particular key. -## -## t: the table to query. -## -## key: the key to check for existence. -## -## Returns: true if the key exists in the table. -function BrokerComm::table_contains%(t: BrokerComm::Data, key: BrokerComm::Data%): bool +function Broker::__table_contains%(t: Broker::Data, key: Broker::Data%): bool %{ auto& v = bro_broker::require_data_type(t->AsRecordVal(), TYPE_TABLE, frame); @@ -377,17 +223,7 @@ function BrokerComm::table_contains%(t: BrokerComm::Data, key: BrokerComm::Data% return new Val(v.find(k) != v.end(), TYPE_BOOL); %} -## Insert a key-value pair into a table. -## -## t: the table to modify. -## -## key: the key at which to insert the value. -## -## val: the value to insert. -## -## Returns: true if the key-value pair was inserted, or false if the key -## already existed in the table. -function BrokerComm::table_insert%(t: BrokerComm::Data, key: BrokerComm::Data, val: BrokerComm::Data%): BrokerComm::Data +function Broker::__table_insert%(t: Broker::Data, key: Broker::Data, val: Broker::Data%): Broker::Data %{ auto& table = bro_broker::require_data_type(t->AsRecordVal(), TYPE_TABLE, frame); @@ -404,19 +240,11 @@ function BrokerComm::table_insert%(t: BrokerComm::Data, key: BrokerComm::Data, v catch (const std::out_of_range&) { table[k] = v; - return new RecordVal(BifType::Record::BrokerComm::Data); + return new RecordVal(BifType::Record::Broker::Data); } %} -## Remove a key-value pair from a table. -## -## t: the table to modify. -## -## key: the key to remove from the table. -## -## Returns: the value associated with the key. If the key did not exist, then -## the optional field of the returned record is not set. -function BrokerComm::table_remove%(t: BrokerComm::Data, key: BrokerComm::Data%): BrokerComm::Data +function Broker::__table_remove%(t: Broker::Data, key: Broker::Data%): Broker::Data %{ auto& table = bro_broker::require_data_type(t->AsRecordVal(), TYPE_TABLE, frame); @@ -424,7 +252,7 @@ function BrokerComm::table_remove%(t: BrokerComm::Data, key: BrokerComm::Data%): auto it = table.find(k); if ( it == table.end() ) - return new RecordVal(BifType::Record::BrokerComm::Data); + return new RecordVal(BifType::Record::Broker::Data); else { auto rval = bro_broker::make_data_val(move(it->second)); @@ -433,15 +261,7 @@ function BrokerComm::table_remove%(t: BrokerComm::Data, key: BrokerComm::Data%): } %} -## Retrieve a value from a table. -## -## t: the table to query. -## -## key: the key to lookup. -## -## Returns: the value associated with the key. If the key did not exist, then -## the optional field of the returned record is not set. -function BrokerComm::table_lookup%(t: BrokerComm::Data, key: BrokerComm::Data%): BrokerComm::Data +function Broker::__table_lookup%(t: Broker::Data, key: Broker::Data%): Broker::Data %{ auto& table = bro_broker::require_data_type(t->AsRecordVal(), TYPE_TABLE, frame); @@ -449,42 +269,23 @@ function BrokerComm::table_lookup%(t: BrokerComm::Data, key: BrokerComm::Data%): auto it = table.find(k); if ( it == table.end() ) - return new RecordVal(BifType::Record::BrokerComm::Data); + return new RecordVal(BifType::Record::Broker::Data); else return bro_broker::make_data_val(it->second); %} -## Create an iterator for a table. Note that this makes a copy of the table -## internally to ensure the iterator is always valid. -## -## t: the table to iterate over. -## -## Returns: an iterator. -function BrokerComm::table_iterator%(t: BrokerComm::Data%): opaque of BrokerComm::TableIterator +function Broker::__table_iterator%(t: Broker::Data%): opaque of Broker::TableIterator %{ return new bro_broker::TableIterator(t->AsRecordVal(), TYPE_TABLE, frame); %} -## Check if there are no more elements to iterate over. -## -## it: an iterator. -## -## Returns: true if there are no more elements to iterator over, i.e. -## the iterator is one-past-the-final-element. -function BrokerComm::table_iterator_last%(it: opaque of BrokerComm::TableIterator%): bool +function Broker::__table_iterator_last%(it: opaque of Broker::TableIterator%): bool %{ auto ti = static_cast(it); return new Val(ti->it == ti->dat.end(), TYPE_BOOL); %} -## Advance an iterator. -## -## it: an iterator. -## -## Returns: true if the iterator, after advancing, still references an element -## in the collection. False if the iterator, after advancing, is -## one-past-the-final-element. -function BrokerComm::table_iterator_next%(it: opaque of BrokerComm::TableIterator%): bool +function Broker::__table_iterator_next%(it: opaque of Broker::TableIterator%): bool %{ auto ti = static_cast(it); @@ -495,17 +296,12 @@ function BrokerComm::table_iterator_next%(it: opaque of BrokerComm::TableIterato return new Val(ti->it != ti->dat.end(), TYPE_BOOL); %} -## Retrieve the data at an iterator's current position. -## -## it: an iterator. -## -## Returns: element in the collection that the iterator currently references. -function BrokerComm::table_iterator_value%(it: opaque of BrokerComm::TableIterator%): BrokerComm::TableItem +function Broker::__table_iterator_value%(it: opaque of Broker::TableIterator%): Broker::TableItem %{ auto ti = static_cast(it); - auto rval = new RecordVal(BifType::Record::BrokerComm::TableItem); - auto key_val = new RecordVal(BifType::Record::BrokerComm::Data); - auto val_val = new RecordVal(BifType::Record::BrokerComm::Data); + auto rval = new RecordVal(BifType::Record::Broker::TableItem); + auto key_val = new RecordVal(BifType::Record::Broker::Data); + auto val_val = new RecordVal(BifType::Record::Broker::Data); rval->Assign(0, key_val); rval->Assign(1, val_val); @@ -522,18 +318,12 @@ function BrokerComm::table_iterator_value%(it: opaque of BrokerComm::TableIterat return rval; %} -## Create communication data of type "vector". -function BrokerComm::vector_create%(%): BrokerComm::Data +function Broker::__vector_create%(%): Broker::Data %{ return bro_broker::make_data_val(broker::vector()); %} -## Remove all elements within a vector. -## -## v: the vector to clear. -## -## Returns: always true. -function BrokerComm::vector_clear%(v: BrokerComm::Data%): bool +function Broker::__vector_clear%(v: Broker::Data%): bool %{ auto& vec = bro_broker::require_data_type(v->AsRecordVal(), TYPE_VECTOR, frame); @@ -541,30 +331,14 @@ function BrokerComm::vector_clear%(v: BrokerComm::Data%): bool return new Val(true, TYPE_BOOL); %} -## Get the number of elements within a vector. -## -## v: the vector to query. -## -## Returns: the number of elements in the vector. -function BrokerComm::vector_size%(v: BrokerComm::Data%): count +function Broker::__vector_size%(v: Broker::Data%): count %{ auto& vec = bro_broker::require_data_type(v->AsRecordVal(), TYPE_VECTOR, frame); return new Val(static_cast(vec.size()), TYPE_COUNT); %} -## Insert an element into a vector at a particular position, possibly displacing -## existing elements (insertion always grows the size of the vector by one). -## -## v: the vector to modify. -## -## d: the element to insert. -## -## idx: the index at which to insert the data. If it is greater than the -## current size of the vector, the element is inserted at the end. -## -## Returns: always true. -function BrokerComm::vector_insert%(v: BrokerComm::Data, d: BrokerComm::Data, idx: count%): bool +function Broker::__vector_insert%(v: Broker::Data, d: Broker::Data, idx: count%): bool %{ auto& vec = bro_broker::require_data_type(v->AsRecordVal(), TYPE_VECTOR, frame); @@ -574,101 +348,56 @@ function BrokerComm::vector_insert%(v: BrokerComm::Data, d: BrokerComm::Data, id return new Val(true, TYPE_BOOL); %} -## Replace an element in a vector at a particular position. -## -## v: the vector to modify. -## -## d: the element to insert. -## -## idx: the index to replace. -## -## Returns: the value that was just evicted. If the index was larger than any -## valid index, the optional field of the returned record is not set. -function BrokerComm::vector_replace%(v: BrokerComm::Data, d: BrokerComm::Data, idx: count%): BrokerComm::Data +function Broker::__vector_replace%(v: Broker::Data, d: Broker::Data, idx: count%): Broker::Data %{ auto& vec = bro_broker::require_data_type(v->AsRecordVal(), TYPE_VECTOR, frame); auto& item = bro_broker::opaque_field_to_data(d->AsRecordVal(), frame); if ( idx >= vec.size() ) - return new RecordVal(BifType::Record::BrokerComm::Data); + return new RecordVal(BifType::Record::Broker::Data); auto rval = bro_broker::make_data_val(move(vec[idx])); vec[idx] = item; return rval; %} -## Remove an element from a vector at a particular position. -## -## v: the vector to modify. -## -## idx: the index to remove. -## -## Returns: the value that was just evicted. If the index was larger than any -## valid index, the optional field of the returned record is not set. -function BrokerComm::vector_remove%(v: BrokerComm::Data, idx: count%): BrokerComm::Data +function Broker::__vector_remove%(v: Broker::Data, idx: count%): Broker::Data %{ auto& vec = bro_broker::require_data_type(v->AsRecordVal(), TYPE_VECTOR, frame); if ( idx >= vec.size() ) - return new RecordVal(BifType::Record::BrokerComm::Data); + return new RecordVal(BifType::Record::Broker::Data); auto rval = bro_broker::make_data_val(move(vec[idx])); vec.erase(vec.begin() + idx); return rval; %} -## Lookup an element in a vector at a particular position. -## -## v: the vector to query. -## -## idx: the index to lookup. -## -## Returns: the value at the index. If the index was larger than any -## valid index, the optional field of the returned record is not set. -function BrokerComm::vector_lookup%(v: BrokerComm::Data, idx: count%): BrokerComm::Data +function Broker::__vector_lookup%(v: Broker::Data, idx: count%): Broker::Data %{ auto& vec = bro_broker::require_data_type(v->AsRecordVal(), TYPE_VECTOR, frame); if ( idx >= vec.size() ) - return new RecordVal(BifType::Record::BrokerComm::Data); + return new RecordVal(BifType::Record::Broker::Data); return bro_broker::make_data_val(vec[idx]); %} -## Create an iterator for a vector. Note that this makes a copy of the vector -## internally to ensure the iterator is always valid. -## -## v: the vector to iterate over. -## -## Returns: an iterator. -function BrokerComm::vector_iterator%(v: BrokerComm::Data%): opaque of BrokerComm::VectorIterator +function Broker::__vector_iterator%(v: Broker::Data%): opaque of Broker::VectorIterator %{ return new bro_broker::VectorIterator(v->AsRecordVal(), TYPE_VECTOR, frame); %} -## Check if there are no more elements to iterate over. -## -## it: an iterator. -## -## Returns: true if there are no more elements to iterator over, i.e. -## the iterator is one-past-the-final-element. -function BrokerComm::vector_iterator_last%(it: opaque of BrokerComm::VectorIterator%): bool +function Broker::__vector_iterator_last%(it: opaque of Broker::VectorIterator%): bool %{ auto vi = static_cast(it); return new Val(vi->it == vi->dat.end(), TYPE_BOOL); %} -## Advance an iterator. -## -## it: an iterator. -## -## Returns: true if the iterator, after advancing, still references an element -## in the collection. False if the iterator, after advancing, is -## one-past-the-final-element. -function BrokerComm::vector_iterator_next%(it: opaque of BrokerComm::VectorIterator%): bool +function Broker::__vector_iterator_next%(it: opaque of Broker::VectorIterator%): bool %{ auto vi = static_cast(it); @@ -679,15 +408,10 @@ function BrokerComm::vector_iterator_next%(it: opaque of BrokerComm::VectorItera return new Val(vi->it != vi->dat.end(), TYPE_BOOL); %} -## Retrieve the data at an iterator's current position. -## -## it: an iterator. -## -## Returns: element in the collection that the iterator currently references. -function BrokerComm::vector_iterator_value%(it: opaque of BrokerComm::VectorIterator%): BrokerComm::Data +function Broker::__vector_iterator_value%(it: opaque of Broker::VectorIterator%): Broker::Data %{ auto vi = static_cast(it); - auto rval = new RecordVal(BifType::Record::BrokerComm::Data); + auto rval = new RecordVal(BifType::Record::Broker::Data); if ( vi->it == vi->dat.end() ) { @@ -701,38 +425,19 @@ function BrokerComm::vector_iterator_value%(it: opaque of BrokerComm::VectorIter return rval; %} -## Create communication data of type "record". -## -## sz: the number of fields in the record. -## -## Returns: record data, with all fields uninitialized. -function BrokerComm::record_create%(sz: count%): BrokerComm::Data +function Broker::__record_create%(sz: count%): Broker::Data %{ return bro_broker::make_data_val(broker::record(std::vector(sz))); %} -## Get the number of fields within a record. -## -## r: the record to query. -## -## Returns: the number of fields in the record. -function BrokerComm::record_size%(r: BrokerComm::Data%): count +function Broker::__record_size%(r: Broker::Data%): count %{ auto& v = bro_broker::require_data_type(r->AsRecordVal(), TYPE_RECORD, frame); return new Val(static_cast(v.fields.size()), TYPE_COUNT); %} -## Replace a field in a record at a particular position. -## -## r: the record to modify. -## -## d: the new field value to assign. -## -## idx: the index to replace. -## -## Returns: false if the index was larger than any valid index, else true. -function BrokerComm::record_assign%(r: BrokerComm::Data, d: BrokerComm::Data, idx: count%): bool +function Broker::__record_assign%(r: Broker::Data, d: Broker::Data, idx: count%): bool %{ auto& v = bro_broker::require_data_type(r->AsRecordVal(), TYPE_RECORD, frame); @@ -745,60 +450,32 @@ function BrokerComm::record_assign%(r: BrokerComm::Data, d: BrokerComm::Data, id return new Val(true, TYPE_BOOL); %} -## Lookup a field in a record at a particular position. -## -## r: the record to query. -## -## idx: the index to lookup. -## -## Returns: the value at the index. The optional field of the returned record -## may not be set if the field of the record has no value or if the -## index was not valid. -function BrokerComm::record_lookup%(r: BrokerComm::Data, idx: count%): BrokerComm::Data +function Broker::__record_lookup%(r: Broker::Data, idx: count%): Broker::Data %{ auto& v = bro_broker::require_data_type(r->AsRecordVal(), TYPE_RECORD, frame); if ( idx >= v.size() ) - return new RecordVal(BifType::Record::BrokerComm::Data); + return new RecordVal(BifType::Record::Broker::Data); if ( ! v.fields[idx] ) - return new RecordVal(BifType::Record::BrokerComm::Data); + return new RecordVal(BifType::Record::Broker::Data); return bro_broker::make_data_val(*v.fields[idx]); %} -## Create an iterator for a record. Note that this makes a copy of the record -## internally to ensure the iterator is always valid. -## -## r: the record to iterate over. -## -## Returns: an iterator. -function BrokerComm::record_iterator%(r: BrokerComm::Data%): opaque of BrokerComm::RecordIterator +function Broker::__record_iterator%(r: Broker::Data%): opaque of Broker::RecordIterator %{ return new bro_broker::RecordIterator(r->AsRecordVal(), TYPE_RECORD, frame); %} -## Check if there are no more elements to iterate over. -## -## it: an iterator. -## -## Returns: true if there are no more elements to iterator over, i.e. -## the iterator is one-past-the-final-element. -function BrokerComm::record_iterator_last%(it: opaque of BrokerComm::RecordIterator%): bool +function Broker::__record_iterator_last%(it: opaque of Broker::RecordIterator%): bool %{ auto ri = static_cast(it); return new Val(ri->it == ri->dat.fields.end(), TYPE_BOOL); %} -## Advance an iterator. -## -## it: an iterator. -## -## Returns: true if the iterator, after advancing, still references an element -## in the collection. False if the iterator, after advancing, is -## one-past-the-final-element. -function BrokerComm::record_iterator_next%(it: opaque of BrokerComm::RecordIterator%): bool +function Broker::__record_iterator_next%(it: opaque of Broker::RecordIterator%): bool %{ auto ri = static_cast(it); @@ -809,15 +486,10 @@ function BrokerComm::record_iterator_next%(it: opaque of BrokerComm::RecordItera return new Val(ri->it != ri->dat.fields.end(), TYPE_BOOL); %} -## Retrieve the data at an iterator's current position. -## -## it: an iterator. -## -## Returns: element in the collection that the iterator currently references. -function BrokerComm::record_iterator_value%(it: opaque of BrokerComm::RecordIterator%): BrokerComm::Data +function Broker::__record_iterator_value%(it: opaque of Broker::RecordIterator%): Broker::Data %{ auto ri = static_cast(it); - auto rval = new RecordVal(BifType::Record::BrokerComm::Data); + auto rval = new RecordVal(BifType::Record::Broker::Data); if ( ri->it == ri->dat.fields.end() ) { diff --git a/src/broker/messaging.bif b/src/broker/messaging.bif index 97b794b50e..dadece9681 100644 --- a/src/broker/messaging.bif +++ b/src/broker/messaging.bif @@ -6,209 +6,106 @@ #include "logging/Manager.h" %%} -module BrokerComm; +module Broker; -type BrokerComm::SendFlags: record; +type Broker::SendFlags: record; -type BrokerComm::EventArgs: record; +type Broker::EventArgs: record; ## Used to handle remote print messages from peers that call -## :bro:see:`BrokerComm::print`. -event BrokerComm::print_handler%(msg: string%); +## :bro:see:`Broker::send_print`. +event Broker::print_handler%(msg: string%); -## Print a simple message to any interested peers. The receiver can use -## :bro:see:`BrokerComm::print_handler` to handle messages. -## -## topic: a topic associated with the printed message. -## -## msg: the print message to send to peers. -## -## flags: tune the behavior of how the message is sent. -## -## Returns: true if the message is sent. -function BrokerComm::print%(topic: string, msg: string, - flags: SendFlags &default = SendFlags()%): bool +function Broker::__send_print%(topic: string, msg: string, flags: Broker::SendFlags%): bool %{ auto rval = broker_mgr->Print(topic->CheckString(), msg->CheckString(), flags); return new Val(rval, TYPE_BOOL); %} -## Register interest in all peer print messages that use a certain topic prefix. -## Use :bro:see:`BrokerComm::print_handler` to handle received messages. -## -## topic_prefix: a prefix to match against remote message topics. -## e.g. an empty prefix matches everything and "a" matches -## "alice" and "amy" but not "bob". -## -## Returns: true if it's a new print subscription and it is now registered. -function BrokerComm::subscribe_to_prints%(topic_prefix: string%): bool +function Broker::__subscribe_to_prints%(topic_prefix: string%): bool %{ auto rval = broker_mgr->SubscribeToPrints(topic_prefix->CheckString()); return new Val(rval, TYPE_BOOL); %} -## Unregister interest in all peer print messages that use a topic prefix. -## -## topic_prefix: a prefix previously supplied to a successful call to -## :bro:see:`BrokerComm::subscribe_to_prints`. -## -## Returns: true if interest in the topic prefix is no longer advertised. -function BrokerComm::unsubscribe_to_prints%(topic_prefix: string%): bool +function Broker::__unsubscribe_to_prints%(topic_prefix: string%): bool %{ auto rval = broker_mgr->UnsubscribeToPrints(topic_prefix->CheckString()); return new Val(rval, TYPE_BOOL); %} ## Create a data structure that may be used to send a remote event via -## :bro:see:`BrokerComm::event`. +## :bro:see:`Broker::send_event`. ## ## args: an event, followed by a list of argument values that may be used ## to call it. ## -## Returns: opaque communication data that may be used to send a remote event. -function BrokerComm::event_args%(...%): BrokerComm::EventArgs +## Returns: opaque communication data that may be used to send a remote +## event. +function Broker::event_args%(...%): Broker::EventArgs %{ auto rval = broker_mgr->MakeEventArgs(@ARGS@); return rval; %} -## Send an event to any interested peers. -## -## topic: a topic associated with the event message. -## -## args: event arguments as made by :bro:see:`BrokerComm::event_args`. -## -## flags: tune the behavior of how the message is sent. -## -## Returns: true if the message is sent. -function BrokerComm::event%(topic: string, args: BrokerComm::EventArgs, - flags: SendFlags &default = SendFlags()%): bool +function Broker::__event%(topic: string, args: Broker::EventArgs, flags: Broker::SendFlags%): bool %{ auto rval = broker_mgr->Event(topic->CheckString(), args->AsRecordVal(), flags); return new Val(rval, TYPE_BOOL); %} -## Automatically send an event to any interested peers whenever it is -## locally dispatched (e.g. using "event my_event(...);" in a script). -## -## topic: a topic string associated with the event message. -## Peers advertise interest by registering a subscription to some prefix -## of this topic name. -## -## ev: a Bro event value. -## -## flags: tune the behavior of how the message is sent. -## -## Returns: true if automatic event sending is now enabled. -function BrokerComm::auto_event%(topic: string, ev: any, - flags: SendFlags &default = SendFlags()%): bool +function Broker::__auto_event%(topic: string, ev: any, flags: Broker::SendFlags%): bool %{ auto rval = broker_mgr->AutoEvent(topic->CheckString(), ev, flags); return new Val(rval, TYPE_BOOL); %} -## Stop automatically sending an event to peers upon local dispatch. -## -## topic: a topic originally given to :bro:see:`BrokerComm::auto_event`. -## -## ev: an event originally given to :bro:see:`BrokerComm::auto_event`. -## -## Returns: true if automatic events will not occur for the topic/event pair. -function BrokerComm::auto_event_stop%(topic: string, ev: any%): bool +function Broker::__auto_event_stop%(topic: string, ev: any%): bool %{ auto rval = broker_mgr->AutoEventStop(topic->CheckString(), ev); return new Val(rval, TYPE_BOOL); %} -## Register interest in all peer event messages that use a certain topic prefix. -## -## topic_prefix: a prefix to match against remote message topics. -## e.g. an empty prefix matches everything and "a" matches -## "alice" and "amy" but not "bob". -## -## Returns: true if it's a new event subscription and it is now registered. -function BrokerComm::subscribe_to_events%(topic_prefix: string%): bool +function Broker::__subscribe_to_events%(topic_prefix: string%): bool %{ auto rval = broker_mgr->SubscribeToEvents(topic_prefix->CheckString()); return new Val(rval, TYPE_BOOL); %} -## Unregister interest in all peer event messages that use a topic prefix. -## -## topic_prefix: a prefix previously supplied to a successful call to -## :bro:see:`BrokerComm::subscribe_to_events`. -## -## Returns: true if interest in the topic prefix is no longer advertised. -function BrokerComm::unsubscribe_to_events%(topic_prefix: string%): bool +function Broker::__unsubscribe_to_events%(topic_prefix: string%): bool %{ auto rval = broker_mgr->UnsubscribeToEvents(topic_prefix->CheckString()); return new Val(rval, TYPE_BOOL); %} -## Enable remote logs for a given log stream. -## -## id: the log stream to enable remote logs for. -## -## flags: tune the behavior of how log entry messages are sent. -## -## Returns: true if remote logs are enabled for the stream. -function -BrokerComm::enable_remote_logs%(id: Log::ID, - flags: SendFlags &default = SendFlags()%): bool +function Broker::__enable_remote_logs%(id: Log::ID, flags: Broker::SendFlags%): bool %{ auto rval = log_mgr->EnableRemoteLogs(id->AsEnumVal(), bro_broker::Manager::send_flags_to_int(flags)); return new Val(rval, TYPE_BOOL); %} -## Disable remote logs for a given log stream. -## -## id: the log stream to disable remote logs for. -## -## Returns: true if remote logs are disabled for the stream. -function BrokerComm::disable_remote_logs%(id: Log::ID%): bool +function Broker::__disable_remote_logs%(id: Log::ID%): bool %{ auto rval = log_mgr->DisableRemoteLogs(id->AsEnumVal()); return new Val(rval, TYPE_BOOL); %} -## Check if remote logs are enabled for a given log stream. -## -## id: the log stream to check. -## -## Returns: true if remote logs are enabled for the given stream. -function BrokerComm::remote_logs_enabled%(id: Log::ID%): bool +function Broker::__remote_logs_enabled%(id: Log::ID%): bool %{ auto rval = log_mgr->RemoteLogsAreEnabled(id->AsEnumVal()); return new Val(rval, TYPE_BOOL); %} -## Register interest in all peer log messages that use a certain topic prefix. -## Logs are implicitly sent with topic "bro/log/" and the -## receiving side processes them through the logging framework as usual. -## -## topic_prefix: a prefix to match against remote message topics. -## e.g. an empty prefix matches everything and "a" matches -## "alice" and "amy" but not "bob". -## -## Returns: true if it's a new log subscription and it is now registered. -function BrokerComm::subscribe_to_logs%(topic_prefix: string%): bool +function Broker::__subscribe_to_logs%(topic_prefix: string%): bool %{ auto rval = broker_mgr->SubscribeToLogs(topic_prefix->CheckString()); return new Val(rval, TYPE_BOOL); %} -## Unregister interest in all peer log messages that use a topic prefix. -## Logs are implicitly sent with topic "bro/log/" and the -## receiving side processes them through the logging framework as usual. -## -## topic_prefix: a prefix previously supplied to a successful call to -## :bro:see:`BrokerComm::subscribe_to_logs`. -## -## Returns: true if interest in the topic prefix is no longer advertised. -function BrokerComm::unsubscribe_to_logs%(topic_prefix: string%): bool +function Broker::__unsubscribe_to_logs%(topic_prefix: string%): bool %{ auto rval = broker_mgr->UnsubscribeToLogs(topic_prefix->CheckString()); return new Val(rval, TYPE_BOOL); diff --git a/src/broker/store.bif b/src/broker/store.bif index 853bd1f2d7..6d7ddea6af 100644 --- a/src/broker/store.bif +++ b/src/broker/store.bif @@ -8,13 +8,13 @@ #include "Trigger.h" %%} -module BrokerStore; +module Broker; -type BrokerStore::ExpiryTime: record; +type Broker::ExpiryTime: record; -type BrokerStore::QueryResult: record; +type Broker::QueryResult: record; -type BrokerStore::BackendOptions: record; +type Broker::BackendOptions: record; ## Enumerates the possible storage backends. enum BackendType %{ @@ -23,17 +23,8 @@ enum BackendType %{ ROCKSDB, %} -## Create a master data store which contains key-value pairs. -## -## id: a unique name for the data store. -## -## b: the storage backend to use. -## -## options: tunes how some storage backends operate. -## -## Returns: a handle to the data store. -function BrokerStore::create_master%(id: string, b: BackendType &default = MEMORY, - options: BackendOptions &default = BackendOptions()%): opaque of BrokerStore::Handle +function Broker::__create_master%(id: string, b: BackendType, + options: BackendOptions &default = BackendOptions()%): opaque of Broker::Handle %{ auto id_str = id->CheckString(); auto type = bro_broker::StoreType::MASTER; @@ -46,38 +37,16 @@ function BrokerStore::create_master%(id: string, b: BackendType &default = MEMOR } rval = new bro_broker::StoreHandleVal(id_str, type, - static_cast(b->AsEnum()), + static_cast(b->AsEnum()), options->AsRecordVal()); auto added = broker_mgr->AddStore(rval); assert(added); return rval; %} -## Create a clone of a master data store which may live with a remote peer. -## A clone automatically synchronizes to the master by automatically receiving -## modifications and applying them locally. Direct modifications are not -## possible, they must be sent through the master store, which then -## automatically broadcasts the changes out to clones. But queries may be made -## directly against the local cloned copy, which may be resolved quicker than -## reaching out to a remote master store. -## -## id: the unique name which identifies the master data store. -## -## b: the storage backend to use. -## -## options: tunes how some storage backends operate. -## -## resync: the interval at which to re-attempt synchronizing with the master -## store should the connection be lost. If the clone has not yet -## synchronized for the first time, updates and queries queue up until -## the synchronization completes. After, if the connection to the -## master store is lost, queries continue to use the clone's version, -## but updates will be lost until the master is once again available. -## -## Returns: a handle to the data store. -function BrokerStore::create_clone%(id: string, b: BackendType &default = MEMORY, +function Broker::__create_clone%(id: string, b: BackendType, options: BackendOptions &default = BackendOptions(), - resync: interval &default = 1sec%): opaque of BrokerStore::Handle + resync: interval &default = 1sec%): opaque of Broker::Handle %{ auto id_str = id->CheckString(); auto type = bro_broker::StoreType::CLONE; @@ -90,7 +59,7 @@ function BrokerStore::create_clone%(id: string, b: BackendType &default = MEMORY } rval = new bro_broker::StoreHandleVal(id_str, type, - static_cast(b->AsEnum()), + static_cast(b->AsEnum()), options->AsRecordVal(), std::chrono::duration(resync)); auto added = broker_mgr->AddStore(rval); @@ -98,13 +67,7 @@ function BrokerStore::create_clone%(id: string, b: BackendType &default = MEMORY return rval; %} -## Create a frontend interface to an existing master data store that allows -## querying and updating its contents. -## -## id: the unique name which identifies the master data store. -## -## Returns: a handle to the data store. -function BrokerStore::create_frontend%(id: string%): opaque of BrokerStore::Handle +function Broker::__create_frontend%(id: string%): opaque of Broker::Handle %{ auto id_str = id->CheckString(); auto type = bro_broker::StoreType::FRONTEND; @@ -122,13 +85,7 @@ function BrokerStore::create_frontend%(id: string%): opaque of BrokerStore::Hand return rval; %} -## Close a data store. -## -## h: a data store handle. -## -## Returns: true if store was valid and is now closed. The handle can no -## longer be used for data store operations. -function BrokerStore::close_by_handle%(h: opaque of BrokerStore::Handle%): bool +function Broker::__close_by_handle%(h: opaque of Broker::Handle%): bool %{ auto handle = static_cast(h); @@ -143,20 +100,9 @@ function BrokerStore::close_by_handle%(h: opaque of BrokerStore::Handle%): bool # non-blocking update API # ########################### -## Insert a key-value pair in to the store. -## -## h: the handle of the store to modify. -## -## k: the key to insert. -## -## v: the value to insert. -## -## e: the expiration time of the key-value pair. -## -## Returns: false if the store handle was not valid. -function BrokerStore::insert%(h: opaque of BrokerStore::Handle, - k: BrokerComm::Data, v: BrokerComm::Data, - e: BrokerStore::ExpiryTime &default = BrokerStore::ExpiryTime()%): bool +function Broker::__insert%(h: opaque of Broker::Handle, + k: Broker::Data, v: Broker::Data, + e: Broker::ExpiryTime &default = Broker::ExpiryTime()%): bool %{ auto handle = static_cast(h); @@ -191,14 +137,7 @@ function BrokerStore::insert%(h: opaque of BrokerStore::Handle, return new Val(true, TYPE_BOOL); %} -## Remove a key-value pair from the store. -## -## h: the handle of the store to modify. -## -## k: the key to remove. -## -## Returns: false if the store handle was not valid. -function BrokerStore::erase%(h: opaque of BrokerStore::Handle, k: BrokerComm::Data%): bool +function Broker::__erase%(h: opaque of Broker::Handle, k: Broker::Data%): bool %{ auto handle = static_cast(h); @@ -210,12 +149,7 @@ function BrokerStore::erase%(h: opaque of BrokerStore::Handle, k: BrokerComm::Da return new Val(true, TYPE_BOOL); %} -## Remove all key-value pairs from the store. -## -## h: the handle of the store to modify. -## -## Returns: false if the store handle was not valid. -function BrokerStore::clear%(h: opaque of BrokerStore::Handle%): bool +function Broker::__clear%(h: opaque of Broker::Handle%): bool %{ auto handle = static_cast(h); @@ -226,18 +160,8 @@ function BrokerStore::clear%(h: opaque of BrokerStore::Handle%): bool return new Val(true, TYPE_BOOL); %} -## Increment an integer value in a data store. -## -## h: the handle of the store to modify. -## -## k: the key whose associated value is to be modified. -## -## by: the amount to increment the value by. A non-existent key will first -## create it with an implicit value of zero before incrementing. -## -## Returns: false if the store handle was not valid. -function BrokerStore::increment%(h: opaque of BrokerStore::Handle, - k: BrokerComm::Data, by: int &default = +1%): bool +function Broker::__increment%(h: opaque of Broker::Handle, + k: Broker::Data, by: int &default = +1%): bool %{ auto handle = static_cast(h); @@ -249,18 +173,8 @@ function BrokerStore::increment%(h: opaque of BrokerStore::Handle, return new Val(true, TYPE_BOOL); %} -## Decrement an integer value in a data store. -## -## h: the handle of the store to modify. -## -## k: the key whose associated value is to be modified. -## -## by: the amount to decrement the value by. A non-existent key will first -## create it with an implicit value of zero before decrementing. -## -## Returns: false if the store handle was not valid. -function BrokerStore::decrement%(h: opaque of BrokerStore::Handle, - k: BrokerComm::Data, by: int &default = +1%): bool +function Broker::__decrement%(h: opaque of Broker::Handle, + k: Broker::Data, by: int &default = +1%): bool %{ auto handle = static_cast(h); @@ -272,18 +186,8 @@ function BrokerStore::decrement%(h: opaque of BrokerStore::Handle, return new Val(true, TYPE_BOOL); %} -## Add an element to a set value in a data store. -## -## h: the handle of the store to modify. -## -## k: the key whose associated value is to be modified. -## -## element: the element to add to the set. A non-existent key will first -## create it with an implicit empty set value before modifying. -## -## Returns: false if the store handle was not valid. -function BrokerStore::add_to_set%(h: opaque of BrokerStore::Handle, - k: BrokerComm::Data, element: BrokerComm::Data%): bool +function Broker::__add_to_set%(h: opaque of Broker::Handle, + k: Broker::Data, element: Broker::Data%): bool %{ auto handle = static_cast(h); @@ -296,18 +200,8 @@ function BrokerStore::add_to_set%(h: opaque of BrokerStore::Handle, return new Val(true, TYPE_BOOL); %} -## Remove an element from a set value in a data store. -## -## h: the handle of the store to modify. -## -## k: the key whose associated value is to be modified. -## -## element: the element to remove from the set. A non-existent key will -## implicitly create an empty set value associated with the key. -## -## Returns: false if the store handle was not valid. -function BrokerStore::remove_from_set%(h: opaque of BrokerStore::Handle, - k: BrokerComm::Data, element: BrokerComm::Data%): bool +function Broker::__remove_from_set%(h: opaque of Broker::Handle, + k: Broker::Data, element: Broker::Data%): bool %{ auto handle = static_cast(h); @@ -320,18 +214,8 @@ function BrokerStore::remove_from_set%(h: opaque of BrokerStore::Handle, return new Val(true, TYPE_BOOL); %} -## Add a new item to the head of a vector value in a data store. -## -## h: the handle of store to modify. -## -## k: the key whose associated value is to be modified. -## -## items: the element to insert in to the vector. A non-existent key will first -## create an empty vector value before modifying. -## -## Returns: false if the store handle was not valid. -function BrokerStore::push_left%(h: opaque of BrokerStore::Handle, k: BrokerComm::Data, - items: BrokerComm::DataVector%): bool +function Broker::__push_left%(h: opaque of Broker::Handle, k: Broker::Data, + items: Broker::DataVector%): bool %{ auto handle = static_cast(h); @@ -353,18 +237,8 @@ function BrokerStore::push_left%(h: opaque of BrokerStore::Handle, k: BrokerComm return new Val(true, TYPE_BOOL); %} -## Add a new item to the tail of a vector value in a data store. -## -## h: the handle of store to modify. -## -## k: the key whose associated value is to be modified. -## -## items: the element to insert in to the vector. A non-existent key will first -## create an empty vector value before modifying. -## -## Returns: false if the store handle was not valid. -function BrokerStore::push_right%(h: opaque of BrokerStore::Handle, k: BrokerComm::Data, - items: BrokerComm::DataVector%): bool +function Broker::__push_right%(h: opaque of Broker::Handle, k: Broker::Data, + items: Broker::DataVector%): bool %{ auto handle = static_cast(h); @@ -401,7 +275,7 @@ static bool prepare_for_query(Val* opaque, Frame* frame, if ( ! (*handle)->store ) { reporter->PushLocation(frame->GetCall()->GetLocationInfo()); - reporter->Error("BrokerStore query has an invalid data store"); + reporter->Error("Broker query has an invalid data store"); reporter->PopLocation(); return false; } @@ -411,7 +285,7 @@ static bool prepare_for_query(Val* opaque, Frame* frame, if ( ! trigger ) { reporter->PushLocation(frame->GetCall()->GetLocationInfo()); - reporter->Error("BrokerStore queries can only be called inside when-condition"); + reporter->Error("Broker queries can only be called inside when-condition"); reporter->PopLocation(); return false; } @@ -421,7 +295,7 @@ static bool prepare_for_query(Val* opaque, Frame* frame, if ( *timeout < 0 ) { reporter->PushLocation(frame->GetCall()->GetLocationInfo()); - reporter->Error("BrokerStore queries must specify a timeout block"); + reporter->Error("Broker queries must specify a timeout block"); reporter->PopLocation(); return false; } @@ -437,15 +311,8 @@ static bool prepare_for_query(Val* opaque, Frame* frame, %%} -## Pop the head of a data store vector value. -## -## h: the handle of the store to query. -## -## k: the key associated with the vector to modify. -## -## Returns: the result of the query. -function BrokerStore::pop_left%(h: opaque of BrokerStore::Handle, - k: BrokerComm::Data%): BrokerStore::QueryResult +function Broker::__pop_left%(h: opaque of Broker::Handle, + k: Broker::Data%): Broker::QueryResult %{ if ( ! broker_mgr->Enabled() ) return bro_broker::query_result(); @@ -467,15 +334,8 @@ function BrokerStore::pop_left%(h: opaque of BrokerStore::Handle, return 0; %} -## Pop the tail of a data store vector value. -## -## h: the handle of the store to query. -## -## k: the key associated with the vector to modify. -## -## Returns: the result of the query. -function BrokerStore::pop_right%(h: opaque of BrokerStore::Handle, - k: BrokerComm::Data%): BrokerStore::QueryResult +function Broker::__pop_right%(h: opaque of Broker::Handle, + k: Broker::Data%): Broker::QueryResult %{ if ( ! broker_mgr->Enabled() ) return bro_broker::query_result(); @@ -497,15 +357,8 @@ function BrokerStore::pop_right%(h: opaque of BrokerStore::Handle, return 0; %} -## Lookup the value associated with a key in a data store. -## -## h: the handle of the store to query. -## -## k: the key to lookup. -## -## Returns: the result of the query. -function BrokerStore::lookup%(h: opaque of BrokerStore::Handle, - k: BrokerComm::Data%): BrokerStore::QueryResult +function Broker::__lookup%(h: opaque of Broker::Handle, + k: Broker::Data%): Broker::QueryResult %{ if ( ! broker_mgr->Enabled() ) return bro_broker::query_result(); @@ -527,15 +380,8 @@ function BrokerStore::lookup%(h: opaque of BrokerStore::Handle, return 0; %} -## Check if a data store contains a given key. -## -## h: the handle of the store to query. -## -## k: the key to check for existence. -## -## Returns: the result of the query (uses :bro:see:`BrokerComm::BOOL`). -function BrokerStore::exists%(h: opaque of BrokerStore::Handle, - k: BrokerComm::Data%): BrokerStore::QueryResult +function Broker::__exists%(h: opaque of Broker::Handle, + k: Broker::Data%): Broker::QueryResult %{ if ( ! broker_mgr->Enabled() ) return bro_broker::query_result(); @@ -557,12 +403,7 @@ function BrokerStore::exists%(h: opaque of BrokerStore::Handle, return 0; %} -## Retrieve all keys in a data store. -## -## h: the handle of the store to query. -## -## Returns: the result of the query (uses :bro:see:`BrokerComm::VECTOR`). -function BrokerStore::keys%(h: opaque of BrokerStore::Handle%): BrokerStore::QueryResult +function Broker::__keys%(h: opaque of Broker::Handle%): Broker::QueryResult %{ double timeout; bro_broker::StoreQueryCallback* cb; @@ -575,12 +416,7 @@ function BrokerStore::keys%(h: opaque of BrokerStore::Handle%): BrokerStore::Que return 0; %} -## Get the number of key-value pairs in a data store. -## -## h: the handle of the store to query. -## -## Returns: the result of the query (uses :bro:see:`BrokerComm::COUNT`). -function BrokerStore::size%(h: opaque of BrokerStore::Handle%): BrokerStore::QueryResult +function Broker::__size%(h: opaque of Broker::Handle%): Broker::QueryResult %{ if ( ! broker_mgr->Enabled() ) return bro_broker::query_result(); diff --git a/src/broxygen/Configuration.cc b/src/broxygen/Configuration.cc index 264e8e6fcb..4780e6ad99 100644 --- a/src/broxygen/Configuration.cc +++ b/src/broxygen/Configuration.cc @@ -65,7 +65,7 @@ Config::Config(const string& arg_file, const string& delim) Target* target = target_factory.Create(tokens[0], tokens[2], tokens[1]); if ( ! target ) - reporter->FatalError("unkown Broxygen target type: %s", + reporter->FatalError("unknown Broxygen target type: %s", tokens[0].c_str()); targets.push_back(target); diff --git a/src/cq.c b/src/cq.c index 8005544400..24f474d928 100644 --- a/src/cq.c +++ b/src/cq.c @@ -42,6 +42,7 @@ struct cq_handle { int lowmark; /* low bucket threshold */ int nextbucket; /* next bucket to check */ int noresize; /* don't resize while we're resizing */ + uint64_t cumulative_num; /* cumulative entries ever enqueued */ double lastpri; /* last priority */ double ysize; /* length of a year */ double bwidth; /* width of each bucket */ @@ -175,6 +176,9 @@ cq_enqueue(register struct cq_handle *hp, register double pri, } bp->pri = pri; bp->cookie = cookie; + + ++hp->cumulative_num; + if (++hp->qlen > hp->max_qlen) hp->max_qlen = hp->qlen; #ifdef DEBUG @@ -414,6 +418,12 @@ cq_max_size(struct cq_handle *hp) return hp->max_qlen; } +uint64_t +cq_cumulative_num(struct cq_handle *hp) +{ + return hp->cumulative_num; +} + /* Return without doing anything if we fail to allocate a new bucket array */ static int cq_resize(register struct cq_handle *hp, register int grow) diff --git a/src/cq.h b/src/cq.h index 540cccde74..152a7da536 100644 --- a/src/cq.h +++ b/src/cq.h @@ -1,3 +1,6 @@ + +#include + struct cq_handle *cq_init(double, double); void cq_destroy(struct cq_handle *); int cq_enqueue(struct cq_handle *, double, void *); @@ -5,6 +8,7 @@ void *cq_dequeue(struct cq_handle *, double); void *cq_remove(struct cq_handle *, double, void *); int cq_size(struct cq_handle *); int cq_max_size(struct cq_handle *); +uint64_t cq_cumulative_num(struct cq_handle *); unsigned int cq_memory_allocation(void); #ifdef DEBUG void cq_debug(struct cq_handle *, int); diff --git a/src/event.bif b/src/event.bif index ff6ec059fb..49afb86fa4 100644 --- a/src/event.bif +++ b/src/event.bif @@ -306,10 +306,10 @@ event packet_contents%(c: connection, contents: string%); ## t2: The new payload. ## ## tcp_flags: A string with the TCP flags of the packet triggering the -## inconsistency. In the string, each character corresponds to one set flag, -## as follows: ``S`` -> SYN; ``F`` -> FIN; ``R`` -> RST; ``A`` -> ACK; ``P`` -> -## PUSH. This string will not always be set, only if the information is available; -## it's "best effort". +## inconsistency. In the string, each character corresponds to one +## set flag, as follows: ``S`` -> SYN; ``F`` -> FIN; ``R`` -> RST; +## ``A`` -> ACK; ``P`` -> PUSH. This string will not always be set, +## only if the information is available; it's "best effort". ## ## .. bro:see:: tcp_rexmit tcp_contents event rexmit_inconsistency%(c: connection, t1: string, t2: string, tcp_flags: string%); @@ -366,26 +366,6 @@ event ack_above_hole%(c: connection%); ## the two. event content_gap%(c: connection, is_orig: bool, seq: count, length: count%); -## Summarizes the amount of missing TCP payload at regular intervals. -## Internally, Bro tracks (1) the number of :bro:id:`ack_above_hole` events, -## including the number of bytes missing; and (2) the total number of TCP -## acks seen, with the total volume of bytes that have been acked. This event -## reports these statistics in :bro:id:`gap_report_freq` intervals for the -## purpose of determining packet loss. -## -## dt: The time that has passed since the last ``gap_report`` interval. -## -## info: The gap statistics. -## -## .. bro:see:: content_gap ack_above_hole -## -## .. note:: -## -## Bro comes with a script :doc:`/scripts/policy/misc/capture-loss.bro` that -## uses this event to estimate packet loss and report when a predefined -## threshold is exceeded. -event gap_report%(dt: interval, info: gap_info%); - ## Generated when a protocol analyzer confirms that a connection is indeed ## using that protocol. Bro's dynamic protocol detection heuristically activates ## analyzers as soon as it believes a connection *could* be using a particular diff --git a/src/file_analysis/FileReassembler.cc b/src/file_analysis/FileReassembler.cc index 8b678e5209..ba15086320 100644 --- a/src/file_analysis/FileReassembler.cc +++ b/src/file_analysis/FileReassembler.cc @@ -8,7 +8,7 @@ namespace file_analysis { class File; FileReassembler::FileReassembler(File *f, uint64 starting_offset) - : Reassembler(starting_offset), the_file(f), flushing(false) + : Reassembler(starting_offset, REASSEM_FILE), the_file(f), flushing(false) { } diff --git a/src/file_analysis/Manager.h b/src/file_analysis/Manager.h index 93c8e7f613..bcc8ac5dd2 100644 --- a/src/file_analysis/Manager.h +++ b/src/file_analysis/Manager.h @@ -302,6 +302,15 @@ public: */ std::string DetectMIME(const u_char* data, uint64 len) const; + uint64 CurrentFiles() + { return id_map.Length(); } + + uint64 MaxFiles() + { return id_map.MaxLength(); } + + uint64 CumulativeFiles() + { return id_map.NumCumulativeInserts(); } + protected: friend class FileTimer; diff --git a/src/file_analysis/analyzer/x509/X509.cc b/src/file_analysis/analyzer/x509/X509.cc index e8ea5cb7b4..ebf7b1d04f 100644 --- a/src/file_analysis/analyzer/x509/X509.cc +++ b/src/file_analysis/analyzer/x509/X509.cc @@ -543,7 +543,7 @@ double file_analysis::X509::GetTimeFromAsn1(const ASN1_TIME* atime, const char* } // year is first two digits in YY format. Buffer expects YYYY format. - if ( pString[0] - '0' < 50 ) // RFC 2459 4.1.2.5.1 + if ( pString[0] < '5' ) // RFC 2459 4.1.2.5.1 { *(pBuffer++) = '2'; *(pBuffer++) = '0'; diff --git a/src/iosource/Packet.cc b/src/iosource/Packet.cc index 2aa7fa58c7..c75b62a832 100644 --- a/src/iosource/Packet.cc +++ b/src/iosource/Packet.cc @@ -428,15 +428,6 @@ void Packet::ProcessLayer2() RecordVal* Packet::BuildPktHdrVal() const { - static RecordType* l2_hdr_type = 0; - static RecordType* raw_pkt_hdr_type = 0; - - if ( ! raw_pkt_hdr_type ) - { - raw_pkt_hdr_type = internal_type("raw_pkt_hdr")->AsRecordType(); - l2_hdr_type = internal_type("l2_hdr")->AsRecordType(); - } - RecordVal* pkt_hdr = new RecordVal(raw_pkt_hdr_type); RecordVal* l2_hdr = new RecordVal(l2_hdr_type); diff --git a/src/iosource/PktSrc.cc b/src/iosource/PktSrc.cc index 8db9db6ef1..025432eba3 100644 --- a/src/iosource/PktSrc.cc +++ b/src/iosource/PktSrc.cc @@ -91,7 +91,7 @@ void PktSrc::Opened(const Properties& arg_props) { char buf[512]; safe_snprintf(buf, sizeof(buf), - "unknown data link type 0x%x", props.link_type); + "unknown data link type 0x%x", arg_props.link_type); Error(buf); Close(); return; diff --git a/src/main.cc b/src/main.cc index 73181c82f2..a0615d75da 100644 --- a/src/main.cc +++ b/src/main.cc @@ -1172,8 +1172,8 @@ int main(int argc, char** argv) double time_net_start = current_time(true);; - unsigned int mem_net_start_total; - unsigned int mem_net_start_malloced; + uint64 mem_net_start_total; + uint64 mem_net_start_malloced; if ( time_bro ) { @@ -1181,7 +1181,7 @@ int main(int argc, char** argv) fprintf(stderr, "# initialization %.6f\n", time_net_start - time_start); - fprintf(stderr, "# initialization %uM/%uM\n", + fprintf(stderr, "# initialization %" PRIu64 "M/%" PRIu64 "M\n", mem_net_start_total / 1024 / 1024, mem_net_start_malloced / 1024 / 1024); } @@ -1190,8 +1190,8 @@ int main(int argc, char** argv) double time_net_done = current_time(true);; - unsigned int mem_net_done_total; - unsigned int mem_net_done_malloced; + uint64 mem_net_done_total; + uint64 mem_net_done_malloced; if ( time_bro ) { @@ -1200,7 +1200,7 @@ int main(int argc, char** argv) fprintf(stderr, "# total time %.6f, processing %.6f\n", time_net_done - time_start, time_net_done - time_net_start); - fprintf(stderr, "# total mem %uM/%uM, processing %uM/%uM\n", + fprintf(stderr, "# total mem %" PRId64 "M/%" PRId64 "M, processing %" PRId64 "M/%" PRId64 "M\n", mem_net_done_total / 1024 / 1024, mem_net_done_malloced / 1024 / 1024, (mem_net_done_total - mem_net_start_total) / 1024 / 1024, diff --git a/src/nb_dns.c b/src/nb_dns.c index 1e5d427924..35059ab4f0 100644 --- a/src/nb_dns.c +++ b/src/nb_dns.c @@ -389,7 +389,7 @@ nb_dns_addr_request2(register struct nb_dns_info *nd, char *addrp, default: snprintf(errstr, NB_DNS_ERRSIZE, - "nb_dns_addr_request2(): uknown address family %d", af); + "nb_dns_addr_request2(): unknown address family %d", af); return (-1); } diff --git a/src/parse.y b/src/parse.y index c67732835f..f9eb7cbe9b 100644 --- a/src/parse.y +++ b/src/parse.y @@ -1474,11 +1474,20 @@ event: TOK_ID '(' opt_expr_list ')' { set_location(@1, @4); - $$ = new EventExpr($1, $3); - ID* id = lookup_ID($1, current_module.c_str()); - if ( id && id->IsDeprecated() ) - reporter->Warning("deprecated (%s)", id->Name()); + ID* id = lookup_ID($1, current_module.c_str()); + if ( id ) + { + if ( ! id->IsGlobal() ) + { + yyerror(fmt("local identifier \"%s\" cannot be used to reference an event", $1)); + YYERROR; + } + if ( id->IsDeprecated() ) + reporter->Warning("deprecated (%s)", id->Name()); + } + + $$ = new EventExpr($1, $3); } ; diff --git a/src/stats.bif b/src/stats.bif new file mode 100644 index 0000000000..e901b5e777 --- /dev/null +++ b/src/stats.bif @@ -0,0 +1,422 @@ + +%%{ // C segment +#include "util.h" +#include "threading/Manager.h" + +RecordType* ProcStats; +RecordType* NetStats; +RecordType* MatcherStats; +RecordType* ReassemblerStats; +RecordType* DNSStats; +RecordType* ConnStats; +RecordType* GapStats; +RecordType* EventStats; +RecordType* ThreadStats; +RecordType* TimerStats; +RecordType* FileAnalysisStats; +%%} + +## Returns packet capture statistics. Statistics include the number of +## packets *(i)* received by Bro, *(ii)* dropped, and *(iii)* seen on the +## link (not always available). +## +## Returns: A record of packet statistics. +## +## .. bro:see:: get_conn_stats +## get_dns_stats +## get_event_stats +## get_file_analysis_stats +## get_gap_stats +## get_matcher_stats +## get_proc_stats +## get_reassembler_stats +## get_thread_stats +## get_timer_stats +function get_net_stats%(%): NetStats + %{ + uint64 recv = 0; + uint64 drop = 0; + uint64 link = 0; + uint64 bytes_recv = 0; + + const iosource::Manager::PktSrcList& pkt_srcs(iosource_mgr->GetPktSrcs()); + + for ( iosource::Manager::PktSrcList::const_iterator i = pkt_srcs.begin(); + i != pkt_srcs.end(); i++ ) + { + iosource::PktSrc* ps = *i; + + struct iosource::PktSrc::Stats stat; + ps->Statistics(&stat); + recv += stat.received; + drop += stat.dropped; + link += stat.link; + bytes_recv += stat.bytes_received; + } + + RecordVal* r = new RecordVal(NetStats); + int n = 0; + + r->Assign(n++, new Val(recv, TYPE_COUNT)); + r->Assign(n++, new Val(drop, TYPE_COUNT)); + r->Assign(n++, new Val(link, TYPE_COUNT)); + r->Assign(n++, new Val(bytes_recv, TYPE_COUNT)); + + return r; + %} + +## Returns Bro traffic statistics. +## +## Returns: A record with connection and packet statistics. +## +## .. bro:see:: get_dns_stats +## get_event_stats +## get_file_analysis_stats +## get_gap_stats +## get_matcher_stats +## get_net_stats +## get_proc_stats +## get_reassembler_stats +## get_thread_stats +## get_timer_stats +function get_conn_stats%(%): ConnStats + %{ + RecordVal* r = new RecordVal(ConnStats); + int n = 0; + + r->Assign(n++, new Val(Connection::TotalConnections(), TYPE_COUNT)); + r->Assign(n++, new Val(Connection::CurrentConnections(), TYPE_COUNT)); + r->Assign(n++, new Val(Connection::CurrentExternalConnections(), TYPE_COUNT)); + r->Assign(n++, new Val(sessions->CurrentConnections(), TYPE_COUNT)); + + SessionStats s; + if ( sessions ) + sessions->GetStats(s); + +#define ADD_STAT(x) \ + r->Assign(n++, new Val(unsigned(sessions ? x : 0), TYPE_COUNT)); + + ADD_STAT(s.num_packets); + ADD_STAT(s.num_fragments); + ADD_STAT(s.max_fragments); + ADD_STAT(s.num_TCP_conns); + ADD_STAT(s.max_TCP_conns); + ADD_STAT(s.cumulative_TCP_conns); + ADD_STAT(s.num_UDP_conns); + ADD_STAT(s.max_UDP_conns); + ADD_STAT(s.cumulative_UDP_conns); + ADD_STAT(s.num_ICMP_conns); + ADD_STAT(s.max_ICMP_conns); + ADD_STAT(s.cumulative_ICMP_conns); + + r->Assign(n++, new Val(killed_by_inactivity, TYPE_COUNT)); + + return r; + %} + +## Returns Bro process statistics. +## +## Returns: A record with process statistics. +## +## .. bro:see:: get_conn_stats +## get_dns_stats +## get_event_stats +## get_file_analysis_stats +## get_gap_stats +## get_matcher_stats +## get_net_stats +## get_reassembler_stats +## get_thread_stats +## get_timer_stats +function get_proc_stats%(%): ProcStats + %{ + struct rusage ru; + if ( getrusage(RUSAGE_SELF, &ru) < 0 ) + reporter->InternalError("getrusage() failed in get_proc_stats()"); + + RecordVal* r = new RecordVal(ProcStats); + int n = 0; + + double elapsed_time = current_time() - bro_start_time; + double user_time = + double(ru.ru_utime.tv_sec) + double(ru.ru_utime.tv_usec) / 1e6; + double system_time = + double(ru.ru_stime.tv_sec) + double(ru.ru_stime.tv_usec) / 1e6; + +#ifdef DEBUG + r->Assign(n++, new Val(1, TYPE_COUNT)); +#else + r->Assign(n++, new Val(0, TYPE_COUNT)); +#endif + + r->Assign(n++, new Val(bro_start_time, TYPE_TIME)); + + r->Assign(n++, new IntervalVal(elapsed_time, Seconds)); + r->Assign(n++, new IntervalVal(user_time, Seconds)); + r->Assign(n++, new IntervalVal(system_time, Seconds)); + + uint64 total_mem; + get_memory_usage(&total_mem, NULL); + r->Assign(n++, new Val(unsigned(total_mem), TYPE_COUNT)); + + r->Assign(n++, new Val(unsigned(ru.ru_minflt), TYPE_COUNT)); + r->Assign(n++, new Val(unsigned(ru.ru_majflt), TYPE_COUNT)); + r->Assign(n++, new Val(unsigned(ru.ru_nswap), TYPE_COUNT)); + r->Assign(n++, new Val(unsigned(ru.ru_inblock), TYPE_COUNT)); + r->Assign(n++, new Val(unsigned(ru.ru_oublock), TYPE_COUNT)); + r->Assign(n++, new Val(unsigned(ru.ru_nivcsw), TYPE_COUNT)); + + return r; + %} + +## Returns statistics about the event engine. +## +## Returns: A record with event engine statistics. +## +## .. bro:see:: get_conn_stats +## get_dns_stats +## get_file_analysis_stats +## get_gap_stats +## get_matcher_stats +## get_net_stats +## get_proc_stats +## get_reassembler_stats +## get_thread_stats +## get_timer_stats +function get_event_stats%(%): EventStats + %{ + RecordVal* r = new RecordVal(EventStats); + int n = 0; + + r->Assign(n++, new Val(num_events_queued, TYPE_COUNT)); + r->Assign(n++, new Val(num_events_dispatched, TYPE_COUNT)); + + return r; + %} + +## Returns statistics about reassembler usage. +## +## Returns: A record with reassembler statistics. +## +## .. bro:see:: get_conn_stats +## get_dns_stats +## get_event_stats +## get_file_analysis_stats +## get_gap_stats +## get_matcher_stats +## get_net_stats +## get_proc_stats +## get_thread_stats +## get_timer_stats +function get_reassembler_stats%(%): ReassemblerStats + %{ + RecordVal* r = new RecordVal(ReassemblerStats); + int n = 0; + + r->Assign(n++, new Val(Reassembler::MemoryAllocation(REASSEM_FILE), TYPE_COUNT)); + r->Assign(n++, new Val(Reassembler::MemoryAllocation(REASSEM_FRAG), TYPE_COUNT)); + r->Assign(n++, new Val(Reassembler::MemoryAllocation(REASSEM_TCP), TYPE_COUNT)); + r->Assign(n++, new Val(Reassembler::MemoryAllocation(REASSEM_UNKNOWN), TYPE_COUNT)); + + return r; + %} + +## Returns statistics about DNS lookup activity. +## +## Returns: A record with DNS lookup statistics. +## +## .. bro:see:: get_conn_stats +## get_event_stats +## get_file_analysis_stats +## get_gap_stats +## get_matcher_stats +## get_net_stats +## get_proc_stats +## get_reassembler_stats +## get_thread_stats +## get_timer_stats +function get_dns_stats%(%): DNSStats + %{ + RecordVal* r = new RecordVal(DNSStats); + int n = 0; + + DNS_Mgr::Stats dstats; + dns_mgr->GetStats(&dstats); + + r->Assign(n++, new Val(unsigned(dstats.requests), TYPE_COUNT)); + r->Assign(n++, new Val(unsigned(dstats.successful), TYPE_COUNT)); + r->Assign(n++, new Val(unsigned(dstats.failed), TYPE_COUNT)); + r->Assign(n++, new Val(unsigned(dstats.pending), TYPE_COUNT)); + r->Assign(n++, new Val(unsigned(dstats.cached_hosts), TYPE_COUNT)); + r->Assign(n++, new Val(unsigned(dstats.cached_addresses), TYPE_COUNT)); + + return r; + %} + +## Returns statistics about timer usage. +## +## Returns: A record with timer usage statistics. +## +## .. bro:see:: get_conn_stats +## get_dns_stats +## get_event_stats +## get_file_analysis_stats +## get_gap_stats +## get_matcher_stats +## get_net_stats +## get_proc_stats +## get_reassembler_stats +## get_thread_stats +function get_timer_stats%(%): TimerStats + %{ + RecordVal* r = new RecordVal(TimerStats); + int n = 0; + + r->Assign(n++, new Val(unsigned(timer_mgr->Size()), TYPE_COUNT)); + r->Assign(n++, new Val(unsigned(timer_mgr->PeakSize()), TYPE_COUNT)); + r->Assign(n++, new Val(timer_mgr->CumulativeNum(), TYPE_COUNT)); + + return r; + %} + +## Returns statistics about file analysis. +## +## Returns: A record with file analysis statistics. +## +## .. bro:see:: get_conn_stats +## get_dns_stats +## get_event_stats +## get_gap_stats +## get_matcher_stats +## get_net_stats +## get_proc_stats +## get_reassembler_stats +## get_thread_stats +## get_timer_stats +function get_file_analysis_stats%(%): FileAnalysisStats + %{ + RecordVal* r = new RecordVal(FileAnalysisStats); + int n = 0; + + r->Assign(n++, new Val(file_mgr->CurrentFiles(), TYPE_COUNT)); + r->Assign(n++, new Val(file_mgr->MaxFiles(), TYPE_COUNT)); + r->Assign(n++, new Val(file_mgr->CumulativeFiles(), TYPE_COUNT)); + + return r; + %} + +## Returns statistics about thread usage. +## +## Returns: A record with thread usage statistics. +## +## .. bro:see:: get_conn_stats +## get_dns_stats +## get_event_stats +## get_file_analysis_stats +## get_gap_stats +## get_matcher_stats +## get_net_stats +## get_proc_stats +## get_reassembler_stats +## get_timer_stats +function get_thread_stats%(%): ThreadStats + %{ + RecordVal* r = new RecordVal(ThreadStats); + int n = 0; + + r->Assign(n++, new Val(thread_mgr->NumThreads(), TYPE_COUNT)); + + return r; + %} + +## Returns statistics about TCP gaps. +## +## Returns: A record with TCP gap statistics. +## +## .. bro:see:: get_conn_stats +## get_dns_stats +## get_event_stats +## get_file_analysis_stats +## get_matcher_stats +## get_net_stats +## get_proc_stats +## get_reassembler_stats +## get_thread_stats +## get_timer_stats +function get_gap_stats%(%): GapStats + %{ + RecordVal* r = new RecordVal(GapStats); + int n = 0; + + r->Assign(n++, new Val(tot_ack_events, TYPE_COUNT)); + r->Assign(n++, new Val(tot_ack_bytes, TYPE_COUNT)); + r->Assign(n++, new Val(tot_gap_events, TYPE_COUNT)); + r->Assign(n++, new Val(tot_gap_bytes, TYPE_COUNT)); + + return r; + %} + +## Returns statistics about the regular expression engine. Statistics include +## the number of distinct matchers, DFA states, DFA state transitions, memory +## usage of DFA states, cache hits/misses, and average number of NFA states +## across all matchers. +## +## Returns: A record with matcher statistics. +## +## .. bro:see:: get_conn_stats +## get_dns_stats +## get_event_stats +## get_file_analysis_stats +## get_gap_stats +## get_net_stats +## get_proc_stats +## get_reassembler_stats +## get_thread_stats +## get_timer_stats +function get_matcher_stats%(%): MatcherStats + %{ + RecordVal* r = new RecordVal(MatcherStats); + int n = 0; + + RuleMatcher::Stats s; + memset(&s, 0, sizeof(s)); + if ( rule_matcher ) + rule_matcher->GetStats(&s); + + r->Assign(n++, new Val(s.matchers, TYPE_COUNT)); + r->Assign(n++, new Val(s.nfa_states, TYPE_COUNT)); + r->Assign(n++, new Val(s.dfa_states, TYPE_COUNT)); + r->Assign(n++, new Val(s.computed, TYPE_COUNT)); + r->Assign(n++, new Val(s.mem, TYPE_COUNT)); + r->Assign(n++, new Val(s.hits, TYPE_COUNT)); + r->Assign(n++, new Val(s.misses, TYPE_COUNT)); + + return r; + %} + +# function get_broker_stats%(%): BrokerStats +# %{ +# RecordVal* r = new RecordVal(CommunicationStats); +# int n = 0; +# +# #ifdef ENABLE_BROKER +# auto cs = broker_mgr->ConsumeStatistics(); +# +# r->Assign(n++, new Val(cs.outgoing_peer_count, TYPE_COUNT)); +# r->Assign(n++, new Val(cs.data_store_count, TYPE_COUNT)); +# r->Assign(n++, new Val(cs.pending_query_count, TYPE_COUNT)); +# r->Assign(n++, new Val(cs.response_count, TYPE_COUNT)); +# r->Assign(n++, new Val(cs.outgoing_conn_status_count, TYPE_COUNT)); +# r->Assign(n++, new Val(cs.incoming_conn_status_count, TYPE_COUNT)); +# r->Assign(n++, new Val(cs.report_count, TYPE_COUNT)); +# +# //for ( const auto& s : cs.print_count ) +# // file->Write(fmt(" %-25s prints dequeued=%zu\n", s.first.data(), s.second)); +# //for ( const auto& s : cs.event_count ) +# // file->Write(fmt(" %-25s events dequeued=%zu\n", s.first.data(), s.second)); +# //for ( const auto& s : cs.log_count ) +# // file->Write(fmt(" %-25s logs dequeued=%zu\n", s.first.data(), s.second)); +# #endif +# +# return r; +# %} diff --git a/src/threading/formatters/JSON.cc b/src/threading/formatters/JSON.cc index 3558baee5c..45c7be3e93 100644 --- a/src/threading/formatters/JSON.cc +++ b/src/threading/formatters/JSON.cc @@ -116,21 +116,28 @@ bool JSON::Describe(ODesc* desc, Value* val, const string& name) const { char buffer[40]; char buffer2[40]; - time_t t = time_t(val->val.double_val); + time_t the_time = time_t(val->val.double_val); + struct tm t; - if ( strftime(buffer, sizeof(buffer), "%Y-%m-%dT%H:%M:%S", gmtime(&t)) > 0 ) + desc->AddRaw("\"", 1); + + if ( ! gmtime_r(&the_time, &t) || + ! strftime(buffer, sizeof(buffer), "%Y-%m-%dT%H:%M:%S", &t) ) + { + GetThread()->Error(GetThread()->Fmt("json formatter: failure getting time: (%" PRIu64 ")", val->val.double_val)); + // This was a failure, doesn't really matter what gets put here + // but it should probably stand out... + desc->Add("2000-01-01T00:00:00.000000"); + } + else { double integ; double frac = modf(val->val.double_val, &integ); snprintf(buffer2, sizeof(buffer2), "%s.%06.0fZ", buffer, frac * 1000000); - desc->AddRaw("\"", 1); desc->Add(buffer2); - desc->AddRaw("\"", 1); } - else - GetThread()->Error(GetThread()->Fmt("strftime error for JSON: %" PRIu64)); - + desc->AddRaw("\"", 1); } else if ( timestamps == TS_EPOCH ) diff --git a/src/util.cc b/src/util.cc index 0ea89beb90..e6015cc20a 100644 --- a/src/util.cc +++ b/src/util.cc @@ -14,6 +14,11 @@ # endif #endif +#ifdef HAVE_DARWIN +#include +#include +#endif + #include #include #include @@ -571,7 +576,14 @@ const char* fmt_access_time(double t) { static char buf[256]; time_t time = (time_t) t; - strftime(buf, sizeof(buf), "%d/%m-%H:%M", localtime(&time)); + struct tm ts; + + if ( ! localtime_r(&time, &ts) ) + { + reporter->InternalError("unable to get time"); + } + + strftime(buf, sizeof(buf), "%d/%m-%H:%M", &ts); return buf; } @@ -1611,23 +1623,35 @@ extern "C" void out_of_memory(const char* where) abort(); } -void get_memory_usage(unsigned int* total, unsigned int* malloced) +void get_memory_usage(uint64* total, uint64* malloced) { - unsigned int ret_total; + uint64 ret_total; #ifdef HAVE_MALLINFO struct mallinfo mi = mallinfo(); if ( malloced ) *malloced = mi.uordblks; - #endif +#ifdef HAVE_DARWIN + struct mach_task_basic_info t_info; + mach_msg_type_number_t t_info_count = MACH_TASK_BASIC_INFO; + + if ( KERN_SUCCESS != task_info(mach_task_self(), + MACH_TASK_BASIC_INFO, + (task_info_t)&t_info, + &t_info_count) ) + ret_total = 0; + else + ret_total = t_info.resident_size; +#else struct rusage r; getrusage(RUSAGE_SELF, &r); // In KB. ret_total = r.ru_maxrss * 1024; +#endif if ( total ) *total = ret_total; diff --git a/src/util.h b/src/util.h index 15d1a059cd..70095fba8d 100644 --- a/src/util.h +++ b/src/util.h @@ -499,8 +499,7 @@ inline int safe_vsnprintf(char* str, size_t size, const char* format, va_list al // Returns total memory allocations and (if available) amount actually // handed out by malloc. -extern void get_memory_usage(unsigned int* total, - unsigned int* malloced); +extern void get_memory_usage(uint64* total, uint64* malloced); // Class to be used as a third argument for STL maps to be able to use // char*'s as keys. Otherwise the pointer values will be compared instead of diff --git a/testing/btest/Baseline/bifs.fmt/out b/testing/btest/Baseline/bifs.fmt/out index 5f380c1b22..2572f924fb 100644 --- a/testing/btest/Baseline/bifs.fmt/out +++ b/testing/btest/Baseline/bifs.fmt/out @@ -45,11 +45,6 @@ test 310 310 2 -1 2 2 -1 -2 -2 -1 2 diff --git a/testing/btest/Baseline/bifs.get_current_packet_header/output b/testing/btest/Baseline/bifs.get_current_packet_header/output new file mode 100644 index 0000000000..761a248077 --- /dev/null +++ b/testing/btest/Baseline/bifs.get_current_packet_header/output @@ -0,0 +1 @@ +[l2=[encap=LINK_ETHERNET, len=78, cap_len=78, src=00:00:00:00:00:00, dst=ff:ff:ff:ff:ff:ff, vlan=, inner_vlan=, eth_type=34525, proto=L3_IPV6], ip=, ip6=[class=0, flow=0, len=24, nxt=58, hlim=255, src=fe80::dead, dst=fe80::beef, exts=[]], tcp=, udp=, icmp=[icmp_type=135]] diff --git a/testing/btest/Baseline/broker.clone_store/clone.clone.out b/testing/btest/Baseline/broker.clone_store/clone.clone.out index 570f3f25ca..3db1dd4e00 100644 --- a/testing/btest/Baseline/broker.clone_store/clone.clone.out +++ b/testing/btest/Baseline/broker.clone_store/clone.clone.out @@ -1,5 +1,5 @@ -clone keys, [status=BrokerStore::SUCCESS, result=[d=broker::data{[one, two, myset, myvec]}]] -lookup, one, [status=BrokerStore::SUCCESS, result=[d=broker::data{111}]] -lookup, myset, [status=BrokerStore::SUCCESS, result=[d=broker::data{{a, c, d}}]] -lookup, two, [status=BrokerStore::SUCCESS, result=[d=broker::data{222}]] -lookup, myvec, [status=BrokerStore::SUCCESS, result=[d=broker::data{[delta, alpha, beta, gamma, omega]}]] +clone keys, [status=Broker::SUCCESS, result=[d=broker::data{[one, two, myset, myvec]}]] +lookup, two, [status=Broker::SUCCESS, result=[d=broker::data{222}]] +lookup, one, [status=Broker::SUCCESS, result=[d=broker::data{111}]] +lookup, myvec, [status=Broker::SUCCESS, result=[d=broker::data{[delta, alpha, beta, gamma, omega]}]] +lookup, myset, [status=Broker::SUCCESS, result=[d=broker::data{{a, c, d}}]] diff --git a/testing/btest/Baseline/broker.connection_updates/recv.recv.out b/testing/btest/Baseline/broker.connection_updates/recv.recv.out index 714cbfbac4..d246bf153f 100644 --- a/testing/btest/Baseline/broker.connection_updates/recv.recv.out +++ b/testing/btest/Baseline/broker.connection_updates/recv.recv.out @@ -1,2 +1,2 @@ -BrokerComm::incoming_connection_established, connector -BrokerComm::incoming_connection_broken, connector +Broker::incoming_connection_established, connector +Broker::incoming_connection_broken, connector diff --git a/testing/btest/Baseline/broker.connection_updates/send.send.out b/testing/btest/Baseline/broker.connection_updates/send.send.out index 61c988d1c8..205782c8f0 100644 --- a/testing/btest/Baseline/broker.connection_updates/send.send.out +++ b/testing/btest/Baseline/broker.connection_updates/send.send.out @@ -1 +1 @@ -BrokerComm::outgoing_connection_established, 127.0.0.1, 9999/tcp, listener +Broker::outgoing_connection_established, 127.0.0.1, 9999/tcp, listener diff --git a/testing/btest/Baseline/broker.data/out b/testing/btest/Baseline/broker.data/out index 628870144a..8703ca6a0c 100644 --- a/testing/btest/Baseline/broker.data/out +++ b/testing/btest/Baseline/broker.data/out @@ -1,18 +1,18 @@ -BrokerComm::BOOL -BrokerComm::INT -BrokerComm::COUNT -BrokerComm::DOUBLE -BrokerComm::STRING -BrokerComm::ADDR -BrokerComm::SUBNET -BrokerComm::PORT -BrokerComm::TIME -BrokerComm::INTERVAL -BrokerComm::ENUM -BrokerComm::SET -BrokerComm::TABLE -BrokerComm::VECTOR -BrokerComm::RECORD +Broker::BOOL +Broker::INT +Broker::COUNT +Broker::DOUBLE +Broker::STRING +Broker::ADDR +Broker::SUBNET +Broker::PORT +Broker::TIME +Broker::INTERVAL +Broker::ENUM +Broker::SET +Broker::TABLE +Broker::VECTOR +Broker::RECORD *************************** T F @@ -29,13 +29,22 @@ hello 22/tcp 42.0 180.0 -BrokerComm::BOOL -*************************** +Broker::BOOL { two, one, three } +{ +[two] = 2, +[one] = 1, +[three] = 3 +} +[zero, one, two] +[a=, b=bee, c=1] +[a=test, b=bee, c=1] +[a=test, b=testagain, c=1] +*************************** 0 T 1 @@ -43,19 +52,20 @@ T F T 2 +F +2 T 1 F { bye } +T 0 -*************************** { -[two] = 2, -[one] = 1, -[three] = 3 + } +*************************** 0 [d=] 1 @@ -69,8 +79,14 @@ F 37 [d=broker::data{42}] 1 +[d=] +1 +T +0 +{ + +} *************************** -[zero, one, two] 0 T T @@ -85,10 +101,10 @@ T [d=broker::data{bah}] [hi, salutations, greetings] 3 +T +0 +[] *************************** -[a=, b=bee, c=1] -[a=test, b=bee, c=1] -[a=test, b=testagain, c=1] 3 T T @@ -97,3 +113,6 @@ T [d=broker::data{hello}] [d=broker::data{37}] 3 +T +3 +[d=broker::data{goodbye}] diff --git a/testing/btest/Baseline/broker.master_store/master.out b/testing/btest/Baseline/broker.master_store/master.out index 4208503151..1983d0bccc 100644 --- a/testing/btest/Baseline/broker.master_store/master.out +++ b/testing/btest/Baseline/broker.master_store/master.out @@ -1,14 +1,14 @@ -lookup(two): [status=BrokerStore::SUCCESS, result=[d=broker::data{222}]] -lookup(four): [status=BrokerStore::SUCCESS, result=[d=]] -lookup(myset): [status=BrokerStore::SUCCESS, result=[d=broker::data{{a, c, d}}]] -lookup(one): [status=BrokerStore::SUCCESS, result=[d=broker::data{111}]] -lookup(myvec): [status=BrokerStore::SUCCESS, result=[d=broker::data{[delta, alpha, beta, gamma, omega]}]] -exists(one): [status=BrokerStore::SUCCESS, result=[d=broker::data{1}]] -exists(two): [status=BrokerStore::SUCCESS, result=[d=broker::data{0}]] -exists(myset): [status=BrokerStore::SUCCESS, result=[d=broker::data{1}]] -exists(four): [status=BrokerStore::SUCCESS, result=[d=broker::data{0}]] -pop_right(myvec): [status=BrokerStore::SUCCESS, result=[d=broker::data{omega}]] -pop_left(myvec): [status=BrokerStore::SUCCESS, result=[d=broker::data{delta}]] -keys: [status=BrokerStore::SUCCESS, result=[d=broker::data{[myvec, myset, one]}]] -size: [status=BrokerStore::SUCCESS, result=[d=broker::data{3}]] -size (after clear): [status=BrokerStore::SUCCESS, result=[d=broker::data{0}]] +lookup(two): [status=Broker::SUCCESS, result=[d=broker::data{222}]] +lookup(myset): [status=Broker::SUCCESS, result=[d=broker::data{{a, c, d}}]] +lookup(one): [status=Broker::SUCCESS, result=[d=broker::data{111}]] +lookup(myvec): [status=Broker::SUCCESS, result=[d=broker::data{[delta, alpha, beta, gamma, omega]}]] +lookup(four): [status=Broker::SUCCESS, result=[d=]] +exists(two): [status=Broker::SUCCESS, result=[d=broker::data{0}]] +exists(myset): [status=Broker::SUCCESS, result=[d=broker::data{1}]] +exists(one): [status=Broker::SUCCESS, result=[d=broker::data{1}]] +exists(four): [status=Broker::SUCCESS, result=[d=broker::data{0}]] +pop_left(myvec): [status=Broker::SUCCESS, result=[d=broker::data{delta}]] +pop_right(myvec): [status=Broker::SUCCESS, result=[d=broker::data{omega}]] +keys: [status=Broker::SUCCESS, result=[d=broker::data{[myvec, myset, one]}]] +size: [status=Broker::SUCCESS, result=[d=broker::data{3}]] +size (after clear): [status=Broker::SUCCESS, result=[d=broker::data{0}]] diff --git a/testing/btest/Baseline/broker.remote_event/send.send.out b/testing/btest/Baseline/broker.remote_event/send.send.out index a29c1ecd1e..2d61135abe 100644 --- a/testing/btest/Baseline/broker.remote_event/send.send.out +++ b/testing/btest/Baseline/broker.remote_event/send.send.out @@ -1,4 +1,4 @@ -BrokerComm::outgoing_connection_established, 127.0.0.1, 9999/tcp +Broker::outgoing_connection_established, 127.0.0.1, 9999/tcp got event msg, pong, 0 got auto event msg, ping, 0 got event msg, pong, 1 diff --git a/testing/btest/Baseline/broker.remote_log/recv.recv.out b/testing/btest/Baseline/broker.remote_log/recv.recv.out index ef9cb8402d..2f4a31df51 100644 --- a/testing/btest/Baseline/broker.remote_log/recv.recv.out +++ b/testing/btest/Baseline/broker.remote_log/recv.recv.out @@ -1,6 +1,6 @@ -wrote log, [msg=ping, num=0, nolog=no] -wrote log, [msg=ping, num=1, nolog=no] -wrote log, [msg=ping, num=2, nolog=no] -wrote log, [msg=ping, num=3, nolog=no] -wrote log, [msg=ping, num=4, nolog=no] -wrote log, [msg=ping, num=5, nolog=no] +wrote log, [msg=ping, nolog=no, num=0] +wrote log, [msg=ping, nolog=no, num=1] +wrote log, [msg=ping, nolog=no, num=2] +wrote log, [msg=ping, nolog=no, num=3] +wrote log, [msg=ping, nolog=no, num=4] +wrote log, [msg=ping, nolog=no, num=5] diff --git a/testing/btest/Baseline/broker.remote_log/send.send.out b/testing/btest/Baseline/broker.remote_log/send.send.out index d97ef33af1..632279e697 100644 --- a/testing/btest/Baseline/broker.remote_log/send.send.out +++ b/testing/btest/Baseline/broker.remote_log/send.send.out @@ -1 +1 @@ -BrokerComm::outgoing_connection_established, 127.0.0.1, 9999/tcp +Broker::outgoing_connection_established, 127.0.0.1, 9999/tcp diff --git a/testing/btest/Baseline/broker.remote_print/send.send.out b/testing/btest/Baseline/broker.remote_print/send.send.out index 65d8ee79b7..861dd64a8a 100644 --- a/testing/btest/Baseline/broker.remote_print/send.send.out +++ b/testing/btest/Baseline/broker.remote_print/send.send.out @@ -1,4 +1,4 @@ -BrokerComm::outgoing_connection_established, 127.0.0.1, 9999/tcp +Broker::outgoing_connection_established, 127.0.0.1, 9999/tcp got print msg, pong 0 got print msg, pong 1 got print msg, pong 2 diff --git a/testing/btest/Baseline/core.ipv6_zero_len_ah/output b/testing/btest/Baseline/core.ipv6_zero_len_ah/output index d8db6a4c48..585011acbe 100644 --- a/testing/btest/Baseline/core.ipv6_zero_len_ah/output +++ b/testing/btest/Baseline/core.ipv6_zero_len_ah/output @@ -1,2 +1,2 @@ [orig_h=2000:1300::1, orig_p=128/icmp, resp_h=2000:1300::2, resp_p=129/icmp] -[ip=, ip6=[class=0, flow=0, len=166, nxt=51, hlim=255, src=2000:1300::1, dst=2000:1300::2, exts=[[id=51, hopopts=, dstopts=, routing=, fragment=, ah=[nxt=58, len=0, rsv=0, spi=0, seq=, data=], esp=, mobility=]]], tcp=, udp=, icmp=] +[ip=, ip6=[class=0, flow=0, len=166, nxt=51, hlim=255, src=2000:1300::1, dst=2000:1300::2, exts=[[id=51, hopopts=, dstopts=, routing=, fragment=, ah=[nxt=58, len=0, rsv=0, spi=0, seq=, data=], esp=, mobility=]]], tcp=, udp=, icmp=[icmp_type=128]] diff --git a/testing/btest/Baseline/core.leaks.broker.clone_store/clone.clone.out b/testing/btest/Baseline/core.leaks.broker.clone_store/clone.clone.out index 017537fea9..ef997abeb8 100644 --- a/testing/btest/Baseline/core.leaks.broker.clone_store/clone.clone.out +++ b/testing/btest/Baseline/core.leaks.broker.clone_store/clone.clone.out @@ -1,5 +1,5 @@ -clone keys, [status=BrokerStore::SUCCESS, result=[d=broker::data{[one, two, myset, myvec]}]] -lookup, one, [status=BrokerStore::SUCCESS, result=[d=broker::data{111}]] -lookup, two, [status=BrokerStore::SUCCESS, result=[d=broker::data{222}]] -lookup, myset, [status=BrokerStore::SUCCESS, result=[d=broker::data{{a, c, d}}]] -lookup, myvec, [status=BrokerStore::SUCCESS, result=[d=broker::data{[delta, alpha, beta, gamma, omega]}]] +clone keys, [status=Broker::SUCCESS, result=[d=broker::data{[one, two, myset, myvec]}]] +lookup, one, [status=Broker::SUCCESS, result=[d=broker::data{111}]] +lookup, two, [status=Broker::SUCCESS, result=[d=broker::data{222}]] +lookup, myset, [status=Broker::SUCCESS, result=[d=broker::data{{a, c, d}}]] +lookup, myvec, [status=Broker::SUCCESS, result=[d=broker::data{[delta, alpha, beta, gamma, omega]}]] diff --git a/testing/btest/Baseline/core.leaks.broker.data/bro..stdout b/testing/btest/Baseline/core.leaks.broker.data/bro..stdout index 628870144a..8703ca6a0c 100644 --- a/testing/btest/Baseline/core.leaks.broker.data/bro..stdout +++ b/testing/btest/Baseline/core.leaks.broker.data/bro..stdout @@ -1,18 +1,18 @@ -BrokerComm::BOOL -BrokerComm::INT -BrokerComm::COUNT -BrokerComm::DOUBLE -BrokerComm::STRING -BrokerComm::ADDR -BrokerComm::SUBNET -BrokerComm::PORT -BrokerComm::TIME -BrokerComm::INTERVAL -BrokerComm::ENUM -BrokerComm::SET -BrokerComm::TABLE -BrokerComm::VECTOR -BrokerComm::RECORD +Broker::BOOL +Broker::INT +Broker::COUNT +Broker::DOUBLE +Broker::STRING +Broker::ADDR +Broker::SUBNET +Broker::PORT +Broker::TIME +Broker::INTERVAL +Broker::ENUM +Broker::SET +Broker::TABLE +Broker::VECTOR +Broker::RECORD *************************** T F @@ -29,13 +29,22 @@ hello 22/tcp 42.0 180.0 -BrokerComm::BOOL -*************************** +Broker::BOOL { two, one, three } +{ +[two] = 2, +[one] = 1, +[three] = 3 +} +[zero, one, two] +[a=, b=bee, c=1] +[a=test, b=bee, c=1] +[a=test, b=testagain, c=1] +*************************** 0 T 1 @@ -43,19 +52,20 @@ T F T 2 +F +2 T 1 F { bye } +T 0 -*************************** { -[two] = 2, -[one] = 1, -[three] = 3 + } +*************************** 0 [d=] 1 @@ -69,8 +79,14 @@ F 37 [d=broker::data{42}] 1 +[d=] +1 +T +0 +{ + +} *************************** -[zero, one, two] 0 T T @@ -85,10 +101,10 @@ T [d=broker::data{bah}] [hi, salutations, greetings] 3 +T +0 +[] *************************** -[a=, b=bee, c=1] -[a=test, b=bee, c=1] -[a=test, b=testagain, c=1] 3 T T @@ -97,3 +113,6 @@ T [d=broker::data{hello}] [d=broker::data{37}] 3 +T +3 +[d=broker::data{goodbye}] diff --git a/testing/btest/Baseline/core.leaks.broker.master_store/bro..stdout b/testing/btest/Baseline/core.leaks.broker.master_store/bro..stdout index 4208503151..9eebc797e5 100644 --- a/testing/btest/Baseline/core.leaks.broker.master_store/bro..stdout +++ b/testing/btest/Baseline/core.leaks.broker.master_store/bro..stdout @@ -1,14 +1,14 @@ -lookup(two): [status=BrokerStore::SUCCESS, result=[d=broker::data{222}]] -lookup(four): [status=BrokerStore::SUCCESS, result=[d=]] -lookup(myset): [status=BrokerStore::SUCCESS, result=[d=broker::data{{a, c, d}}]] -lookup(one): [status=BrokerStore::SUCCESS, result=[d=broker::data{111}]] -lookup(myvec): [status=BrokerStore::SUCCESS, result=[d=broker::data{[delta, alpha, beta, gamma, omega]}]] -exists(one): [status=BrokerStore::SUCCESS, result=[d=broker::data{1}]] -exists(two): [status=BrokerStore::SUCCESS, result=[d=broker::data{0}]] -exists(myset): [status=BrokerStore::SUCCESS, result=[d=broker::data{1}]] -exists(four): [status=BrokerStore::SUCCESS, result=[d=broker::data{0}]] -pop_right(myvec): [status=BrokerStore::SUCCESS, result=[d=broker::data{omega}]] -pop_left(myvec): [status=BrokerStore::SUCCESS, result=[d=broker::data{delta}]] -keys: [status=BrokerStore::SUCCESS, result=[d=broker::data{[myvec, myset, one]}]] -size: [status=BrokerStore::SUCCESS, result=[d=broker::data{3}]] -size (after clear): [status=BrokerStore::SUCCESS, result=[d=broker::data{0}]] +lookup(two): [status=Broker::SUCCESS, result=[d=broker::data{222}]] +lookup(four): [status=Broker::SUCCESS, result=[d=]] +lookup(myset): [status=Broker::SUCCESS, result=[d=broker::data{{a, c, d}}]] +lookup(one): [status=Broker::SUCCESS, result=[d=broker::data{111}]] +lookup(myvec): [status=Broker::SUCCESS, result=[d=broker::data{[delta, alpha, beta, gamma, omega]}]] +exists(one): [status=Broker::SUCCESS, result=[d=broker::data{1}]] +exists(two): [status=Broker::SUCCESS, result=[d=broker::data{0}]] +exists(myset): [status=Broker::SUCCESS, result=[d=broker::data{1}]] +exists(four): [status=Broker::SUCCESS, result=[d=broker::data{0}]] +pop_right(myvec): [status=Broker::SUCCESS, result=[d=broker::data{omega}]] +pop_left(myvec): [status=Broker::SUCCESS, result=[d=broker::data{delta}]] +keys: [status=Broker::SUCCESS, result=[d=broker::data{[myvec, myset, one]}]] +size: [status=Broker::SUCCESS, result=[d=broker::data{3}]] +size (after clear): [status=Broker::SUCCESS, result=[d=broker::data{0}]] diff --git a/testing/btest/Baseline/core.leaks.broker.remote_event/send.send.out b/testing/btest/Baseline/core.leaks.broker.remote_event/send.send.out index a29c1ecd1e..2d61135abe 100644 --- a/testing/btest/Baseline/core.leaks.broker.remote_event/send.send.out +++ b/testing/btest/Baseline/core.leaks.broker.remote_event/send.send.out @@ -1,4 +1,4 @@ -BrokerComm::outgoing_connection_established, 127.0.0.1, 9999/tcp +Broker::outgoing_connection_established, 127.0.0.1, 9999/tcp got event msg, pong, 0 got auto event msg, ping, 0 got event msg, pong, 1 diff --git a/testing/btest/Baseline/core.leaks.broker.remote_log/send.send.out b/testing/btest/Baseline/core.leaks.broker.remote_log/send.send.out index d97ef33af1..632279e697 100644 --- a/testing/btest/Baseline/core.leaks.broker.remote_log/send.send.out +++ b/testing/btest/Baseline/core.leaks.broker.remote_log/send.send.out @@ -1 +1 @@ -BrokerComm::outgoing_connection_established, 127.0.0.1, 9999/tcp +Broker::outgoing_connection_established, 127.0.0.1, 9999/tcp diff --git a/testing/btest/Baseline/core.leaks.broker.remote_print/send.send.out b/testing/btest/Baseline/core.leaks.broker.remote_print/send.send.out index 65d8ee79b7..861dd64a8a 100644 --- a/testing/btest/Baseline/core.leaks.broker.remote_print/send.send.out +++ b/testing/btest/Baseline/core.leaks.broker.remote_print/send.send.out @@ -1,4 +1,4 @@ -BrokerComm::outgoing_connection_established, 127.0.0.1, 9999/tcp +Broker::outgoing_connection_established, 127.0.0.1, 9999/tcp got print msg, pong 0 got print msg, pong 1 got print msg, pong 2 diff --git a/testing/btest/Baseline/core.print-bpf-filters/output2 b/testing/btest/Baseline/core.print-bpf-filters/output2 index ac140925fc..3321684b43 100644 --- a/testing/btest/Baseline/core.print-bpf-filters/output2 +++ b/testing/btest/Baseline/core.print-bpf-filters/output2 @@ -1,5 +1,6 @@ 2 1080 1 137 +1 143 1 1434 1 161 1 162 @@ -20,7 +21,9 @@ 1 5060 1 5072 1 514 +1 5222 1 5223 +1 5269 2 53 1 5353 1 5355 @@ -47,8 +50,8 @@ 1 992 1 993 1 995 -54 and -53 or -54 port -36 tcp +57 and +56 or +57 port +39 tcp 18 udp diff --git a/testing/btest/Baseline/coverage.bare-load-baseline/canonified_loaded_scripts.log b/testing/btest/Baseline/coverage.bare-load-baseline/canonified_loaded_scripts.log index 0a30ac0a71..05b7adcd11 100644 --- a/testing/btest/Baseline/coverage.bare-load-baseline/canonified_loaded_scripts.log +++ b/testing/btest/Baseline/coverage.bare-load-baseline/canonified_loaded_scripts.log @@ -3,7 +3,7 @@ #empty_field (empty) #unset_field - #path loaded_scripts -#open 2016-04-13-04-57-15 +#open 2016-05-02-20-39-26 #fields name #types string scripts/base/init-bare.bro @@ -17,6 +17,11 @@ scripts/base/init-bare.bro build/scripts/base/bif/event.bif.bro scripts/base/frameworks/broker/__load__.bro scripts/base/frameworks/broker/main.bro + build/scripts/base/bif/comm.bif.bro + build/scripts/base/bif/messaging.bif.bro + scripts/base/frameworks/broker/store.bro + build/scripts/base/bif/data.bif.bro + build/scripts/base/bif/store.bif.bro scripts/base/frameworks/logging/__load__.bro scripts/base/frameworks/logging/main.bro build/scripts/base/bif/logging.bif.bro @@ -45,15 +50,12 @@ scripts/base/init-bare.bro scripts/base/utils/patterns.bro scripts/base/frameworks/files/magic/__load__.bro build/scripts/base/bif/__load__.bro + build/scripts/base/bif/stats.bif.bro build/scripts/base/bif/broxygen.bif.bro build/scripts/base/bif/functions.bif.bro build/scripts/base/bif/bloom-filter.bif.bro build/scripts/base/bif/cardinality-counter.bif.bro build/scripts/base/bif/top-k.bif.bro - build/scripts/base/bif/comm.bif.bro - build/scripts/base/bif/data.bif.bro - build/scripts/base/bif/messaging.bif.bro - build/scripts/base/bif/store.bif.bro build/scripts/base/bif/plugins/__load__.bro build/scripts/base/bif/plugins/Bro_ARP.events.bif.bro build/scripts/base/bif/plugins/Bro_AYIYA.events.bif.bro @@ -75,6 +77,7 @@ scripts/base/init-bare.bro build/scripts/base/bif/plugins/Bro_HTTP.functions.bif.bro build/scripts/base/bif/plugins/Bro_ICMP.events.bif.bro build/scripts/base/bif/plugins/Bro_Ident.events.bif.bro + build/scripts/base/bif/plugins/Bro_IMAP.events.bif.bro build/scripts/base/bif/plugins/Bro_InterConn.events.bif.bro build/scripts/base/bif/plugins/Bro_IRC.events.bif.bro build/scripts/base/bif/plugins/Bro_KRB.events.bif.bro @@ -109,6 +112,7 @@ scripts/base/init-bare.bro build/scripts/base/bif/plugins/Bro_TCP.functions.bif.bro build/scripts/base/bif/plugins/Bro_Teredo.events.bif.bro build/scripts/base/bif/plugins/Bro_UDP.events.bif.bro + build/scripts/base/bif/plugins/Bro_XMPP.events.bif.bro build/scripts/base/bif/plugins/Bro_ZIP.events.bif.bro build/scripts/base/bif/plugins/Bro_FileEntropy.events.bif.bro build/scripts/base/bif/plugins/Bro_FileExtract.events.bif.bro @@ -130,4 +134,4 @@ scripts/base/init-bare.bro build/scripts/base/bif/plugins/Bro_SQLiteWriter.sqlite.bif.bro scripts/policy/misc/loaded-scripts.bro scripts/base/utils/paths.bro -#close 2016-04-13-04-57-15 +#close 2016-05-02-20-39-26 diff --git a/testing/btest/Baseline/coverage.default-load-baseline/canonified_loaded_scripts.log b/testing/btest/Baseline/coverage.default-load-baseline/canonified_loaded_scripts.log index 8b8dca3b12..d07c1727a0 100644 --- a/testing/btest/Baseline/coverage.default-load-baseline/canonified_loaded_scripts.log +++ b/testing/btest/Baseline/coverage.default-load-baseline/canonified_loaded_scripts.log @@ -3,7 +3,7 @@ #empty_field (empty) #unset_field - #path loaded_scripts -#open 2016-04-13-04-57-25 +#open 2016-04-29-20-49-25 #fields name #types string scripts/base/init-bare.bro @@ -17,6 +17,11 @@ scripts/base/init-bare.bro build/scripts/base/bif/event.bif.bro scripts/base/frameworks/broker/__load__.bro scripts/base/frameworks/broker/main.bro + build/scripts/base/bif/comm.bif.bro + build/scripts/base/bif/messaging.bif.bro + scripts/base/frameworks/broker/store.bro + build/scripts/base/bif/data.bif.bro + build/scripts/base/bif/store.bif.bro scripts/base/frameworks/logging/__load__.bro scripts/base/frameworks/logging/main.bro build/scripts/base/bif/logging.bif.bro @@ -45,15 +50,12 @@ scripts/base/init-bare.bro scripts/base/utils/patterns.bro scripts/base/frameworks/files/magic/__load__.bro build/scripts/base/bif/__load__.bro + build/scripts/base/bif/stats.bif.bro build/scripts/base/bif/broxygen.bif.bro build/scripts/base/bif/functions.bif.bro build/scripts/base/bif/bloom-filter.bif.bro build/scripts/base/bif/cardinality-counter.bif.bro build/scripts/base/bif/top-k.bif.bro - build/scripts/base/bif/comm.bif.bro - build/scripts/base/bif/data.bif.bro - build/scripts/base/bif/messaging.bif.bro - build/scripts/base/bif/store.bif.bro build/scripts/base/bif/plugins/__load__.bro build/scripts/base/bif/plugins/Bro_ARP.events.bif.bro build/scripts/base/bif/plugins/Bro_AYIYA.events.bif.bro @@ -75,6 +77,7 @@ scripts/base/init-bare.bro build/scripts/base/bif/plugins/Bro_HTTP.functions.bif.bro build/scripts/base/bif/plugins/Bro_ICMP.events.bif.bro build/scripts/base/bif/plugins/Bro_Ident.events.bif.bro + build/scripts/base/bif/plugins/Bro_IMAP.events.bif.bro build/scripts/base/bif/plugins/Bro_InterConn.events.bif.bro build/scripts/base/bif/plugins/Bro_IRC.events.bif.bro build/scripts/base/bif/plugins/Bro_KRB.events.bif.bro @@ -109,6 +112,7 @@ scripts/base/init-bare.bro build/scripts/base/bif/plugins/Bro_TCP.functions.bif.bro build/scripts/base/bif/plugins/Bro_Teredo.events.bif.bro build/scripts/base/bif/plugins/Bro_UDP.events.bif.bro + build/scripts/base/bif/plugins/Bro_XMPP.events.bif.bro build/scripts/base/bif/plugins/Bro_ZIP.events.bif.bro build/scripts/base/bif/plugins/Bro_FileEntropy.events.bif.bro build/scripts/base/bif/plugins/Bro_FileExtract.events.bif.bro @@ -251,6 +255,8 @@ scripts/base/init-default.bro scripts/base/protocols/http/entities.bro scripts/base/protocols/http/utils.bro scripts/base/protocols/http/files.bro + scripts/base/protocols/imap/__load__.bro + scripts/base/protocols/imap/main.bro scripts/base/protocols/irc/__load__.bro scripts/base/protocols/irc/main.bro scripts/base/protocols/irc/dcc-send.bro @@ -291,6 +297,8 @@ scripts/base/init-default.bro scripts/base/protocols/syslog/consts.bro scripts/base/protocols/syslog/main.bro scripts/base/protocols/tunnels/__load__.bro + scripts/base/protocols/xmpp/__load__.bro + scripts/base/protocols/xmpp/main.bro scripts/base/files/pe/__load__.bro scripts/base/files/pe/consts.bro scripts/base/files/pe/main.bro @@ -301,4 +309,4 @@ scripts/base/init-default.bro scripts/base/misc/find-checksum-offloading.bro scripts/base/misc/find-filtered-trace.bro scripts/policy/misc/loaded-scripts.bro -#close 2016-04-13-04-57-25 +#close 2016-05-02-20-39-35 diff --git a/testing/btest/Baseline/doc.sphinx.include-doc_frameworks_broker_connecting-connector_bro/output b/testing/btest/Baseline/doc.sphinx.include-doc_frameworks_broker_connecting-connector_bro/output index 042b8999f3..c4cbde045c 100644 --- a/testing/btest/Baseline/doc.sphinx.include-doc_frameworks_broker_connecting-connector_bro/output +++ b/testing/btest/Baseline/doc.sphinx.include-doc_frameworks_broker_connecting-connector_bro/output @@ -4,19 +4,19 @@ connecting-connector.bro const broker_port: port = 9999/tcp &redef; redef exit_only_after_terminate = T; -redef BrokerComm::endpoint_name = "connector"; +redef Broker::endpoint_name = "connector"; event bro_init() { - BrokerComm::enable(); - BrokerComm::connect("127.0.0.1", broker_port, 1sec); + Broker::enable(); + Broker::connect("127.0.0.1", broker_port, 1sec); } -event BrokerComm::outgoing_connection_established(peer_address: string, +event Broker::outgoing_connection_established(peer_address: string, peer_port: port, peer_name: string) { - print "BrokerComm::outgoing_connection_established", + print "Broker::outgoing_connection_established", peer_address, peer_port, peer_name; terminate(); } diff --git a/testing/btest/Baseline/doc.sphinx.include-doc_frameworks_broker_connecting-listener_bro/output b/testing/btest/Baseline/doc.sphinx.include-doc_frameworks_broker_connecting-listener_bro/output index 33e3df2330..8ea85569c9 100644 --- a/testing/btest/Baseline/doc.sphinx.include-doc_frameworks_broker_connecting-listener_bro/output +++ b/testing/btest/Baseline/doc.sphinx.include-doc_frameworks_broker_connecting-listener_bro/output @@ -4,21 +4,21 @@ connecting-listener.bro const broker_port: port = 9999/tcp &redef; redef exit_only_after_terminate = T; -redef BrokerComm::endpoint_name = "listener"; +redef Broker::endpoint_name = "listener"; event bro_init() { - BrokerComm::enable(); - BrokerComm::listen(broker_port, "127.0.0.1"); + Broker::enable(); + Broker::listen(broker_port, "127.0.0.1"); } -event BrokerComm::incoming_connection_established(peer_name: string) +event Broker::incoming_connection_established(peer_name: string) { - print "BrokerComm::incoming_connection_established", peer_name; + print "Broker::incoming_connection_established", peer_name; } -event BrokerComm::incoming_connection_broken(peer_name: string) +event Broker::incoming_connection_broken(peer_name: string) { - print "BrokerComm::incoming_connection_broken", peer_name; + print "Broker::incoming_connection_broken", peer_name; terminate(); } diff --git a/testing/btest/Baseline/doc.sphinx.include-doc_frameworks_broker_events-connector_bro/output b/testing/btest/Baseline/doc.sphinx.include-doc_frameworks_broker_events-connector_bro/output index fe97fdb4ce..d7a0e64be2 100644 --- a/testing/btest/Baseline/doc.sphinx.include-doc_frameworks_broker_events-connector_bro/output +++ b/testing/btest/Baseline/doc.sphinx.include-doc_frameworks_broker_events-connector_bro/output @@ -4,31 +4,31 @@ events-connector.bro const broker_port: port = 9999/tcp &redef; redef exit_only_after_terminate = T; -redef BrokerComm::endpoint_name = "connector"; +redef Broker::endpoint_name = "connector"; global my_event: event(msg: string, c: count); global my_auto_event: event(msg: string, c: count); event bro_init() { - BrokerComm::enable(); - BrokerComm::connect("127.0.0.1", broker_port, 1sec); - BrokerComm::auto_event("bro/event/my_auto_event", my_auto_event); + Broker::enable(); + Broker::connect("127.0.0.1", broker_port, 1sec); + Broker::auto_event("bro/event/my_auto_event", my_auto_event); } -event BrokerComm::outgoing_connection_established(peer_address: string, +event Broker::outgoing_connection_established(peer_address: string, peer_port: port, peer_name: string) { - print "BrokerComm::outgoing_connection_established", + print "Broker::outgoing_connection_established", peer_address, peer_port, peer_name; - BrokerComm::event("bro/event/my_event", BrokerComm::event_args(my_event, "hi", 0)); + Broker::send_event("bro/event/my_event", Broker::event_args(my_event, "hi", 0)); event my_auto_event("stuff", 88); - BrokerComm::event("bro/event/my_event", BrokerComm::event_args(my_event, "...", 1)); + Broker::send_event("bro/event/my_event", Broker::event_args(my_event, "...", 1)); event my_auto_event("more stuff", 51); - BrokerComm::event("bro/event/my_event", BrokerComm::event_args(my_event, "bye", 2)); + Broker::send_event("bro/event/my_event", Broker::event_args(my_event, "bye", 2)); } -event BrokerComm::outgoing_connection_broken(peer_address: string, +event Broker::outgoing_connection_broken(peer_address: string, peer_port: port) { terminate(); diff --git a/testing/btest/Baseline/doc.sphinx.include-doc_frameworks_broker_events-listener_bro/output b/testing/btest/Baseline/doc.sphinx.include-doc_frameworks_broker_events-listener_bro/output index 9f004692cb..640722cac0 100644 --- a/testing/btest/Baseline/doc.sphinx.include-doc_frameworks_broker_events-listener_bro/output +++ b/testing/btest/Baseline/doc.sphinx.include-doc_frameworks_broker_events-listener_bro/output @@ -4,21 +4,21 @@ events-listener.bro const broker_port: port = 9999/tcp &redef; redef exit_only_after_terminate = T; -redef BrokerComm::endpoint_name = "listener"; +redef Broker::endpoint_name = "listener"; global msg_count = 0; global my_event: event(msg: string, c: count); global my_auto_event: event(msg: string, c: count); event bro_init() { - BrokerComm::enable(); - BrokerComm::subscribe_to_events("bro/event/"); - BrokerComm::listen(broker_port, "127.0.0.1"); + Broker::enable(); + Broker::subscribe_to_events("bro/event/"); + Broker::listen(broker_port, "127.0.0.1"); } -event BrokerComm::incoming_connection_established(peer_name: string) +event Broker::incoming_connection_established(peer_name: string) { - print "BrokerComm::incoming_connection_established", peer_name; + print "Broker::incoming_connection_established", peer_name; } event my_event(msg: string, c: count) diff --git a/testing/btest/Baseline/doc.sphinx.include-doc_frameworks_broker_logs-connector_bro/output b/testing/btest/Baseline/doc.sphinx.include-doc_frameworks_broker_logs-connector_bro/output index 6884d5e4d6..907d712c88 100644 --- a/testing/btest/Baseline/doc.sphinx.include-doc_frameworks_broker_logs-connector_bro/output +++ b/testing/btest/Baseline/doc.sphinx.include-doc_frameworks_broker_logs-connector_bro/output @@ -6,16 +6,16 @@ logs-connector.bro const broker_port: port = 9999/tcp &redef; redef exit_only_after_terminate = T; -redef BrokerComm::endpoint_name = "connector"; +redef Broker::endpoint_name = "connector"; redef Log::enable_local_logging = F; redef Log::enable_remote_logging = F; global n = 0; event bro_init() { - BrokerComm::enable(); - BrokerComm::enable_remote_logs(Test::LOG); - BrokerComm::connect("127.0.0.1", broker_port, 1sec); + Broker::enable(); + Broker::enable_remote_logs(Test::LOG); + Broker::connect("127.0.0.1", broker_port, 1sec); } event do_write() @@ -28,16 +28,16 @@ event do_write() event do_write(); } -event BrokerComm::outgoing_connection_established(peer_address: string, +event Broker::outgoing_connection_established(peer_address: string, peer_port: port, peer_name: string) { - print "BrokerComm::outgoing_connection_established", + print "Broker::outgoing_connection_established", peer_address, peer_port, peer_name; event do_write(); } -event BrokerComm::outgoing_connection_broken(peer_address: string, +event Broker::outgoing_connection_broken(peer_address: string, peer_port: port) { terminate(); diff --git a/testing/btest/Baseline/doc.sphinx.include-doc_frameworks_broker_logs-listener_bro/output b/testing/btest/Baseline/doc.sphinx.include-doc_frameworks_broker_logs-listener_bro/output index 1610bde502..de6abbf5a0 100644 --- a/testing/btest/Baseline/doc.sphinx.include-doc_frameworks_broker_logs-listener_bro/output +++ b/testing/btest/Baseline/doc.sphinx.include-doc_frameworks_broker_logs-listener_bro/output @@ -6,18 +6,18 @@ logs-listener.bro const broker_port: port = 9999/tcp &redef; redef exit_only_after_terminate = T; -redef BrokerComm::endpoint_name = "listener"; +redef Broker::endpoint_name = "listener"; event bro_init() { - BrokerComm::enable(); - BrokerComm::subscribe_to_logs("bro/log/Test::LOG"); - BrokerComm::listen(broker_port, "127.0.0.1"); + Broker::enable(); + Broker::subscribe_to_logs("bro/log/Test::LOG"); + Broker::listen(broker_port, "127.0.0.1"); } -event BrokerComm::incoming_connection_established(peer_name: string) +event Broker::incoming_connection_established(peer_name: string) { - print "BrokerComm::incoming_connection_established", peer_name; + print "Broker::incoming_connection_established", peer_name; } event Test::log_test(rec: Test::Info) diff --git a/testing/btest/Baseline/doc.sphinx.include-doc_frameworks_broker_printing-connector_bro/output b/testing/btest/Baseline/doc.sphinx.include-doc_frameworks_broker_printing-connector_bro/output index 86ad4f459f..91ee179fe6 100644 --- a/testing/btest/Baseline/doc.sphinx.include-doc_frameworks_broker_printing-connector_bro/output +++ b/testing/btest/Baseline/doc.sphinx.include-doc_frameworks_broker_printing-connector_bro/output @@ -4,26 +4,26 @@ printing-connector.bro const broker_port: port = 9999/tcp &redef; redef exit_only_after_terminate = T; -redef BrokerComm::endpoint_name = "connector"; +redef Broker::endpoint_name = "connector"; event bro_init() { - BrokerComm::enable(); - BrokerComm::connect("127.0.0.1", broker_port, 1sec); + Broker::enable(); + Broker::connect("127.0.0.1", broker_port, 1sec); } -event BrokerComm::outgoing_connection_established(peer_address: string, +event Broker::outgoing_connection_established(peer_address: string, peer_port: port, peer_name: string) { - print "BrokerComm::outgoing_connection_established", + print "Broker::outgoing_connection_established", peer_address, peer_port, peer_name; - BrokerComm::print("bro/print/hi", "hello"); - BrokerComm::print("bro/print/stuff", "..."); - BrokerComm::print("bro/print/bye", "goodbye"); + Broker::send_print("bro/print/hi", "hello"); + Broker::send_print("bro/print/stuff", "..."); + Broker::send_print("bro/print/bye", "goodbye"); } -event BrokerComm::outgoing_connection_broken(peer_address: string, +event Broker::outgoing_connection_broken(peer_address: string, peer_port: port) { terminate(); diff --git a/testing/btest/Baseline/doc.sphinx.include-doc_frameworks_broker_printing-listener_bro/output b/testing/btest/Baseline/doc.sphinx.include-doc_frameworks_broker_printing-listener_bro/output index fb416612ab..37e4d0eae9 100644 --- a/testing/btest/Baseline/doc.sphinx.include-doc_frameworks_broker_printing-listener_bro/output +++ b/testing/btest/Baseline/doc.sphinx.include-doc_frameworks_broker_printing-listener_bro/output @@ -4,22 +4,22 @@ printing-listener.bro const broker_port: port = 9999/tcp &redef; redef exit_only_after_terminate = T; -redef BrokerComm::endpoint_name = "listener"; +redef Broker::endpoint_name = "listener"; global msg_count = 0; event bro_init() { - BrokerComm::enable(); - BrokerComm::subscribe_to_prints("bro/print/"); - BrokerComm::listen(broker_port, "127.0.0.1"); + Broker::enable(); + Broker::subscribe_to_prints("bro/print/"); + Broker::listen(broker_port, "127.0.0.1"); } -event BrokerComm::incoming_connection_established(peer_name: string) +event Broker::incoming_connection_established(peer_name: string) { - print "BrokerComm::incoming_connection_established", peer_name; + print "Broker::incoming_connection_established", peer_name; } -event BrokerComm::print_handler(msg: string) +event Broker::print_handler(msg: string) { ++msg_count; print "got print message", msg; diff --git a/testing/btest/Baseline/doc.sphinx.include-doc_frameworks_broker_stores-connector_bro/output b/testing/btest/Baseline/doc.sphinx.include-doc_frameworks_broker_stores-connector_bro/output index 6ca9e3b49b..74b59467e7 100644 --- a/testing/btest/Baseline/doc.sphinx.include-doc_frameworks_broker_stores-connector_bro/output +++ b/testing/btest/Baseline/doc.sphinx.include-doc_frameworks_broker_stores-connector_bro/output @@ -5,42 +5,42 @@ stores-connector.bro const broker_port: port = 9999/tcp &redef; redef exit_only_after_terminate = T; -global h: opaque of BrokerStore::Handle; +global h: opaque of Broker::Handle; -function dv(d: BrokerComm::Data): BrokerComm::DataVector +function dv(d: Broker::Data): Broker::DataVector { - local rval: BrokerComm::DataVector; + local rval: Broker::DataVector; rval[0] = d; return rval; } global ready: event(); -event BrokerComm::outgoing_connection_broken(peer_address: string, +event Broker::outgoing_connection_broken(peer_address: string, peer_port: port) { terminate(); } -event BrokerComm::outgoing_connection_established(peer_address: string, +event Broker::outgoing_connection_established(peer_address: string, peer_port: port, peer_name: string) { local myset: set[string] = {"a", "b", "c"}; local myvec: vector of string = {"alpha", "beta", "gamma"}; - h = BrokerStore::create_master("mystore"); - BrokerStore::insert(h, BrokerComm::data("one"), BrokerComm::data(110)); - BrokerStore::insert(h, BrokerComm::data("two"), BrokerComm::data(223)); - BrokerStore::insert(h, BrokerComm::data("myset"), BrokerComm::data(myset)); - BrokerStore::insert(h, BrokerComm::data("myvec"), BrokerComm::data(myvec)); - BrokerStore::increment(h, BrokerComm::data("one")); - BrokerStore::decrement(h, BrokerComm::data("two")); - BrokerStore::add_to_set(h, BrokerComm::data("myset"), BrokerComm::data("d")); - BrokerStore::remove_from_set(h, BrokerComm::data("myset"), BrokerComm::data("b")); - BrokerStore::push_left(h, BrokerComm::data("myvec"), dv(BrokerComm::data("delta"))); - BrokerStore::push_right(h, BrokerComm::data("myvec"), dv(BrokerComm::data("omega"))); + h = Broker::create_master("mystore"); + Broker::insert(h, Broker::data("one"), Broker::data(110)); + Broker::insert(h, Broker::data("two"), Broker::data(223)); + Broker::insert(h, Broker::data("myset"), Broker::data(myset)); + Broker::insert(h, Broker::data("myvec"), Broker::data(myvec)); + Broker::increment(h, Broker::data("one")); + Broker::decrement(h, Broker::data("two")); + Broker::add_to_set(h, Broker::data("myset"), Broker::data("d")); + Broker::remove_from_set(h, Broker::data("myset"), Broker::data("b")); + Broker::push_left(h, Broker::data("myvec"), dv(Broker::data("delta"))); + Broker::push_right(h, Broker::data("myvec"), dv(Broker::data("omega"))); - when ( local res = BrokerStore::size(h) ) + when ( local res = Broker::size(h) ) { print "master size", res; event ready(); @@ -51,7 +51,7 @@ event BrokerComm::outgoing_connection_established(peer_address: string, event bro_init() { - BrokerComm::enable(); - BrokerComm::connect("127.0.0.1", broker_port, 1secs); - BrokerComm::auto_event("bro/event/ready", ready); + Broker::enable(); + Broker::connect("127.0.0.1", broker_port, 1secs); + Broker::auto_event("bro/event/ready", ready); } diff --git a/testing/btest/Baseline/doc.sphinx.include-doc_frameworks_broker_stores-listener_bro/output b/testing/btest/Baseline/doc.sphinx.include-doc_frameworks_broker_stores-listener_bro/output index 6942ec17d2..8dadbc803c 100644 --- a/testing/btest/Baseline/doc.sphinx.include-doc_frameworks_broker_stores-listener_bro/output +++ b/testing/btest/Baseline/doc.sphinx.include-doc_frameworks_broker_stores-listener_bro/output @@ -5,13 +5,13 @@ stores-listener.bro const broker_port: port = 9999/tcp &redef; redef exit_only_after_terminate = T; -global h: opaque of BrokerStore::Handle; +global h: opaque of Broker::Handle; global expected_key_count = 4; global key_count = 0; function do_lookup(key: string) { - when ( local res = BrokerStore::lookup(h, BrokerComm::data(key)) ) + when ( local res = Broker::lookup(h, Broker::data(key)) ) { ++key_count; print "lookup", key, res; @@ -25,15 +25,15 @@ function do_lookup(key: string) event ready() { - h = BrokerStore::create_clone("mystore"); + h = Broker::create_clone("mystore"); - when ( local res = BrokerStore::keys(h) ) + when ( local res = Broker::keys(h) ) { print "clone keys", res; - do_lookup(BrokerComm::refine_to_string(BrokerComm::vector_lookup(res$result, 0))); - do_lookup(BrokerComm::refine_to_string(BrokerComm::vector_lookup(res$result, 1))); - do_lookup(BrokerComm::refine_to_string(BrokerComm::vector_lookup(res$result, 2))); - do_lookup(BrokerComm::refine_to_string(BrokerComm::vector_lookup(res$result, 3))); + do_lookup(Broker::refine_to_string(Broker::vector_lookup(res$result, 0))); + do_lookup(Broker::refine_to_string(Broker::vector_lookup(res$result, 1))); + do_lookup(Broker::refine_to_string(Broker::vector_lookup(res$result, 2))); + do_lookup(Broker::refine_to_string(Broker::vector_lookup(res$result, 3))); } timeout 10sec { print "timeout"; } @@ -41,7 +41,7 @@ event ready() event bro_init() { - BrokerComm::enable(); - BrokerComm::subscribe_to_events("bro/event/ready"); - BrokerComm::listen(broker_port, "127.0.0.1"); + Broker::enable(); + Broker::subscribe_to_events("bro/event/ready"); + Broker::listen(broker_port, "127.0.0.1"); } diff --git a/testing/btest/Baseline/doc.sphinx.include-doc_frameworks_broker_testlog_bro/output b/testing/btest/Baseline/doc.sphinx.include-doc_frameworks_broker_testlog_bro/output index c87fc3cd6f..d5a92417dc 100644 --- a/testing/btest/Baseline/doc.sphinx.include-doc_frameworks_broker_testlog_bro/output +++ b/testing/btest/Baseline/doc.sphinx.include-doc_frameworks_broker_testlog_bro/output @@ -17,6 +17,6 @@ export { event bro_init() &priority=5 { - BrokerComm::enable(); + Broker::enable(); Log::create_stream(Test::LOG, [$columns=Test::Info, $ev=log_test, $path="test"]); } diff --git a/testing/btest/Baseline/doc.sphinx.include-scripts_base_protocols_conn_main_bro/output b/testing/btest/Baseline/doc.sphinx.include-doc_scripting_data_type_record_bro/output similarity index 97% rename from testing/btest/Baseline/doc.sphinx.include-scripts_base_protocols_conn_main_bro/output rename to testing/btest/Baseline/doc.sphinx.include-doc_scripting_data_type_record_bro/output index 83e9d5bea1..6d8760700a 100644 --- a/testing/btest/Baseline/doc.sphinx.include-scripts_base_protocols_conn_main_bro/output +++ b/testing/btest/Baseline/doc.sphinx.include-doc_scripting_data_type_record_bro/output @@ -1,6 +1,6 @@ # @TEST-EXEC: cat %INPUT >output && btest-diff output -main.bro +data_type_record.bro module Conn; diff --git a/testing/btest/Baseline/doc.sphinx.include-scripts_base_protocols_http_main_bro/output b/testing/btest/Baseline/doc.sphinx.include-doc_scripting_http_main_bro/output similarity index 93% rename from testing/btest/Baseline/doc.sphinx.include-scripts_base_protocols_http_main_bro/output rename to testing/btest/Baseline/doc.sphinx.include-doc_scripting_http_main_bro/output index e3f7a39429..9f49450799 100644 --- a/testing/btest/Baseline/doc.sphinx.include-scripts_base_protocols_http_main_bro/output +++ b/testing/btest/Baseline/doc.sphinx.include-doc_scripting_http_main_bro/output @@ -1,6 +1,6 @@ # @TEST-EXEC: cat %INPUT >output && btest-diff output -main.bro +http_main.bro module HTTP; diff --git a/testing/btest/Baseline/language.event-local-var/out b/testing/btest/Baseline/language.event-local-var/out new file mode 100644 index 0000000000..2802c45d69 --- /dev/null +++ b/testing/btest/Baseline/language.event-local-var/out @@ -0,0 +1 @@ +error in /home/jgras/devel/bro/testing/btest/.tmp/language.event-local-var/event-local-var.bro, line 15: local identifier "v" cannot be used to reference an event, at or near ")" diff --git a/testing/btest/Baseline/language.event/out b/testing/btest/Baseline/language.event/out index 41c3e0d717..14fa9c1e8a 100644 --- a/testing/btest/Baseline/language.event/out +++ b/testing/btest/Baseline/language.event/out @@ -1,6 +1,7 @@ event statement event part1 event part2 +assign event variable (6) schedule statement in bro_init schedule statement in global schedule statement another in bro_init diff --git a/testing/btest/Baseline/plugins.hooks/output b/testing/btest/Baseline/plugins.hooks/output index 5e771791ea..4535f8c366 100644 --- a/testing/btest/Baseline/plugins.hooks/output +++ b/testing/btest/Baseline/plugins.hooks/output @@ -25,6 +25,7 @@ 0.000000 MetaHookPost CallFunction(Analyzer::__register_for_port, , (Analyzer::ANALYZER_HTTP, 8080/tcp)) -> 0.000000 MetaHookPost CallFunction(Analyzer::__register_for_port, , (Analyzer::ANALYZER_HTTP, 81/tcp)) -> 0.000000 MetaHookPost CallFunction(Analyzer::__register_for_port, , (Analyzer::ANALYZER_HTTP, 8888/tcp)) -> +0.000000 MetaHookPost CallFunction(Analyzer::__register_for_port, , (Analyzer::ANALYZER_IMAP, 143/tcp)) -> 0.000000 MetaHookPost CallFunction(Analyzer::__register_for_port, , (Analyzer::ANALYZER_IRC, 6666/tcp)) -> 0.000000 MetaHookPost CallFunction(Analyzer::__register_for_port, , (Analyzer::ANALYZER_IRC, 6667/tcp)) -> 0.000000 MetaHookPost CallFunction(Analyzer::__register_for_port, , (Analyzer::ANALYZER_IRC, 6668/tcp)) -> @@ -56,6 +57,8 @@ 0.000000 MetaHookPost CallFunction(Analyzer::__register_for_port, , (Analyzer::ANALYZER_SSL, 995/tcp)) -> 0.000000 MetaHookPost CallFunction(Analyzer::__register_for_port, , (Analyzer::ANALYZER_SYSLOG, 514/udp)) -> 0.000000 MetaHookPost CallFunction(Analyzer::__register_for_port, , (Analyzer::ANALYZER_TEREDO, 3544/udp)) -> +0.000000 MetaHookPost CallFunction(Analyzer::__register_for_port, , (Analyzer::ANALYZER_XMPP, 5222/tcp)) -> +0.000000 MetaHookPost CallFunction(Analyzer::__register_for_port, , (Analyzer::ANALYZER_XMPP, 5269/tcp)) -> 0.000000 MetaHookPost CallFunction(Analyzer::disable_analyzer, , (Analyzer::ANALYZER_BACKDOOR)) -> 0.000000 MetaHookPost CallFunction(Analyzer::disable_analyzer, , (Analyzer::ANALYZER_INTERCONN)) -> 0.000000 MetaHookPost CallFunction(Analyzer::disable_analyzer, , (Analyzer::ANALYZER_STEPPINGSTONE)) -> @@ -83,6 +86,7 @@ 0.000000 MetaHookPost CallFunction(Analyzer::register_for_port, , (Analyzer::ANALYZER_HTTP, 8080/tcp)) -> 0.000000 MetaHookPost CallFunction(Analyzer::register_for_port, , (Analyzer::ANALYZER_HTTP, 81/tcp)) -> 0.000000 MetaHookPost CallFunction(Analyzer::register_for_port, , (Analyzer::ANALYZER_HTTP, 8888/tcp)) -> +0.000000 MetaHookPost CallFunction(Analyzer::register_for_port, , (Analyzer::ANALYZER_IMAP, 143/tcp)) -> 0.000000 MetaHookPost CallFunction(Analyzer::register_for_port, , (Analyzer::ANALYZER_IRC, 6666/tcp)) -> 0.000000 MetaHookPost CallFunction(Analyzer::register_for_port, , (Analyzer::ANALYZER_IRC, 6667/tcp)) -> 0.000000 MetaHookPost CallFunction(Analyzer::register_for_port, , (Analyzer::ANALYZER_IRC, 6668/tcp)) -> @@ -114,6 +118,8 @@ 0.000000 MetaHookPost CallFunction(Analyzer::register_for_port, , (Analyzer::ANALYZER_SSL, 995/tcp)) -> 0.000000 MetaHookPost CallFunction(Analyzer::register_for_port, , (Analyzer::ANALYZER_SYSLOG, 514/udp)) -> 0.000000 MetaHookPost CallFunction(Analyzer::register_for_port, , (Analyzer::ANALYZER_TEREDO, 3544/udp)) -> +0.000000 MetaHookPost CallFunction(Analyzer::register_for_port, , (Analyzer::ANALYZER_XMPP, 5222/tcp)) -> +0.000000 MetaHookPost CallFunction(Analyzer::register_for_port, , (Analyzer::ANALYZER_XMPP, 5269/tcp)) -> 0.000000 MetaHookPost CallFunction(Analyzer::register_for_ports, , (Analyzer::ANALYZER_AYIYA, {5072/udp})) -> 0.000000 MetaHookPost CallFunction(Analyzer::register_for_ports, , (Analyzer::ANALYZER_DHCP, {67<...>/udp})) -> 0.000000 MetaHookPost CallFunction(Analyzer::register_for_ports, , (Analyzer::ANALYZER_DNP3_TCP, {20000<...>/tcp})) -> @@ -122,6 +128,7 @@ 0.000000 MetaHookPost CallFunction(Analyzer::register_for_ports, , (Analyzer::ANALYZER_FTP, {2811<...>/tcp})) -> 0.000000 MetaHookPost CallFunction(Analyzer::register_for_ports, , (Analyzer::ANALYZER_GTPV1, {2152<...>/udp})) -> 0.000000 MetaHookPost CallFunction(Analyzer::register_for_ports, , (Analyzer::ANALYZER_HTTP, {631<...>/tcp})) -> +0.000000 MetaHookPost CallFunction(Analyzer::register_for_ports, , (Analyzer::ANALYZER_IMAP, {143/tcp})) -> 0.000000 MetaHookPost CallFunction(Analyzer::register_for_ports, , (Analyzer::ANALYZER_IRC, {6669<...>/tcp})) -> 0.000000 MetaHookPost CallFunction(Analyzer::register_for_ports, , (Analyzer::ANALYZER_KRB, {88/udp})) -> 0.000000 MetaHookPost CallFunction(Analyzer::register_for_ports, , (Analyzer::ANALYZER_KRB_TCP, {88/tcp})) -> @@ -137,6 +144,7 @@ 0.000000 MetaHookPost CallFunction(Analyzer::register_for_ports, , (Analyzer::ANALYZER_SSL, {5223<...>/tcp})) -> 0.000000 MetaHookPost CallFunction(Analyzer::register_for_ports, , (Analyzer::ANALYZER_SYSLOG, {514/udp})) -> 0.000000 MetaHookPost CallFunction(Analyzer::register_for_ports, , (Analyzer::ANALYZER_TEREDO, {3544/udp})) -> +0.000000 MetaHookPost CallFunction(Analyzer::register_for_ports, , (Analyzer::ANALYZER_XMPP, {5222<...>/tcp})) -> 0.000000 MetaHookPost CallFunction(Cluster::is_enabled, , ()) -> 0.000000 MetaHookPost CallFunction(Cluster::is_enabled, , ()) -> 0.000000 MetaHookPost CallFunction(Files::register_analyzer_add_callback, , (Files::ANALYZER_EXTRACT, FileExtract::on_add{ if (!FileExtract::args?$extract_filename) FileExtract::args$extract_filename = cat(extract-, FileExtract::f$last_active, -, FileExtract::f$source, -, FileExtract::f$id)FileExtract::f$info$extracted = FileExtract::args$extract_filenameFileExtract::args$extract_filename = build_path_compressed(FileExtract::prefix, FileExtract::args$extract_filename)mkdir(FileExtract::prefix)})) -> @@ -230,7 +238,7 @@ 0.000000 MetaHookPost CallFunction(Log::__create_stream, , (Weird::LOG, [columns=, ev=Weird::log_weird, path=weird])) -> 0.000000 MetaHookPost CallFunction(Log::__create_stream, , (X509::LOG, [columns=, ev=X509::log_x509, path=x509])) -> 0.000000 MetaHookPost CallFunction(Log::__create_stream, , (mysql::LOG, [columns=, ev=MySQL::log_mysql, path=mysql])) -> -0.000000 MetaHookPost CallFunction(Log::__write, , (PacketFilter::LOG, [ts=1460523470.220624, node=bro, filter=ip or not ip, init=T, success=T])) -> +0.000000 MetaHookPost CallFunction(Log::__write, , (PacketFilter::LOG, [ts=1462646849.582646, node=bro, filter=ip or not ip, init=T, success=T])) -> 0.000000 MetaHookPost CallFunction(Log::add_default_filter, , (Cluster::LOG)) -> 0.000000 MetaHookPost CallFunction(Log::add_default_filter, , (Communication::LOG)) -> 0.000000 MetaHookPost CallFunction(Log::add_default_filter, , (Conn::LOG)) -> @@ -351,7 +359,7 @@ 0.000000 MetaHookPost CallFunction(Log::create_stream, , (Weird::LOG, [columns=, ev=Weird::log_weird, path=weird])) -> 0.000000 MetaHookPost CallFunction(Log::create_stream, , (X509::LOG, [columns=, ev=X509::log_x509, path=x509])) -> 0.000000 MetaHookPost CallFunction(Log::create_stream, , (mysql::LOG, [columns=, ev=MySQL::log_mysql, path=mysql])) -> -0.000000 MetaHookPost CallFunction(Log::write, , (PacketFilter::LOG, [ts=1460523470.220624, node=bro, filter=ip or not ip, init=T, success=T])) -> +0.000000 MetaHookPost CallFunction(Log::write, , (PacketFilter::LOG, [ts=1462646849.582646, node=bro, filter=ip or not ip, init=T, success=T])) -> 0.000000 MetaHookPost CallFunction(NetControl::check_plugins, , ()) -> 0.000000 MetaHookPost CallFunction(NetControl::init, , ()) -> 0.000000 MetaHookPost CallFunction(Notice::want_pp, , ()) -> @@ -416,6 +424,7 @@ 0.000000 MetaHookPost LoadFile(./Bro_HTTP.events.bif.bro) -> -1 0.000000 MetaHookPost LoadFile(./Bro_HTTP.functions.bif.bro) -> -1 0.000000 MetaHookPost LoadFile(./Bro_ICMP.events.bif.bro) -> -1 +0.000000 MetaHookPost LoadFile(./Bro_IMAP.events.bif.bro) -> -1 0.000000 MetaHookPost LoadFile(./Bro_IRC.events.bif.bro) -> -1 0.000000 MetaHookPost LoadFile(./Bro_Ident.events.bif.bro) -> -1 0.000000 MetaHookPost LoadFile(./Bro_InterConn.events.bif.bro) -> -1 @@ -463,6 +472,7 @@ 0.000000 MetaHookPost LoadFile(./Bro_X509.events.bif.bro) -> -1 0.000000 MetaHookPost LoadFile(./Bro_X509.functions.bif.bro) -> -1 0.000000 MetaHookPost LoadFile(./Bro_X509.types.bif.bro) -> -1 +0.000000 MetaHookPost LoadFile(./Bro_XMPP.events.bif.bro) -> -1 0.000000 MetaHookPost LoadFile(./Bro_ZIP.events.bif.bro) -> -1 0.000000 MetaHookPost LoadFile(./acld) -> -1 0.000000 MetaHookPost LoadFile(./addrs) -> -1 @@ -523,7 +533,9 @@ 0.000000 MetaHookPost LoadFile(./sftp) -> -1 0.000000 MetaHookPost LoadFile(./shunt) -> -1 0.000000 MetaHookPost LoadFile(./site) -> -1 +0.000000 MetaHookPost LoadFile(./stats.bif.bro) -> -1 0.000000 MetaHookPost LoadFile(./std-dev) -> -1 +0.000000 MetaHookPost LoadFile(./store) -> -1 0.000000 MetaHookPost LoadFile(./store.bif.bro) -> -1 0.000000 MetaHookPost LoadFile(./strings.bif.bro) -> -1 0.000000 MetaHookPost LoadFile(./sum) -> -1 @@ -565,11 +577,13 @@ 0.000000 MetaHookPost LoadFile(base<...>/bro.bif) -> -1 0.000000 MetaHookPost LoadFile(base<...>/broker) -> -1 0.000000 MetaHookPost LoadFile(base<...>/cluster) -> -1 +0.000000 MetaHookPost LoadFile(base<...>/comm.bif) -> -1 0.000000 MetaHookPost LoadFile(base<...>/communication) -> -1 0.000000 MetaHookPost LoadFile(base<...>/conn) -> -1 0.000000 MetaHookPost LoadFile(base<...>/conn-ids) -> -1 0.000000 MetaHookPost LoadFile(base<...>/const.bif.bro) -> -1 0.000000 MetaHookPost LoadFile(base<...>/control) -> -1 +0.000000 MetaHookPost LoadFile(base<...>/data.bif) -> -1 0.000000 MetaHookPost LoadFile(base<...>/dhcp) -> -1 0.000000 MetaHookPost LoadFile(base<...>/dir) -> -1 0.000000 MetaHookPost LoadFile(base<...>/directions-and-hosts) -> -1 @@ -586,6 +600,7 @@ 0.000000 MetaHookPost LoadFile(base<...>/ftp) -> -1 0.000000 MetaHookPost LoadFile(base<...>/hash) -> -1 0.000000 MetaHookPost LoadFile(base<...>/http) -> -1 +0.000000 MetaHookPost LoadFile(base<...>/imap) -> -1 0.000000 MetaHookPost LoadFile(base<...>/input) -> -1 0.000000 MetaHookPost LoadFile(base<...>/input.bif) -> -1 0.000000 MetaHookPost LoadFile(base<...>/intel) -> -1 @@ -595,6 +610,7 @@ 0.000000 MetaHookPost LoadFile(base<...>/logging) -> -1 0.000000 MetaHookPost LoadFile(base<...>/logging.bif) -> -1 0.000000 MetaHookPost LoadFile(base<...>/main) -> -1 +0.000000 MetaHookPost LoadFile(base<...>/messaging.bif) -> -1 0.000000 MetaHookPost LoadFile(base<...>/modbus) -> -1 0.000000 MetaHookPost LoadFile(base<...>/mysql) -> -1 0.000000 MetaHookPost LoadFile(base<...>/netcontrol) -> -1 @@ -622,6 +638,7 @@ 0.000000 MetaHookPost LoadFile(base<...>/software) -> -1 0.000000 MetaHookPost LoadFile(base<...>/ssh) -> -1 0.000000 MetaHookPost LoadFile(base<...>/ssl) -> -1 +0.000000 MetaHookPost LoadFile(base<...>/store.bif) -> -1 0.000000 MetaHookPost LoadFile(base<...>/strings) -> -1 0.000000 MetaHookPost LoadFile(base<...>/strings.bif) -> -1 0.000000 MetaHookPost LoadFile(base<...>/sumstats) -> -1 @@ -634,6 +651,7 @@ 0.000000 MetaHookPost LoadFile(base<...>/urls) -> -1 0.000000 MetaHookPost LoadFile(base<...>/utils) -> -1 0.000000 MetaHookPost LoadFile(base<...>/x509) -> -1 +0.000000 MetaHookPost LoadFile(base<...>/xmpp) -> -1 0.000000 MetaHookPost QueueEvent(NetControl::init()) -> false 0.000000 MetaHookPost QueueEvent(bro_init()) -> false 0.000000 MetaHookPost QueueEvent(filter_change_tracking()) -> false @@ -664,6 +682,7 @@ 0.000000 MetaHookPre CallFunction(Analyzer::__register_for_port, , (Analyzer::ANALYZER_HTTP, 8080/tcp)) 0.000000 MetaHookPre CallFunction(Analyzer::__register_for_port, , (Analyzer::ANALYZER_HTTP, 81/tcp)) 0.000000 MetaHookPre CallFunction(Analyzer::__register_for_port, , (Analyzer::ANALYZER_HTTP, 8888/tcp)) +0.000000 MetaHookPre CallFunction(Analyzer::__register_for_port, , (Analyzer::ANALYZER_IMAP, 143/tcp)) 0.000000 MetaHookPre CallFunction(Analyzer::__register_for_port, , (Analyzer::ANALYZER_IRC, 6666/tcp)) 0.000000 MetaHookPre CallFunction(Analyzer::__register_for_port, , (Analyzer::ANALYZER_IRC, 6667/tcp)) 0.000000 MetaHookPre CallFunction(Analyzer::__register_for_port, , (Analyzer::ANALYZER_IRC, 6668/tcp)) @@ -695,6 +714,8 @@ 0.000000 MetaHookPre CallFunction(Analyzer::__register_for_port, , (Analyzer::ANALYZER_SSL, 995/tcp)) 0.000000 MetaHookPre CallFunction(Analyzer::__register_for_port, , (Analyzer::ANALYZER_SYSLOG, 514/udp)) 0.000000 MetaHookPre CallFunction(Analyzer::__register_for_port, , (Analyzer::ANALYZER_TEREDO, 3544/udp)) +0.000000 MetaHookPre CallFunction(Analyzer::__register_for_port, , (Analyzer::ANALYZER_XMPP, 5222/tcp)) +0.000000 MetaHookPre CallFunction(Analyzer::__register_for_port, , (Analyzer::ANALYZER_XMPP, 5269/tcp)) 0.000000 MetaHookPre CallFunction(Analyzer::disable_analyzer, , (Analyzer::ANALYZER_BACKDOOR)) 0.000000 MetaHookPre CallFunction(Analyzer::disable_analyzer, , (Analyzer::ANALYZER_INTERCONN)) 0.000000 MetaHookPre CallFunction(Analyzer::disable_analyzer, , (Analyzer::ANALYZER_STEPPINGSTONE)) @@ -722,6 +743,7 @@ 0.000000 MetaHookPre CallFunction(Analyzer::register_for_port, , (Analyzer::ANALYZER_HTTP, 8080/tcp)) 0.000000 MetaHookPre CallFunction(Analyzer::register_for_port, , (Analyzer::ANALYZER_HTTP, 81/tcp)) 0.000000 MetaHookPre CallFunction(Analyzer::register_for_port, , (Analyzer::ANALYZER_HTTP, 8888/tcp)) +0.000000 MetaHookPre CallFunction(Analyzer::register_for_port, , (Analyzer::ANALYZER_IMAP, 143/tcp)) 0.000000 MetaHookPre CallFunction(Analyzer::register_for_port, , (Analyzer::ANALYZER_IRC, 6666/tcp)) 0.000000 MetaHookPre CallFunction(Analyzer::register_for_port, , (Analyzer::ANALYZER_IRC, 6667/tcp)) 0.000000 MetaHookPre CallFunction(Analyzer::register_for_port, , (Analyzer::ANALYZER_IRC, 6668/tcp)) @@ -753,6 +775,8 @@ 0.000000 MetaHookPre CallFunction(Analyzer::register_for_port, , (Analyzer::ANALYZER_SSL, 995/tcp)) 0.000000 MetaHookPre CallFunction(Analyzer::register_for_port, , (Analyzer::ANALYZER_SYSLOG, 514/udp)) 0.000000 MetaHookPre CallFunction(Analyzer::register_for_port, , (Analyzer::ANALYZER_TEREDO, 3544/udp)) +0.000000 MetaHookPre CallFunction(Analyzer::register_for_port, , (Analyzer::ANALYZER_XMPP, 5222/tcp)) +0.000000 MetaHookPre CallFunction(Analyzer::register_for_port, , (Analyzer::ANALYZER_XMPP, 5269/tcp)) 0.000000 MetaHookPre CallFunction(Analyzer::register_for_ports, , (Analyzer::ANALYZER_AYIYA, {5072/udp})) 0.000000 MetaHookPre CallFunction(Analyzer::register_for_ports, , (Analyzer::ANALYZER_DHCP, {67<...>/udp})) 0.000000 MetaHookPre CallFunction(Analyzer::register_for_ports, , (Analyzer::ANALYZER_DNP3_TCP, {20000<...>/tcp})) @@ -761,6 +785,7 @@ 0.000000 MetaHookPre CallFunction(Analyzer::register_for_ports, , (Analyzer::ANALYZER_FTP, {2811<...>/tcp})) 0.000000 MetaHookPre CallFunction(Analyzer::register_for_ports, , (Analyzer::ANALYZER_GTPV1, {2152<...>/udp})) 0.000000 MetaHookPre CallFunction(Analyzer::register_for_ports, , (Analyzer::ANALYZER_HTTP, {631<...>/tcp})) +0.000000 MetaHookPre CallFunction(Analyzer::register_for_ports, , (Analyzer::ANALYZER_IMAP, {143/tcp})) 0.000000 MetaHookPre CallFunction(Analyzer::register_for_ports, , (Analyzer::ANALYZER_IRC, {6669<...>/tcp})) 0.000000 MetaHookPre CallFunction(Analyzer::register_for_ports, , (Analyzer::ANALYZER_KRB, {88/udp})) 0.000000 MetaHookPre CallFunction(Analyzer::register_for_ports, , (Analyzer::ANALYZER_KRB_TCP, {88/tcp})) @@ -776,6 +801,7 @@ 0.000000 MetaHookPre CallFunction(Analyzer::register_for_ports, , (Analyzer::ANALYZER_SSL, {5223<...>/tcp})) 0.000000 MetaHookPre CallFunction(Analyzer::register_for_ports, , (Analyzer::ANALYZER_SYSLOG, {514/udp})) 0.000000 MetaHookPre CallFunction(Analyzer::register_for_ports, , (Analyzer::ANALYZER_TEREDO, {3544/udp})) +0.000000 MetaHookPre CallFunction(Analyzer::register_for_ports, , (Analyzer::ANALYZER_XMPP, {5222<...>/tcp})) 0.000000 MetaHookPre CallFunction(Cluster::is_enabled, , ()) 0.000000 MetaHookPre CallFunction(Cluster::is_enabled, , ()) 0.000000 MetaHookPre CallFunction(Files::register_analyzer_add_callback, , (Files::ANALYZER_EXTRACT, FileExtract::on_add{ if (!FileExtract::args?$extract_filename) FileExtract::args$extract_filename = cat(extract-, FileExtract::f$last_active, -, FileExtract::f$source, -, FileExtract::f$id)FileExtract::f$info$extracted = FileExtract::args$extract_filenameFileExtract::args$extract_filename = build_path_compressed(FileExtract::prefix, FileExtract::args$extract_filename)mkdir(FileExtract::prefix)})) @@ -869,7 +895,7 @@ 0.000000 MetaHookPre CallFunction(Log::__create_stream, , (Weird::LOG, [columns=, ev=Weird::log_weird, path=weird])) 0.000000 MetaHookPre CallFunction(Log::__create_stream, , (X509::LOG, [columns=, ev=X509::log_x509, path=x509])) 0.000000 MetaHookPre CallFunction(Log::__create_stream, , (mysql::LOG, [columns=, ev=MySQL::log_mysql, path=mysql])) -0.000000 MetaHookPre CallFunction(Log::__write, , (PacketFilter::LOG, [ts=1460523470.220624, node=bro, filter=ip or not ip, init=T, success=T])) +0.000000 MetaHookPre CallFunction(Log::__write, , (PacketFilter::LOG, [ts=1462646849.582646, node=bro, filter=ip or not ip, init=T, success=T])) 0.000000 MetaHookPre CallFunction(Log::add_default_filter, , (Cluster::LOG)) 0.000000 MetaHookPre CallFunction(Log::add_default_filter, , (Communication::LOG)) 0.000000 MetaHookPre CallFunction(Log::add_default_filter, , (Conn::LOG)) @@ -990,7 +1016,7 @@ 0.000000 MetaHookPre CallFunction(Log::create_stream, , (Weird::LOG, [columns=, ev=Weird::log_weird, path=weird])) 0.000000 MetaHookPre CallFunction(Log::create_stream, , (X509::LOG, [columns=, ev=X509::log_x509, path=x509])) 0.000000 MetaHookPre CallFunction(Log::create_stream, , (mysql::LOG, [columns=, ev=MySQL::log_mysql, path=mysql])) -0.000000 MetaHookPre CallFunction(Log::write, , (PacketFilter::LOG, [ts=1460523470.220624, node=bro, filter=ip or not ip, init=T, success=T])) +0.000000 MetaHookPre CallFunction(Log::write, , (PacketFilter::LOG, [ts=1462646849.582646, node=bro, filter=ip or not ip, init=T, success=T])) 0.000000 MetaHookPre CallFunction(NetControl::check_plugins, , ()) 0.000000 MetaHookPre CallFunction(NetControl::init, , ()) 0.000000 MetaHookPre CallFunction(Notice::want_pp, , ()) @@ -1055,6 +1081,7 @@ 0.000000 MetaHookPre LoadFile(./Bro_HTTP.events.bif.bro) 0.000000 MetaHookPre LoadFile(./Bro_HTTP.functions.bif.bro) 0.000000 MetaHookPre LoadFile(./Bro_ICMP.events.bif.bro) +0.000000 MetaHookPre LoadFile(./Bro_IMAP.events.bif.bro) 0.000000 MetaHookPre LoadFile(./Bro_IRC.events.bif.bro) 0.000000 MetaHookPre LoadFile(./Bro_Ident.events.bif.bro) 0.000000 MetaHookPre LoadFile(./Bro_InterConn.events.bif.bro) @@ -1102,6 +1129,7 @@ 0.000000 MetaHookPre LoadFile(./Bro_X509.events.bif.bro) 0.000000 MetaHookPre LoadFile(./Bro_X509.functions.bif.bro) 0.000000 MetaHookPre LoadFile(./Bro_X509.types.bif.bro) +0.000000 MetaHookPre LoadFile(./Bro_XMPP.events.bif.bro) 0.000000 MetaHookPre LoadFile(./Bro_ZIP.events.bif.bro) 0.000000 MetaHookPre LoadFile(./acld) 0.000000 MetaHookPre LoadFile(./addrs) @@ -1162,7 +1190,9 @@ 0.000000 MetaHookPre LoadFile(./sftp) 0.000000 MetaHookPre LoadFile(./shunt) 0.000000 MetaHookPre LoadFile(./site) +0.000000 MetaHookPre LoadFile(./stats.bif.bro) 0.000000 MetaHookPre LoadFile(./std-dev) +0.000000 MetaHookPre LoadFile(./store) 0.000000 MetaHookPre LoadFile(./store.bif.bro) 0.000000 MetaHookPre LoadFile(./strings.bif.bro) 0.000000 MetaHookPre LoadFile(./sum) @@ -1204,11 +1234,13 @@ 0.000000 MetaHookPre LoadFile(base<...>/bro.bif) 0.000000 MetaHookPre LoadFile(base<...>/broker) 0.000000 MetaHookPre LoadFile(base<...>/cluster) +0.000000 MetaHookPre LoadFile(base<...>/comm.bif) 0.000000 MetaHookPre LoadFile(base<...>/communication) 0.000000 MetaHookPre LoadFile(base<...>/conn) 0.000000 MetaHookPre LoadFile(base<...>/conn-ids) 0.000000 MetaHookPre LoadFile(base<...>/const.bif.bro) 0.000000 MetaHookPre LoadFile(base<...>/control) +0.000000 MetaHookPre LoadFile(base<...>/data.bif) 0.000000 MetaHookPre LoadFile(base<...>/dhcp) 0.000000 MetaHookPre LoadFile(base<...>/dir) 0.000000 MetaHookPre LoadFile(base<...>/directions-and-hosts) @@ -1225,6 +1257,7 @@ 0.000000 MetaHookPre LoadFile(base<...>/ftp) 0.000000 MetaHookPre LoadFile(base<...>/hash) 0.000000 MetaHookPre LoadFile(base<...>/http) +0.000000 MetaHookPre LoadFile(base<...>/imap) 0.000000 MetaHookPre LoadFile(base<...>/input) 0.000000 MetaHookPre LoadFile(base<...>/input.bif) 0.000000 MetaHookPre LoadFile(base<...>/intel) @@ -1234,6 +1267,7 @@ 0.000000 MetaHookPre LoadFile(base<...>/logging) 0.000000 MetaHookPre LoadFile(base<...>/logging.bif) 0.000000 MetaHookPre LoadFile(base<...>/main) +0.000000 MetaHookPre LoadFile(base<...>/messaging.bif) 0.000000 MetaHookPre LoadFile(base<...>/modbus) 0.000000 MetaHookPre LoadFile(base<...>/mysql) 0.000000 MetaHookPre LoadFile(base<...>/netcontrol) @@ -1261,6 +1295,7 @@ 0.000000 MetaHookPre LoadFile(base<...>/software) 0.000000 MetaHookPre LoadFile(base<...>/ssh) 0.000000 MetaHookPre LoadFile(base<...>/ssl) +0.000000 MetaHookPre LoadFile(base<...>/store.bif) 0.000000 MetaHookPre LoadFile(base<...>/strings) 0.000000 MetaHookPre LoadFile(base<...>/strings.bif) 0.000000 MetaHookPre LoadFile(base<...>/sumstats) @@ -1273,6 +1308,7 @@ 0.000000 MetaHookPre LoadFile(base<...>/urls) 0.000000 MetaHookPre LoadFile(base<...>/utils) 0.000000 MetaHookPre LoadFile(base<...>/x509) +0.000000 MetaHookPre LoadFile(base<...>/xmpp) 0.000000 MetaHookPre QueueEvent(NetControl::init()) 0.000000 MetaHookPre QueueEvent(bro_init()) 0.000000 MetaHookPre QueueEvent(filter_change_tracking()) @@ -1303,6 +1339,7 @@ 0.000000 | HookCallFunction Analyzer::__register_for_port(Analyzer::ANALYZER_HTTP, 8080/tcp) 0.000000 | HookCallFunction Analyzer::__register_for_port(Analyzer::ANALYZER_HTTP, 81/tcp) 0.000000 | HookCallFunction Analyzer::__register_for_port(Analyzer::ANALYZER_HTTP, 8888/tcp) +0.000000 | HookCallFunction Analyzer::__register_for_port(Analyzer::ANALYZER_IMAP, 143/tcp) 0.000000 | HookCallFunction Analyzer::__register_for_port(Analyzer::ANALYZER_IRC, 6666/tcp) 0.000000 | HookCallFunction Analyzer::__register_for_port(Analyzer::ANALYZER_IRC, 6667/tcp) 0.000000 | HookCallFunction Analyzer::__register_for_port(Analyzer::ANALYZER_IRC, 6668/tcp) @@ -1334,6 +1371,8 @@ 0.000000 | HookCallFunction Analyzer::__register_for_port(Analyzer::ANALYZER_SSL, 995/tcp) 0.000000 | HookCallFunction Analyzer::__register_for_port(Analyzer::ANALYZER_SYSLOG, 514/udp) 0.000000 | HookCallFunction Analyzer::__register_for_port(Analyzer::ANALYZER_TEREDO, 3544/udp) +0.000000 | HookCallFunction Analyzer::__register_for_port(Analyzer::ANALYZER_XMPP, 5222/tcp) +0.000000 | HookCallFunction Analyzer::__register_for_port(Analyzer::ANALYZER_XMPP, 5269/tcp) 0.000000 | HookCallFunction Analyzer::disable_analyzer(Analyzer::ANALYZER_BACKDOOR) 0.000000 | HookCallFunction Analyzer::disable_analyzer(Analyzer::ANALYZER_INTERCONN) 0.000000 | HookCallFunction Analyzer::disable_analyzer(Analyzer::ANALYZER_STEPPINGSTONE) @@ -1361,6 +1400,7 @@ 0.000000 | HookCallFunction Analyzer::register_for_port(Analyzer::ANALYZER_HTTP, 8080/tcp) 0.000000 | HookCallFunction Analyzer::register_for_port(Analyzer::ANALYZER_HTTP, 81/tcp) 0.000000 | HookCallFunction Analyzer::register_for_port(Analyzer::ANALYZER_HTTP, 8888/tcp) +0.000000 | HookCallFunction Analyzer::register_for_port(Analyzer::ANALYZER_IMAP, 143/tcp) 0.000000 | HookCallFunction Analyzer::register_for_port(Analyzer::ANALYZER_IRC, 6666/tcp) 0.000000 | HookCallFunction Analyzer::register_for_port(Analyzer::ANALYZER_IRC, 6667/tcp) 0.000000 | HookCallFunction Analyzer::register_for_port(Analyzer::ANALYZER_IRC, 6668/tcp) @@ -1392,6 +1432,8 @@ 0.000000 | HookCallFunction Analyzer::register_for_port(Analyzer::ANALYZER_SSL, 995/tcp) 0.000000 | HookCallFunction Analyzer::register_for_port(Analyzer::ANALYZER_SYSLOG, 514/udp) 0.000000 | HookCallFunction Analyzer::register_for_port(Analyzer::ANALYZER_TEREDO, 3544/udp) +0.000000 | HookCallFunction Analyzer::register_for_port(Analyzer::ANALYZER_XMPP, 5222/tcp) +0.000000 | HookCallFunction Analyzer::register_for_port(Analyzer::ANALYZER_XMPP, 5269/tcp) 0.000000 | HookCallFunction Analyzer::register_for_ports(Analyzer::ANALYZER_AYIYA, {5072/udp}) 0.000000 | HookCallFunction Analyzer::register_for_ports(Analyzer::ANALYZER_DHCP, {67<...>/udp}) 0.000000 | HookCallFunction Analyzer::register_for_ports(Analyzer::ANALYZER_DNP3_TCP, {20000<...>/tcp}) @@ -1400,6 +1442,7 @@ 0.000000 | HookCallFunction Analyzer::register_for_ports(Analyzer::ANALYZER_FTP, {2811<...>/tcp}) 0.000000 | HookCallFunction Analyzer::register_for_ports(Analyzer::ANALYZER_GTPV1, {2152<...>/udp}) 0.000000 | HookCallFunction Analyzer::register_for_ports(Analyzer::ANALYZER_HTTP, {631<...>/tcp}) +0.000000 | HookCallFunction Analyzer::register_for_ports(Analyzer::ANALYZER_IMAP, {143/tcp}) 0.000000 | HookCallFunction Analyzer::register_for_ports(Analyzer::ANALYZER_IRC, {6669<...>/tcp}) 0.000000 | HookCallFunction Analyzer::register_for_ports(Analyzer::ANALYZER_KRB, {88/udp}) 0.000000 | HookCallFunction Analyzer::register_for_ports(Analyzer::ANALYZER_KRB_TCP, {88/tcp}) @@ -1415,6 +1458,7 @@ 0.000000 | HookCallFunction Analyzer::register_for_ports(Analyzer::ANALYZER_SSL, {5223<...>/tcp}) 0.000000 | HookCallFunction Analyzer::register_for_ports(Analyzer::ANALYZER_SYSLOG, {514/udp}) 0.000000 | HookCallFunction Analyzer::register_for_ports(Analyzer::ANALYZER_TEREDO, {3544/udp}) +0.000000 | HookCallFunction Analyzer::register_for_ports(Analyzer::ANALYZER_XMPP, {5222<...>/tcp}) 0.000000 | HookCallFunction Cluster::is_enabled() 0.000000 | HookCallFunction Files::register_analyzer_add_callback(Files::ANALYZER_EXTRACT, FileExtract::on_add{ if (!FileExtract::args?$extract_filename) FileExtract::args$extract_filename = cat(extract-, FileExtract::f$last_active, -, FileExtract::f$source, -, FileExtract::f$id)FileExtract::f$info$extracted = FileExtract::args$extract_filenameFileExtract::args$extract_filename = build_path_compressed(FileExtract::prefix, FileExtract::args$extract_filename)mkdir(FileExtract::prefix)}) 0.000000 | HookCallFunction Files::register_for_mime_type(Files::ANALYZER_PE, application/x-dosexec) @@ -1507,7 +1551,7 @@ 0.000000 | HookCallFunction Log::__create_stream(Weird::LOG, [columns=, ev=Weird::log_weird, path=weird]) 0.000000 | HookCallFunction Log::__create_stream(X509::LOG, [columns=, ev=X509::log_x509, path=x509]) 0.000000 | HookCallFunction Log::__create_stream(mysql::LOG, [columns=, ev=MySQL::log_mysql, path=mysql]) -0.000000 | HookCallFunction Log::__write(PacketFilter::LOG, [ts=1460523470.220624, node=bro, filter=ip or not ip, init=T, success=T]) +0.000000 | HookCallFunction Log::__write(PacketFilter::LOG, [ts=1462646849.582646, node=bro, filter=ip or not ip, init=T, success=T]) 0.000000 | HookCallFunction Log::add_default_filter(Cluster::LOG) 0.000000 | HookCallFunction Log::add_default_filter(Communication::LOG) 0.000000 | HookCallFunction Log::add_default_filter(Conn::LOG) @@ -1628,7 +1672,7 @@ 0.000000 | HookCallFunction Log::create_stream(Weird::LOG, [columns=, ev=Weird::log_weird, path=weird]) 0.000000 | HookCallFunction Log::create_stream(X509::LOG, [columns=, ev=X509::log_x509, path=x509]) 0.000000 | HookCallFunction Log::create_stream(mysql::LOG, [columns=, ev=MySQL::log_mysql, path=mysql]) -0.000000 | HookCallFunction Log::write(PacketFilter::LOG, [ts=1460523470.220624, node=bro, filter=ip or not ip, init=T, success=T]) +0.000000 | HookCallFunction Log::write(PacketFilter::LOG, [ts=1462646849.582646, node=bro, filter=ip or not ip, init=T, success=T]) 0.000000 | HookCallFunction NetControl::check_plugins() 0.000000 | HookCallFunction NetControl::init() 0.000000 | HookCallFunction Notice::want_pp() @@ -1676,7 +1720,7 @@ 1362692526.869344 MetaHookPost CallFunction(ChecksumOffloading::check, , ()) -> 1362692526.869344 MetaHookPost CallFunction(NetControl::check_conn, , (141.142.228.5)) -> 1362692526.869344 MetaHookPost CallFunction(filter_change_tracking, , ()) -> -1362692526.869344 MetaHookPost CallFunction(net_stats, , ()) -> +1362692526.869344 MetaHookPost CallFunction(get_net_stats, , ()) -> 1362692526.869344 MetaHookPost CallFunction(new_connection, , ([id=[orig_h=141.142.228.5, orig_p=59856<...>/tcp], orig=[size=0, state=0, num_pkts=0, num_bytes_ip=0, flow_label=0], resp=[size=0, state=0, num_pkts=0, num_bytes_ip=0, flow_label=0], start_time=1362692526.869344, duration=0.0, service={}, history=, uid=CXWv6p3arKYeMETxOg, tunnel=, vlan=, inner_vlan=, dpd=, conn=, extract_orig=F, extract_resp=F, thresholds=, dhcp=, dnp3=, dns=, dns_state=, ftp=, ftp_data_reuse=F, ssl=, http=, http_state=, irc=, krb=, modbus=, mysql=, radius=, rdp=, rfb=, sip=, sip_state=, snmp=, smtp=, smtp_state=, socks=, ssh=, syslog=])) -> 1362692526.869344 MetaHookPost DrainEvents() -> 1362692526.869344 MetaHookPost QueueEvent(ChecksumOffloading::check()) -> false @@ -1687,7 +1731,7 @@ 1362692526.869344 MetaHookPre CallFunction(ChecksumOffloading::check, , ()) 1362692526.869344 MetaHookPre CallFunction(NetControl::check_conn, , (141.142.228.5)) 1362692526.869344 MetaHookPre CallFunction(filter_change_tracking, , ()) -1362692526.869344 MetaHookPre CallFunction(net_stats, , ()) +1362692526.869344 MetaHookPre CallFunction(get_net_stats, , ()) 1362692526.869344 MetaHookPre CallFunction(new_connection, , ([id=[orig_h=141.142.228.5, orig_p=59856<...>/tcp], orig=[size=0, state=0, num_pkts=0, num_bytes_ip=0, flow_label=0], resp=[size=0, state=0, num_pkts=0, num_bytes_ip=0, flow_label=0], start_time=1362692526.869344, duration=0.0, service={}, history=, uid=CXWv6p3arKYeMETxOg, tunnel=, vlan=, inner_vlan=, dpd=, conn=, extract_orig=F, extract_resp=F, thresholds=, dhcp=, dnp3=, dns=, dns_state=, ftp=, ftp_data_reuse=F, ssl=, http=, http_state=, irc=, krb=, modbus=, mysql=, radius=, rdp=, rfb=, sip=, sip_state=, snmp=, smtp=, smtp_state=, socks=, ssh=, syslog=])) 1362692526.869344 MetaHookPre DrainEvents() 1362692526.869344 MetaHookPre QueueEvent(ChecksumOffloading::check()) @@ -1699,7 +1743,7 @@ 1362692526.869344 | HookCallFunction ChecksumOffloading::check() 1362692526.869344 | HookCallFunction NetControl::check_conn(141.142.228.5) 1362692526.869344 | HookCallFunction filter_change_tracking() -1362692526.869344 | HookCallFunction net_stats() +1362692526.869344 | HookCallFunction get_net_stats() 1362692526.869344 | HookCallFunction new_connection([id=[orig_h=141.142.228.5, orig_p=59856<...>/tcp], orig=[size=0, state=0, num_pkts=0, num_bytes_ip=0, flow_label=0], resp=[size=0, state=0, num_pkts=0, num_bytes_ip=0, flow_label=0], start_time=1362692526.869344, duration=0.0, service={}, history=, uid=CXWv6p3arKYeMETxOg, tunnel=, vlan=, inner_vlan=, dpd=, conn=, extract_orig=F, extract_resp=F, thresholds=, dhcp=, dnp3=, dns=, dns_state=, ftp=, ftp_data_reuse=F, ssl=, http=, http_state=, irc=, krb=, modbus=, mysql=, radius=, rdp=, rfb=, sip=, sip_state=, snmp=, smtp=, smtp_state=, socks=, ssh=, syslog=]) 1362692526.869344 | HookDrainEvents 1362692526.869344 | HookQueueEvent ChecksumOffloading::check() @@ -2104,11 +2148,11 @@ 1362692527.080972 MetaHookPost CallFunction(filter_change_tracking, , ()) -> 1362692527.080972 MetaHookPost CallFunction(fmt, , (%s:%d > %s:%d, 141.142.228.5, 59856<...>/tcp)) -> 1362692527.080972 MetaHookPost CallFunction(get_file_handle, , (Analyzer::ANALYZER_HTTP, [id=[orig_h=141.142.228.5, orig_p=59856<...>/plain], current_entity=, orig_mime_depth=1, resp_mime_depth=1], http_state=[pending={}, current_request=1, current_response=1, trans_depth=1], irc=, krb=, modbus=, mysql=, radius=, rdp=, rfb=, sip=, sip_state=, snmp=, smtp=, smtp_state=, socks=, ssh=, syslog=], T)) -> +1362692527.080972 MetaHookPost CallFunction(get_net_stats, , ()) -> 1362692527.080972 MetaHookPost CallFunction(get_port_transport_proto, , (80/tcp)) -> 1362692527.080972 MetaHookPost CallFunction(id_string, , ([orig_h=141.142.228.5, orig_p=59856<...>/tcp])) -> 1362692527.080972 MetaHookPost CallFunction(is_tcp_port, , (59856/tcp)) -> 1362692527.080972 MetaHookPost CallFunction(net_done, , (1362692527.080972)) -> -1362692527.080972 MetaHookPost CallFunction(net_stats, , ()) -> 1362692527.080972 MetaHookPost CallFunction(reading_traces, , ()) -> 1362692527.080972 MetaHookPost CallFunction(set_file_handle, , (Analyzer::ANALYZER_HTTP1362692526.869344T11141.142.228.5:59856 > 192.150.187.43:80)) -> 1362692527.080972 MetaHookPost CallFunction(sub_bytes, , (HTTP, 0, 1)) -> @@ -2134,11 +2178,11 @@ 1362692527.080972 MetaHookPre CallFunction(filter_change_tracking, , ()) 1362692527.080972 MetaHookPre CallFunction(fmt, , (%s:%d > %s:%d, 141.142.228.5, 59856<...>/tcp)) 1362692527.080972 MetaHookPre CallFunction(get_file_handle, , (Analyzer::ANALYZER_HTTP, [id=[orig_h=141.142.228.5, orig_p=59856<...>/plain], current_entity=, orig_mime_depth=1, resp_mime_depth=1], http_state=[pending={}, current_request=1, current_response=1, trans_depth=1], irc=, krb=, modbus=, mysql=, radius=, rdp=, rfb=, sip=, sip_state=, snmp=, smtp=, smtp_state=, socks=, ssh=, syslog=], T)) +1362692527.080972 MetaHookPre CallFunction(get_net_stats, , ()) 1362692527.080972 MetaHookPre CallFunction(get_port_transport_proto, , (80/tcp)) 1362692527.080972 MetaHookPre CallFunction(id_string, , ([orig_h=141.142.228.5, orig_p=59856<...>/tcp])) 1362692527.080972 MetaHookPre CallFunction(is_tcp_port, , (59856/tcp)) 1362692527.080972 MetaHookPre CallFunction(net_done, , (1362692527.080972)) -1362692527.080972 MetaHookPre CallFunction(net_stats, , ()) 1362692527.080972 MetaHookPre CallFunction(reading_traces, , ()) 1362692527.080972 MetaHookPre CallFunction(set_file_handle, , (Analyzer::ANALYZER_HTTP1362692526.869344T11141.142.228.5:59856 > 192.150.187.43:80)) 1362692527.080972 MetaHookPre CallFunction(sub_bytes, , (HTTP, 0, 1)) @@ -2165,11 +2209,11 @@ 1362692527.080972 | HookCallFunction filter_change_tracking() 1362692527.080972 | HookCallFunction fmt(%s:%d > %s:%d, 141.142.228.5, 59856<...>/tcp) 1362692527.080972 | HookCallFunction get_file_handle(Analyzer::ANALYZER_HTTP, [id=[orig_h=141.142.228.5, orig_p=59856<...>/plain], current_entity=, orig_mime_depth=1, resp_mime_depth=1], http_state=[pending={}, current_request=1, current_response=1, trans_depth=1], irc=, krb=, modbus=, mysql=, radius=, rdp=, rfb=, sip=, sip_state=, snmp=, smtp=, smtp_state=, socks=, ssh=, syslog=], T) +1362692527.080972 | HookCallFunction get_net_stats() 1362692527.080972 | HookCallFunction get_port_transport_proto(80/tcp) 1362692527.080972 | HookCallFunction id_string([orig_h=141.142.228.5, orig_p=59856<...>/tcp]) 1362692527.080972 | HookCallFunction is_tcp_port(59856/tcp) 1362692527.080972 | HookCallFunction net_done(1362692527.080972) -1362692527.080972 | HookCallFunction net_stats() 1362692527.080972 | HookCallFunction reading_traces() 1362692527.080972 | HookCallFunction set_file_handle(Analyzer::ANALYZER_HTTP1362692526.869344T11141.142.228.5:59856 > 192.150.187.43:80) 1362692527.080972 | HookCallFunction sub_bytes(HTTP, 0, 1) diff --git a/testing/btest/Baseline/scripts.base.files.x509.1999/x509.log b/testing/btest/Baseline/scripts.base.files.x509.1999/x509.log new file mode 100644 index 0000000000..60bd109b5d --- /dev/null +++ b/testing/btest/Baseline/scripts.base.files.x509.1999/x509.log @@ -0,0 +1,12 @@ +#separator \x09 +#set_separator , +#empty_field (empty) +#unset_field - +#path x509 +#open 2016-04-26-19-27-59 +#fields ts id certificate.version certificate.serial certificate.subject certificate.issuer certificate.not_valid_before certificate.not_valid_after certificate.key_alg certificate.sig_alg certificate.key_type certificate.key_length certificate.exponent certificate.curve san.dns san.uri san.email san.ip basic_constraints.ca basic_constraints.path_len +#types time string count string string string time time string string string count string string vector[string] vector[string] vector[string] vector[addr] bool count +1461697070.246986 Feyr3x4h8S7yqikqYd 3 339D9ED8E73927C9 CN=imap.gmx.net,emailAddress=server-certs@1und1.de,L=Montabaur,ST=Rhineland-Palatinate,O=1&1 Mail & Media GmbH,C=DE CN=TeleSec ServerPass DE-1,street=Untere Industriestr. 20,L=Netphen,postalCode=57250,ST=NRW,OU=T-Systems Trust Center,O=T-Systems International GmbH,C=DE 1384251451.000000 1479427199.000000 rsaEncryption sha1WithRSAEncryption rsa 2048 65537 - imap.gmx.net,imap.gmx.de - - - F - +1461697070.246986 FdSwvBrmfL9It607b 3 21B6777E8CBD0EA8 CN=TeleSec ServerPass DE-1,street=Untere Industriestr. 20,L=Netphen,postalCode=57250,ST=NRW,OU=T-Systems Trust Center,O=T-Systems International GmbH,C=DE CN=Deutsche Telekom Root CA 2,OU=T-TeleSec Trust Center,O=Deutsche Telekom AG,C=DE 1362146309.000000 1562716740.000000 rsaEncryption sha1WithRSAEncryption rsa 2048 65537 - - - - - T 0 +1461697070.246986 F7YtKFoAux1T0Ycb3 3 26 CN=Deutsche Telekom Root CA 2,OU=T-TeleSec Trust Center,O=Deutsche Telekom AG,C=DE CN=Deutsche Telekom Root CA 2,OU=T-TeleSec Trust Center,O=Deutsche Telekom AG,C=DE 931522260.000000 1562716740.000000 rsaEncryption sha1WithRSAEncryption rsa 2048 65537 - - - - - T 5 +#close 2016-04-26-19-27-59 diff --git a/testing/btest/Baseline/scripts.base.frameworks.netcontrol.acld-hook/recv.recv.out b/testing/btest/Baseline/scripts.base.frameworks.netcontrol.acld-hook/recv.recv.out index d36130b29b..d6d5c32fb2 100644 --- a/testing/btest/Baseline/scripts.base.frameworks.netcontrol.acld-hook/recv.recv.out +++ b/testing/btest/Baseline/scripts.base.frameworks.netcontrol.acld-hook/recv.recv.out @@ -1,4 +1,4 @@ -BrokerComm::incoming_connection_established +Broker::incoming_connection_established add_rule, 0, [ty=NetControl::FLOW, conn=, flow=[src_h=192.168.18.50/32, src_p=, dst_h=74.125.239.97/32, dst_p=, src_m=, dst_m=], ip=, mac=], NetControl::DROP, [command=blockhosthost, cookie=2, arg=192.168.18.50 74.125.239.97, comment=here] add_rule, 0, [ty=NetControl::FLOW, conn=, flow=[src_h=, src_p=, dst_h=, dst_p=443/tcp, src_m=, dst_m=], ip=, mac=], NetControl::DROP, [command=droptcpport, cookie=3, arg=443, comment=there] add_rule, 0, [ty=NetControl::ADDRESS, conn=, flow=, ip=192.168.18.50/32, mac=], NetControl::DROP, [command=nullzero, cookie=4, arg=192.168.18.50/32, comment=] diff --git a/testing/btest/Baseline/scripts.base.frameworks.netcontrol.acld-hook/send.send.out b/testing/btest/Baseline/scripts.base.frameworks.netcontrol.acld-hook/send.send.out index fd7f00bb7c..5d8cb431f4 100644 --- a/testing/btest/Baseline/scripts.base.frameworks.netcontrol.acld-hook/send.send.out +++ b/testing/btest/Baseline/scripts.base.frameworks.netcontrol.acld-hook/send.send.out @@ -1,4 +1,4 @@ -BrokerComm::outgoing_connection_established, 127.0.0.1, 9999/tcp +Broker::outgoing_connection_established, 127.0.0.1, 9999/tcp rule added, [ty=NetControl::FLOW, conn=, flow=[src_h=192.168.18.50/32, src_p=, dst_h=74.125.239.97/32, dst_p=, src_m=, dst_m=], ip=, mac=], NetControl::DROP rule added, [ty=NetControl::FLOW, conn=, flow=[src_h=, src_p=, dst_h=, dst_p=443/tcp, src_m=, dst_m=], ip=, mac=], NetControl::DROP rule added, [ty=NetControl::ADDRESS, conn=, flow=, ip=192.168.18.50/32, mac=], NetControl::DROP diff --git a/testing/btest/Baseline/scripts.base.frameworks.netcontrol.acld/recv.recv.out b/testing/btest/Baseline/scripts.base.frameworks.netcontrol.acld/recv.recv.out index 6890484529..f75f20ea28 100644 --- a/testing/btest/Baseline/scripts.base.frameworks.netcontrol.acld/recv.recv.out +++ b/testing/btest/Baseline/scripts.base.frameworks.netcontrol.acld/recv.recv.out @@ -1,4 +1,4 @@ -BrokerComm::incoming_connection_established +Broker::incoming_connection_established add_rule, 0, [ty=NetControl::FLOW, conn=, flow=[src_h=192.168.18.50/32, src_p=, dst_h=74.125.239.97/32, dst_p=, src_m=, dst_m=], ip=, mac=], NetControl::DROP, [command=blockhosthost, cookie=2, arg=192.168.18.50 74.125.239.97, comment=here] add_rule, 0, [ty=NetControl::FLOW, conn=, flow=[src_h=, src_p=, dst_h=, dst_p=443/tcp, src_m=, dst_m=], ip=, mac=], NetControl::DROP, [command=droptcpport, cookie=3, arg=443, comment=there] add_rule, 0, [ty=NetControl::ADDRESS, conn=, flow=, ip=192.168.18.50/32, mac=], NetControl::DROP, [command=drop, cookie=4, arg=192.168.18.50/32, comment=] diff --git a/testing/btest/Baseline/scripts.base.frameworks.netcontrol.acld/send.send.out b/testing/btest/Baseline/scripts.base.frameworks.netcontrol.acld/send.send.out index fd7f00bb7c..5d8cb431f4 100644 --- a/testing/btest/Baseline/scripts.base.frameworks.netcontrol.acld/send.send.out +++ b/testing/btest/Baseline/scripts.base.frameworks.netcontrol.acld/send.send.out @@ -1,4 +1,4 @@ -BrokerComm::outgoing_connection_established, 127.0.0.1, 9999/tcp +Broker::outgoing_connection_established, 127.0.0.1, 9999/tcp rule added, [ty=NetControl::FLOW, conn=, flow=[src_h=192.168.18.50/32, src_p=, dst_h=74.125.239.97/32, dst_p=, src_m=, dst_m=], ip=, mac=], NetControl::DROP rule added, [ty=NetControl::FLOW, conn=, flow=[src_h=, src_p=, dst_h=, dst_p=443/tcp, src_m=, dst_m=], ip=, mac=], NetControl::DROP rule added, [ty=NetControl::ADDRESS, conn=, flow=, ip=192.168.18.50/32, mac=], NetControl::DROP diff --git a/testing/btest/Baseline/scripts.base.frameworks.netcontrol.broker/recv.recv.out b/testing/btest/Baseline/scripts.base.frameworks.netcontrol.broker/recv.recv.out index 3b02eef7c7..74c5f3499c 100644 --- a/testing/btest/Baseline/scripts.base.frameworks.netcontrol.broker/recv.recv.out +++ b/testing/btest/Baseline/scripts.base.frameworks.netcontrol.broker/recv.recv.out @@ -1,4 +1,4 @@ -BrokerComm::incoming_connection_established +Broker::incoming_connection_established add_rule, 0, [ty=NetControl::FLOW, conn=, flow=[src_h=10.10.1.4/32, src_p=1470/tcp, dst_h=74.53.140.153/32, dst_p=25/tcp, src_m=, dst_m=], ip=, mac=], NetControl::DROP add_rule, 0, [ty=NetControl::ADDRESS, conn=, flow=, ip=10.10.1.4/32, mac=], NetControl::DROP remove_rule, 0, [ty=NetControl::FLOW, conn=, flow=[src_h=10.10.1.4/32, src_p=1470/tcp, dst_h=74.53.140.153/32, dst_p=25/tcp, src_m=, dst_m=], ip=, mac=], NetControl::DROP diff --git a/testing/btest/Baseline/scripts.base.frameworks.netcontrol.broker/send.send.out b/testing/btest/Baseline/scripts.base.frameworks.netcontrol.broker/send.send.out index 31d94be31e..fb086ee0e7 100644 --- a/testing/btest/Baseline/scripts.base.frameworks.netcontrol.broker/send.send.out +++ b/testing/btest/Baseline/scripts.base.frameworks.netcontrol.broker/send.send.out @@ -1,4 +1,4 @@ -BrokerComm::outgoing_connection_established, 127.0.0.1, 9999/tcp +Broker::outgoing_connection_established, 127.0.0.1, 9999/tcp rule added, [ty=NetControl::FLOW, conn=, flow=[src_h=10.10.1.4/32, src_p=1470/tcp, dst_h=74.53.140.153/32, dst_p=25/tcp, src_m=, dst_m=], ip=, mac=], NetControl::DROP rule added, [ty=NetControl::ADDRESS, conn=, flow=, ip=10.10.1.4/32, mac=], NetControl::DROP rule timeout, [ty=NetControl::FLOW, conn=, flow=[src_h=10.10.1.4/32, src_p=1470/tcp, dst_h=74.53.140.153/32, dst_p=25/tcp, src_m=, dst_m=], ip=, mac=], NetControl::DROP, [duration=, packet_count=, byte_count=] diff --git a/testing/btest/Baseline/scripts.base.frameworks.openflow.broker-basic/recv.recv.out b/testing/btest/Baseline/scripts.base.frameworks.openflow.broker-basic/recv.recv.out index ec3b038bd9..b1c2ed5050 100644 --- a/testing/btest/Baseline/scripts.base.frameworks.openflow.broker-basic/recv.recv.out +++ b/testing/btest/Baseline/scripts.base.frameworks.openflow.broker-basic/recv.recv.out @@ -1,4 +1,4 @@ -BrokerComm::incoming_connection_established +Broker::incoming_connection_established flow_clear, 42 got flow_mod, 42, [in_port=, dl_src=, dl_dst=, dl_vlan=, dl_vlan_pcp=, dl_type=, nw_tos=, nw_proto=, nw_src=, nw_dst=, tp_src=, tp_dst=], [cookie=4398046511105, table_id=, command=OpenFlow::OFPFC_ADD, idle_timeout=0, hard_timeout=0, priority=0, out_port=, out_group=, flags=0, actions=[out_ports=[3, 7], vlan_vid=, vlan_pcp=, vlan_strip=F, dl_src=, dl_dst=, nw_tos=, nw_src=, nw_dst=, tp_src=, tp_dst=]] got flow_mod, 42, [in_port=, dl_src=, dl_dst=, dl_vlan=, dl_vlan_pcp=, dl_type=2048, nw_tos=, nw_proto=6, nw_src=10.10.1.4/32, nw_dst=74.53.140.153/32, tp_src=1470, tp_dst=25], [cookie=4398046511146, table_id=, command=OpenFlow::OFPFC_ADD, idle_timeout=30, hard_timeout=0, priority=5, out_port=, out_group=, flags=0, actions=[out_ports=[], vlan_vid=, vlan_pcp=, vlan_strip=F, dl_src=, dl_dst=, nw_tos=, nw_src=, nw_dst=, tp_src=, tp_dst=]] diff --git a/testing/btest/Baseline/scripts.base.frameworks.openflow.broker-basic/send.send.out b/testing/btest/Baseline/scripts.base.frameworks.openflow.broker-basic/send.send.out index d81ed49aee..5f4fadfb81 100644 --- a/testing/btest/Baseline/scripts.base.frameworks.openflow.broker-basic/send.send.out +++ b/testing/btest/Baseline/scripts.base.frameworks.openflow.broker-basic/send.send.out @@ -1,4 +1,4 @@ -BrokerComm::outgoing_connection_established, 127.0.0.1, 9999/tcp +Broker::outgoing_connection_established, 127.0.0.1, 9999/tcp Flow_mod_success Flow_mod_failure connection established diff --git a/testing/btest/Baseline/scripts.base.protocols.arp.basic/.stdout b/testing/btest/Baseline/scripts.base.protocols.arp.basic/.stdout new file mode 100644 index 0000000000..d45f9ba0d7 --- /dev/null +++ b/testing/btest/Baseline/scripts.base.protocols.arp.basic/.stdout @@ -0,0 +1,2 @@ +78:31:c1:c6:3f:c2, ff:ff:ff:ff:ff:ff, 10.0.0.2, 78:31:c1:c6:3f:c2, 10.0.0.1, 00:00:00:00:00:00 +f8:ed:a5:c0:a4:f1, 78:31:c1:c6:3f:c2, 10.0.0.1, f8:ed:a5:c0:a4:f1, 10.0.0.2, 78:31:c1:c6:3f:c2 diff --git a/testing/btest/Baseline/scripts.base.protocols.conn.new_connection_contents/.stdout b/testing/btest/Baseline/scripts.base.protocols.conn.new_connection_contents/.stdout new file mode 100644 index 0000000000..1581730b33 --- /dev/null +++ b/testing/btest/Baseline/scripts.base.protocols.conn.new_connection_contents/.stdout @@ -0,0 +1,2 @@ +new_connection_contents for [orig_h=192.168.1.77, orig_p=57640/tcp, resp_h=66.198.80.67, resp_p=6667/tcp] +new_connection_contents for [orig_h=192.168.1.77, orig_p=57655/tcp, resp_h=209.197.168.151, resp_p=1024/tcp] diff --git a/testing/btest/Baseline/scripts.base.protocols.dns.caa/.stdout b/testing/btest/Baseline/scripts.base.protocols.dns.caa/.stdout new file mode 100644 index 0000000000..4ba72f24b4 --- /dev/null +++ b/testing/btest/Baseline/scripts.base.protocols.dns.caa/.stdout @@ -0,0 +1 @@ +0, issue, symantec.com diff --git a/testing/btest/Baseline/scripts.base.protocols.dns.huge-ttl/.stdout b/testing/btest/Baseline/scripts.base.protocols.dns.huge-ttl/.stdout new file mode 100644 index 0000000000..99f7325c23 --- /dev/null +++ b/testing/btest/Baseline/scripts.base.protocols.dns.huge-ttl/.stdout @@ -0,0 +1,8 @@ +[answer_type=1, query=us.v27.distributed.net, qtype=1, qclass=1, TTL=49710.0 days 6.0 hrs 28.0 mins 15.0 secs] +[answer_type=1, query=us.v27.distributed.net, qtype=1, qclass=1, TTL=15.0 mins] +[answer_type=1, query=us.v27.distributed.net, qtype=1, qclass=1, TTL=15.0 mins] +[answer_type=1, query=us.v27.distributed.net, qtype=1, qclass=1, TTL=15.0 mins] +[answer_type=1, query=us.v27.distributed.net, qtype=1, qclass=1, TTL=15.0 mins] +[answer_type=1, query=us.v27.distributed.net, qtype=1, qclass=1, TTL=15.0 mins] +[answer_type=1, query=us.v27.distributed.net, qtype=1, qclass=1, TTL=15.0 mins] +[answer_type=1, query=us.v27.distributed.net, qtype=1, qclass=1, TTL=15.0 mins] diff --git a/testing/btest/Baseline/scripts.base.protocols.imap.capabilities/.stdout b/testing/btest/Baseline/scripts.base.protocols.imap.capabilities/.stdout new file mode 100644 index 0000000000..bf69e13682 --- /dev/null +++ b/testing/btest/Baseline/scripts.base.protocols.imap.capabilities/.stdout @@ -0,0 +1 @@ +[IMAP4rev1, CHILDREN, ENABLE, ID, IDLE, LIST-EXTENDED, LIST-STATUS, LITERAL+, MOVE, NAMESPACE, SASL-IR, SORT, SPECIAL-USE, THREAD=ORDEREDSUBJECT, UIDPLUS, UNSELECT, WITHIN, STARTTLS, AUTH=LOGIN, AUTH=PLAIN] diff --git a/testing/btest/Baseline/scripts.base.protocols.imap.starttls/.stdout b/testing/btest/Baseline/scripts.base.protocols.imap.starttls/.stdout new file mode 100644 index 0000000000..5fbafd3ab3 --- /dev/null +++ b/testing/btest/Baseline/scripts.base.protocols.imap.starttls/.stdout @@ -0,0 +1 @@ +Tls started for connection diff --git a/testing/btest/Baseline/scripts.base.protocols.imap.starttls/conn.log b/testing/btest/Baseline/scripts.base.protocols.imap.starttls/conn.log new file mode 100644 index 0000000000..0ae19c2fda --- /dev/null +++ b/testing/btest/Baseline/scripts.base.protocols.imap.starttls/conn.log @@ -0,0 +1,10 @@ +#separator \x09 +#set_separator , +#empty_field (empty) +#unset_field - +#path conn +#open 2015-07-22-17-31-02 +#fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p proto service duration orig_bytes resp_bytes conn_state local_orig local_resp missed_bytes history orig_pkts orig_ip_bytes resp_pkts resp_ip_bytes tunnel_parents +#types time string addr port addr port enum string interval count count string bool bool count string count count count count set[string] +1437584567.812552 CXWv6p3arKYeMETxOg 192.168.17.53 49640 212.227.17.186 143 tcp ssl,imap 2.827002 540 5653 SF - - 0 ShAdDafFr 18 1284 14 6225 (empty) +#close 2015-07-22-17-31-02 diff --git a/testing/btest/Baseline/scripts.base.protocols.imap.starttls/ssl.log b/testing/btest/Baseline/scripts.base.protocols.imap.starttls/ssl.log new file mode 100644 index 0000000000..aefbf3d41e --- /dev/null +++ b/testing/btest/Baseline/scripts.base.protocols.imap.starttls/ssl.log @@ -0,0 +1,10 @@ +#separator \x09 +#set_separator , +#empty_field (empty) +#unset_field - +#path ssl +#open 2015-07-22-17-31-02 +#fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p version cipher curve server_name resumed last_alert next_protocol established cert_chain_fuids client_cert_chain_fuids subject issuer client_subject client_issuer +#types time string addr port addr port string string string string bool string string bool vector[string] vector[string] string string string string +1437584568.570497 CXWv6p3arKYeMETxOg 192.168.17.53 49640 212.227.17.186 143 TLSv12 TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 secp256r1 - F - - T FOWmhO3rUj3SEB5RTb,FjH9n52SzEIJ9UoVK9,FisDHa396LIaZadgG9 (empty) CN=imap.gmx.net,emailAddress=server-certs@1und1.de,L=Montabaur,ST=Rhineland-Palatinate,O=1&1 Mail & Media GmbH,C=DE CN=TeleSec ServerPass DE-1,street=Untere Industriestr. 20,L=Netphen,postalCode=57250,ST=NRW,OU=T-Systems Trust Center,O=T-Systems International GmbH,C=DE - - +#close 2015-07-22-17-31-02 diff --git a/testing/btest/Baseline/scripts.base.protocols.imap.starttls/x509.log b/testing/btest/Baseline/scripts.base.protocols.imap.starttls/x509.log new file mode 100644 index 0000000000..6d1be68725 --- /dev/null +++ b/testing/btest/Baseline/scripts.base.protocols.imap.starttls/x509.log @@ -0,0 +1,12 @@ +#separator \x09 +#set_separator , +#empty_field (empty) +#unset_field - +#path x509 +#open 2015-07-22-17-31-02 +#fields ts id certificate.version certificate.serial certificate.subject certificate.issuer certificate.not_valid_before certificate.not_valid_after certificate.key_alg certificate.sig_alg certificate.key_type certificate.key_length certificate.exponent certificate.curve san.dns san.uri san.email san.ip basic_constraints.ca basic_constraints.path_len +#types time string count string string string time time string string string count string string vector[string] vector[string] vector[string] vector[addr] bool count +1437584568.769690 FOWmhO3rUj3SEB5RTb 3 339D9ED8E73927C9 CN=imap.gmx.net,emailAddress=server-certs@1und1.de,L=Montabaur,ST=Rhineland-Palatinate,O=1&1 Mail & Media GmbH,C=DE CN=TeleSec ServerPass DE-1,street=Untere Industriestr. 20,L=Netphen,postalCode=57250,ST=NRW,OU=T-Systems Trust Center,O=T-Systems International GmbH,C=DE 1384251451.000000 1479427199.000000 rsaEncryption sha1WithRSAEncryption rsa 2048 65537 - imap.gmx.net,imap.gmx.de - - - F - +1437584568.769690 FjH9n52SzEIJ9UoVK9 3 21B6777E8CBD0EA8 CN=TeleSec ServerPass DE-1,street=Untere Industriestr. 20,L=Netphen,postalCode=57250,ST=NRW,OU=T-Systems Trust Center,O=T-Systems International GmbH,C=DE CN=Deutsche Telekom Root CA 2,OU=T-TeleSec Trust Center,O=Deutsche Telekom AG,C=DE 1362146309.000000 1562716740.000000 rsaEncryption sha1WithRSAEncryption rsa 2048 65537 - - - - - T 0 +1437584568.769690 FisDHa396LIaZadgG9 3 26 CN=Deutsche Telekom Root CA 2,OU=T-TeleSec Trust Center,O=Deutsche Telekom AG,C=DE CN=Deutsche Telekom Root CA 2,OU=T-TeleSec Trust Center,O=Deutsche Telekom AG,C=DE 931522260.000000 1562716740.000000 rsaEncryption sha1WithRSAEncryption rsa 2048 65537 - - - - - T 5 +#close 2015-07-22-17-31-02 diff --git a/testing/btest/Baseline/scripts.base.protocols.ssl.dtls-stun-dpd/dpd.log b/testing/btest/Baseline/scripts.base.protocols.ssl.dtls-stun-dpd/dpd.log new file mode 100644 index 0000000000..e69de29bb2 diff --git a/testing/btest/Baseline/scripts.base.protocols.ssl.dtls-stun-dpd/ssl.log b/testing/btest/Baseline/scripts.base.protocols.ssl.dtls-stun-dpd/ssl.log new file mode 100644 index 0000000000..0328a5e982 --- /dev/null +++ b/testing/btest/Baseline/scripts.base.protocols.ssl.dtls-stun-dpd/ssl.log @@ -0,0 +1,10 @@ +#separator \x09 +#set_separator , +#empty_field (empty) +#unset_field - +#path ssl +#open 2016-05-17-23-36-28 +#fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p version cipher curve server_name resumed last_alert next_protocol established cert_chain_fuids client_cert_chain_fuids subject issuer client_subject client_issuer +#types time string addr port addr port string string string string bool string string bool vector[string] vector[string] string string string string +1463527314.688817 CXWv6p3arKYeMETxOg 192.168.6.82 51462 74.201.205.9 43044 DTLSv10 TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA secp256r1 - F - - T Fk1e6E3pbe7faF41T5 FjQcYL1EtJ5VueihC7 CN=mantis.tokbox.com,O=Tokbox,L=San Francisco,ST=California,C=US CN=mantis.tokbox.com,O=Tokbox,L=San Francisco,ST=California,C=US CN=a CN=a +#close 2016-05-17-23-36-28 diff --git a/testing/btest/Baseline/scripts.base.protocols.xmpp.client-dpd/ssl.log b/testing/btest/Baseline/scripts.base.protocols.xmpp.client-dpd/ssl.log new file mode 100644 index 0000000000..0ce11b2e6f --- /dev/null +++ b/testing/btest/Baseline/scripts.base.protocols.xmpp.client-dpd/ssl.log @@ -0,0 +1,10 @@ +#separator \x09 +#set_separator , +#empty_field (empty) +#unset_field - +#path ssl +#open 2015-07-21-20-08-11 +#fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p version cipher curve server_name resumed last_alert next_protocol established cert_chain_fuids client_cert_chain_fuids subject issuer client_subject client_issuer +#types time string addr port addr port string string string string bool string string bool vector[string] vector[string] string string string string +1437091702.232293 CXWv6p3arKYeMETxOg 198.128.203.95 56048 146.255.57.229 5222 TLSv12 TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 secp256r1 - F - - T F5Nz2G1vSZQ0QXM2s8,FUw8omi2keRxShDUa (empty) CN=jabber.ccc.de,O=Chaos Computer Club e.V.,L=Hamburg,ST=Hamburg,C=DE emailAddress=support@cacert.org,CN=CA Cert Signing Authority,OU=http://www.cacert.org,O=Root CA - - +#close 2015-07-21-20-08-11 diff --git a/testing/btest/Baseline/scripts.base.protocols.xmpp.server-dialback-dpd/ssl.log b/testing/btest/Baseline/scripts.base.protocols.xmpp.server-dialback-dpd/ssl.log new file mode 100644 index 0000000000..15641ba5b0 --- /dev/null +++ b/testing/btest/Baseline/scripts.base.protocols.xmpp.server-dialback-dpd/ssl.log @@ -0,0 +1,10 @@ +#separator \x09 +#set_separator , +#empty_field (empty) +#unset_field - +#path ssl +#open 2015-07-21-20-18-36 +#fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p version cipher curve server_name resumed last_alert next_protocol established cert_chain_fuids client_cert_chain_fuids subject issuer client_subject client_issuer +#types time string addr port addr port string string string string bool string string bool vector[string] vector[string] string string string string +1437506779.381295 CXWv6p3arKYeMETxOg 184.73.173.246 1193 104.236.167.107 5269 TLSv12 TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 secp384r1 - F - - T FLFr7Z1TXmFDv9FwC2,FydVem3ToAkEIAHD29,FK07OA1VxtQi69Irde F3D2e62Vxl7iTnwbA4,FUCD5w4ABMG5N0YvSi,FxWUEd3mgvThYO2uod,FGOrVE2laVCPsCLMF6 CN=www.0xxon.net,OU=Free SSL,OU=Domain Control Validated CN=COMODO RSA Domain Validation Secure Server CA,O=COMODO CA Limited,L=Salford,ST=Greater Manchester,C=GB CN=*.hosted.im,OU=Domain Control Validated CN=Go Daddy Secure Certificate Authority - G2,OU=http://certs.godaddy.com/repository/,O=GoDaddy.com\\, Inc.,L=Scottsdale,ST=Arizona,C=US +#close 2015-07-21-20-18-36 diff --git a/testing/btest/Baseline/scripts.base.protocols.xmpp.starttls/conn.log b/testing/btest/Baseline/scripts.base.protocols.xmpp.starttls/conn.log new file mode 100644 index 0000000000..2f5bd2f66d --- /dev/null +++ b/testing/btest/Baseline/scripts.base.protocols.xmpp.starttls/conn.log @@ -0,0 +1,10 @@ +#separator \x09 +#set_separator , +#empty_field (empty) +#unset_field - +#path conn +#open 2015-07-21-18-55-16 +#fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p proto service duration orig_bytes resp_bytes conn_state local_orig local_resp missed_bytes history orig_pkts orig_ip_bytes resp_pkts resp_ip_bytes tunnel_parents +#types time string addr port addr port enum string interval count count string bool bool count string count count count count set[string] +1437091701.732171 CXWv6p3arKYeMETxOg 198.128.203.95 56048 146.255.57.229 5222 tcp ssl,xmpp 2.213218 676 4678 SF - - 0 ShADadfFr 19 1676 15 5442 (empty) +#close 2015-07-21-18-55-16 diff --git a/testing/btest/Baseline/scripts.base.protocols.xmpp.starttls/ssl.log b/testing/btest/Baseline/scripts.base.protocols.xmpp.starttls/ssl.log new file mode 100644 index 0000000000..f67ea92631 --- /dev/null +++ b/testing/btest/Baseline/scripts.base.protocols.xmpp.starttls/ssl.log @@ -0,0 +1,10 @@ +#separator \x09 +#set_separator , +#empty_field (empty) +#unset_field - +#path ssl +#open 2015-07-21-18-55-16 +#fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p version cipher curve server_name resumed last_alert next_protocol established cert_chain_fuids client_cert_chain_fuids subject issuer client_subject client_issuer +#types time string addr port addr port string string string string bool string string bool vector[string] vector[string] string string string string +1437091702.232293 CXWv6p3arKYeMETxOg 198.128.203.95 56048 146.255.57.229 5222 TLSv12 TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 secp256r1 - F - - T F5Nz2G1vSZQ0QXM2s8,FUw8omi2keRxShDUa (empty) CN=jabber.ccc.de,O=Chaos Computer Club e.V.,L=Hamburg,ST=Hamburg,C=DE emailAddress=support@cacert.org,CN=CA Cert Signing Authority,OU=http://www.cacert.org,O=Root CA - - +#close 2015-07-21-18-55-16 diff --git a/testing/btest/Baseline/scripts.base.protocols.xmpp.starttls/x509.log b/testing/btest/Baseline/scripts.base.protocols.xmpp.starttls/x509.log new file mode 100644 index 0000000000..4a49298e8a --- /dev/null +++ b/testing/btest/Baseline/scripts.base.protocols.xmpp.starttls/x509.log @@ -0,0 +1,11 @@ +#separator \x09 +#set_separator , +#empty_field (empty) +#unset_field - +#path x509 +#open 2015-07-21-18-55-16 +#fields ts id certificate.version certificate.serial certificate.subject certificate.issuer certificate.not_valid_before certificate.not_valid_after certificate.key_alg certificate.sig_alg certificate.key_type certificate.key_length certificate.exponent certificate.curve san.dns san.uri san.email san.ip basic_constraints.ca basic_constraints.path_len +#types time string count string string string time time string string string count string string vector[string] vector[string] vector[string] vector[addr] bool count +1437091702.407347 F5Nz2G1vSZQ0QXM2s8 3 0DF4F2 CN=jabber.ccc.de,O=Chaos Computer Club e.V.,L=Hamburg,ST=Hamburg,C=DE emailAddress=support@cacert.org,CN=CA Cert Signing Authority,OU=http://www.cacert.org,O=Root CA 1382043019.000000 1445115019.000000 rsaEncryption sha1WithRSAEncryption rsa 2048 65537 - jabber.ccc.de,conference.jabber.ccc.de,jabberd.jabber.ccc.de,pubsub.jabber.ccc.de,vjud.jabber.ccc.de - - - F - +1437091702.407347 FUw8omi2keRxShDUa 3 00 emailAddress=support@cacert.org,CN=CA Cert Signing Authority,OU=http://www.cacert.org,O=Root CA emailAddress=support@cacert.org,CN=CA Cert Signing Authority,OU=http://www.cacert.org,O=Root CA 1049027389.000000 1995712189.000000 rsaEncryption md5WithRSAEncryption rsa 4096 65537 - - - - - T - +#close 2015-07-21-18-55-16 diff --git a/testing/btest/Baseline/scripts.policy.frameworks.intel.seen.certs/intel-all.log b/testing/btest/Baseline/scripts.policy.frameworks.intel.seen.certs/intel-all.log index ba1afe4239..0cac337cf3 100644 --- a/testing/btest/Baseline/scripts.policy.frameworks.intel.seen.certs/intel-all.log +++ b/testing/btest/Baseline/scripts.policy.frameworks.intel.seen.certs/intel-all.log @@ -3,20 +3,23 @@ #empty_field (empty) #unset_field - #path intel -#open 2015-03-14-01-47-46 +#open 2016-04-25-23-53-37 #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p fuid file_mime_type file_desc seen.indicator seen.indicator_type seen.where seen.node sources #types time string addr port addr port string string string string enum enum string set[string] 1416942644.593119 CXWv6p3arKYeMETxOg 192.168.4.149 49422 23.92.19.75 443 F0txuw2pvrkZOn04a8 application/pkix-cert 23.92.19.75:443/tcp www.pantz.org Intel::DOMAIN X509::IN_CERT bro source1 -#close 2015-03-14-01-47-46 +#close 2016-04-25-23-53-37 #separator \x09 #set_separator , #empty_field (empty) #unset_field - #path intel -#open 2015-03-14-01-47-46 +#open 2016-04-25-23-53-38 #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p fuid file_mime_type file_desc seen.indicator seen.indicator_type seen.where seen.node sources #types time string addr port addr port string string string string enum enum string set[string] -1170717505.934612 CXWv6p3arKYeMETxOg 192.150.187.164 58868 194.127.84.106 443 - - - www.dresdner-privat.de Intel::DOMAIN X509::IN_CERT bro source1 -1170717509.082241 CjhGID4nQcgTWjvg4c 192.150.187.164 58869 194.127.84.106 443 - - - www.dresdner-privat.de Intel::DOMAIN X509::IN_CERT bro source1 -1170717512.108799 CCvvfg3TEfuqmmG4bh 192.150.187.164 58870 194.127.84.106 443 - - - www.dresdner-privat.de Intel::DOMAIN X509::IN_CERT bro source1 -#close 2015-03-14-01-47-46 +1170717505.735416 CXWv6p3arKYeMETxOg 192.150.187.164 58868 194.127.84.106 443 FeCwNK3rzqPnZ7eBQ5 application/pkix-cert 194.127.84.106:443/tcp 2c322ae2b7fe91391345e070b63668978bb1c9da Intel::CERT_HASH X509::IN_CERT bro source1 +1170717505.934612 CXWv6p3arKYeMETxOg 192.150.187.164 58868 194.127.84.106 443 FeCwNK3rzqPnZ7eBQ5 - - www.dresdner-privat.de Intel::DOMAIN X509::IN_CERT bro source1 +1170717508.883051 CjhGID4nQcgTWjvg4c 192.150.187.164 58869 194.127.84.106 443 FjkLnG4s34DVZlaBNc application/pkix-cert 194.127.84.106:443/tcp 2c322ae2b7fe91391345e070b63668978bb1c9da Intel::CERT_HASH X509::IN_CERT bro source1 +1170717509.082241 CjhGID4nQcgTWjvg4c 192.150.187.164 58869 194.127.84.106 443 FjkLnG4s34DVZlaBNc - - www.dresdner-privat.de Intel::DOMAIN X509::IN_CERT bro source1 +1170717511.909717 CCvvfg3TEfuqmmG4bh 192.150.187.164 58870 194.127.84.106 443 FQXAWgI2FB5STbrff application/pkix-cert 194.127.84.106:443/tcp 2c322ae2b7fe91391345e070b63668978bb1c9da Intel::CERT_HASH X509::IN_CERT bro source1 +1170717512.108799 CCvvfg3TEfuqmmG4bh 192.150.187.164 58870 194.127.84.106 443 FQXAWgI2FB5STbrff - - www.dresdner-privat.de Intel::DOMAIN X509::IN_CERT bro source1 +#close 2016-04-25-23-53-38 diff --git a/testing/btest/Traces/arp-who-has.pcap b/testing/btest/Traces/arp-who-has.pcap new file mode 100644 index 0000000000..085dddf1fe Binary files /dev/null and b/testing/btest/Traces/arp-who-has.pcap differ diff --git a/testing/btest/Traces/dns-caa.pcap b/testing/btest/Traces/dns-caa.pcap new file mode 100644 index 0000000000..7409c0347b Binary files /dev/null and b/testing/btest/Traces/dns-caa.pcap differ diff --git a/testing/btest/Traces/dns-huge-ttl.pcap b/testing/btest/Traces/dns-huge-ttl.pcap new file mode 100644 index 0000000000..27849b904b Binary files /dev/null and b/testing/btest/Traces/dns-huge-ttl.pcap differ diff --git a/testing/btest/Traces/tls/imap-starttls.pcap b/testing/btest/Traces/tls/imap-starttls.pcap new file mode 100644 index 0000000000..f6bfe5458d Binary files /dev/null and b/testing/btest/Traces/tls/imap-starttls.pcap differ diff --git a/testing/btest/Traces/tls/telesec.pcap b/testing/btest/Traces/tls/telesec.pcap new file mode 100644 index 0000000000..0f27b68d59 Binary files /dev/null and b/testing/btest/Traces/tls/telesec.pcap differ diff --git a/testing/btest/Traces/tls/webrtc-stun.pcap b/testing/btest/Traces/tls/webrtc-stun.pcap new file mode 100644 index 0000000000..6eb5f90372 Binary files /dev/null and b/testing/btest/Traces/tls/webrtc-stun.pcap differ diff --git a/testing/btest/Traces/tls/xmpp-dialback-starttls.pcap b/testing/btest/Traces/tls/xmpp-dialback-starttls.pcap new file mode 100644 index 0000000000..ad55c6eceb Binary files /dev/null and b/testing/btest/Traces/tls/xmpp-dialback-starttls.pcap differ diff --git a/testing/btest/Traces/tls/xmpp-starttls.pcap b/testing/btest/Traces/tls/xmpp-starttls.pcap new file mode 100644 index 0000000000..b4a7ee61e1 Binary files /dev/null and b/testing/btest/Traces/tls/xmpp-starttls.pcap differ diff --git a/testing/btest/bifs/fmt.bro b/testing/btest/bifs/fmt.bro index 93607c2740..7fc4dc38d7 100644 --- a/testing/btest/bifs/fmt.bro +++ b/testing/btest/bifs/fmt.bro @@ -65,26 +65,16 @@ event bro_init() print fmt("%.3g", 3.1e+2); print fmt("%.7g", 3.1e+2); - # Tests comparing "%As" and "%s" (the string length is printed instead - # of the string itself because the print command does its own escaping) - local s0 = "\x00\x07"; - local s1 = fmt("%As", s0); # expands \x00 to "\0" - local s2 = fmt("%s", s0); # expands \x00 to "\0", and \x07 to "^G" + # Tests of "%s" with non-printable characters (the string length is printed + # instead of the string itself because the print command does its own + # escaping) + local s0 = "\x00\x1f"; + local s1 = fmt("%s", s0); print |s0|; print |s1|; - print |s2|; - - s0 = "\x07\x1f"; - s1 = fmt("%As", s0); - s2 = fmt("%s", s0); # expands \x07 to "^G", and \x1f to "\x1f" - print |s0|; - print |s1|; - print |s2|; s0 = "\x7f\xff"; - s1 = fmt("%As", s0); - s2 = fmt("%s", s0); # expands \x7f to "^?", and \xff to "\xff" + s1 = fmt("%s", s0); print |s0|; print |s1|; - print |s2|; } diff --git a/testing/btest/bifs/get_current_packet_header.bro b/testing/btest/bifs/get_current_packet_header.bro new file mode 100644 index 0000000000..24144545ef --- /dev/null +++ b/testing/btest/bifs/get_current_packet_header.bro @@ -0,0 +1,8 @@ +# @TEST-EXEC: bro -C -r $TRACES/icmp/icmp6-neighbor-solicit.pcap %INPUT > output +# @TEST-EXEC: btest-diff output + +event icmp_neighbor_solicitation(c: connection, icmp: icmp_conn, tgt: addr, options: icmp6_nd_options) + { + local hdr: raw_pkt_hdr = get_current_packet_header(); + print fmt("%s", hdr); + } \ No newline at end of file diff --git a/testing/btest/bifs/net_stats_trace.test b/testing/btest/bifs/net_stats_trace.test index fcf3e9ba0d..cd9ee52a27 100644 --- a/testing/btest/bifs/net_stats_trace.test +++ b/testing/btest/bifs/net_stats_trace.test @@ -4,5 +4,5 @@ event bro_done() { - print net_stats(); + print get_net_stats(); } diff --git a/testing/btest/bifs/resource_usage.bro b/testing/btest/bifs/resource_usage.bro deleted file mode 100644 index 5cf3f0f962..0000000000 --- a/testing/btest/bifs/resource_usage.bro +++ /dev/null @@ -1,9 +0,0 @@ -# -# @TEST-EXEC: bro -b %INPUT - -event bro_init() - { - local a = resource_usage(); - if ( a$version != bro_version() ) - exit(1); - } diff --git a/testing/btest/broker/clone_store.bro b/testing/btest/broker/clone_store.bro index 1973595bab..1ed35826dc 100644 --- a/testing/btest/broker/clone_store.bro +++ b/testing/btest/broker/clone_store.bro @@ -1,8 +1,8 @@ # @TEST-SERIALIZE: brokercomm -# @TEST-REQUIRES: grep -q ENABLE_BROKER $BUILD/CMakeCache.txt +# @TEST-REQUIRES: grep -q ENABLE_BROKER:BOOL=true $BUILD/CMakeCache.txt -# @TEST-EXEC: btest-bg-run clone "bro -b -r $TRACES/wikipedia.trace ../clone.bro broker_port=$BROKER_PORT >clone.out" -# @TEST-EXEC: btest-bg-run master "bro -b -r $TRACES/wikipedia.trace ../master.bro broker_port=$BROKER_PORT >master.out" +# @TEST-EXEC: btest-bg-run clone "bro -b ../clone.bro broker_port=$BROKER_PORT >clone.out" +# @TEST-EXEC: btest-bg-run master "bro -b ../master.bro broker_port=$BROKER_PORT >master.out" # @TEST-EXEC: btest-bg-wait 60 # @TEST-EXEC: TEST_DIFF_CANONIFIER=$SCRIPTS/diff-sort btest-diff clone/clone.out @@ -13,7 +13,7 @@ const broker_port: port &redef; redef exit_only_after_terminate = T; -global h: opaque of BrokerStore::Handle; +global h: opaque of Broker::Handle; global expected_key_count = 4; global key_count = 0; @@ -21,7 +21,7 @@ global query_timeout = 30sec; function do_lookup(key: string) { - when ( local res = BrokerStore::lookup(h, BrokerComm::data(key)) ) + when ( local res = Broker::lookup(h, Broker::data(key)) ) { ++key_count; print "lookup", key, res; @@ -38,15 +38,15 @@ function do_lookup(key: string) event ready() { - h = BrokerStore::create_clone("mystore"); + h = Broker::create_clone("mystore"); - when ( local res = BrokerStore::keys(h) ) + when ( local res = Broker::keys(h) ) { print "clone keys", res; - do_lookup(BrokerComm::refine_to_string(BrokerComm::vector_lookup(res$result, 0))); - do_lookup(BrokerComm::refine_to_string(BrokerComm::vector_lookup(res$result, 1))); - do_lookup(BrokerComm::refine_to_string(BrokerComm::vector_lookup(res$result, 2))); - do_lookup(BrokerComm::refine_to_string(BrokerComm::vector_lookup(res$result, 3))); + do_lookup(Broker::refine_to_string(Broker::vector_lookup(res$result, 0))); + do_lookup(Broker::refine_to_string(Broker::vector_lookup(res$result, 1))); + do_lookup(Broker::refine_to_string(Broker::vector_lookup(res$result, 2))); + do_lookup(Broker::refine_to_string(Broker::vector_lookup(res$result, 3))); } timeout query_timeout { @@ -57,9 +57,9 @@ event ready() event bro_init() { - BrokerComm::enable(); - BrokerComm::subscribe_to_events("bro/event/ready"); - BrokerComm::listen(broker_port, "127.0.0.1"); + Broker::enable(); + Broker::subscribe_to_events("bro/event/ready"); + Broker::listen(broker_port, "127.0.0.1"); } @TEST-END-FILE @@ -71,42 +71,42 @@ global query_timeout = 15sec; const broker_port: port &redef; redef exit_only_after_terminate = T; -global h: opaque of BrokerStore::Handle; +global h: opaque of Broker::Handle; -function dv(d: BrokerComm::Data): BrokerComm::DataVector +function dv(d: Broker::Data): Broker::DataVector { - local rval: BrokerComm::DataVector; + local rval: Broker::DataVector; rval[0] = d; return rval; } global ready: event(); -event BrokerComm::outgoing_connection_broken(peer_address: string, +event Broker::outgoing_connection_broken(peer_address: string, peer_port: port) { terminate(); } -event BrokerComm::outgoing_connection_established(peer_address: string, +event Broker::outgoing_connection_established(peer_address: string, peer_port: port, peer_name: string) { local myset: set[string] = {"a", "b", "c"}; local myvec: vector of string = {"alpha", "beta", "gamma"}; - h = BrokerStore::create_master("mystore"); - BrokerStore::insert(h, BrokerComm::data("one"), BrokerComm::data(110)); - BrokerStore::insert(h, BrokerComm::data("two"), BrokerComm::data(223)); - BrokerStore::insert(h, BrokerComm::data("myset"), BrokerComm::data(myset)); - BrokerStore::insert(h, BrokerComm::data("myvec"), BrokerComm::data(myvec)); - BrokerStore::increment(h, BrokerComm::data("one")); - BrokerStore::decrement(h, BrokerComm::data("two")); - BrokerStore::add_to_set(h, BrokerComm::data("myset"), BrokerComm::data("d")); - BrokerStore::remove_from_set(h, BrokerComm::data("myset"), BrokerComm::data("b")); - BrokerStore::push_left(h, BrokerComm::data("myvec"), dv(BrokerComm::data("delta"))); - BrokerStore::push_right(h, BrokerComm::data("myvec"), dv(BrokerComm::data("omega"))); + h = Broker::create_master("mystore"); + Broker::insert(h, Broker::data("one"), Broker::data(110)); + Broker::insert(h, Broker::data("two"), Broker::data(223)); + Broker::insert(h, Broker::data("myset"), Broker::data(myset)); + Broker::insert(h, Broker::data("myvec"), Broker::data(myvec)); + Broker::increment(h, Broker::data("one")); + Broker::decrement(h, Broker::data("two")); + Broker::add_to_set(h, Broker::data("myset"), Broker::data("d")); + Broker::remove_from_set(h, Broker::data("myset"), Broker::data("b")); + Broker::push_left(h, Broker::data("myvec"), dv(Broker::data("delta"))); + Broker::push_right(h, Broker::data("myvec"), dv(Broker::data("omega"))); - when ( local res = BrokerStore::size(h) ) + when ( local res = Broker::size(h) ) { event ready(); } timeout query_timeout { @@ -117,9 +117,9 @@ event BrokerComm::outgoing_connection_established(peer_address: string, event bro_init() { - BrokerComm::enable(); - BrokerComm::auto_event("bro/event/ready", ready); - BrokerComm::connect("127.0.0.1", broker_port, 1secs); + Broker::enable(); + Broker::auto_event("bro/event/ready", ready); + Broker::connect("127.0.0.1", broker_port, 1secs); } @TEST-END-FILE diff --git a/testing/btest/broker/connection_updates.bro b/testing/btest/broker/connection_updates.bro index 1bbe90ccb5..d431a59dbe 100644 --- a/testing/btest/broker/connection_updates.bro +++ b/testing/btest/broker/connection_updates.bro @@ -1,5 +1,5 @@ # @TEST-SERIALIZE: brokercomm -# @TEST-REQUIRES: grep -q ENABLE_BROKER $BUILD/CMakeCache.txt +# @TEST-REQUIRES: grep -q ENABLE_BROKER:BOOL=true $BUILD/CMakeCache.txt # @TEST-EXEC: btest-bg-run recv "bro -b ../recv.bro broker_port=$BROKER_PORT >recv.out" # @TEST-EXEC: btest-bg-run send "bro -b ../send.bro broker_port=$BROKER_PORT >send.out" @@ -12,22 +12,22 @@ const broker_port: port &redef; redef exit_only_after_terminate = T; -redef BrokerComm::endpoint_name = "listener"; +redef Broker::endpoint_name = "listener"; event bro_init() { - BrokerComm::enable(); - BrokerComm::listen(broker_port, "127.0.0.1"); + Broker::enable(); + Broker::listen(broker_port, "127.0.0.1"); } -event BrokerComm::incoming_connection_established(peer_name: string) +event Broker::incoming_connection_established(peer_name: string) { - print "BrokerComm::incoming_connection_established", peer_name;; + print "Broker::incoming_connection_established", peer_name; } -event BrokerComm::incoming_connection_broken(peer_name: string) +event Broker::incoming_connection_broken(peer_name: string) { - print "BrokerComm::incoming_connection_broken", peer_name;; + print "Broker::incoming_connection_broken", peer_name; terminate(); } @@ -37,20 +37,20 @@ event BrokerComm::incoming_connection_broken(peer_name: string) const broker_port: port &redef; redef exit_only_after_terminate = T; -redef BrokerComm::endpoint_name = "connector"; +redef Broker::endpoint_name = "connector"; event bro_init() { - BrokerComm::enable(); - BrokerComm::connect("127.0.0.1", broker_port, 1sec); + Broker::enable(); + Broker::connect("127.0.0.1", broker_port, 1sec); } -event BrokerComm::outgoing_connection_established(peer_address: string, +event Broker::outgoing_connection_established(peer_address: string, peer_port: port, peer_name: string) { - print "BrokerComm::outgoing_connection_established", - peer_address, peer_port, peer_name;; + print "Broker::outgoing_connection_established", + peer_address, peer_port, peer_name; terminate(); } diff --git a/testing/btest/broker/data.bro b/testing/btest/broker/data.bro index bac7242c85..49474e3a5a 100644 --- a/testing/btest/broker/data.bro +++ b/testing/btest/broker/data.bro @@ -1,4 +1,4 @@ -# @TEST-REQUIRES: grep -q ENABLE_BROKER $BUILD/CMakeCache.txt +# @TEST-REQUIRES: grep -q ENABLE_BROKER:BOOL=true $BUILD/CMakeCache.txt # @TEST-EXEC: bro -b %INPUT >out # @TEST-EXEC: btest-diff out @@ -13,210 +13,243 @@ type bro_record : record { c: count; }; -function comm_record_to_bro_record_recurse(it: opaque of BrokerComm::RecordIterator, +function broker_to_bro_record_recurse(it: opaque of Broker::RecordIterator, rval: bro_record, idx: count): bro_record { - if ( BrokerComm::record_iterator_last(it) ) + if ( Broker::record_iterator_last(it) ) return rval; - local field_value = BrokerComm::record_iterator_value(it); + local field_value = Broker::record_iterator_value(it); if ( field_value?$d ) switch ( idx ) { case 0: - rval$a = BrokerComm::refine_to_string(field_value); + rval$a = Broker::refine_to_string(field_value); break; case 1: - rval$b = BrokerComm::refine_to_string(field_value); + rval$b = Broker::refine_to_string(field_value); break; case 2: - rval$c = BrokerComm::refine_to_count(field_value); + rval$c = Broker::refine_to_count(field_value); break; }; ++idx; - BrokerComm::record_iterator_next(it); - return comm_record_to_bro_record_recurse(it, rval, idx); + Broker::record_iterator_next(it); + return broker_to_bro_record_recurse(it, rval, idx); } -function comm_record_to_bro_record(d: BrokerComm::Data): bro_record +function broker_to_bro_record(d: Broker::Data): bro_record { - return comm_record_to_bro_record_recurse(BrokerComm::record_iterator(d), + return broker_to_bro_record_recurse(Broker::record_iterator(d), bro_record($c = 0), 0); } function -comm_set_to_bro_set_recurse(it: opaque of BrokerComm::SetIterator, +broker_to_bro_set_recurse(it: opaque of Broker::SetIterator, rval: bro_set): bro_set { - if ( BrokerComm::set_iterator_last(it) ) + if ( Broker::set_iterator_last(it) ) return rval; - add rval[BrokerComm::refine_to_string(BrokerComm::set_iterator_value(it))]; - BrokerComm::set_iterator_next(it); - return comm_set_to_bro_set_recurse(it, rval); + add rval[Broker::refine_to_string(Broker::set_iterator_value(it))]; + Broker::set_iterator_next(it); + return broker_to_bro_set_recurse(it, rval); } -function comm_set_to_bro_set(d: BrokerComm::Data): bro_set +function broker_to_bro_set(d: Broker::Data): bro_set { - return comm_set_to_bro_set_recurse(BrokerComm::set_iterator(d), bro_set()); + return broker_to_bro_set_recurse(Broker::set_iterator(d), bro_set()); } function -comm_table_to_bro_table_recurse(it: opaque of BrokerComm::TableIterator, +broker_to_bro_table_recurse(it: opaque of Broker::TableIterator, rval: bro_table): bro_table { - if ( BrokerComm::table_iterator_last(it) ) + if ( Broker::table_iterator_last(it) ) return rval; - local item = BrokerComm::table_iterator_value(it); - rval[BrokerComm::refine_to_string(item$key)] = BrokerComm::refine_to_count(item$val); - BrokerComm::table_iterator_next(it); - return comm_table_to_bro_table_recurse(it, rval); + local item = Broker::table_iterator_value(it); + rval[Broker::refine_to_string(item$key)] = Broker::refine_to_count(item$val); + Broker::table_iterator_next(it); + return broker_to_bro_table_recurse(it, rval); } -function comm_table_to_bro_table(d: BrokerComm::Data): bro_table +function broker_to_bro_table(d: Broker::Data): bro_table { - return comm_table_to_bro_table_recurse(BrokerComm::table_iterator(d), + return broker_to_bro_table_recurse(Broker::table_iterator(d), bro_table()); } -function comm_vector_to_bro_vector_recurse(it: opaque of BrokerComm::VectorIterator, +function broker_to_bro_vector_recurse(it: opaque of Broker::VectorIterator, rval: bro_vector): bro_vector { - if ( BrokerComm::vector_iterator_last(it) ) + if ( Broker::vector_iterator_last(it) ) return rval; - rval[|rval|] = BrokerComm::refine_to_string(BrokerComm::vector_iterator_value(it)); - BrokerComm::vector_iterator_next(it); - return comm_vector_to_bro_vector_recurse(it, rval); + rval[|rval|] = Broker::refine_to_string(Broker::vector_iterator_value(it)); + Broker::vector_iterator_next(it); + return broker_to_bro_vector_recurse(it, rval); } -function comm_vector_to_bro_vector(d: BrokerComm::Data): bro_vector +function broker_to_bro_vector(d: Broker::Data): bro_vector { - return comm_vector_to_bro_vector_recurse(BrokerComm::vector_iterator(d), + return broker_to_bro_vector_recurse(Broker::vector_iterator(d), bro_vector()); } event bro_init() { -BrokerComm::enable(); -print BrokerComm::data_type(BrokerComm::data(T)); -print BrokerComm::data_type(BrokerComm::data(+1)); -print BrokerComm::data_type(BrokerComm::data(1)); -print BrokerComm::data_type(BrokerComm::data(1.1)); -print BrokerComm::data_type(BrokerComm::data("1 (how creative)")); -print BrokerComm::data_type(BrokerComm::data(1.1.1.1)); -print BrokerComm::data_type(BrokerComm::data(1.1.1.1/1)); -print BrokerComm::data_type(BrokerComm::data(1/udp)); -print BrokerComm::data_type(BrokerComm::data(double_to_time(1))); -print BrokerComm::data_type(BrokerComm::data(1sec)); -print BrokerComm::data_type(BrokerComm::data(BrokerComm::BOOL)); +Broker::enable(); + +### Print every broker data type + +print Broker::data_type(Broker::data(T)); +print Broker::data_type(Broker::data(+1)); +print Broker::data_type(Broker::data(1)); +print Broker::data_type(Broker::data(1.1)); +print Broker::data_type(Broker::data("1 (how creative)")); +print Broker::data_type(Broker::data(1.1.1.1)); +print Broker::data_type(Broker::data(1.1.1.1/1)); +print Broker::data_type(Broker::data(1/udp)); +print Broker::data_type(Broker::data(double_to_time(1))); +print Broker::data_type(Broker::data(1sec)); +print Broker::data_type(Broker::data(Broker::BOOL)); local s: bro_set = bro_set("one", "two", "three"); local t: bro_table = bro_table(["one"] = 1, ["two"] = 2, ["three"] = 3); local v: bro_vector = bro_vector("zero", "one", "two"); local r: bro_record = bro_record($c = 1); -print BrokerComm::data_type(BrokerComm::data(s)); -print BrokerComm::data_type(BrokerComm::data(t)); -print BrokerComm::data_type(BrokerComm::data(v)); -print BrokerComm::data_type(BrokerComm::data(r)); +print Broker::data_type(Broker::data(s)); +print Broker::data_type(Broker::data(t)); +print Broker::data_type(Broker::data(v)); +print Broker::data_type(Broker::data(r)); print "***************************"; -print BrokerComm::refine_to_bool(BrokerComm::data(T)); -print BrokerComm::refine_to_bool(BrokerComm::data(F)); -print BrokerComm::refine_to_int(BrokerComm::data(+1)); -print BrokerComm::refine_to_int(BrokerComm::data(+0)); -print BrokerComm::refine_to_int(BrokerComm::data(-1)); -print BrokerComm::refine_to_count(BrokerComm::data(1)); -print BrokerComm::refine_to_count(BrokerComm::data(0)); -print BrokerComm::refine_to_double(BrokerComm::data(1.1)); -print BrokerComm::refine_to_double(BrokerComm::data(-11.1)); -print BrokerComm::refine_to_string(BrokerComm::data("hello")); -print BrokerComm::refine_to_addr(BrokerComm::data(1.2.3.4)); -print BrokerComm::refine_to_subnet(BrokerComm::data(192.168.1.1/16)); -print BrokerComm::refine_to_port(BrokerComm::data(22/tcp)); -print BrokerComm::refine_to_time(BrokerComm::data(double_to_time(42))); -print BrokerComm::refine_to_interval(BrokerComm::data(3min)); -print BrokerComm::refine_to_enum_name(BrokerComm::data(BrokerComm::BOOL)); +### Convert a Bro value to a broker value, then print the result -print "***************************"; +print Broker::refine_to_bool(Broker::data(T)); +print Broker::refine_to_bool(Broker::data(F)); +print Broker::refine_to_int(Broker::data(+1)); +print Broker::refine_to_int(Broker::data(+0)); +print Broker::refine_to_int(Broker::data(-1)); +print Broker::refine_to_count(Broker::data(1)); +print Broker::refine_to_count(Broker::data(0)); +print Broker::refine_to_double(Broker::data(1.1)); +print Broker::refine_to_double(Broker::data(-11.1)); +print Broker::refine_to_string(Broker::data("hello")); +print Broker::refine_to_addr(Broker::data(1.2.3.4)); +print Broker::refine_to_subnet(Broker::data(192.168.1.1/16)); +print Broker::refine_to_port(Broker::data(22/tcp)); +print Broker::refine_to_time(Broker::data(double_to_time(42))); +print Broker::refine_to_interval(Broker::data(3min)); +print Broker::refine_to_enum_name(Broker::data(Broker::BOOL)); -local cs = BrokerComm::data(s); -print comm_set_to_bro_set(cs); -cs = BrokerComm::set_create(); -print BrokerComm::set_size(cs); -print BrokerComm::set_insert(cs, BrokerComm::data("hi")); -print BrokerComm::set_size(cs); -print BrokerComm::set_contains(cs, BrokerComm::data("hi")); -print BrokerComm::set_contains(cs, BrokerComm::data("bye")); -print BrokerComm::set_insert(cs, BrokerComm::data("bye")); -print BrokerComm::set_size(cs); -print BrokerComm::set_remove(cs, BrokerComm::data("hi")); -print BrokerComm::set_size(cs); -print BrokerComm::set_remove(cs, BrokerComm::data("hi")); -print comm_set_to_bro_set(cs); -BrokerComm::set_clear(cs); -print BrokerComm::set_size(cs); +local cs = Broker::data(s); +print broker_to_bro_set(cs); -print "***************************"; +local ct = Broker::data(t); +print broker_to_bro_table(ct); -local ct = BrokerComm::data(t); -print comm_table_to_bro_table(ct); -ct = BrokerComm::table_create(); -print BrokerComm::table_size(ct); -print BrokerComm::table_insert(ct, BrokerComm::data("hi"), BrokerComm::data(42)); -print BrokerComm::table_size(ct); -print BrokerComm::table_contains(ct, BrokerComm::data("hi")); -print BrokerComm::refine_to_count(BrokerComm::table_lookup(ct, BrokerComm::data("hi"))); -print BrokerComm::table_contains(ct, BrokerComm::data("bye")); -print BrokerComm::table_insert(ct, BrokerComm::data("bye"), BrokerComm::data(7)); -print BrokerComm::table_size(ct); -print BrokerComm::table_insert(ct, BrokerComm::data("bye"), BrokerComm::data(37)); -print BrokerComm::table_size(ct); -print BrokerComm::refine_to_count(BrokerComm::table_lookup(ct, BrokerComm::data("bye"))); -print BrokerComm::table_remove(ct, BrokerComm::data("hi")); -print BrokerComm::table_size(ct); +local cv = Broker::data(v); +print broker_to_bro_vector(cv); -print "***************************"; +local cr = Broker::data(r); +print broker_to_bro_record(cr); -local cv = BrokerComm::data(v); -print comm_vector_to_bro_vector(cv); -cv = BrokerComm::vector_create(); -print BrokerComm::vector_size(cv); -print BrokerComm::vector_insert(cv, BrokerComm::data("hi"), 0); -print BrokerComm::vector_insert(cv, BrokerComm::data("hello"), 1); -print BrokerComm::vector_insert(cv, BrokerComm::data("greetings"), 2); -print BrokerComm::vector_insert(cv, BrokerComm::data("salutations"), 1); -print comm_vector_to_bro_vector(cv); -print BrokerComm::vector_size(cv); -print BrokerComm::vector_replace(cv, BrokerComm::data("bah"), 2); -print BrokerComm::vector_lookup(cv, 2); -print BrokerComm::vector_lookup(cv, 0); -print comm_vector_to_bro_vector(cv); -print BrokerComm::vector_remove(cv, 2); -print comm_vector_to_bro_vector(cv); -print BrokerComm::vector_size(cv); - -print "***************************"; - -local cr = BrokerComm::data(r); -print comm_record_to_bro_record(cr); r$a = "test"; -cr = BrokerComm::data(r); -print comm_record_to_bro_record(cr); +cr = Broker::data(r); +print broker_to_bro_record(cr); + r$b = "testagain"; -cr = BrokerComm::data(r); -print comm_record_to_bro_record(cr); -cr = BrokerComm::record_create(3); -print BrokerComm::record_size(cr); -print BrokerComm::record_assign(cr, BrokerComm::data("hi"), 0); -print BrokerComm::record_assign(cr, BrokerComm::data("hello"), 1); -print BrokerComm::record_assign(cr, BrokerComm::data(37), 2); -print BrokerComm::record_lookup(cr, 0); -print BrokerComm::record_lookup(cr, 1); -print BrokerComm::record_lookup(cr, 2); -print BrokerComm::record_size(cr); +cr = Broker::data(r); +print broker_to_bro_record(cr); + +print "***************************"; + +### Test the broker set BIFs + +cs = Broker::set_create(); +print Broker::set_size(cs); +print Broker::set_insert(cs, Broker::data("hi")); +print Broker::set_size(cs); +print Broker::set_contains(cs, Broker::data("hi")); +print Broker::set_contains(cs, Broker::data("bye")); +print Broker::set_insert(cs, Broker::data("bye")); +print Broker::set_size(cs); +print Broker::set_insert(cs, Broker::data("bye")); +print Broker::set_size(cs); +print Broker::set_remove(cs, Broker::data("hi")); +print Broker::set_size(cs); +print Broker::set_remove(cs, Broker::data("hi")); +print broker_to_bro_set(cs); +print Broker::set_clear(cs); +print Broker::set_size(cs); +print broker_to_bro_set(cs); + +print "***************************"; + +### Test the broker table BIFs + +ct = Broker::table_create(); +print Broker::table_size(ct); +print Broker::table_insert(ct, Broker::data("hi"), Broker::data(42)); +print Broker::table_size(ct); +print Broker::table_contains(ct, Broker::data("hi")); +print Broker::refine_to_count(Broker::table_lookup(ct, Broker::data("hi"))); +print Broker::table_contains(ct, Broker::data("bye")); +print Broker::table_insert(ct, Broker::data("bye"), Broker::data(7)); +print Broker::table_size(ct); +print Broker::table_insert(ct, Broker::data("bye"), Broker::data(37)); +print Broker::table_size(ct); +print Broker::refine_to_count(Broker::table_lookup(ct, Broker::data("bye"))); +print Broker::table_remove(ct, Broker::data("hi")); +print Broker::table_size(ct); +print Broker::table_remove(ct, Broker::data("hi")); +print Broker::table_size(ct); +print Broker::table_clear(ct); +print Broker::table_size(ct); +print broker_to_bro_table(ct); + +print "***************************"; + +### Test the broker vector BIFs + +cv = Broker::vector_create(); +print Broker::vector_size(cv); +print Broker::vector_insert(cv, Broker::data("hi"), 0); +print Broker::vector_insert(cv, Broker::data("hello"), 1); +print Broker::vector_insert(cv, Broker::data("greetings"), 2); +print Broker::vector_insert(cv, Broker::data("salutations"), 1); +print broker_to_bro_vector(cv); +print Broker::vector_size(cv); +print Broker::vector_replace(cv, Broker::data("bah"), 2); +print Broker::vector_lookup(cv, 2); +print Broker::vector_lookup(cv, 0); +print broker_to_bro_vector(cv); +print Broker::vector_remove(cv, 2); +print broker_to_bro_vector(cv); +print Broker::vector_size(cv); +print Broker::vector_clear(cv); +print Broker::vector_size(cv); +print broker_to_bro_vector(cv); + +print "***************************"; + +### Test the broker record BIFs + +cr = Broker::record_create(3); +print Broker::record_size(cr); +print Broker::record_assign(cr, Broker::data("hi"), 0); +print Broker::record_assign(cr, Broker::data("hello"), 1); +print Broker::record_assign(cr, Broker::data(37), 2); +print Broker::record_lookup(cr, 0); +print Broker::record_lookup(cr, 1); +print Broker::record_lookup(cr, 2); +print Broker::record_size(cr); +print Broker::record_assign(cr, Broker::data("goodbye"), 1); +print Broker::record_size(cr); +print Broker::record_lookup(cr, 1); } diff --git a/testing/btest/broker/enable-and-exit.bro b/testing/btest/broker/enable-and-exit.bro index 9f45672bb6..78800b31b0 100644 --- a/testing/btest/broker/enable-and-exit.bro +++ b/testing/btest/broker/enable-and-exit.bro @@ -1,4 +1,4 @@ -# @TEST-REQUIRES: grep -q ENABLE_BROKER $BUILD/CMakeCache.txt +# @TEST-REQUIRES: grep -q ENABLE_BROKER:BOOL=true $BUILD/CMakeCache.txt # @TEST-EXEC: bro -b %INPUT >output # @TEST-EXEC: btest-diff output @@ -11,7 +11,7 @@ event terminate_me() { } event bro_init() { - BrokerComm::enable(); + Broker::enable(); print "1"; schedule 1sec { terminate_me() }; diff --git a/testing/btest/broker/master_store.bro b/testing/btest/broker/master_store.bro index 2536addc0f..09f0f82880 100644 --- a/testing/btest/broker/master_store.bro +++ b/testing/btest/broker/master_store.bro @@ -1,4 +1,4 @@ -# @TEST-REQUIRES: grep -q ENABLE_BROKER $BUILD/CMakeCache.txt +# @TEST-REQUIRES: grep -q ENABLE_BROKER:BOOL=true $BUILD/CMakeCache.txt # @TEST-EXEC: btest-bg-run master "bro -b %INPUT >out" # @TEST-EXEC: btest-bg-wait 60 @@ -6,7 +6,7 @@ redef exit_only_after_terminate = T; -global h: opaque of BrokerStore::Handle; +global h: opaque of Broker::Handle; global lookup_count = 0; const lookup_expect_count = 5; global exists_count = 0; @@ -20,13 +20,13 @@ global query_timeout = 30sec; event test_clear() { - BrokerStore::clear(h); + Broker::clear(h); event test_size("after clear"); } event test_size(where: string) { - when ( local res = BrokerStore::size(h) ) + when ( local res = Broker::size(h) ) { if ( where == "" ) { @@ -52,7 +52,7 @@ event test_size(where: string) event test_keys() { - when ( local res = BrokerStore::keys(h) ) + when ( local res = Broker::keys(h) ) { print fmt("keys: %s", res); event test_size(); @@ -66,7 +66,7 @@ event test_keys() event test_pop(key: string) { - when ( local lres = BrokerStore::pop_left(h, BrokerComm::data(key)) ) + when ( local lres = Broker::pop_left(h, Broker::data(key)) ) { print fmt("pop_left(%s): %s", key, lres); ++pop_count; @@ -83,7 +83,7 @@ event test_pop(key: string) event test_keys(); } - when ( local rres = BrokerStore::pop_right(h, BrokerComm::data(key)) ) + when ( local rres = Broker::pop_right(h, Broker::data(key)) ) { print fmt("pop_right(%s): %s", key, rres); ++pop_count; @@ -103,7 +103,7 @@ event test_pop(key: string) function do_exists(key: string) { - when ( local res = BrokerStore::exists(h, BrokerComm::data(key)) ) + when ( local res = Broker::exists(h, Broker::data(key)) ) { print fmt("exists(%s): %s", key, res); ++exists_count; @@ -123,7 +123,7 @@ function do_exists(key: string) event test_erase() { - BrokerStore::erase(h, BrokerComm::data("two")); + Broker::erase(h, Broker::data("two")); do_exists("one"); do_exists("two"); do_exists("myset"); @@ -132,7 +132,7 @@ event test_erase() function do_lookup(key: string) { - when ( local res = BrokerStore::lookup(h, BrokerComm::data(key)) ) + when ( local res = Broker::lookup(h, Broker::data(key)) ) { print fmt("lookup(%s): %s", key, res); ++lookup_count; @@ -150,29 +150,29 @@ function do_lookup(key: string) } } -function dv(d: BrokerComm::Data): BrokerComm::DataVector +function dv(d: Broker::Data): Broker::DataVector { - local rval: BrokerComm::DataVector; + local rval: Broker::DataVector; rval[0] = d; return rval; } event bro_init() { - BrokerComm::enable(); + Broker::enable(); local myset: set[string] = {"a", "b", "c"}; local myvec: vector of string = {"alpha", "beta", "gamma"}; - h = BrokerStore::create_master("master"); - BrokerStore::insert(h, BrokerComm::data("one"), BrokerComm::data(110)); - BrokerStore::insert(h, BrokerComm::data("two"), BrokerComm::data(223)); - BrokerStore::insert(h, BrokerComm::data("myset"), BrokerComm::data(myset)); - BrokerStore::insert(h, BrokerComm::data("myvec"), BrokerComm::data(myvec)); - BrokerStore::increment(h, BrokerComm::data("one")); - BrokerStore::decrement(h, BrokerComm::data("two")); - BrokerStore::add_to_set(h, BrokerComm::data("myset"), BrokerComm::data("d")); - BrokerStore::remove_from_set(h, BrokerComm::data("myset"), BrokerComm::data("b")); - BrokerStore::push_left(h, BrokerComm::data("myvec"), dv(BrokerComm::data("delta"))); - BrokerStore::push_right(h, BrokerComm::data("myvec"), dv(BrokerComm::data("omega"))); + h = Broker::create_master("master"); + Broker::insert(h, Broker::data("one"), Broker::data(110)); + Broker::insert(h, Broker::data("two"), Broker::data(223)); + Broker::insert(h, Broker::data("myset"), Broker::data(myset)); + Broker::insert(h, Broker::data("myvec"), Broker::data(myvec)); + Broker::increment(h, Broker::data("one")); + Broker::decrement(h, Broker::data("two")); + Broker::add_to_set(h, Broker::data("myset"), Broker::data("d")); + Broker::remove_from_set(h, Broker::data("myset"), Broker::data("b")); + Broker::push_left(h, Broker::data("myvec"), dv(Broker::data("delta"))); + Broker::push_right(h, Broker::data("myvec"), dv(Broker::data("omega"))); do_lookup("one"); do_lookup("two"); do_lookup("myset"); diff --git a/testing/btest/broker/remote_event.test b/testing/btest/broker/remote_event.test index 6dbf8e77a0..5118f1a5e8 100644 --- a/testing/btest/broker/remote_event.test +++ b/testing/btest/broker/remote_event.test @@ -1,5 +1,5 @@ # @TEST-SERIALIZE: brokercomm -# @TEST-REQUIRES: grep -q ENABLE_BROKER $BUILD/CMakeCache.txt +# @TEST-REQUIRES: grep -q ENABLE_BROKER:BOOL=true $BUILD/CMakeCache.txt # @TEST-EXEC: btest-bg-run recv "bro -b ../recv.bro broker_port=$BROKER_PORT >recv.out" # @TEST-EXEC: btest-bg-run send "bro -b ../send.bro broker_port=$BROKER_PORT >send.out" @@ -18,10 +18,10 @@ global auto_event_handler: event(msg: string, c: count); event bro_init() { - BrokerComm::enable(); - BrokerComm::subscribe_to_events("bro/event/"); - BrokerComm::auto_event("bro/event/my_topic", auto_event_handler); - BrokerComm::listen(broker_port, "127.0.0.1"); + Broker::enable(); + Broker::subscribe_to_events("bro/event/"); + Broker::auto_event("bro/event/my_topic", auto_event_handler); + Broker::listen(broker_port, "127.0.0.1"); } global event_count = 0; @@ -39,8 +39,8 @@ event event_handler(msg: string, n: count) } event auto_event_handler(msg, n); - local args = BrokerComm::event_args(event_handler, "pong", n); - BrokerComm::event("bro/event/my_topic", args); + local args = Broker::event_args(event_handler, "pong", n); + Broker::send_event("bro/event/my_topic", args); } @TEST-END-FILE @@ -55,24 +55,24 @@ global auto_event_handler: event(msg: string, c: count); event bro_init() { - BrokerComm::enable(); - BrokerComm::subscribe_to_events("bro/event/my_topic"); - BrokerComm::connect("127.0.0.1", broker_port, 1secs); + Broker::enable(); + Broker::subscribe_to_events("bro/event/my_topic"); + Broker::connect("127.0.0.1", broker_port, 1secs); } global event_count = 0; -event BrokerComm::outgoing_connection_established(peer_address: string, +event Broker::outgoing_connection_established(peer_address: string, peer_port: port, peer_name: string) { - print "BrokerComm::outgoing_connection_established", peer_address, peer_port; - local args = BrokerComm::event_args(event_handler, "ping", event_count); - BrokerComm::event("bro/event/hi", args); + print "Broker::outgoing_connection_established", peer_address, peer_port; + local args = Broker::event_args(event_handler, "ping", event_count); + Broker::send_event("bro/event/hi", args); ++event_count; } -event BrokerComm::outgoing_connection_broken(peer_address: string, +event Broker::outgoing_connection_broken(peer_address: string, peer_port: port) { terminate(); @@ -81,8 +81,8 @@ event BrokerComm::outgoing_connection_broken(peer_address: string, event event_handler(msg: string, n: count) { print "got event msg", msg, n; - local args = BrokerComm::event_args(event_handler, "ping", event_count); - BrokerComm::event("bro/event/hi", args); + local args = Broker::event_args(event_handler, "ping", event_count); + Broker::send_event("bro/event/hi", args); ++event_count; } diff --git a/testing/btest/broker/remote_log.test b/testing/btest/broker/remote_log.test index d481f0ae25..5881ad6d92 100644 --- a/testing/btest/broker/remote_log.test +++ b/testing/btest/broker/remote_log.test @@ -1,5 +1,5 @@ # @TEST-SERIALIZE: brokercomm -# @TEST-REQUIRES: grep -q ENABLE_BROKER $BUILD/CMakeCache.txt +# @TEST-REQUIRES: grep -q ENABLE_BROKER:BOOL=true $BUILD/CMakeCache.txt # @TEST-EXEC: btest-bg-run recv "bro -b ../common.bro ../recv.bro broker_port=$BROKER_PORT >recv.out" # @TEST-EXEC: btest-bg-run send "bro -b ../common.bro ../send.bro broker_port=$BROKER_PORT >send.out" @@ -19,8 +19,8 @@ export { type Info: record { msg: string &log; - num: count &log; nolog: string &default="no"; + num: count &log; }; global log_test: event(rec: Test::Info); @@ -28,7 +28,7 @@ export { event bro_init() &priority=5 { - BrokerComm::enable(); + Broker::enable(); Log::create_stream(Test::LOG, [$columns=Test::Info, $ev=log_test]); } @@ -41,8 +41,8 @@ redef exit_only_after_terminate = T; event bro_init() { - BrokerComm::subscribe_to_logs("bro/log/"); - BrokerComm::listen(broker_port, "127.0.0.1"); + Broker::subscribe_to_logs("bro/log/"); + Broker::listen(broker_port, "127.0.0.1"); } event Test::log_test(rec: Test::Info) @@ -62,8 +62,8 @@ redef exit_only_after_terminate = T; event bro_init() { - BrokerComm::enable_remote_logs(Test::LOG); - BrokerComm::connect("127.0.0.1", broker_port, 1secs); + Broker::enable_remote_logs(Test::LOG); + Broker::connect("127.0.0.1", broker_port, 1secs); } global n = 0; @@ -80,15 +80,15 @@ event do_write() } } -event BrokerComm::outgoing_connection_established(peer_address: string, +event Broker::outgoing_connection_established(peer_address: string, peer_port: port, peer_name: string) { - print "BrokerComm::outgoing_connection_established", peer_address, peer_port; + print "Broker::outgoing_connection_established", peer_address, peer_port; event do_write(); } -event BrokerComm::outgoing_connection_broken(peer_address: string, +event Broker::outgoing_connection_broken(peer_address: string, peer_port: port) { terminate(); diff --git a/testing/btest/broker/remote_print.test b/testing/btest/broker/remote_print.test index b6430ec3be..c64e70fedc 100644 --- a/testing/btest/broker/remote_print.test +++ b/testing/btest/broker/remote_print.test @@ -1,5 +1,5 @@ # @TEST-SERIALIZE: brokercomm -# @TEST-REQUIRES: grep -q ENABLE_BROKER $BUILD/CMakeCache.txt +# @TEST-REQUIRES: grep -q ENABLE_BROKER:BOOL=true $BUILD/CMakeCache.txt # @TEST-EXEC: btest-bg-run recv "bro -b ../recv.bro broker_port=$BROKER_PORT >recv.out" # @TEST-EXEC: btest-bg-run send "bro -b ../send.bro broker_port=$BROKER_PORT >send.out" @@ -15,16 +15,16 @@ redef exit_only_after_terminate = T; event bro_init() { - BrokerComm::enable(); - BrokerComm::listen(broker_port, "127.0.0.1"); - BrokerComm::subscribe_to_prints("bro/print/"); + Broker::enable(); + Broker::subscribe_to_prints("bro/print/"); + Broker::listen(broker_port, "127.0.0.1"); } global messages_to_recv = 6; global messages_sent = 0; global messages_recv = 0; -event BrokerComm::print_handler(msg: string) +event Broker::print_handler(msg: string) { ++messages_recv; print "got print msg", msg; @@ -35,7 +35,7 @@ event BrokerComm::print_handler(msg: string) return; } - BrokerComm::print("bro/print/my_topic", fmt("pong %d", messages_sent)); + Broker::send_print("bro/print/my_topic", fmt("pong %d", messages_sent)); ++messages_sent; } @@ -48,35 +48,35 @@ redef exit_only_after_terminate = T; event bro_init() { - BrokerComm::enable(); - BrokerComm::subscribe_to_prints("bro/print/my_topic"); - BrokerComm::connect("127.0.0.1", broker_port, 1secs); + Broker::enable(); + Broker::subscribe_to_prints("bro/print/my_topic"); + Broker::connect("127.0.0.1", broker_port, 1secs); } global messages_sent = 0; global messages_recv = 0; global peer_disconnected = F; -event BrokerComm::outgoing_connection_established(peer_address: string, +event Broker::outgoing_connection_established(peer_address: string, peer_port: port, peer_name: string) { - print "BrokerComm::outgoing_connection_established", peer_address, peer_port; - BrokerComm::print("bro/print/hi", fmt("ping %d", messages_sent)); + print "Broker::outgoing_connection_established", peer_address, peer_port; + Broker::send_print("bro/print/hi", fmt("ping %d", messages_sent)); ++messages_sent; } -event BrokerComm::outgoing_connection_broken(peer_address: string, +event Broker::outgoing_connection_broken(peer_address: string, peer_port: port) { terminate(); } -event BrokerComm::print_handler(msg: string) +event Broker::print_handler(msg: string) { ++messages_recv; print "got print msg", msg; - BrokerComm::print("bro/print/hi", fmt("ping %d", messages_sent)); + Broker::send_print("bro/print/hi", fmt("ping %d", messages_sent)); ++messages_sent; } diff --git a/testing/btest/core/leaks/broker/clone_store.bro b/testing/btest/core/leaks/broker/clone_store.bro index 06df81e1d5..c3b11a7a0d 100644 --- a/testing/btest/core/leaks/broker/clone_store.bro +++ b/testing/btest/core/leaks/broker/clone_store.bro @@ -1,5 +1,5 @@ # @TEST-SERIALIZE: brokercomm -# @TEST-REQUIRES: grep -q ENABLE_BROKER $BUILD/CMakeCache.txt +# @TEST-REQUIRES: grep -q ENABLE_BROKER:BOOL=true $BUILD/CMakeCache.txt # @TEST-REQUIRES: bro --help 2>&1 | grep -q mem-leaks # @TEST-GROUP: leak @@ -14,13 +14,13 @@ const broker_port: port &redef; redef exit_only_after_terminate = T; -global h: opaque of BrokerStore::Handle; +global h: opaque of Broker::Handle; global expected_key_count = 4; global key_count = 0; function do_lookup(key: string) { - when ( local res = BrokerStore::lookup(h, BrokerComm::data(key)) ) + when ( local res = Broker::lookup(h, Broker::data(key)) ) { ++key_count; print "lookup", key, res; @@ -34,15 +34,15 @@ function do_lookup(key: string) event ready() { - h = BrokerStore::create_clone("mystore"); + h = Broker::create_clone("mystore"); - when ( local res = BrokerStore::keys(h) ) + when ( local res = Broker::keys(h) ) { print "clone keys", res; - do_lookup(BrokerComm::refine_to_string(BrokerComm::vector_lookup(res$result, 0))); - do_lookup(BrokerComm::refine_to_string(BrokerComm::vector_lookup(res$result, 1))); - do_lookup(BrokerComm::refine_to_string(BrokerComm::vector_lookup(res$result, 2))); - do_lookup(BrokerComm::refine_to_string(BrokerComm::vector_lookup(res$result, 3))); + do_lookup(Broker::refine_to_string(Broker::vector_lookup(res$result, 0))); + do_lookup(Broker::refine_to_string(Broker::vector_lookup(res$result, 1))); + do_lookup(Broker::refine_to_string(Broker::vector_lookup(res$result, 2))); + do_lookup(Broker::refine_to_string(Broker::vector_lookup(res$result, 3))); } timeout 10sec { print "timeout"; } @@ -50,9 +50,9 @@ event ready() event bro_init() { - BrokerComm::enable(); - BrokerComm::listen(broker_port, "127.0.0.1"); - BrokerComm::subscribe_to_events("bro/event/ready"); + Broker::enable(); + Broker::subscribe_to_events("bro/event/ready"); + Broker::listen(broker_port, "127.0.0.1"); } @TEST-END-FILE @@ -62,41 +62,41 @@ event bro_init() const broker_port: port &redef; redef exit_only_after_terminate = T; -global h: opaque of BrokerStore::Handle; +global h: opaque of Broker::Handle; -function dv(d: BrokerComm::Data): BrokerComm::DataVector +function dv(d: Broker::Data): Broker::DataVector { - local rval: BrokerComm::DataVector; + local rval: Broker::DataVector; rval[0] = d; return rval; } global ready: event(); -event BrokerComm::outgoing_connection_broken(peer_address: string, +event Broker::outgoing_connection_broken(peer_address: string, peer_port: port) { terminate(); } -event BrokerComm::outgoing_connection_established(peer_address: string, +event Broker::outgoing_connection_established(peer_address: string, peer_port: port, peer_name: string) { local myset: set[string] = {"a", "b", "c"}; local myvec: vector of string = {"alpha", "beta", "gamma"}; - BrokerStore::insert(h, BrokerComm::data("one"), BrokerComm::data(110)); - BrokerStore::insert(h, BrokerComm::data("two"), BrokerComm::data(223)); - BrokerStore::insert(h, BrokerComm::data("myset"), BrokerComm::data(myset)); - BrokerStore::insert(h, BrokerComm::data("myvec"), BrokerComm::data(myvec)); - BrokerStore::increment(h, BrokerComm::data("one")); - BrokerStore::decrement(h, BrokerComm::data("two")); - BrokerStore::add_to_set(h, BrokerComm::data("myset"), BrokerComm::data("d")); - BrokerStore::remove_from_set(h, BrokerComm::data("myset"), BrokerComm::data("b")); - BrokerStore::push_left(h, BrokerComm::data("myvec"), dv(BrokerComm::data("delta"))); - BrokerStore::push_right(h, BrokerComm::data("myvec"), dv(BrokerComm::data("omega"))); + Broker::insert(h, Broker::data("one"), Broker::data(110)); + Broker::insert(h, Broker::data("two"), Broker::data(223)); + Broker::insert(h, Broker::data("myset"), Broker::data(myset)); + Broker::insert(h, Broker::data("myvec"), Broker::data(myvec)); + Broker::increment(h, Broker::data("one")); + Broker::decrement(h, Broker::data("two")); + Broker::add_to_set(h, Broker::data("myset"), Broker::data("d")); + Broker::remove_from_set(h, Broker::data("myset"), Broker::data("b")); + Broker::push_left(h, Broker::data("myvec"), dv(Broker::data("delta"))); + Broker::push_right(h, Broker::data("myvec"), dv(Broker::data("omega"))); - when ( local res = BrokerStore::size(h) ) + when ( local res = Broker::size(h) ) { event ready(); } timeout 10sec { print "timeout"; } @@ -104,10 +104,10 @@ event BrokerComm::outgoing_connection_established(peer_address: string, event bro_init() { - BrokerComm::enable(); - h = BrokerStore::create_master("mystore"); - BrokerComm::connect("127.0.0.1", broker_port, 1secs); - BrokerComm::auto_event("bro/event/ready", ready); + Broker::enable(); + Broker::auto_event("bro/event/ready", ready); + h = Broker::create_master("mystore"); + Broker::connect("127.0.0.1", broker_port, 1secs); } @TEST-END-FILE diff --git a/testing/btest/core/leaks/broker/data.bro b/testing/btest/core/leaks/broker/data.bro index d4f6402ae3..d67c879fbf 100644 --- a/testing/btest/core/leaks/broker/data.bro +++ b/testing/btest/core/leaks/broker/data.bro @@ -1,4 +1,4 @@ -# @TEST-REQUIRES: grep -q ENABLE_BROKER $BUILD/CMakeCache.txt +# @TEST-REQUIRES: grep -q ENABLE_BROKER:BOOL=true $BUILD/CMakeCache.txt # @TEST-REQUIRES: bro --help 2>&1 | grep -q mem-leaks # @TEST-GROUP: leaks @@ -16,218 +16,251 @@ type bro_record : record { c: count; }; -function comm_record_to_bro_record_recurse(it: opaque of BrokerComm::RecordIterator, +function broker_to_bro_record_recurse(it: opaque of Broker::RecordIterator, rval: bro_record, idx: count): bro_record { - if ( BrokerComm::record_iterator_last(it) ) + if ( Broker::record_iterator_last(it) ) return rval; - local field_value = BrokerComm::record_iterator_value(it); + local field_value = Broker::record_iterator_value(it); if ( field_value?$d ) switch ( idx ) { case 0: - rval$a = BrokerComm::refine_to_string(field_value); + rval$a = Broker::refine_to_string(field_value); break; case 1: - rval$b = BrokerComm::refine_to_string(field_value); + rval$b = Broker::refine_to_string(field_value); break; case 2: - rval$c = BrokerComm::refine_to_count(field_value); + rval$c = Broker::refine_to_count(field_value); break; }; ++idx; - BrokerComm::record_iterator_next(it); - return comm_record_to_bro_record_recurse(it, rval, idx); + Broker::record_iterator_next(it); + return broker_to_bro_record_recurse(it, rval, idx); } -function comm_record_to_bro_record(d: BrokerComm::Data): bro_record +function broker_to_bro_record(d: Broker::Data): bro_record { - return comm_record_to_bro_record_recurse(BrokerComm::record_iterator(d), + return broker_to_bro_record_recurse(Broker::record_iterator(d), bro_record($c = 0), 0); } function -comm_set_to_bro_set_recurse(it: opaque of BrokerComm::SetIterator, +broker_to_bro_set_recurse(it: opaque of Broker::SetIterator, rval: bro_set): bro_set { - if ( BrokerComm::set_iterator_last(it) ) + if ( Broker::set_iterator_last(it) ) return rval; - add rval[BrokerComm::refine_to_string(BrokerComm::set_iterator_value(it))]; - BrokerComm::set_iterator_next(it); - return comm_set_to_bro_set_recurse(it, rval); + add rval[Broker::refine_to_string(Broker::set_iterator_value(it))]; + Broker::set_iterator_next(it); + return broker_to_bro_set_recurse(it, rval); } -function comm_set_to_bro_set(d: BrokerComm::Data): bro_set +function broker_to_bro_set(d: Broker::Data): bro_set { - return comm_set_to_bro_set_recurse(BrokerComm::set_iterator(d), bro_set()); + return broker_to_bro_set_recurse(Broker::set_iterator(d), bro_set()); } function -comm_table_to_bro_table_recurse(it: opaque of BrokerComm::TableIterator, +broker_to_bro_table_recurse(it: opaque of Broker::TableIterator, rval: bro_table): bro_table { - if ( BrokerComm::table_iterator_last(it) ) + if ( Broker::table_iterator_last(it) ) return rval; - local item = BrokerComm::table_iterator_value(it); - rval[BrokerComm::refine_to_string(item$key)] = BrokerComm::refine_to_count(item$val); - BrokerComm::table_iterator_next(it); - return comm_table_to_bro_table_recurse(it, rval); + local item = Broker::table_iterator_value(it); + rval[Broker::refine_to_string(item$key)] = Broker::refine_to_count(item$val); + Broker::table_iterator_next(it); + return broker_to_bro_table_recurse(it, rval); } -function comm_table_to_bro_table(d: BrokerComm::Data): bro_table +function broker_to_bro_table(d: Broker::Data): bro_table { - return comm_table_to_bro_table_recurse(BrokerComm::table_iterator(d), + return broker_to_bro_table_recurse(Broker::table_iterator(d), bro_table()); } -function comm_vector_to_bro_vector_recurse(it: opaque of BrokerComm::VectorIterator, +function broker_to_bro_vector_recurse(it: opaque of Broker::VectorIterator, rval: bro_vector): bro_vector { - if ( BrokerComm::vector_iterator_last(it) ) + if ( Broker::vector_iterator_last(it) ) return rval; - rval[|rval|] = BrokerComm::refine_to_string(BrokerComm::vector_iterator_value(it)); - BrokerComm::vector_iterator_next(it); - return comm_vector_to_bro_vector_recurse(it, rval); + rval[|rval|] = Broker::refine_to_string(Broker::vector_iterator_value(it)); + Broker::vector_iterator_next(it); + return broker_to_bro_vector_recurse(it, rval); } -function comm_vector_to_bro_vector(d: BrokerComm::Data): bro_vector +function broker_to_bro_vector(d: Broker::Data): bro_vector { - return comm_vector_to_bro_vector_recurse(BrokerComm::vector_iterator(d), + return broker_to_bro_vector_recurse(Broker::vector_iterator(d), bro_vector()); } event bro_init() - { -BrokerComm::enable(); - } +{ +Broker::enable(); +} global did_it = F; event new_connection(c: connection) - { +{ if ( did_it ) return; did_it = T; -print BrokerComm::data_type(BrokerComm::data(T)); -print BrokerComm::data_type(BrokerComm::data(+1)); -print BrokerComm::data_type(BrokerComm::data(1)); -print BrokerComm::data_type(BrokerComm::data(1.1)); -print BrokerComm::data_type(BrokerComm::data("1 (how creative)")); -print BrokerComm::data_type(BrokerComm::data(1.1.1.1)); -print BrokerComm::data_type(BrokerComm::data(1.1.1.1/1)); -print BrokerComm::data_type(BrokerComm::data(1/udp)); -print BrokerComm::data_type(BrokerComm::data(double_to_time(1))); -print BrokerComm::data_type(BrokerComm::data(1sec)); -print BrokerComm::data_type(BrokerComm::data(BrokerComm::BOOL)); + +### Print every broker data type + +print Broker::data_type(Broker::data(T)); +print Broker::data_type(Broker::data(+1)); +print Broker::data_type(Broker::data(1)); +print Broker::data_type(Broker::data(1.1)); +print Broker::data_type(Broker::data("1 (how creative)")); +print Broker::data_type(Broker::data(1.1.1.1)); +print Broker::data_type(Broker::data(1.1.1.1/1)); +print Broker::data_type(Broker::data(1/udp)); +print Broker::data_type(Broker::data(double_to_time(1))); +print Broker::data_type(Broker::data(1sec)); +print Broker::data_type(Broker::data(Broker::BOOL)); local s: bro_set = bro_set("one", "two", "three"); local t: bro_table = bro_table(["one"] = 1, ["two"] = 2, ["three"] = 3); local v: bro_vector = bro_vector("zero", "one", "two"); local r: bro_record = bro_record($c = 1); -print BrokerComm::data_type(BrokerComm::data(s)); -print BrokerComm::data_type(BrokerComm::data(t)); -print BrokerComm::data_type(BrokerComm::data(v)); -print BrokerComm::data_type(BrokerComm::data(r)); +print Broker::data_type(Broker::data(s)); +print Broker::data_type(Broker::data(t)); +print Broker::data_type(Broker::data(v)); +print Broker::data_type(Broker::data(r)); print "***************************"; -print BrokerComm::refine_to_bool(BrokerComm::data(T)); -print BrokerComm::refine_to_bool(BrokerComm::data(F)); -print BrokerComm::refine_to_int(BrokerComm::data(+1)); -print BrokerComm::refine_to_int(BrokerComm::data(+0)); -print BrokerComm::refine_to_int(BrokerComm::data(-1)); -print BrokerComm::refine_to_count(BrokerComm::data(1)); -print BrokerComm::refine_to_count(BrokerComm::data(0)); -print BrokerComm::refine_to_double(BrokerComm::data(1.1)); -print BrokerComm::refine_to_double(BrokerComm::data(-11.1)); -print BrokerComm::refine_to_string(BrokerComm::data("hello")); -print BrokerComm::refine_to_addr(BrokerComm::data(1.2.3.4)); -print BrokerComm::refine_to_subnet(BrokerComm::data(192.168.1.1/16)); -print BrokerComm::refine_to_port(BrokerComm::data(22/tcp)); -print BrokerComm::refine_to_time(BrokerComm::data(double_to_time(42))); -print BrokerComm::refine_to_interval(BrokerComm::data(3min)); -print BrokerComm::refine_to_enum_name(BrokerComm::data(BrokerComm::BOOL)); +### Convert a Bro value to a broker value, then print the result -print "***************************"; +print Broker::refine_to_bool(Broker::data(T)); +print Broker::refine_to_bool(Broker::data(F)); +print Broker::refine_to_int(Broker::data(+1)); +print Broker::refine_to_int(Broker::data(+0)); +print Broker::refine_to_int(Broker::data(-1)); +print Broker::refine_to_count(Broker::data(1)); +print Broker::refine_to_count(Broker::data(0)); +print Broker::refine_to_double(Broker::data(1.1)); +print Broker::refine_to_double(Broker::data(-11.1)); +print Broker::refine_to_string(Broker::data("hello")); +print Broker::refine_to_addr(Broker::data(1.2.3.4)); +print Broker::refine_to_subnet(Broker::data(192.168.1.1/16)); +print Broker::refine_to_port(Broker::data(22/tcp)); +print Broker::refine_to_time(Broker::data(double_to_time(42))); +print Broker::refine_to_interval(Broker::data(3min)); +print Broker::refine_to_enum_name(Broker::data(Broker::BOOL)); -local cs = BrokerComm::data(s); -print comm_set_to_bro_set(cs); -cs = BrokerComm::set_create(); -print BrokerComm::set_size(cs); -print BrokerComm::set_insert(cs, BrokerComm::data("hi")); -print BrokerComm::set_size(cs); -print BrokerComm::set_contains(cs, BrokerComm::data("hi")); -print BrokerComm::set_contains(cs, BrokerComm::data("bye")); -print BrokerComm::set_insert(cs, BrokerComm::data("bye")); -print BrokerComm::set_size(cs); -print BrokerComm::set_remove(cs, BrokerComm::data("hi")); -print BrokerComm::set_size(cs); -print BrokerComm::set_remove(cs, BrokerComm::data("hi")); -print comm_set_to_bro_set(cs); -BrokerComm::set_clear(cs); -print BrokerComm::set_size(cs); +local cs = Broker::data(s); +print broker_to_bro_set(cs); -print "***************************"; +local ct = Broker::data(t); +print broker_to_bro_table(ct); -local ct = BrokerComm::data(t); -print comm_table_to_bro_table(ct); -ct = BrokerComm::table_create(); -print BrokerComm::table_size(ct); -print BrokerComm::table_insert(ct, BrokerComm::data("hi"), BrokerComm::data(42)); -print BrokerComm::table_size(ct); -print BrokerComm::table_contains(ct, BrokerComm::data("hi")); -print BrokerComm::refine_to_count(BrokerComm::table_lookup(ct, BrokerComm::data("hi"))); -print BrokerComm::table_contains(ct, BrokerComm::data("bye")); -print BrokerComm::table_insert(ct, BrokerComm::data("bye"), BrokerComm::data(7)); -print BrokerComm::table_size(ct); -print BrokerComm::table_insert(ct, BrokerComm::data("bye"), BrokerComm::data(37)); -print BrokerComm::table_size(ct); -print BrokerComm::refine_to_count(BrokerComm::table_lookup(ct, BrokerComm::data("bye"))); -print BrokerComm::table_remove(ct, BrokerComm::data("hi")); -print BrokerComm::table_size(ct); +local cv = Broker::data(v); +print broker_to_bro_vector(cv); -print "***************************"; +local cr = Broker::data(r); +print broker_to_bro_record(cr); -local cv = BrokerComm::data(v); -print comm_vector_to_bro_vector(cv); -cv = BrokerComm::vector_create(); -print BrokerComm::vector_size(cv); -print BrokerComm::vector_insert(cv, BrokerComm::data("hi"), 0); -print BrokerComm::vector_insert(cv, BrokerComm::data("hello"), 1); -print BrokerComm::vector_insert(cv, BrokerComm::data("greetings"), 2); -print BrokerComm::vector_insert(cv, BrokerComm::data("salutations"), 1); -print comm_vector_to_bro_vector(cv); -print BrokerComm::vector_size(cv); -print BrokerComm::vector_replace(cv, BrokerComm::data("bah"), 2); -print BrokerComm::vector_lookup(cv, 2); -print BrokerComm::vector_lookup(cv, 0); -print comm_vector_to_bro_vector(cv); -print BrokerComm::vector_remove(cv, 2); -print comm_vector_to_bro_vector(cv); -print BrokerComm::vector_size(cv); - -print "***************************"; - -local cr = BrokerComm::data(r); -print comm_record_to_bro_record(cr); r$a = "test"; -cr = BrokerComm::data(r); -print comm_record_to_bro_record(cr); +cr = Broker::data(r); +print broker_to_bro_record(cr); + r$b = "testagain"; -cr = BrokerComm::data(r); -print comm_record_to_bro_record(cr); -cr = BrokerComm::record_create(3); -print BrokerComm::record_size(cr); -print BrokerComm::record_assign(cr, BrokerComm::data("hi"), 0); -print BrokerComm::record_assign(cr, BrokerComm::data("hello"), 1); -print BrokerComm::record_assign(cr, BrokerComm::data(37), 2); -print BrokerComm::record_lookup(cr, 0); -print BrokerComm::record_lookup(cr, 1); -print BrokerComm::record_lookup(cr, 2); -print BrokerComm::record_size(cr); +cr = Broker::data(r); +print broker_to_bro_record(cr); + +print "***************************"; + +### Test the broker set BIFs + +cs = Broker::set_create(); +print Broker::set_size(cs); +print Broker::set_insert(cs, Broker::data("hi")); +print Broker::set_size(cs); +print Broker::set_contains(cs, Broker::data("hi")); +print Broker::set_contains(cs, Broker::data("bye")); +print Broker::set_insert(cs, Broker::data("bye")); +print Broker::set_size(cs); +print Broker::set_insert(cs, Broker::data("bye")); +print Broker::set_size(cs); +print Broker::set_remove(cs, Broker::data("hi")); +print Broker::set_size(cs); +print Broker::set_remove(cs, Broker::data("hi")); +print broker_to_bro_set(cs); +print Broker::set_clear(cs); +print Broker::set_size(cs); +print broker_to_bro_set(cs); + +print "***************************"; + +### Test the broker table BIFs + +ct = Broker::table_create(); +print Broker::table_size(ct); +print Broker::table_insert(ct, Broker::data("hi"), Broker::data(42)); +print Broker::table_size(ct); +print Broker::table_contains(ct, Broker::data("hi")); +print Broker::refine_to_count(Broker::table_lookup(ct, Broker::data("hi"))); +print Broker::table_contains(ct, Broker::data("bye")); +print Broker::table_insert(ct, Broker::data("bye"), Broker::data(7)); +print Broker::table_size(ct); +print Broker::table_insert(ct, Broker::data("bye"), Broker::data(37)); +print Broker::table_size(ct); +print Broker::refine_to_count(Broker::table_lookup(ct, Broker::data("bye"))); +print Broker::table_remove(ct, Broker::data("hi")); +print Broker::table_size(ct); +print Broker::table_remove(ct, Broker::data("hi")); +print Broker::table_size(ct); +print Broker::table_clear(ct); +print Broker::table_size(ct); +print broker_to_bro_table(ct); + +print "***************************"; + +### Test the broker vector BIFs + +cv = Broker::vector_create(); +print Broker::vector_size(cv); +print Broker::vector_insert(cv, Broker::data("hi"), 0); +print Broker::vector_insert(cv, Broker::data("hello"), 1); +print Broker::vector_insert(cv, Broker::data("greetings"), 2); +print Broker::vector_insert(cv, Broker::data("salutations"), 1); +print broker_to_bro_vector(cv); +print Broker::vector_size(cv); +print Broker::vector_replace(cv, Broker::data("bah"), 2); +print Broker::vector_lookup(cv, 2); +print Broker::vector_lookup(cv, 0); +print broker_to_bro_vector(cv); +print Broker::vector_remove(cv, 2); +print broker_to_bro_vector(cv); +print Broker::vector_size(cv); +print Broker::vector_clear(cv); +print Broker::vector_size(cv); +print broker_to_bro_vector(cv); + +print "***************************"; + +### Test the broker record BIFs + +cr = Broker::record_create(3); +print Broker::record_size(cr); +print Broker::record_assign(cr, Broker::data("hi"), 0); +print Broker::record_assign(cr, Broker::data("hello"), 1); +print Broker::record_assign(cr, Broker::data(37), 2); +print Broker::record_lookup(cr, 0); +print Broker::record_lookup(cr, 1); +print Broker::record_lookup(cr, 2); +print Broker::record_size(cr); +print Broker::record_assign(cr, Broker::data("goodbye"), 1); +print Broker::record_size(cr); +print Broker::record_lookup(cr, 1); } diff --git a/testing/btest/core/leaks/broker/master_store.bro b/testing/btest/core/leaks/broker/master_store.bro index 19c63236f5..11f32b49ae 100644 --- a/testing/btest/core/leaks/broker/master_store.bro +++ b/testing/btest/core/leaks/broker/master_store.bro @@ -1,4 +1,4 @@ -# @TEST-REQUIRES: grep -q ENABLE_BROKER $BUILD/CMakeCache.txt +# @TEST-REQUIRES: grep -q ENABLE_BROKER:BOOL=true $BUILD/CMakeCache.txt # @TEST-REQUIRES: bro --help 2>&1 | grep -q mem-leaks # @TEST-GROUP: leaks @@ -8,7 +8,7 @@ redef exit_only_after_terminate = T; -global h: opaque of BrokerStore::Handle; +global h: opaque of Broker::Handle; global lookup_count = 0; const lookup_expect_count = 5; global exists_count = 0; @@ -20,13 +20,13 @@ global test_size: event(where: string &default = ""); event test_clear() { - BrokerStore::clear(h); + Broker::clear(h); event test_size("after clear"); } event test_size(where: string) { - when ( local res = BrokerStore::size(h) ) + when ( local res = Broker::size(h) ) { if ( where == "" ) { @@ -45,7 +45,7 @@ event test_size(where: string) event test_keys() { - when ( local res = BrokerStore::keys(h) ) + when ( local res = Broker::keys(h) ) { print fmt("keys: %s", res); event test_size(); @@ -56,7 +56,7 @@ event test_keys() event test_pop(key: string) { - when ( local lres = BrokerStore::pop_left(h, BrokerComm::data(key)) ) + when ( local lres = Broker::pop_left(h, Broker::data(key)) ) { print fmt("pop_left(%s): %s", key, lres); ++pop_count; @@ -67,7 +67,7 @@ event test_pop(key: string) timeout 10sec { print "timeout"; } - when ( local rres = BrokerStore::pop_right(h, BrokerComm::data(key)) ) + when ( local rres = Broker::pop_right(h, Broker::data(key)) ) { print fmt("pop_right(%s): %s", key, rres); ++pop_count; @@ -81,7 +81,7 @@ event test_pop(key: string) function do_exists(key: string) { - when ( local res = BrokerStore::exists(h, BrokerComm::data(key)) ) + when ( local res = Broker::exists(h, Broker::data(key)) ) { print fmt("exists(%s): %s", key, res); ++exists_count; @@ -95,7 +95,7 @@ function do_exists(key: string) event test_erase() { - BrokerStore::erase(h, BrokerComm::data("two")); + Broker::erase(h, Broker::data("two")); do_exists("one"); do_exists("two"); do_exists("myset"); @@ -104,7 +104,7 @@ event test_erase() function do_lookup(key: string) { - when ( local res = BrokerStore::lookup(h, BrokerComm::data(key)) ) + when ( local res = Broker::lookup(h, Broker::data(key)) ) { print fmt("lookup(%s): %s", key, res); ++lookup_count; @@ -116,9 +116,9 @@ function do_lookup(key: string) { print "timeout"; } } -function dv(d: BrokerComm::Data): BrokerComm::DataVector +function dv(d: Broker::Data): Broker::DataVector { - local rval: BrokerComm::DataVector; + local rval: Broker::DataVector; rval[0] = d; return rval; } @@ -127,8 +127,8 @@ global did_it = F; event bro_init() { - BrokerComm::enable(); - h = BrokerStore::create_master("master"); + Broker::enable(); + h = Broker::create_master("master"); } event new_connection(c: connection) @@ -137,16 +137,16 @@ event new_connection(c: connection) did_it = T; local myset: set[string] = {"a", "b", "c"}; local myvec: vector of string = {"alpha", "beta", "gamma"}; - BrokerStore::insert(h, BrokerComm::data("one"), BrokerComm::data(110)); - BrokerStore::insert(h, BrokerComm::data("two"), BrokerComm::data(223)); - BrokerStore::insert(h, BrokerComm::data("myset"), BrokerComm::data(myset)); - BrokerStore::insert(h, BrokerComm::data("myvec"), BrokerComm::data(myvec)); - BrokerStore::increment(h, BrokerComm::data("one")); - BrokerStore::decrement(h, BrokerComm::data("two")); - BrokerStore::add_to_set(h, BrokerComm::data("myset"), BrokerComm::data("d")); - BrokerStore::remove_from_set(h, BrokerComm::data("myset"), BrokerComm::data("b")); - BrokerStore::push_left(h, BrokerComm::data("myvec"), dv(BrokerComm::data("delta"))); - BrokerStore::push_right(h, BrokerComm::data("myvec"), dv(BrokerComm::data("omega"))); + Broker::insert(h, Broker::data("one"), Broker::data(110)); + Broker::insert(h, Broker::data("two"), Broker::data(223)); + Broker::insert(h, Broker::data("myset"), Broker::data(myset)); + Broker::insert(h, Broker::data("myvec"), Broker::data(myvec)); + Broker::increment(h, Broker::data("one")); + Broker::decrement(h, Broker::data("two")); + Broker::add_to_set(h, Broker::data("myset"), Broker::data("d")); + Broker::remove_from_set(h, Broker::data("myset"), Broker::data("b")); + Broker::push_left(h, Broker::data("myvec"), dv(Broker::data("delta"))); + Broker::push_right(h, Broker::data("myvec"), dv(Broker::data("omega"))); do_lookup("one"); do_lookup("two"); do_lookup("myset"); diff --git a/testing/btest/core/leaks/broker/remote_event.test b/testing/btest/core/leaks/broker/remote_event.test index 243d3b04d3..3f63fcba76 100644 --- a/testing/btest/core/leaks/broker/remote_event.test +++ b/testing/btest/core/leaks/broker/remote_event.test @@ -1,5 +1,5 @@ # @TEST-SERIALIZE: brokercomm -# @TEST-REQUIRES: grep -q ENABLE_BROKER $BUILD/CMakeCache.txt +# @TEST-REQUIRES: grep -q ENABLE_BROKER:BOOL=true $BUILD/CMakeCache.txt # @TEST-REQUIRES: bro --help 2>&1 | grep -q mem-leaks # @TEST-GROUP: leak @@ -20,10 +20,10 @@ global auto_event_handler: event(msg: string, c: count); event bro_init() { - BrokerComm::enable(); - BrokerComm::listen(broker_port, "127.0.0.1"); - BrokerComm::subscribe_to_events("bro/event/"); - BrokerComm::auto_event("bro/event/my_topic", auto_event_handler); + Broker::enable(); + Broker::subscribe_to_events("bro/event/"); + Broker::auto_event("bro/event/my_topic", auto_event_handler); + Broker::listen(broker_port, "127.0.0.1"); } global event_count = 0; @@ -41,8 +41,8 @@ event event_handler(msg: string, n: count) } event auto_event_handler(msg, n); - local args = BrokerComm::event_args(event_handler, "pong", n); - BrokerComm::event("bro/event/my_topic", args); + local args = Broker::event_args(event_handler, "pong", n); + Broker::send_event("bro/event/my_topic", args); } @TEST-END-FILE @@ -57,24 +57,24 @@ global auto_event_handler: event(msg: string, c: count); event bro_init() { - BrokerComm::enable(); - BrokerComm::subscribe_to_events("bro/event/my_topic"); - BrokerComm::connect("127.0.0.1", broker_port, 1secs); + Broker::enable(); + Broker::subscribe_to_events("bro/event/my_topic"); + Broker::connect("127.0.0.1", broker_port, 1secs); } global event_count = 0; -event BrokerComm::outgoing_connection_established(peer_address: string, +event Broker::outgoing_connection_established(peer_address: string, peer_port: port, peer_name: string) { - print "BrokerComm::outgoing_connection_established", peer_address, peer_port; - local args = BrokerComm::event_args(event_handler, "ping", event_count); - BrokerComm::event("bro/event/hi", args); + print "Broker::outgoing_connection_established", peer_address, peer_port; + local args = Broker::event_args(event_handler, "ping", event_count); + Broker::send_event("bro/event/hi", args); ++event_count; } -event BrokerComm::outgoing_connection_broken(peer_address: string, +event Broker::outgoing_connection_broken(peer_address: string, peer_port: port) { terminate(); @@ -83,8 +83,8 @@ event BrokerComm::outgoing_connection_broken(peer_address: string, event event_handler(msg: string, n: count) { print "got event msg", msg, n; - local args = BrokerComm::event_args(event_handler, "ping", event_count); - BrokerComm::event("bro/event/hi", args); + local args = Broker::event_args(event_handler, "ping", event_count); + Broker::send_event("bro/event/hi", args); ++event_count; } diff --git a/testing/btest/core/leaks/broker/remote_log.test b/testing/btest/core/leaks/broker/remote_log.test index f6c0c41fda..baeab906f1 100644 --- a/testing/btest/core/leaks/broker/remote_log.test +++ b/testing/btest/core/leaks/broker/remote_log.test @@ -1,5 +1,5 @@ # @TEST-SERIALIZE: brokercomm -# @TEST-REQUIRES: grep -q ENABLE_BROKER $BUILD/CMakeCache.txt +# @TEST-REQUIRES: grep -q ENABLE_BROKER:BOOL=true $BUILD/CMakeCache.txt # @TEST-REQUIRES: bro --help 2>&1 | grep -q mem-leaks # @TEST-GROUP: leak @@ -29,7 +29,7 @@ export { event bro_init() &priority=5 { - BrokerComm::enable(); + Broker::enable(); Log::create_stream(Test::LOG, [$columns=Test::Info, $ev=log_test]); } @@ -42,8 +42,8 @@ redef exit_only_after_terminate = T; event bro_init() { - BrokerComm::listen(broker_port, "127.0.0.1"); - BrokerComm::subscribe_to_logs("bro/log/"); + Broker::subscribe_to_logs("bro/log/"); + Broker::listen(broker_port, "127.0.0.1"); } event Test::log_test(rec: Test::Info) @@ -63,8 +63,8 @@ redef exit_only_after_terminate = T; event bro_init() { - BrokerComm::enable_remote_logs(Test::LOG); - BrokerComm::connect("127.0.0.1", broker_port, 1secs); + Broker::enable_remote_logs(Test::LOG); + Broker::connect("127.0.0.1", broker_port, 1secs); } global n = 0; @@ -81,15 +81,15 @@ event do_write() } } -event BrokerComm::outgoing_connection_established(peer_address: string, +event Broker::outgoing_connection_established(peer_address: string, peer_port: port, peer_name: string) { - print "BrokerComm::outgoing_connection_established", peer_address, peer_port; + print "Broker::outgoing_connection_established", peer_address, peer_port; event do_write(); } -event BrokerComm::outgoing_connection_broken(peer_address: string, +event Broker::outgoing_connection_broken(peer_address: string, peer_port: port) { terminate(); diff --git a/testing/btest/core/leaks/broker/remote_print.test b/testing/btest/core/leaks/broker/remote_print.test index e77881c694..26e6317034 100644 --- a/testing/btest/core/leaks/broker/remote_print.test +++ b/testing/btest/core/leaks/broker/remote_print.test @@ -1,5 +1,5 @@ # @TEST-SERIALIZE: brokercomm -# @TEST-REQUIRES: grep -q ENABLE_BROKER $BUILD/CMakeCache.txt +# @TEST-REQUIRES: grep -q ENABLE_BROKER:BOOL=true $BUILD/CMakeCache.txt # @TEST-REQUIRES: bro --help 2>&1 | grep -q mem-leaks # @TEST-GROUP: leak @@ -17,16 +17,16 @@ redef exit_only_after_terminate = T; event bro_init() { - BrokerComm::enable(); - BrokerComm::listen(broker_port, "127.0.0.1"); - BrokerComm::subscribe_to_prints("bro/print/"); + Broker::enable(); + Broker::subscribe_to_prints("bro/print/"); + Broker::listen(broker_port, "127.0.0.1"); } global messages_to_recv = 6; global messages_sent = 0; global messages_recv = 0; -event BrokerComm::print_handler(msg: string) +event Broker::print_handler(msg: string) { ++messages_recv; print "got print msg", msg; @@ -37,7 +37,7 @@ event BrokerComm::print_handler(msg: string) return; } - BrokerComm::print("bro/print/my_topic", fmt("pong %d", messages_sent)); + Broker::send_print("bro/print/my_topic", fmt("pong %d", messages_sent)); ++messages_sent; } @@ -50,35 +50,35 @@ redef exit_only_after_terminate = T; event bro_init() { - BrokerComm::enable(); - BrokerComm::subscribe_to_prints("bro/print/my_topic"); - BrokerComm::connect("127.0.0.1", broker_port, 1secs); + Broker::enable(); + Broker::subscribe_to_prints("bro/print/my_topic"); + Broker::connect("127.0.0.1", broker_port, 1secs); } global messages_sent = 0; global messages_recv = 0; global peer_disconnected = F; -event BrokerComm::outgoing_connection_established(peer_address: string, +event Broker::outgoing_connection_established(peer_address: string, peer_port: port, peer_name: string) { - print "BrokerComm::outgoing_connection_established", peer_address, peer_port; - BrokerComm::print("bro/print/hi", fmt("ping %d", messages_sent)); + print "Broker::outgoing_connection_established", peer_address, peer_port; + Broker::send_print("bro/print/hi", fmt("ping %d", messages_sent)); ++messages_sent; } -event BrokerComm::outgoing_connection_broken(peer_address: string, +event Broker::outgoing_connection_broken(peer_address: string, peer_port: port) { terminate(); } -event BrokerComm::print_handler(msg: string) +event Broker::print_handler(msg: string) { ++messages_recv; print "got print msg", msg; - BrokerComm::print("bro/print/hi", fmt("ping %d", messages_sent)); + Broker::send_print("bro/print/hi", fmt("ping %d", messages_sent)); ++messages_sent; } diff --git a/testing/btest/doc/sphinx/include-doc_frameworks_broker_connecting-connector_bro.btest b/testing/btest/doc/sphinx/include-doc_frameworks_broker_connecting-connector_bro.btest index 042b8999f3..c4cbde045c 100644 --- a/testing/btest/doc/sphinx/include-doc_frameworks_broker_connecting-connector_bro.btest +++ b/testing/btest/doc/sphinx/include-doc_frameworks_broker_connecting-connector_bro.btest @@ -4,19 +4,19 @@ connecting-connector.bro const broker_port: port = 9999/tcp &redef; redef exit_only_after_terminate = T; -redef BrokerComm::endpoint_name = "connector"; +redef Broker::endpoint_name = "connector"; event bro_init() { - BrokerComm::enable(); - BrokerComm::connect("127.0.0.1", broker_port, 1sec); + Broker::enable(); + Broker::connect("127.0.0.1", broker_port, 1sec); } -event BrokerComm::outgoing_connection_established(peer_address: string, +event Broker::outgoing_connection_established(peer_address: string, peer_port: port, peer_name: string) { - print "BrokerComm::outgoing_connection_established", + print "Broker::outgoing_connection_established", peer_address, peer_port, peer_name; terminate(); } diff --git a/testing/btest/doc/sphinx/include-doc_frameworks_broker_connecting-listener_bro.btest b/testing/btest/doc/sphinx/include-doc_frameworks_broker_connecting-listener_bro.btest index 33e3df2330..8ea85569c9 100644 --- a/testing/btest/doc/sphinx/include-doc_frameworks_broker_connecting-listener_bro.btest +++ b/testing/btest/doc/sphinx/include-doc_frameworks_broker_connecting-listener_bro.btest @@ -4,21 +4,21 @@ connecting-listener.bro const broker_port: port = 9999/tcp &redef; redef exit_only_after_terminate = T; -redef BrokerComm::endpoint_name = "listener"; +redef Broker::endpoint_name = "listener"; event bro_init() { - BrokerComm::enable(); - BrokerComm::listen(broker_port, "127.0.0.1"); + Broker::enable(); + Broker::listen(broker_port, "127.0.0.1"); } -event BrokerComm::incoming_connection_established(peer_name: string) +event Broker::incoming_connection_established(peer_name: string) { - print "BrokerComm::incoming_connection_established", peer_name; + print "Broker::incoming_connection_established", peer_name; } -event BrokerComm::incoming_connection_broken(peer_name: string) +event Broker::incoming_connection_broken(peer_name: string) { - print "BrokerComm::incoming_connection_broken", peer_name; + print "Broker::incoming_connection_broken", peer_name; terminate(); } diff --git a/testing/btest/doc/sphinx/include-doc_frameworks_broker_events-connector_bro.btest b/testing/btest/doc/sphinx/include-doc_frameworks_broker_events-connector_bro.btest index fe97fdb4ce..d7a0e64be2 100644 --- a/testing/btest/doc/sphinx/include-doc_frameworks_broker_events-connector_bro.btest +++ b/testing/btest/doc/sphinx/include-doc_frameworks_broker_events-connector_bro.btest @@ -4,31 +4,31 @@ events-connector.bro const broker_port: port = 9999/tcp &redef; redef exit_only_after_terminate = T; -redef BrokerComm::endpoint_name = "connector"; +redef Broker::endpoint_name = "connector"; global my_event: event(msg: string, c: count); global my_auto_event: event(msg: string, c: count); event bro_init() { - BrokerComm::enable(); - BrokerComm::connect("127.0.0.1", broker_port, 1sec); - BrokerComm::auto_event("bro/event/my_auto_event", my_auto_event); + Broker::enable(); + Broker::connect("127.0.0.1", broker_port, 1sec); + Broker::auto_event("bro/event/my_auto_event", my_auto_event); } -event BrokerComm::outgoing_connection_established(peer_address: string, +event Broker::outgoing_connection_established(peer_address: string, peer_port: port, peer_name: string) { - print "BrokerComm::outgoing_connection_established", + print "Broker::outgoing_connection_established", peer_address, peer_port, peer_name; - BrokerComm::event("bro/event/my_event", BrokerComm::event_args(my_event, "hi", 0)); + Broker::send_event("bro/event/my_event", Broker::event_args(my_event, "hi", 0)); event my_auto_event("stuff", 88); - BrokerComm::event("bro/event/my_event", BrokerComm::event_args(my_event, "...", 1)); + Broker::send_event("bro/event/my_event", Broker::event_args(my_event, "...", 1)); event my_auto_event("more stuff", 51); - BrokerComm::event("bro/event/my_event", BrokerComm::event_args(my_event, "bye", 2)); + Broker::send_event("bro/event/my_event", Broker::event_args(my_event, "bye", 2)); } -event BrokerComm::outgoing_connection_broken(peer_address: string, +event Broker::outgoing_connection_broken(peer_address: string, peer_port: port) { terminate(); diff --git a/testing/btest/doc/sphinx/include-doc_frameworks_broker_events-listener_bro.btest b/testing/btest/doc/sphinx/include-doc_frameworks_broker_events-listener_bro.btest index 9f004692cb..640722cac0 100644 --- a/testing/btest/doc/sphinx/include-doc_frameworks_broker_events-listener_bro.btest +++ b/testing/btest/doc/sphinx/include-doc_frameworks_broker_events-listener_bro.btest @@ -4,21 +4,21 @@ events-listener.bro const broker_port: port = 9999/tcp &redef; redef exit_only_after_terminate = T; -redef BrokerComm::endpoint_name = "listener"; +redef Broker::endpoint_name = "listener"; global msg_count = 0; global my_event: event(msg: string, c: count); global my_auto_event: event(msg: string, c: count); event bro_init() { - BrokerComm::enable(); - BrokerComm::subscribe_to_events("bro/event/"); - BrokerComm::listen(broker_port, "127.0.0.1"); + Broker::enable(); + Broker::subscribe_to_events("bro/event/"); + Broker::listen(broker_port, "127.0.0.1"); } -event BrokerComm::incoming_connection_established(peer_name: string) +event Broker::incoming_connection_established(peer_name: string) { - print "BrokerComm::incoming_connection_established", peer_name; + print "Broker::incoming_connection_established", peer_name; } event my_event(msg: string, c: count) diff --git a/testing/btest/doc/sphinx/include-doc_frameworks_broker_logs-connector_bro.btest b/testing/btest/doc/sphinx/include-doc_frameworks_broker_logs-connector_bro.btest index 6884d5e4d6..907d712c88 100644 --- a/testing/btest/doc/sphinx/include-doc_frameworks_broker_logs-connector_bro.btest +++ b/testing/btest/doc/sphinx/include-doc_frameworks_broker_logs-connector_bro.btest @@ -6,16 +6,16 @@ logs-connector.bro const broker_port: port = 9999/tcp &redef; redef exit_only_after_terminate = T; -redef BrokerComm::endpoint_name = "connector"; +redef Broker::endpoint_name = "connector"; redef Log::enable_local_logging = F; redef Log::enable_remote_logging = F; global n = 0; event bro_init() { - BrokerComm::enable(); - BrokerComm::enable_remote_logs(Test::LOG); - BrokerComm::connect("127.0.0.1", broker_port, 1sec); + Broker::enable(); + Broker::enable_remote_logs(Test::LOG); + Broker::connect("127.0.0.1", broker_port, 1sec); } event do_write() @@ -28,16 +28,16 @@ event do_write() event do_write(); } -event BrokerComm::outgoing_connection_established(peer_address: string, +event Broker::outgoing_connection_established(peer_address: string, peer_port: port, peer_name: string) { - print "BrokerComm::outgoing_connection_established", + print "Broker::outgoing_connection_established", peer_address, peer_port, peer_name; event do_write(); } -event BrokerComm::outgoing_connection_broken(peer_address: string, +event Broker::outgoing_connection_broken(peer_address: string, peer_port: port) { terminate(); diff --git a/testing/btest/doc/sphinx/include-doc_frameworks_broker_logs-listener_bro.btest b/testing/btest/doc/sphinx/include-doc_frameworks_broker_logs-listener_bro.btest index 1610bde502..de6abbf5a0 100644 --- a/testing/btest/doc/sphinx/include-doc_frameworks_broker_logs-listener_bro.btest +++ b/testing/btest/doc/sphinx/include-doc_frameworks_broker_logs-listener_bro.btest @@ -6,18 +6,18 @@ logs-listener.bro const broker_port: port = 9999/tcp &redef; redef exit_only_after_terminate = T; -redef BrokerComm::endpoint_name = "listener"; +redef Broker::endpoint_name = "listener"; event bro_init() { - BrokerComm::enable(); - BrokerComm::subscribe_to_logs("bro/log/Test::LOG"); - BrokerComm::listen(broker_port, "127.0.0.1"); + Broker::enable(); + Broker::subscribe_to_logs("bro/log/Test::LOG"); + Broker::listen(broker_port, "127.0.0.1"); } -event BrokerComm::incoming_connection_established(peer_name: string) +event Broker::incoming_connection_established(peer_name: string) { - print "BrokerComm::incoming_connection_established", peer_name; + print "Broker::incoming_connection_established", peer_name; } event Test::log_test(rec: Test::Info) diff --git a/testing/btest/doc/sphinx/include-doc_frameworks_broker_printing-connector_bro.btest b/testing/btest/doc/sphinx/include-doc_frameworks_broker_printing-connector_bro.btest index 86ad4f459f..91ee179fe6 100644 --- a/testing/btest/doc/sphinx/include-doc_frameworks_broker_printing-connector_bro.btest +++ b/testing/btest/doc/sphinx/include-doc_frameworks_broker_printing-connector_bro.btest @@ -4,26 +4,26 @@ printing-connector.bro const broker_port: port = 9999/tcp &redef; redef exit_only_after_terminate = T; -redef BrokerComm::endpoint_name = "connector"; +redef Broker::endpoint_name = "connector"; event bro_init() { - BrokerComm::enable(); - BrokerComm::connect("127.0.0.1", broker_port, 1sec); + Broker::enable(); + Broker::connect("127.0.0.1", broker_port, 1sec); } -event BrokerComm::outgoing_connection_established(peer_address: string, +event Broker::outgoing_connection_established(peer_address: string, peer_port: port, peer_name: string) { - print "BrokerComm::outgoing_connection_established", + print "Broker::outgoing_connection_established", peer_address, peer_port, peer_name; - BrokerComm::print("bro/print/hi", "hello"); - BrokerComm::print("bro/print/stuff", "..."); - BrokerComm::print("bro/print/bye", "goodbye"); + Broker::send_print("bro/print/hi", "hello"); + Broker::send_print("bro/print/stuff", "..."); + Broker::send_print("bro/print/bye", "goodbye"); } -event BrokerComm::outgoing_connection_broken(peer_address: string, +event Broker::outgoing_connection_broken(peer_address: string, peer_port: port) { terminate(); diff --git a/testing/btest/doc/sphinx/include-doc_frameworks_broker_printing-listener_bro.btest b/testing/btest/doc/sphinx/include-doc_frameworks_broker_printing-listener_bro.btest index fb416612ab..37e4d0eae9 100644 --- a/testing/btest/doc/sphinx/include-doc_frameworks_broker_printing-listener_bro.btest +++ b/testing/btest/doc/sphinx/include-doc_frameworks_broker_printing-listener_bro.btest @@ -4,22 +4,22 @@ printing-listener.bro const broker_port: port = 9999/tcp &redef; redef exit_only_after_terminate = T; -redef BrokerComm::endpoint_name = "listener"; +redef Broker::endpoint_name = "listener"; global msg_count = 0; event bro_init() { - BrokerComm::enable(); - BrokerComm::subscribe_to_prints("bro/print/"); - BrokerComm::listen(broker_port, "127.0.0.1"); + Broker::enable(); + Broker::subscribe_to_prints("bro/print/"); + Broker::listen(broker_port, "127.0.0.1"); } -event BrokerComm::incoming_connection_established(peer_name: string) +event Broker::incoming_connection_established(peer_name: string) { - print "BrokerComm::incoming_connection_established", peer_name; + print "Broker::incoming_connection_established", peer_name; } -event BrokerComm::print_handler(msg: string) +event Broker::print_handler(msg: string) { ++msg_count; print "got print message", msg; diff --git a/testing/btest/doc/sphinx/include-doc_frameworks_broker_stores-connector_bro.btest b/testing/btest/doc/sphinx/include-doc_frameworks_broker_stores-connector_bro.btest index 6ca9e3b49b..74b59467e7 100644 --- a/testing/btest/doc/sphinx/include-doc_frameworks_broker_stores-connector_bro.btest +++ b/testing/btest/doc/sphinx/include-doc_frameworks_broker_stores-connector_bro.btest @@ -5,42 +5,42 @@ stores-connector.bro const broker_port: port = 9999/tcp &redef; redef exit_only_after_terminate = T; -global h: opaque of BrokerStore::Handle; +global h: opaque of Broker::Handle; -function dv(d: BrokerComm::Data): BrokerComm::DataVector +function dv(d: Broker::Data): Broker::DataVector { - local rval: BrokerComm::DataVector; + local rval: Broker::DataVector; rval[0] = d; return rval; } global ready: event(); -event BrokerComm::outgoing_connection_broken(peer_address: string, +event Broker::outgoing_connection_broken(peer_address: string, peer_port: port) { terminate(); } -event BrokerComm::outgoing_connection_established(peer_address: string, +event Broker::outgoing_connection_established(peer_address: string, peer_port: port, peer_name: string) { local myset: set[string] = {"a", "b", "c"}; local myvec: vector of string = {"alpha", "beta", "gamma"}; - h = BrokerStore::create_master("mystore"); - BrokerStore::insert(h, BrokerComm::data("one"), BrokerComm::data(110)); - BrokerStore::insert(h, BrokerComm::data("two"), BrokerComm::data(223)); - BrokerStore::insert(h, BrokerComm::data("myset"), BrokerComm::data(myset)); - BrokerStore::insert(h, BrokerComm::data("myvec"), BrokerComm::data(myvec)); - BrokerStore::increment(h, BrokerComm::data("one")); - BrokerStore::decrement(h, BrokerComm::data("two")); - BrokerStore::add_to_set(h, BrokerComm::data("myset"), BrokerComm::data("d")); - BrokerStore::remove_from_set(h, BrokerComm::data("myset"), BrokerComm::data("b")); - BrokerStore::push_left(h, BrokerComm::data("myvec"), dv(BrokerComm::data("delta"))); - BrokerStore::push_right(h, BrokerComm::data("myvec"), dv(BrokerComm::data("omega"))); + h = Broker::create_master("mystore"); + Broker::insert(h, Broker::data("one"), Broker::data(110)); + Broker::insert(h, Broker::data("two"), Broker::data(223)); + Broker::insert(h, Broker::data("myset"), Broker::data(myset)); + Broker::insert(h, Broker::data("myvec"), Broker::data(myvec)); + Broker::increment(h, Broker::data("one")); + Broker::decrement(h, Broker::data("two")); + Broker::add_to_set(h, Broker::data("myset"), Broker::data("d")); + Broker::remove_from_set(h, Broker::data("myset"), Broker::data("b")); + Broker::push_left(h, Broker::data("myvec"), dv(Broker::data("delta"))); + Broker::push_right(h, Broker::data("myvec"), dv(Broker::data("omega"))); - when ( local res = BrokerStore::size(h) ) + when ( local res = Broker::size(h) ) { print "master size", res; event ready(); @@ -51,7 +51,7 @@ event BrokerComm::outgoing_connection_established(peer_address: string, event bro_init() { - BrokerComm::enable(); - BrokerComm::connect("127.0.0.1", broker_port, 1secs); - BrokerComm::auto_event("bro/event/ready", ready); + Broker::enable(); + Broker::connect("127.0.0.1", broker_port, 1secs); + Broker::auto_event("bro/event/ready", ready); } diff --git a/testing/btest/doc/sphinx/include-doc_frameworks_broker_stores-listener_bro.btest b/testing/btest/doc/sphinx/include-doc_frameworks_broker_stores-listener_bro.btest index 6942ec17d2..8dadbc803c 100644 --- a/testing/btest/doc/sphinx/include-doc_frameworks_broker_stores-listener_bro.btest +++ b/testing/btest/doc/sphinx/include-doc_frameworks_broker_stores-listener_bro.btest @@ -5,13 +5,13 @@ stores-listener.bro const broker_port: port = 9999/tcp &redef; redef exit_only_after_terminate = T; -global h: opaque of BrokerStore::Handle; +global h: opaque of Broker::Handle; global expected_key_count = 4; global key_count = 0; function do_lookup(key: string) { - when ( local res = BrokerStore::lookup(h, BrokerComm::data(key)) ) + when ( local res = Broker::lookup(h, Broker::data(key)) ) { ++key_count; print "lookup", key, res; @@ -25,15 +25,15 @@ function do_lookup(key: string) event ready() { - h = BrokerStore::create_clone("mystore"); + h = Broker::create_clone("mystore"); - when ( local res = BrokerStore::keys(h) ) + when ( local res = Broker::keys(h) ) { print "clone keys", res; - do_lookup(BrokerComm::refine_to_string(BrokerComm::vector_lookup(res$result, 0))); - do_lookup(BrokerComm::refine_to_string(BrokerComm::vector_lookup(res$result, 1))); - do_lookup(BrokerComm::refine_to_string(BrokerComm::vector_lookup(res$result, 2))); - do_lookup(BrokerComm::refine_to_string(BrokerComm::vector_lookup(res$result, 3))); + do_lookup(Broker::refine_to_string(Broker::vector_lookup(res$result, 0))); + do_lookup(Broker::refine_to_string(Broker::vector_lookup(res$result, 1))); + do_lookup(Broker::refine_to_string(Broker::vector_lookup(res$result, 2))); + do_lookup(Broker::refine_to_string(Broker::vector_lookup(res$result, 3))); } timeout 10sec { print "timeout"; } @@ -41,7 +41,7 @@ event ready() event bro_init() { - BrokerComm::enable(); - BrokerComm::subscribe_to_events("bro/event/ready"); - BrokerComm::listen(broker_port, "127.0.0.1"); + Broker::enable(); + Broker::subscribe_to_events("bro/event/ready"); + Broker::listen(broker_port, "127.0.0.1"); } diff --git a/testing/btest/doc/sphinx/include-doc_frameworks_broker_testlog_bro.btest b/testing/btest/doc/sphinx/include-doc_frameworks_broker_testlog_bro.btest index c87fc3cd6f..d5a92417dc 100644 --- a/testing/btest/doc/sphinx/include-doc_frameworks_broker_testlog_bro.btest +++ b/testing/btest/doc/sphinx/include-doc_frameworks_broker_testlog_bro.btest @@ -17,6 +17,6 @@ export { event bro_init() &priority=5 { - BrokerComm::enable(); + Broker::enable(); Log::create_stream(Test::LOG, [$columns=Test::Info, $ev=log_test, $path="test"]); } diff --git a/testing/btest/doc/sphinx/include-scripts_base_protocols_conn_main_bro.btest b/testing/btest/doc/sphinx/include-doc_scripting_data_type_record_bro.btest similarity index 97% rename from testing/btest/doc/sphinx/include-scripts_base_protocols_conn_main_bro.btest rename to testing/btest/doc/sphinx/include-doc_scripting_data_type_record_bro.btest index 83e9d5bea1..6d8760700a 100644 --- a/testing/btest/doc/sphinx/include-scripts_base_protocols_conn_main_bro.btest +++ b/testing/btest/doc/sphinx/include-doc_scripting_data_type_record_bro.btest @@ -1,6 +1,6 @@ # @TEST-EXEC: cat %INPUT >output && btest-diff output -main.bro +data_type_record.bro module Conn; diff --git a/testing/btest/doc/sphinx/include-scripts_base_protocols_http_main_bro.btest b/testing/btest/doc/sphinx/include-doc_scripting_http_main_bro.btest similarity index 93% rename from testing/btest/doc/sphinx/include-scripts_base_protocols_http_main_bro.btest rename to testing/btest/doc/sphinx/include-doc_scripting_http_main_bro.btest index e3f7a39429..9f49450799 100644 --- a/testing/btest/doc/sphinx/include-scripts_base_protocols_http_main_bro.btest +++ b/testing/btest/doc/sphinx/include-doc_scripting_http_main_bro.btest @@ -1,6 +1,6 @@ # @TEST-EXEC: cat %INPUT >output && btest-diff output -main.bro +http_main.bro module HTTP; diff --git a/testing/btest/language/event-local-var.bro b/testing/btest/language/event-local-var.bro new file mode 100644 index 0000000000..d4dd9d19a5 --- /dev/null +++ b/testing/btest/language/event-local-var.bro @@ -0,0 +1,16 @@ +# @TEST-EXEC-FAIL: bro -b %INPUT 2> out +# @TEST-EXEC: TEST_DIFF_CANONIFIER=$SCRIPTS/diff-remove-abspath btest-diff out + + +event e1(num: count) + { + print fmt("event 1: %s", num); + } + +event bro_init() +{ + # Test assigning a local event variable to an event + local v: event(num: count); + v = e1; + schedule 1sec { v(6) }; # This should fail +} diff --git a/testing/btest/language/event.bro b/testing/btest/language/event.bro index 39a3e0da48..d4eef24731 100644 --- a/testing/btest/language/event.bro +++ b/testing/btest/language/event.bro @@ -21,7 +21,7 @@ event e3(test: string) event e4(num: count) { - print "assign event variable"; + print fmt("assign event variable (%s)", num); } # Note: the name of this event is intentionally the same as one above @@ -30,6 +30,8 @@ event e3(test: string) print "event part2"; } +global e5: event(num: count); + event bro_init() { # Test calling an event with "event" statement @@ -43,9 +45,8 @@ event bro_init() event e3("foo"); # Test assigning an event variable to an event - local e5: event(num: count); e5 = e4; - event e5(6); # TODO: this does not do anything + event e5(6); } # scheduling in outside of an event handler shouldn't crash. diff --git a/testing/btest/plugins/file-plugin/CMakeLists.txt b/testing/btest/plugins/file-plugin/CMakeLists.txt index 4823ddb08f..1d0941d9da 100644 --- a/testing/btest/plugins/file-plugin/CMakeLists.txt +++ b/testing/btest/plugins/file-plugin/CMakeLists.txt @@ -9,6 +9,9 @@ endif () set(CMAKE_MODULE_PATH ${BRO_DIST}/cmake) +find_package(OpenSSL) +include_directories(${OPENSSL_INCLUDE_DIR}) + include(BroPlugin) bro_plugin_begin(Demo Foo) diff --git a/testing/btest/plugins/protocol-plugin/CMakeLists.txt b/testing/btest/plugins/protocol-plugin/CMakeLists.txt index 4bc8460c06..a10fff1d67 100644 --- a/testing/btest/plugins/protocol-plugin/CMakeLists.txt +++ b/testing/btest/plugins/protocol-plugin/CMakeLists.txt @@ -9,6 +9,9 @@ endif () set(CMAKE_MODULE_PATH ${BRO_DIST}/cmake) +find_package(OpenSSL) +include_directories(${OPENSSL_INCLUDE_DIR}) + include(BroPlugin) bro_plugin_begin(Demo Foo) diff --git a/testing/btest/scripts/base/files/x509/1999.test b/testing/btest/scripts/base/files/x509/1999.test new file mode 100644 index 0000000000..7c1ab7971f --- /dev/null +++ b/testing/btest/scripts/base/files/x509/1999.test @@ -0,0 +1,5 @@ +# Test that the timestamp of a pre-y-2000 certificate is correctly parsed + +# @TEST-EXEC: bro -r $TRACES/tls/telesec.pcap +# @TEST-EXEC: btest-diff x509.log + diff --git a/testing/btest/scripts/base/frameworks/netcontrol/acld-hook.bro b/testing/btest/scripts/base/frameworks/netcontrol/acld-hook.bro index 566739c0b7..e131ec1dc0 100644 --- a/testing/btest/scripts/base/frameworks/netcontrol/acld-hook.bro +++ b/testing/btest/scripts/base/frameworks/netcontrol/acld-hook.bro @@ -1,5 +1,5 @@ # @TEST-SERIALIZE: brokercomm -# @TEST-REQUIRES: grep -q ENABLE_BROKER $BUILD/CMakeCache.txt +# @TEST-REQUIRES: grep -q ENABLE_BROKER:BOOL=true $BUILD/CMakeCache.txt # @TEST-EXEC: btest-bg-run recv "bro -b ../recv.bro broker_port=$BROKER_PORT >recv.out" # @TEST-EXEC: btest-bg-run send "bro -b -r $TRACES/tls/ecdhe.pcap --pseudo-realtime ../send.bro broker_port=$BROKER_PORT >send.out" @@ -26,14 +26,14 @@ event NetControl::init_done() continue_processing(); } -event BrokerComm::outgoing_connection_established(peer_address: string, +event Broker::outgoing_connection_established(peer_address: string, peer_port: port, peer_name: string) { - print "BrokerComm::outgoing_connection_established", peer_address, peer_port; + print "Broker::outgoing_connection_established", peer_address, peer_port; } -event BrokerComm::outgoing_connection_broken(peer_address: string, +event Broker::outgoing_connection_broken(peer_address: string, peer_port: port) { terminate(); @@ -91,28 +91,28 @@ redef exit_only_after_terminate = T; event bro_init() { - BrokerComm::enable(); - BrokerComm::subscribe_to_events("bro/event/netcontroltest"); - BrokerComm::listen(broker_port, "127.0.0.1"); + Broker::enable(); + Broker::subscribe_to_events("bro/event/netcontroltest"); + Broker::listen(broker_port, "127.0.0.1"); } -event BrokerComm::incoming_connection_established(peer_name: string) +event Broker::incoming_connection_established(peer_name: string) { - print "BrokerComm::incoming_connection_established"; + print "Broker::incoming_connection_established"; } event NetControl::acld_add_rule(id: count, r: NetControl::Rule, ar: NetControl::AclRule) { print "add_rule", id, r$entity, r$ty, ar; - BrokerComm::event("bro/event/netcontroltest", BrokerComm::event_args(NetControl::acld_rule_added, id, r, ar$command)); + Broker::send_event("bro/event/netcontroltest", Broker::event_args(NetControl::acld_rule_added, id, r, ar$command)); } event NetControl::acld_remove_rule(id: count, r: NetControl::Rule, ar: NetControl::AclRule) { print "remove_rule", id, r$entity, r$ty, ar; - BrokerComm::event("bro/event/netcontroltest", BrokerComm::event_args(NetControl::acld_rule_removed, id, r, ar$command)); + Broker::send_event("bro/event/netcontroltest", Broker::event_args(NetControl::acld_rule_removed, id, r, ar$command)); if ( r$cid == 4 ) terminate(); diff --git a/testing/btest/scripts/base/frameworks/netcontrol/acld.bro b/testing/btest/scripts/base/frameworks/netcontrol/acld.bro index dfeaee1055..a509b23c00 100644 --- a/testing/btest/scripts/base/frameworks/netcontrol/acld.bro +++ b/testing/btest/scripts/base/frameworks/netcontrol/acld.bro @@ -1,5 +1,5 @@ # @TEST-SERIALIZE: brokercomm -# @TEST-REQUIRES: grep -q ENABLE_BROKER $BUILD/CMakeCache.txt +# @TEST-REQUIRES: grep -q ENABLE_BROKER:BOOL=true $BUILD/CMakeCache.txt # @TEST-EXEC: btest-bg-run recv "bro -b ../recv.bro broker_port=$BROKER_PORT >recv.out" # @TEST-EXEC: btest-bg-run send "bro -b -r $TRACES/tls/ecdhe.pcap --pseudo-realtime ../send.bro broker_port=$BROKER_PORT >send.out" @@ -21,11 +21,11 @@ event NetControl::init() NetControl::activate(netcontrol_acld, 0); } -event BrokerComm::outgoing_connection_established(peer_address: string, +event Broker::outgoing_connection_established(peer_address: string, peer_port: port, peer_name: string) { - print "BrokerComm::outgoing_connection_established", peer_address, peer_port; + print "Broker::outgoing_connection_established", peer_address, peer_port; } event NetControl::init_done() @@ -33,7 +33,7 @@ event NetControl::init_done() continue_processing(); } -event BrokerComm::outgoing_connection_broken(peer_address: string, +event Broker::outgoing_connection_broken(peer_address: string, peer_port: port) { terminate(); @@ -84,28 +84,28 @@ redef exit_only_after_terminate = T; event bro_init() { - BrokerComm::enable(); - BrokerComm::subscribe_to_events("bro/event/netcontroltest"); - BrokerComm::listen(broker_port, "127.0.0.1"); + Broker::enable(); + Broker::subscribe_to_events("bro/event/netcontroltest"); + Broker::listen(broker_port, "127.0.0.1"); } -event BrokerComm::incoming_connection_established(peer_name: string) +event Broker::incoming_connection_established(peer_name: string) { - print "BrokerComm::incoming_connection_established"; + print "Broker::incoming_connection_established"; } event NetControl::acld_add_rule(id: count, r: NetControl::Rule, ar: NetControl::AclRule) { print "add_rule", id, r$entity, r$ty, ar; - BrokerComm::event("bro/event/netcontroltest", BrokerComm::event_args(NetControl::acld_rule_added, id, r, ar$command)); + Broker::send_event("bro/event/netcontroltest", Broker::event_args(NetControl::acld_rule_added, id, r, ar$command)); } event NetControl::acld_remove_rule(id: count, r: NetControl::Rule, ar: NetControl::AclRule) { print "remove_rule", id, r$entity, r$ty, ar; - BrokerComm::event("bro/event/netcontroltest", BrokerComm::event_args(NetControl::acld_rule_removed, id, r, ar$command)); + Broker::send_event("bro/event/netcontroltest", Broker::event_args(NetControl::acld_rule_removed, id, r, ar$command)); if ( r$cid == 4 ) terminate(); diff --git a/testing/btest/scripts/base/frameworks/netcontrol/broker.bro b/testing/btest/scripts/base/frameworks/netcontrol/broker.bro index 56a76433f2..f9328a458d 100644 --- a/testing/btest/scripts/base/frameworks/netcontrol/broker.bro +++ b/testing/btest/scripts/base/frameworks/netcontrol/broker.bro @@ -1,5 +1,5 @@ # @TEST-SERIALIZE: brokercomm -# @TEST-REQUIRES: grep -q ENABLE_BROKER $BUILD/CMakeCache.txt +# @TEST-REQUIRES: grep -q ENABLE_BROKER:BOOL=true $BUILD/CMakeCache.txt # @TEST-EXEC: btest-bg-run recv "bro -b ../recv.bro broker_port=$BROKER_PORT >recv.out" # @TEST-EXEC: btest-bg-run send "bro -b -r $TRACES/smtp.trace --pseudo-realtime ../send.bro broker_port=$BROKER_PORT >send.out" @@ -27,14 +27,14 @@ event NetControl::init_done() continue_processing(); } -event BrokerComm::outgoing_connection_established(peer_address: string, +event Broker::outgoing_connection_established(peer_address: string, peer_port: port, peer_name: string) { - print "BrokerComm::outgoing_connection_established", peer_address, peer_port; + print "Broker::outgoing_connection_established", peer_address, peer_port; } -event BrokerComm::outgoing_connection_broken(peer_address: string, +event Broker::outgoing_connection_broken(peer_address: string, peer_port: port) { terminate(); @@ -75,29 +75,29 @@ redef exit_only_after_terminate = T; event bro_init() { - BrokerComm::enable(); - BrokerComm::subscribe_to_events("bro/event/netcontroltest"); - BrokerComm::listen(broker_port, "127.0.0.1"); + Broker::enable(); + Broker::subscribe_to_events("bro/event/netcontroltest"); + Broker::listen(broker_port, "127.0.0.1"); } -event BrokerComm::incoming_connection_established(peer_name: string) +event Broker::incoming_connection_established(peer_name: string) { - print "BrokerComm::incoming_connection_established"; + print "Broker::incoming_connection_established"; } event NetControl::broker_add_rule(id: count, r: NetControl::Rule) { print "add_rule", id, r$entity, r$ty; - BrokerComm::event("bro/event/netcontroltest", BrokerComm::event_args(NetControl::broker_rule_added, id, r, "")); + Broker::send_event("bro/event/netcontroltest", Broker::event_args(NetControl::broker_rule_added, id, r, "")); } event NetControl::broker_remove_rule(id: count, r: NetControl::Rule) { print "remove_rule", id, r$entity, r$ty; - BrokerComm::event("bro/event/netcontroltest", BrokerComm::event_args(NetControl::broker_rule_timeout, id, r, NetControl::FlowInfo())); - BrokerComm::event("bro/event/netcontroltest", BrokerComm::event_args(NetControl::broker_rule_removed, id, r, "")); + Broker::send_event("bro/event/netcontroltest", Broker::event_args(NetControl::broker_rule_timeout, id, r, NetControl::FlowInfo())); + Broker::send_event("bro/event/netcontroltest", Broker::event_args(NetControl::broker_rule_removed, id, r, "")); if ( r$cid == 3 ) terminate(); diff --git a/testing/btest/scripts/base/frameworks/openflow/broker-basic.bro b/testing/btest/scripts/base/frameworks/openflow/broker-basic.bro index e973517d44..9250590013 100644 --- a/testing/btest/scripts/base/frameworks/openflow/broker-basic.bro +++ b/testing/btest/scripts/base/frameworks/openflow/broker-basic.bro @@ -1,5 +1,5 @@ # @TEST-SERIALIZE: brokercomm -# @TEST-REQUIRES: grep -q ENABLE_BROKER $BUILD/CMakeCache.txt +# @TEST-REQUIRES: grep -q ENABLE_BROKER:BOOL=true $BUILD/CMakeCache.txt # @TEST-EXEC: btest-bg-run recv "bro -b ../recv.bro broker_port=$BROKER_PORT >recv.out" # @TEST-EXEC: btest-bg-run send "bro -b -r $TRACES/smtp.trace --pseudo-realtime ../send.bro broker_port=$BROKER_PORT >send.out" @@ -23,11 +23,11 @@ event bro_init() of_controller = OpenFlow::broker_new("broker1", 127.0.0.1, broker_port, "bro/event/openflow", 42); } -event BrokerComm::outgoing_connection_established(peer_address: string, +event Broker::outgoing_connection_established(peer_address: string, peer_port: port, peer_name: string) { - print "BrokerComm::outgoing_connection_established", peer_address, peer_port; + print "Broker::outgoing_connection_established", peer_address, peer_port; } event OpenFlow::controller_activated(name: string, controller: OpenFlow::Controller) @@ -37,7 +37,7 @@ event OpenFlow::controller_activated(name: string, controller: OpenFlow::Control OpenFlow::flow_mod(of_controller, [], [$cookie=OpenFlow::generate_cookie(1), $command=OpenFlow::OFPFC_ADD, $actions=[$out_ports=vector(3, 7)]]); } -event BrokerComm::outgoing_connection_broken(peer_address: string, +event Broker::outgoing_connection_broken(peer_address: string, peer_port: port) { terminate(); @@ -83,14 +83,14 @@ global msg_count: count = 0; event bro_init() { - BrokerComm::enable(); - BrokerComm::subscribe_to_events("bro/event/openflow"); - BrokerComm::listen(broker_port, "127.0.0.1"); + Broker::enable(); + Broker::subscribe_to_events("bro/event/openflow"); + Broker::listen(broker_port, "127.0.0.1"); } -event BrokerComm::incoming_connection_established(peer_name: string) +event Broker::incoming_connection_established(peer_name: string) { - print "BrokerComm::incoming_connection_established"; + print "Broker::incoming_connection_established"; } function got_message() @@ -104,8 +104,8 @@ function got_message() event OpenFlow::broker_flow_mod(name: string, dpid: count, match: OpenFlow::ofp_match, flow_mod: OpenFlow::ofp_flow_mod) { print "got flow_mod", dpid, match, flow_mod; - BrokerComm::event("bro/event/openflow", BrokerComm::event_args(OpenFlow::flow_mod_success, name, match, flow_mod, "")); - BrokerComm::event("bro/event/openflow", BrokerComm::event_args(OpenFlow::flow_mod_failure, name, match, flow_mod, "")); + Broker::send_event("bro/event/openflow", Broker::event_args(OpenFlow::flow_mod_success, name, match, flow_mod, "")); + Broker::send_event("bro/event/openflow", Broker::event_args(OpenFlow::flow_mod_failure, name, match, flow_mod, "")); got_message(); } diff --git a/testing/btest/scripts/base/protocols/arp/basic.test b/testing/btest/scripts/base/protocols/arp/basic.test new file mode 100644 index 0000000000..9ef1404567 --- /dev/null +++ b/testing/btest/scripts/base/protocols/arp/basic.test @@ -0,0 +1,13 @@ +# @TEST-EXEC: bro -r $TRACES/arp-who-has.pcap %INPUT +# @TEST-EXEC: btest-diff .stdout + +event arp_request(mac_src: string, mac_dst: string, SPA: addr, SHA: string, TPA: addr, THA: string) + { + print mac_src, mac_dst, SPA, SHA, TPA, THA; + } + +event arp_reply(mac_src: string, mac_dst: string, SPA: addr, SHA: string, TPA: addr, THA: string) + { + print mac_src, mac_dst, SPA, SHA, TPA, THA; + } + diff --git a/testing/btest/scripts/base/protocols/conn/new_connection_contents.bro b/testing/btest/scripts/base/protocols/conn/new_connection_contents.bro new file mode 100644 index 0000000000..42919f6f13 --- /dev/null +++ b/testing/btest/scripts/base/protocols/conn/new_connection_contents.bro @@ -0,0 +1,7 @@ +# @TEST-EXEC: bro -r $TRACES/irc-dcc-send.trace %INPUT +# @TEST-EXEC: btest-diff .stdout + +event new_connection_contents(c: connection) + { + print fmt("new_connection_contents for %s", cat(c$id)); + } diff --git a/testing/btest/scripts/base/protocols/dns/caa.bro b/testing/btest/scripts/base/protocols/dns/caa.bro new file mode 100644 index 0000000000..9a0f4701de --- /dev/null +++ b/testing/btest/scripts/base/protocols/dns/caa.bro @@ -0,0 +1,7 @@ +# @TEST-EXEC: bro -r $TRACES/dns-caa.pcap %INPUT +# @TEST-EXEC: btest-diff .stdout + +event dns_CAA_reply(c: connection, msg: dns_msg, ans: dns_answer, flags: count, tag: string, value: string) + { + print flags,tag,value; + } diff --git a/testing/btest/scripts/base/protocols/dns/huge-ttl.bro b/testing/btest/scripts/base/protocols/dns/huge-ttl.bro new file mode 100644 index 0000000000..ee6a76e978 --- /dev/null +++ b/testing/btest/scripts/base/protocols/dns/huge-ttl.bro @@ -0,0 +1,7 @@ +# @TEST-EXEC: bro -r $TRACES/dns-huge-ttl.pcap %INPUT +# @TEST-EXEC: btest-diff .stdout + +event dns_A_reply(c: connection, msg: dns_msg, ans: dns_answer, a: addr) + { + print ans; + } diff --git a/testing/btest/scripts/base/protocols/imap/capabilities.test b/testing/btest/scripts/base/protocols/imap/capabilities.test new file mode 100644 index 0000000000..06bdb56b7d --- /dev/null +++ b/testing/btest/scripts/base/protocols/imap/capabilities.test @@ -0,0 +1,12 @@ +# @TEST-EXEC: bro -b -C -r $TRACES/tls/imap-starttls.pcap %INPUT +# @TEST-EXEC: btest-diff .stdout + +@load base/protocols/ssl +@load base/protocols/conn +@load base/frameworks/dpd +@load base/protocols/imap + +event imap_capabilities(c: connection, capabilities: string_vec) + { + print capabilities; + } diff --git a/testing/btest/scripts/base/protocols/imap/starttls.test b/testing/btest/scripts/base/protocols/imap/starttls.test new file mode 100644 index 0000000000..444c27688a --- /dev/null +++ b/testing/btest/scripts/base/protocols/imap/starttls.test @@ -0,0 +1,15 @@ +# @TEST-EXEC: bro -b -C -r $TRACES/tls/imap-starttls.pcap %INPUT +# @TEST-EXEC: btest-diff conn.log +# @TEST-EXEC: btest-diff ssl.log +# @TEST-EXEC: btest-diff x509.log +# @TEST-EXEC: btest-diff .stdout + +@load base/protocols/ssl +@load base/protocols/conn +@load base/frameworks/dpd +@load base/protocols/imap + +event imap_starttls(c: connection) + { + print "Tls started for connection"; + } diff --git a/testing/btest/scripts/base/protocols/ssl/dtls-stun-dpd.test b/testing/btest/scripts/base/protocols/ssl/dtls-stun-dpd.test new file mode 100644 index 0000000000..e005e82e03 --- /dev/null +++ b/testing/btest/scripts/base/protocols/ssl/dtls-stun-dpd.test @@ -0,0 +1,10 @@ +# @TEST-EXEC: bro -r $TRACES/tls/webrtc-stun.pcap %INPUT +# @TEST-EXEC: btest-diff ssl.log +# @TEST-EXEC: touch dpd.log +# @TEST-EXEC: btest-diff dpd.log + +event ssl_client_hello(c: connection, version: count, possible_ts: time, client_random: string, session_id: string, ciphers: index_vec) + { + print version, client_random, session_id, ciphers; + } + diff --git a/testing/btest/scripts/base/protocols/xmpp/client-dpd.test b/testing/btest/scripts/base/protocols/xmpp/client-dpd.test new file mode 100644 index 0000000000..9c9cc29c8a --- /dev/null +++ b/testing/btest/scripts/base/protocols/xmpp/client-dpd.test @@ -0,0 +1,8 @@ +# @TEST-EXEC: bro -C -b -r $TRACES/tls/xmpp-starttls.pcap %INPUT +# @TEST-EXEC: btest-diff ssl.log + +@load base/frameworks/dpd +@load base/frameworks/signatures +@load base/protocols/ssl +@load base/protocols/conn +@load-sigs base/protocols/xmpp/dpd.sig diff --git a/testing/btest/scripts/base/protocols/xmpp/server-dialback-dpd.test b/testing/btest/scripts/base/protocols/xmpp/server-dialback-dpd.test new file mode 100644 index 0000000000..9483c0cca8 --- /dev/null +++ b/testing/btest/scripts/base/protocols/xmpp/server-dialback-dpd.test @@ -0,0 +1,8 @@ +# @TEST-EXEC: bro -C -b -r $TRACES/tls/xmpp-dialback-starttls.pcap %INPUT +# @TEST-EXEC: btest-diff ssl.log + +@load base/frameworks/dpd +@load base/frameworks/signatures +@load base/protocols/ssl +@load base/protocols/conn +@load-sigs base/protocols/xmpp/dpd.sig diff --git a/testing/btest/scripts/base/protocols/xmpp/starttls.test b/testing/btest/scripts/base/protocols/xmpp/starttls.test new file mode 100644 index 0000000000..f046d49283 --- /dev/null +++ b/testing/btest/scripts/base/protocols/xmpp/starttls.test @@ -0,0 +1,9 @@ +# @TEST-EXEC: bro -C -b -r $TRACES/tls/xmpp-starttls.pcap %INPUT +# @TEST-EXEC: btest-diff conn.log +# @TEST-EXEC: btest-diff ssl.log +# @TEST-EXEC: btest-diff x509.log + +@load base/protocols/conn +@load base/frameworks/dpd +@load base/protocols/ssl +@load base/protocols/xmpp diff --git a/testing/btest/scripts/policy/frameworks/intel/seen/certs.bro b/testing/btest/scripts/policy/frameworks/intel/seen/certs.bro index 2ab4c6a50a..859e3a6b9f 100644 --- a/testing/btest/scripts/policy/frameworks/intel/seen/certs.bro +++ b/testing/btest/scripts/policy/frameworks/intel/seen/certs.bro @@ -8,6 +8,7 @@ #fields indicator indicator_type meta.source meta.desc meta.url www.pantz.org Intel::DOMAIN source1 test entry http://some-data-distributor.com/100000 www.dresdner-privat.de Intel::DOMAIN source1 test entry http://some-data-distributor.com/100000 +2c322ae2b7fe91391345e070b63668978bb1c9da Intel::CERT_HASH source1 test entry http://some-data-distributor.com/100000 @TEST-END-FILE @load base/frameworks/intel