Mirror/zeek
1
0
Fork 0
mirror of https://github.com/zeek/zeek.git synced 2025-10-04 07:38:19 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 2

Merge branch 'master' of https://github.com/zeek/zeek into topic/zeke/closures

Browse source
This commit is contained in:
Zeke Medley 2019-06-21 09:41:41 -07:00
commit 1ed672287b
462 changed files with 7578 additions and 16720 deletions
Download patch file Download diff file Expand all files Collapse all files

183
CHANGES
View file

@ -1,4 +1,187 @@
2.6-500 | 2019-06-20 20:54:15 -0700
* Add new RDP event: rdp_client_cluster_data (Jeff Atkinson)
* Added "options" field to RDP::ClientChannelDef (Jeff Atkinson)
2.6-494 | 2019-06-20 20:24:38 -0700
* Renaming src/StateAccess.{h,cc} to src/Notifier.{h,cc}.
The old names did not reflect the content of the files anymore. (Robin Sommer, Corelight)
* Remove MutableVal, StateAccess classes, enum Opcode. (Robin Sommer, Corelight)
* Redo API for notifiers.
There's now an notifier::Modifiable interface class that class
supposed to signal modifications are to be derived from. This takes
the place of the former MutableValue class and also unifies how Val
and IDs signal modifications. (Robin Sommer, Corelight)
* Redo NotfifierRegistry to no longer rely on StateAccess.
We simplify the API to a simple Modified() operation. (Robin Sommer, Corelight)
* Add new test for when-statement watching global variables. (Robin Sommer, Corelight)
2.6-482 | 2019-06-20 19:57:20 -0700
* Make configure complain if submodules are not checked out. (Johanna Amann, Corelight)
* Improve C++ header includes to improve build time (Jon Siwek, Corelight)
2.6-479 | 2019-06-20 18:31:58 -0700
* Fix TableVal::DoClone to use CloneState cache (Jon Siwek, Corelight)
2.6-478 | 2019-06-20 14:19:11 -0700
* Remove old Broccoli SSL options (Jon Siwek, Corelight)
- ssl_ca_certificate
- ssl_private_key
- ssl_passphrase
2.6-477 | 2019-06-20 14:00:22 -0700
* Remove unused SerialInfo.h and SerialTypes.h headers (Jon Siwek, Corelight)
2.6-476 | 2019-06-20 13:23:22 -0700
* Remove opaque of ocsp_resp. (Johanna Amann, Corelight)
Only used in one event, without any way to use the opaque for anything
else. At this point this just seems like a complication that has no
reason to be there.
* Remove remnants of event serializer. (Johanna Amann, Corelight)
* Reimplement serialization infrastructure for OpaqueVals.
(Robin Sommer, Corelight & Johanna Amann, Corelight)
We need this to sender through Broker, and we also leverage it for
cloning opaques. The serialization methods now produce Broker data
instances directly, and no longer go through the binary formatter.
Summary of the new API for types derived from OpaqueVal:
- Add DECLARE_OPAQUE_VALUE(<class>) to the class declaration
- Add IMPLEMENT_OPAQUE_VALUE(<class>) to the class' implementation file
- Implement these two methods (which are declated by the 1st macro):
- broker::data DoSerialize() const
- bool DoUnserialize(const broker::data& data)
This machinery should work correctly from dynamic plugins as well.
OpaqueVal provides a default implementation of DoClone() as well that
goes through serialization. Derived classes can provide a more
efficient version if they want.
The declaration of the "OpaqueVal" class has moved into the header
file "OpaqueVal.h", along with the new serialization infrastructure.
This is breaking existing code that relies on the location, but
because the API is changing anyways that seems fine.
* Implement a Shallow Clone operation for types. (Johanna Amann, Corelight)
This is needed to track name changes for the documentation.
* Remove old serialization infrastrucutre. (Johanna Amann, Corelight)
2.6-454 | 2019-06-19 09:39:06 -0700
* GH-393: Add slice notation for vectors (Tim Wojtulewicz, Corelight & Jon Siwek, Corelight)
Example Syntax:
local v = vector(1, 2, 3, 4, 5);
v[2:4] = vector(6, 7, 8); # v is now [1, 2, 6, 7, 8, 5]
print v[:4]; # prints [1, 2, 6, 7]
2.6-446 | 2019-06-17 20:26:49 -0700
* Rename bro to zeek in error messages (Daniel Thayer)
2.6-444 | 2019-06-15 19:09:03 -0700
* Add/rewrite NTP support (Vlad Grigorescu and Mauro Palumbo)
2.6-416 | 2019-06-14 20:57:57 -0700
* DNS: Add support for SPF response records (Vlad Grigorescu)
2.6-413 | 2019-06-14 19:51:28 -0700
* GH-406: rename bro.bif to zeek.bif (Jon Siwek, Corelight)
2.6-412 | 2019-06-14 19:26:21 -0700
* GH-387: update Broker topic names to use "zeek/" prefix (Jon Siwek, Corelight)
* GH-323: change builtin plugin namespaces to Zeek (Jon Siwek, Corelight)
2.6-408 | 2019-06-13 11:19:50 -0700
* Fix potential null-dereference in current_time() (Tim Wojtulewicz, Corelight)
* Add --sanitizers configure script to enable Clang sanitizers (Tim Wojtulewicz, Corelight)
2.6-404 | 2019-06-12 15:10:19 -0700
* Rename directories from bro to zeek (Daniel Thayer)
The new default installation prefix is /usr/local/zeek
2.6-400 | 2019-06-07 20:06:33 -0700
* Adapt bro_plugin CMake macros to use zeek_plugin (Jon Siwek, Corelight)
2.6-399 | 2019-06-07 14:02:18 -0700
* Update SSL documentation. (Johanna Amann)
* Support the newer TLS 1.3 key_share extension. (Johanna Amann)
* Include all data of the server-hello random (Johanna Amann)
Before we cut the first 4 bytes, which makes it impossible to recognize
several newer packets (like the hello retry).
* Parse TLS 1.3 pre-shared-key extension. (Johanna Amann)
Adds new events:
- ssl_extension_pre_shared_key_client_hello
- ssl_extension_pre_shared_key_server_hello
2.6-391 | 2019-06-07 17:29:28 +1000
* GH-209: replace "remote_ip" field of radius.log with "tunnel_client".
Also changes type from addr to string. (Jon Siwek, Corelight)
2.6-389 | 2019-06-06 20:02:19 -0700
* Update plugin unit tests to use --zeek-dist (Jon Siwek, Corelight)
2.6-388 | 2019-06-06 19:48:55 -0700
* Change default value of peer_description "zeek" (Jon Siwek, Corelight)
2.6-387 | 2019-06-06 18:51:09 -0700
* Rename Bro to Zeek in Zeekygen-generated documentation (Jon Siwek, Corelight)
2.6-386 | 2019-06-06 17:17:55 -0700
* Add new RDP event: rdp_native_encrytped_data (Anthony Kasza, Corelight)
2.6-384 | 2019-06-06 16:49:14 -0700
* Add new RDP event: rdp_client_security_data (Jeff Atkinson)
2.6-379 | 2019-06-06 11:56:58 -0700
* Improve sqlite logging unit tests (Jon Siwek, Corelight)

44
CMakeLists.txt
View file

@ -24,7 +24,7 @@ endif ()
set(ZEEK_ROOT_DIR ${CMAKE_INSTALL_PREFIX})
if (NOT ZEEK_SCRIPT_INSTALL_PATH)
# set the default Zeek script installation path (user did not specify one)
set(ZEEK_SCRIPT_INSTALL_PATH ${ZEEK_ROOT_DIR}/share/bro)
set(ZEEK_SCRIPT_INSTALL_PATH ${ZEEK_ROOT_DIR}/share/zeek)
endif ()
if (NOT ZEEK_MAN_INSTALL_PATH)
@ -37,7 +37,7 @@ endif ()
get_filename_component(ZEEK_SCRIPT_INSTALL_PATH ${ZEEK_SCRIPT_INSTALL_PATH}
ABSOLUTE)
set(BRO_PLUGIN_INSTALL_PATH ${ZEEK_ROOT_DIR}/lib/bro/plugins CACHE STRING "Installation path for plugins" FORCE)
set(BRO_PLUGIN_INSTALL_PATH ${ZEEK_ROOT_DIR}/lib/zeek/plugins CACHE STRING "Installation path for plugins" FORCE)
configure_file(zeek-path-dev.in ${CMAKE_CURRENT_BINARY_DIR}/zeek-path-dev)
@ -72,6 +72,40 @@ if(${ENABLE_DEBUG})
set(VERSION_C_IDENT "${VERSION_C_IDENT}_debug")
endif()
if ( NOT BINARY_PACKAGING_MODE )
macro(_make_install_dir_symlink _target _link)
install(CODE "
if ( \"\$ENV{DESTDIR}\" STREQUAL \"\" )
if ( EXISTS \"${_target}\" AND NOT EXISTS \"${_link}\" )
message(STATUS \"WARNING: installed ${_link} as symlink to ${_target}\")
execute_process(COMMAND \"${CMAKE_COMMAND}\" -E create_symlink
\"${_target}\" \"${_link}\")
endif ()
endif ()
")
endmacro()
if ( "${CMAKE_INSTALL_PREFIX}" STREQUAL "/usr/local/zeek" )
# If we're installing into the default prefix, check if the
# old default prefix already exists and symlink to it.
# This is done to help keep custom user configuration/installation
# if they're upgrading from a version before Zeek 3.0.
_make_install_dir_symlink("/usr/local/bro" "/usr/local/zeek")
endif ()
# Check whether we need to symlink directories used by versions
# before Zeek 3.0.
_make_install_dir_symlink("${CMAKE_INSTALL_PREFIX}/include/bro" "${CMAKE_INSTALL_PREFIX}/include/zeek")
_make_install_dir_symlink("${CMAKE_INSTALL_PREFIX}/share/bro" "${CMAKE_INSTALL_PREFIX}/share/zeek")
_make_install_dir_symlink("${CMAKE_INSTALL_PREFIX}/lib/bro" "${CMAKE_INSTALL_PREFIX}/lib/zeek")
endif ()
if ( SANITIZERS )
set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -fsanitize=${SANITIZERS} -fno-omit-frame-pointer")
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fsanitize=${SANITIZERS} -fno-omit-frame-pointer")
set(CMAKE_LD_FLAGS "${CMAKE_LD_FLAGS} -fsanitize=${SANITIZERS} -fno-omit-frame-pointer")
endif()
########################################################################
## Dependency Configuration
@ -257,7 +291,7 @@ string(TOLOWER ${CMAKE_BUILD_TYPE} CMAKE_BUILD_TYPE_LOWER)
configure_file(${CMAKE_CURRENT_SOURCE_DIR}/zeek-config.h.in
${CMAKE_CURRENT_BINARY_DIR}/zeek-config.h)
include_directories(${CMAKE_CURRENT_BINARY_DIR})
install(FILES ${CMAKE_CURRENT_BINARY_DIR}/zeek-config.h DESTINATION include/bro)
install(FILES ${CMAKE_CURRENT_BINARY_DIR}/zeek-config.h DESTINATION include/zeek)
if ( CAF_ROOT_DIR )
set(ZEEK_CONFIG_CAF_ROOT_DIR ${CAF_ROOT_DIR})
@ -281,7 +315,7 @@ configure_file(${CMAKE_CURRENT_SOURCE_DIR}/zeek-config.in
${CMAKE_CURRENT_BINARY_DIR}/zeek-config @ONLY)
install(PROGRAMS ${CMAKE_CURRENT_BINARY_DIR}/zeek-config DESTINATION bin)
install(DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}/cmake DESTINATION share/bro
install(DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}/cmake DESTINATION share/zeek
USE_SOURCE_PERMISSIONS)
# Install wrapper script for Bro-to-Zeek renaming.
@ -289,7 +323,7 @@ include(InstallShellScript)
include(InstallSymlink)
InstallShellScript("bin" "zeek-wrapper.in" "zeek-wrapper")
InstallSymlink("${CMAKE_INSTALL_PREFIX}/bin/zeek-wrapper" "${CMAKE_INSTALL_PREFIX}/bin/bro-config")
InstallSymlink("${CMAKE_INSTALL_PREFIX}/include/bro/zeek-config.h" "${CMAKE_INSTALL_PREFIX}/include/bro/bro-config.h")
InstallSymlink("${CMAKE_INSTALL_PREFIX}/include/zeek/zeek-config.h" "${CMAKE_INSTALL_PREFIX}/include/zeek/bro-config.h")
########################################################################
## Recurse on sub-directories

111
NEWS
View file

@ -18,6 +18,9 @@ New Functionality
- dns_NSEC
- dns_NSEC3
- Added support for parsing and logging DNS SPF resource records.
A new ``dns_SPF_reply`` event is also available.
- Zeek's Plugin framework now allows a patch version. If a patch version is not
provided, it will default to 0. To specify this, modify the plugin
Configuration class in your ``src/Plugin.cc`` and set
@ -79,9 +82,35 @@ New Functionality
- The ``/<re>/i`` convenience syntax for case-insensitive patterns is now
also allowed when specifying patterns used in signature files.
- Add a new "client_channels" field to rdp.log based on data parsed from
- New RDP functionality.
- New events:
- rdp_client_network_data
- rdp_client_security_data
- rdp_client_cluster_data
- rdp_native_encrypted_data
- Add a new "client_channels" field to rdp.log based on data parsed from
the Client Network Data (TS_UD_CS_NET) packet. The channel list is also
available in a new event, "rdp_client_network_data".
available in the new ``rdp_client_network_data`` event.
- Add parsing support for TLS 1.3 pre-shared key extension. This info
is available in the events: ``ssl_extension_pre_shared_key_client_hello``
and ``ssl_extension_pre_shared_key_server_hello``.
- Added/re-wrote support for NTP.
- Parsing support for modes 1-7, with parsed structures available in
the ``ntp_message`` event.
- An ntp.log is produced by default, containing data extracted from
NTP messages with modes 1-5.
- Add support for vector slicing operations. For example::
local v = vector(1, 2, 3, 4, 5);
v[2:4] = vector(6, 7, 8); # v is now [1, 2, 6, 7, 8, 5]
print v[:4]; # prints [1, 2, 6, 7]
Changed Functionality
---------------------
@ -105,6 +134,13 @@ Changed Functionality
installation, installing the newer Zeek version over it will retain that
file and even symlink the new ``zeekctl.cfg`` to it.
- The default install prefix is now ``/usr/local/zeek`` instead of
``/usr/local/bro``. If you have an existing installation that used
the previous default and are still using the new default when upgrading,
we'll crate ``/usr/local/zeek`` as a symlink to ``/usr/local/bro``.
Certain subdirectories will also get similar treatment: ``share/bro``,
``include/bro``, and ``lib/bro``.
- ``$prefix/share/bro/site/local.bro`` has been renamed to
``local.zeek``. If you have a ``local.bro`` file from a previous
installation, possibly with customizations made to it, the new
@ -237,6 +273,64 @@ Changed Functionality
@load policy/files/unified2
- The default value of ``peer_description`` has changed from "bro"
to "zeek". This won't effect most users, except for the fact that
this value may appear in several log files, so any external plugins
that have written unit tests that compare baselines of such log
files may need to be updated.
- The "remote_ip" field of "addr" type was removed from radius.log and
replaced with a field named "tunnel_client" of "string" type. The
reason for this is that the Tunnel-Client-Endpoint RADIUS attribute
this data is derived from may also be a FQDN, not just an IP address.
- The ``ssl_server_hello`` event's ``server_random`` parameter has been
changed to always include the full 32-byte field from the
ServerHello. Previously a 4-byte timestamp and 28-byte random data
were parsed separately as some TLS protocol versions specified a
separate timestamp field as part of the full 32-byte random sequence.
- The namespace used by all the builtin plugins that ship with Zeek have
changed to use "Zeek::" instead of "Bro::".
- Any Broker topic names used in scripts shipped with Zeek that
previously were prefixed with "bro/" are now prefixed with "zeek/"
instead.
In the case where external applications were using a "bro/" topic
to send data into a Bro process, a Zeek process still subscribes
to those topics in addition to the equivalently named "zeek/" topic.
In the case where external applications were using a "bro/" topic
to subscribe to remote messages or query data stores, there's no
backwards compatibility and external applications must be changed
to use the new "zeek/" topic. The thought is this change will have
low impact since most data published under "bro/" topic names is
intended for use only as a detail of implementing cluster-enabled
versions of various scripts.
A list of the most relevant/common topic names that could potentially
be used in external applications to consume/query remote data that
one may need to change:
- store names
- bro/known/services
- bro/known/hosts
- bro/known/certs
- cluster nodes
- bro/cluster/<node type>
- bro/cluster/node/<name>
- bro/cluster/nodeid/<id>
- logging
- bro/logs/<stream>
- The ``resp_ref`` argument was removed from the ``ocsp_response_bytes``
event. ``resp_ref`` was not used by anything in the codebase and could not be
passed to any other functions for further processing. The remainder of the
``ocsp_response_bytes`` is unchanged.
Removed Functionality
---------------------
@ -325,6 +419,9 @@ Removed Functionality
- ``log_encryption_key``
- ``state_dir``
- ``state_write_delay``
- ``ssl_ca_certificate``
- ``ssl_private_key``
- ``ssl_passphrase``
- The following constants were used as part of deprecated functionality in version 2.6
or below and are removed from this release:
@ -345,8 +442,18 @@ Removed Functionality
and &rotate_size attributes, which were deprecated in Bro 2.6, was removed. The ``-g``
command-line option (dump-config) which relied on this functionality was also removed.
- Functionality for writing state updates for variables with the
&synchronized attribute was removed. This entails the
``-x`` command-line option (print-state) as well as the
``capture_state_updates`` function.
- Removed the BroControl ``update`` command, which was deprecated in Bro 2.6.
- Functionality for writing/reading binary event streams was
removed. This functionality relied on the old communication code
anc was basically untested. The ``-R`` command-line option (replay)
as well as the ``capture_events`` function were removed.
Deprecated Functionality
------------------------

2
VERSION
View file

@ -1 +1 @@
2.6-379
2.6-500

27
configure vendored
View file

@ -31,9 +31,9 @@ Usage: $0 [OPTION]... [VAR=VALUE]...
(useful for cross-compiling)
Installation Directories:
--prefix=PREFIX installation directory [/usr/local/bro]
--prefix=PREFIX installation directory [/usr/local/zeek]
--scriptdir=PATH root installation directory for Zeek scripts
[PREFIX/share/bro]
[PREFIX/share/zeek]
--localstatedir=PATH when using ZeekControl, path to store log files
and run-time data (within log/ and spool/ subdirs)
[PREFIX]
@ -58,6 +58,7 @@ Usage: $0 [OPTION]... [VAR=VALUE]...
--disable-perftools don't try to build with Google Perftools
--disable-python don't try to build python bindings for Broker
--disable-broker-tests don't try to build Broker unit tests
--sanitizers=SANITIZERS comma-separated list of Clang sanitizers to enable
Required Packages in Non-Standard Locations:
--with-openssl=PATH path to OpenSSL install root
@ -105,6 +106,18 @@ Usage: $0 [OPTION]... [VAR=VALUE]...
sourcedir="$( cd "$( dirname "$0" )" && pwd )"
if [ ! -e "$sourcedir/cmake/COPYING" ] && [ -d "$sourcedir/.git" ]; then
echo "\
You seem to be missing the content of the cmake directory.
This typically means that you performed a non-recursive git clone of
Zeek. To check out the required subdirectories, please execute:
( cd $sourcedir && git submodule update --recursive --init )
" >&2;
exit 1;
fi
# Function to append a CMake cache entry definition to the
# CMakeCacheEntries variable.
# $1 is the cache entry variable name
@ -127,12 +140,12 @@ remove_cache_entry () {
# set defaults
builddir=build
prefix=/usr/local/bro
prefix=/usr/local/zeek
CMakeCacheEntries=""
append_cache_entry CMAKE_INSTALL_PREFIX PATH $prefix
append_cache_entry ZEEK_ROOT_DIR PATH $prefix
append_cache_entry PY_MOD_INSTALL_DIR PATH $prefix/lib/zeekctl
append_cache_entry ZEEK_SCRIPT_INSTALL_PATH STRING $prefix/share/bro
append_cache_entry ZEEK_SCRIPT_INSTALL_PATH STRING $prefix/share/zeek
append_cache_entry ZEEK_ETC_INSTALL_DIR PATH $prefix/etc
append_cache_entry ENABLE_DEBUG BOOL false
append_cache_entry ENABLE_PERFTOOLS BOOL false
@ -144,6 +157,7 @@ append_cache_entry INSTALL_ZEEKCTL BOOL true
append_cache_entry CPACK_SOURCE_IGNORE_FILES STRING
append_cache_entry ENABLE_MOBILE_IPV6 BOOL false
append_cache_entry DISABLE_PERFTOOLS BOOL false
append_cache_entry SANITIZERS STRING ""
# parse arguments
while [ $# -ne 0 ]; do
@ -216,6 +230,9 @@ while [ $# -ne 0 ]; do
append_cache_entry ENABLE_PERFTOOLS BOOL true
append_cache_entry ENABLE_PERFTOOLS_DEBUG BOOL true
;;
--sanitizers=*)
append_cache_entry SANITIZERS STRING $optarg
;;
--enable-jemalloc)
append_cache_entry ENABLE_JEMALLOC BOOL true
;;
@ -321,7 +338,7 @@ while [ $# -ne 0 ]; do
done
if [ "$user_set_scriptdir" != "true" ]; then
append_cache_entry ZEEK_SCRIPT_INSTALL_PATH STRING $prefix/share/bro
append_cache_entry ZEEK_SCRIPT_INSTALL_PATH STRING $prefix/share/zeek
fi
if [ "$user_set_conffilesdir" != "true" ]; then

2
scripts/base/frameworks/broker/main.zeek
View file

@ -113,7 +113,7 @@ export {
## The default topic prefix where logs will be published. The log's stream
## id is appended when writing to a particular stream.
const default_log_topic_prefix = "bro/logs/" &redef;
const default_log_topic_prefix = "zeek/logs/" &redef;
## The default implementation for :zeek:see:`Broker::log_topic`.
function default_log_topic(id: Log::ID, path: string): string

14
scripts/base/frameworks/cluster/main.zeek
View file

@ -17,31 +17,31 @@ export {
## The topic name used for exchanging messages that are relevant to
## logger nodes in a cluster. Used with broker-enabled cluster communication.
const logger_topic = "bro/cluster/logger" &redef;
const logger_topic = "zeek/cluster/logger" &redef;
## The topic name used for exchanging messages that are relevant to
## manager nodes in a cluster. Used with broker-enabled cluster communication.
const manager_topic = "bro/cluster/manager" &redef;
const manager_topic = "zeek/cluster/manager" &redef;
## The topic name used for exchanging messages that are relevant to
## proxy nodes in a cluster. Used with broker-enabled cluster communication.
const proxy_topic = "bro/cluster/proxy" &redef;
const proxy_topic = "zeek/cluster/proxy" &redef;
## The topic name used for exchanging messages that are relevant to
## worker nodes in a cluster. Used with broker-enabled cluster communication.
const worker_topic = "bro/cluster/worker" &redef;
const worker_topic = "zeek/cluster/worker" &redef;
## The topic name used for exchanging messages that are relevant to
## time machine nodes in a cluster. Used with broker-enabled cluster communication.
const time_machine_topic = "bro/cluster/time_machine" &redef;
const time_machine_topic = "zeek/cluster/time_machine" &redef;
## The topic prefix used for exchanging messages that are relevant to
## a named node in a cluster. Used with broker-enabled cluster communication.
const node_topic_prefix = "bro/cluster/node/" &redef;
const node_topic_prefix = "zeek/cluster/node/" &redef;
## The topic prefix used for exchanging messages that are relevant to
## a unique node in a cluster. Used with broker-enabled cluster communication.
const nodeid_topic_prefix = "bro/cluster/nodeid/" &redef;
const nodeid_topic_prefix = "zeek/cluster/nodeid/" &redef;
## Name of the node on which master data stores will be created if no other
## has already been specified by the user in :zeek:see:`Cluster::stores`.

6
scripts/base/frameworks/cluster/pools.zeek
View file

@ -60,17 +60,17 @@ export {
## The specification for :zeek:see:`Cluster::proxy_pool`.
global proxy_pool_spec: PoolSpec =
PoolSpec($topic = "bro/cluster/pool/proxy",
PoolSpec($topic = "zeek/cluster/pool/proxy",
$node_type = Cluster::PROXY) &redef;
## The specification for :zeek:see:`Cluster::worker_pool`.
global worker_pool_spec: PoolSpec =
PoolSpec($topic = "bro/cluster/pool/worker",
PoolSpec($topic = "zeek/cluster/pool/worker",
$node_type = Cluster::WORKER) &redef;
## The specification for :zeek:see:`Cluster::logger_pool`.
global logger_pool_spec: PoolSpec =
PoolSpec($topic = "bro/cluster/pool/logger",
PoolSpec($topic = "zeek/cluster/pool/logger",
$node_type = Cluster::LOGGER) &redef;
## A pool containing all the proxy nodes of a cluster.

269
scripts/base/init-bare.zeek
View file

@ -1113,9 +1113,6 @@ const table_expire_delay = 0.01 secs &redef;
## Time to wait before timing out a DNS request.
const dns_session_timeout = 10 sec &redef;
## Time to wait before timing out an NTP request.
const ntp_session_timeout = 300 sec &redef;
## Time to wait before timing out an RPC request.
const rpc_timeout = 24 sec &redef;
@ -1789,7 +1786,7 @@ type gtp_delete_pdp_ctx_response_elements: record {
};
# Prototypes of Zeek built-in functions.
@load base/bif/bro.bif
@load base/bif/zeek.bif
@load base/bif/stats.bif
@load base/bif/reporter.bif
@load base/bif/strings.bif
@ -2529,26 +2526,6 @@ export {
};
}
module GLOBAL;
## An NTP message.
##
## .. zeek:see:: ntp_message
type ntp_msg: record {
id: count; ##< Message ID.
code: count; ##< Message code.
stratum: count; ##< Stratum.
poll: count; ##< Poll.
precision: int; ##< Precision.
distance: interval; ##< Distance.
dispersion: interval; ##< Dispersion.
ref_t: time; ##< Reference time.
originate_t: time; ##< Originating time.
receive_t: time; ##< Receive time.
xmit_t: time; ##< Send time.
};
module NTLM;
export {
@ -4140,6 +4117,10 @@ export {
SignatureAlgorithm: count; ##< Signature algorithm number
};
type PSKIdentity: record {
identity: string; ##< PSK identity
obfuscated_ticket_age: count;
};
## Number of non-DTLS frames that can occur in a DTLS connection before
## parsing of the connection is suspended.
@ -4161,6 +4142,8 @@ module GLOBAL;
## directly and then remove this alias.
type signature_and_hashalgorithm_vec: vector of SSL::SignatureAndHashAlgorithm;
type psk_identity_vec: vector of SSL::PSKIdentity;
module X509;
export {
type Certificate: record {
@ -4276,10 +4259,28 @@ export {
dig_product_id: string &optional;
};
## The TS_UD_CS_SEC data block contains security-related information used
## to advertise client cryptographic support.
type RDP::ClientSecurityData: record {
## Cryptographic encryption methods supported by the client and used in
## conjunction with Standard RDP Security. Known flags:
##
## - 0x00000001: support for 40-bit session encryption keys
## - 0x00000002: support for 128-bit session encryption keys
## - 0x00000008: support for 56-bit session encryption keys
## - 0x00000010: support for FIPS compliant encryption and MAC methods
encryption_methods: count;
## Only used in French locale and designates the encryption method. If
## non-zero, then encryption_methods should be set to 0.
ext_encryption_methods: count;
};
## Name and flags for a single channel requested by the client.
type RDP::ClientChannelDef: record {
## A unique name for the channel
name: string;
## Channel Def raw options as count
options: count;
## Absence of this flag indicates that this channel is
## a placeholder and that the server MUST NOT set it up.
initialized: bool;
@ -4305,11 +4306,35 @@ export {
persistent: bool;
};
## The TS_UD_CS_CLUSTER data block is sent by the client to the server
## either to advertise that it can support the Server Redirection PDUs
## or to request a connection to a given session identifier.
type RDP::ClientClusterData: record {
## Cluster information flags.
flags: count;
## If the *redir_sessionid_field_valid* flag is set, this field
## contains a valid session identifier to which the client requests
## to connect.
redir_session_id: count;
## The client can receive server session redirection packets.
## If this flag is set, the *svr_session_redir_version_mask*
## field MUST contain the server session redirection version that
## the client supports.
redir_supported: bool;
## The server session redirection version that the client supports.
svr_session_redir_version_mask: count;
## Whether the *redir_session_id* field identifies a session on
## the server to associate with the connection.
redir_sessionid_field_valid: bool;
## The client logged on with a smart card.
redir_smartcard: bool;
};
## The list of channels requested by the client.
type RDP::ClientChannelList: vector of ClientChannelDef;
}
@load base/bif/plugins/Bro_SNMP.types.bif
@load base/bif/plugins/Zeek_SNMP.types.bif
module SNMP;
export {
@ -4431,7 +4456,7 @@ export {
};
}
@load base/bif/plugins/Bro_KRB.types.bif
@load base/bif/plugins/Zeek_KRB.types.bif
module KRB;
export {
@ -4711,22 +4736,6 @@ const report_gaps_for_partial = F &redef;
## controlled for reproducing results.
const exit_only_after_terminate = F &redef;
## The CA certificate file to authorize remote Zeeks/Broccolis.
##
## .. zeek:see:: ssl_private_key ssl_passphrase
const ssl_ca_certificate = "<undefined>" &redef;
## File containing our private key and our certificate.
##
## .. zeek:see:: ssl_ca_certificate ssl_passphrase
const ssl_private_key = "<undefined>" &redef;
## The passphrase for our private key. Keeping this undefined
## causes Zeek to prompt for the passphrase.
##
## .. zeek:see:: ssl_private_key ssl_ca_certificate
const ssl_passphrase = "<undefined>" &redef;
## Default mode for Zeek's user-space dynamic packet filter. If true, packets
## that aren't explicitly allowed through, are dropped from any further
## processing.
@ -4743,7 +4752,7 @@ const packet_filter_default = F &redef;
const sig_max_group_size = 50 &redef;
## Description transmitted to remote communication peers for identification.
const peer_description = "bro" &redef;
const peer_description = "zeek" &redef;
## The number of IO chunks allowed to be buffered between the child
## and parent process of remote communication before Zeek starts dropping
@ -4953,6 +4962,180 @@ export {
const max_frame_size = 65536 &redef;
}
module NTP;
export {
## NTP standard message as defined in :rfc:`5905` for modes 1-5
## This record contains the standard fields used by the NTP protocol
## for standard syncronization operations.
type NTP::StandardMessage: record {
## This value mainly identifies the type of server (primary server,
## secondary server, etc.). Possible values, as in :rfc:`5905`, are:
##
## * 0 -> unspecified or invalid
## * 1 -> primary server (e.g., equipped with a GPS receiver)
## * 2-15 -> secondary server (via NTP)
## * 16 -> unsynchronized
## * 17-255 -> reserved
##
## For stratum 0, a *kiss_code* can be given for debugging and
## monitoring.
stratum: count;
## The maximum interval between successive messages.
poll: interval;
## The precision of the system clock.
precision: interval;
## Root delay. The total round-trip delay to the reference clock.
root_delay: interval;
## Root Dispersion. The total dispersion to the reference clock.
root_disp: interval;
## For stratum 0, four-character ASCII string used for debugging and
## monitoring. Values are defined in :rfc:`1345`.
kiss_code: string &optional;
## Reference ID. For stratum 1, this is the ID assigned to the
## reference clock by IANA.
## For example: GOES, GPS, GAL, etc. (see :rfc:`5905`)
ref_id: string &optional;
## Above stratum 1, when using IPv4, the IP address of the reference
## clock. Note that the NTP protocol did not originally specify a
## large enough field to represent IPv6 addresses, so they use
## the first four bytes of the MD5 hash of the reference clock's
## IPv6 address (i.e. an IPv4 address here is not necessarily IPv4).
ref_addr: addr &optional;
## Reference timestamp. Time when the system clock was last set or
## correct.
ref_time: time;
## Origin timestamp. Time at the client when the request departed for
## the NTP server.
org_time: time;
## Receive timestamp. Time at the server when the request arrived from
## the NTP client.
rec_time: time;
## Transmit timestamp. Time at the server when the response departed
# for the NTP client.
xmt_time: time;
## Key used to designate a secret MD5 key.
key_id: count &optional;
## MD5 hash computed over the key followed by the NTP packet header and
## extension fields.
digest: string &optional;
## Number of extension fields (which are not currently parsed).
num_exts: count &default=0;
};
## NTP control message as defined in :rfc:`1119` for mode=6
## This record contains the fields used by the NTP protocol
## for control operations.
type NTP::ControlMessage: record {
## An integer specifying the command function. Values currently defined:
##
## * 1 read status command/response
## * 2 read variables command/response
## * 3 write variables command/response
## * 4 read clock variables command/response
## * 5 write clock variables command/response
## * 6 set trap address/port command/response
## * 7 trap response
##
## Other values are reserved.
op_code: count;
## The response bit. Set to zero for commands, one for responses.
resp_bit: bool;
## The error bit. Set to zero for normal response, one for error
## response.
err_bit: bool;
## The more bit. Set to zero for last fragment, one for all others.
more_bit: bool;
## The sequence number of the command or response.
sequence: count;
## The current status of the system, peer or clock.
#TODO: this can be further parsed internally
status: count;
## A 16-bit integer identifying a valid association.
association_id: count;
## Message data for the command or response + Authenticator (optional).
data: string &optional;
## This is an integer identifying the cryptographic
## key used to generate the message-authentication code.
key_id: count &optional;
## This is a crypto-checksum computed by the encryption procedure.
crypto_checksum: string &optional;
};
## NTP mode 7 message. Note that this is not defined in any RFC and is
## implementation dependent. We used the official implementation from the
## `NTP official project <www.ntp.org>`_. A mode 7 packet is used
## exchanging data between an NTP server and a client for purposes other
## than time synchronization, e.g. monitoring, statistics gathering and
## configuration. For details see the documentation from the `NTP official
## project <www.ntp.org>`_, code v. ntp-4.2.8p13, in include/ntp_request.h.
type NTP::Mode7Message: record {
## An implementation-specific code which specifies the
## operation to be (which has been) performed and/or the
## format and semantics of the data included in the packet.
req_code: count;
## The authenticated bit. If set, this packet is authenticated.
auth_bit: bool;
## For a multipacket response, contains the sequence
## number of this packet. 0 is the first in the sequence,
## 127 (or less) is the last. The More Bit must be set in
## all packets but the last.
sequence: count;
## The number of the implementation this request code
## is defined by. An implementation number of zero is used
## for requst codes/data formats which all implementations
## agree on. Implementation number 255 is reserved (for
## extensions, in case we run out).
implementation: count;
## Must be 0 for a request. For a response, holds an error
## code relating to the request. If nonzero, the operation
## requested wasn't performed.
##
## * 0 - no error
## * 1 - incompatible implementation number
## * 2 - unimplemented request code
## * 3 - format error (wrong data items, data size, packet size etc.)
## * 4 - no data available (e.g. request for details on unknown peer)
## * 5 - unknown
## * 6 - unknown
## * 7 - authentication failure (i.e. permission denied)
err: count;
## Rest of data
data: string &optional;
};
## NTP message as defined in :rfc:`5905`. Does include fields for mode 7,
## reserved for private use in :rfc:`5905`, but used in some implementation
## for commands such as "monlist".
type NTP::Message: record {
## The NTP version number (1, 2, 3, 4).
version: count;
## The NTP mode being used. Possible values are:
##
## * 1 - symmetric active
## * 2 - symmetric passive
## * 3 - client
## * 4 - server
## * 5 - broadcast
## * 6 - NTP control message
## * 7 - reserved for private use
mode: count;
## If mode 1-5, the standard fields for syncronization operations are
## here. See :rfc:`5905`
std_msg: NTP::StandardMessage &optional;
## If mode 6, the fields for control operations are here.
## See :rfc:`1119`
control_msg: NTP::ControlMessage &optional;
## If mode 7, the fields for extra operations are here.
## Note that this is not defined in any RFC
## and is implementation dependent. We used the official implementation
## from the `NTP official project <www.ntp.org>`_.
## A mode 7 packet is used exchanging data between an NTP server
## and a client for purposes other than time synchronization, e.g.
## monitoring, statistics gathering and configuration.
mode7_msg: NTP::Mode7Message &optional;
};
}
module Cluster;
export {
type Cluster::Pool: record {};

1
scripts/base/init-default.zeek
View file

@ -56,6 +56,7 @@
@load base/protocols/modbus
@load base/protocols/mysql
@load base/protocols/ntlm
@load base/protocols/ntp
@load base/protocols/pop3
@load base/protocols/radius
@load base/protocols/rdp

15
scripts/base/protocols/dns/main.zeek
View file

@ -456,6 +456,21 @@ event dns_TXT_reply(c: connection, msg: dns_msg, ans: dns_answer, strs: string_v
hook DNS::do_reply(c, msg, ans, txt_strings);
}
event dns_SPF_reply(c: connection, msg: dns_msg, ans: dns_answer, strs: string_vec) &priority=5
{
local spf_strings: string = "";
for ( i in strs )
{
if ( i > 0 )
spf_strings += " ";
spf_strings += fmt("SPF %d %s", |strs[i]|, strs[i]);
}
hook DNS::do_reply(c, msg, ans, spf_strings);
}
event dns_AAAA_reply(c: connection, msg: dns_msg, ans: dns_answer, a: addr) &priority=5
{
hook DNS::do_reply(c, msg, ans, fmt("%s", a));

2
scripts/base/protocols/ntp/__load__.zeek Normal file
View file

@ -0,0 +1,2 @@
@load ./main
@load ./consts

15
scripts/base/protocols/ntp/consts.zeek Normal file
View file

@ -0,0 +1,15 @@
module NTP;
export {
## The descriptions of the NTP mode value, as described
## in :rfc:`5905`, Figure 1
const modes: table[count] of string = {
[1] = "symmetric active",
[2] = "symmetric passive",
[3] = "client",
[4] = "server",
[5] = "broadcast server",
[6] = "broadcast client",
[7] = "reserved",
} &default=function(i: count):string { return fmt("unknown-%d", i); } &redef;
}

107
scripts/base/protocols/ntp/main.zeek Normal file
View file

@ -0,0 +1,107 @@
module NTP;
export {
redef enum Log::ID += { LOG };
type Info: record {
## Timestamp for when the event happened.
ts: time &log;
## Unique ID for the connection.
uid: string &log;
## The connection's 4-tuple of endpoint addresses/ports.
id: conn_id &log;
## The NTP version number (1, 2, 3, 4).
version: count &log;
## The NTP mode being used.
mode: count &log;
## The stratum (primary server, secondary server, etc.).
stratum: count &log;
## The maximum interval between successive messages.
poll: interval &log;
## The precision of the system clock.
precision: interval &log;
## Total round-trip delay to the reference clock.
root_delay: interval &log;
## Total dispersion to the reference clock.
root_disp: interval &log;
## For stratum 0, 4 character string used for debugging.
## For stratum 1, ID assigned to the reference clock by IANA.
## Above stratum 1, when using IPv4, the IP address of the reference
## clock. Note that the NTP protocol did not originally specify a
## large enough field to represent IPv6 addresses, so they use
## the first four bytes of the MD5 hash of the reference clock's
## IPv6 address (i.e. an IPv4 address here is not necessarily IPv4).
ref_id: string &log;
## Time when the system clock was last set or correct.
ref_time: time &log;
## Time at the client when the request departed for the NTP server.
org_time: time &log;
## Time at the server when the request arrived from the NTP client.
rec_time: time &log;
## Time at the server when the response departed for the NTP client.
xmt_time: time &log;
## Number of extension fields (which are not currently parsed).
num_exts: count &default=0 &log;
};
## Event that can be handled to access the NTP record as it is sent on
## to the logging framework.
global log_ntp: event(rec: Info);
}
redef record connection += {
ntp: Info &optional;
};
const ports = { 123/udp };
redef likely_server_ports += { ports };
event ntp_message(c: connection, is_orig: bool, msg: NTP::Message) &priority=5
{
local info: Info;
info$ts = network_time();
info$uid = c$uid;
info$id = c$id;
info$version = msg$version;
info$mode = msg$mode;
if ( msg$mode < 6 )
{
info$stratum = msg$std_msg$stratum;
info$poll = msg$std_msg$poll;
info$precision = msg$std_msg$precision;
info$root_delay = msg$std_msg$root_delay;
info$root_disp = msg$std_msg$root_disp;
if ( msg$std_msg?$kiss_code )
info$ref_id = msg$std_msg$kiss_code;
else if ( msg$std_msg?$ref_id )
info$ref_id = msg$std_msg$ref_id;
else if ( msg$std_msg?$ref_addr )
info$ref_id= cat(msg$std_msg$ref_addr);
info$ref_time = msg$std_msg$ref_time;
info$org_time = msg$std_msg$org_time;
info$rec_time = msg$std_msg$rec_time;
info$xmt_time = msg$std_msg$xmt_time;
info$num_exts = msg$std_msg$num_exts;
}
# Copy the present packet info into the connection record
# If more ntp packets are sent on the same connection, the newest one
# will overwrite the previous
c$ntp = info;
}
event ntp_message(c: connection, is_orig: bool, msg: NTP::Message) &priority=-5
{
if ( msg$mode < 6 )
Log::write(NTP::LOG, c$ntp);
}
event zeek_init() &priority=5
{
Analyzer::register_for_ports(Analyzer::ANALYZER_NTP, ports);
Log::create_stream(NTP::LOG, [$columns = Info, $ev = log_ntp]);
}

11
scripts/base/protocols/radius/main.zeek
View file

@ -24,9 +24,10 @@ export {
## and the network access server is not required to honor
## the address.
framed_addr : addr &log &optional;
## Remote IP address, if present. This is collected
## from the Tunnel-Client-Endpoint attribute.
remote_ip : addr &log &optional;
## Address (IPv4, IPv6, or FQDN) of the initiator end of the tunnel,
## if present. This is collected from the Tunnel-Client-Endpoint
## attribute.
tunnel_client: string &log &optional;
## Connect info, if present.
connect_info : string &log &optional;
## Reply message from the server challenge. This is
@ -85,8 +86,8 @@ event radius_message(c: connection, result: RADIUS::Message) &priority=5
c$radius$mac = normalize_mac(result$attributes[31][0]);
# Tunnel-Client-EndPoint (useful for VPNs)
if ( ! c$radius?$remote_ip && 66 in result$attributes )
c$radius$remote_ip = to_addr(result$attributes[66][0]);
if ( ! c$radius?$tunnel_client && 66 in result$attributes )
c$radius$tunnel_client = result$attributes[66][0];
# Connect-Info
if ( ! c$radius?$connect_info && 77 in result$attributes )

2
scripts/policy/protocols/conn/known-hosts.zeek
View file

@ -36,7 +36,7 @@ export {
global host_store: Cluster::StoreInfo;
## The Broker topic name to use for :zeek:see:`Known::host_store`.
const host_store_name = "bro/known/hosts" &redef;
const host_store_name = "zeek/known/hosts" &redef;
## The expiry interval of new entries in :zeek:see:`Known::host_store`.
## This also changes the interval at which hosts get logged.

2
scripts/policy/protocols/conn/known-services.zeek
View file

@ -48,7 +48,7 @@ export {
global service_store: Cluster::StoreInfo;
## The Broker topic name to use for :zeek:see:`Known::service_store`.
const service_store_name = "bro/known/services" &redef;
const service_store_name = "zeek/known/services" &redef;
## The expiry interval of new entries in :zeek:see:`Known::service_store`.
## This also changes the interval at which services get logged.

2
scripts/policy/protocols/ssl/known-certs.zeek
View file

@ -48,7 +48,7 @@ export {
global cert_store: Cluster::StoreInfo;
## The Broker topic name to use for :zeek:see:`Known::cert_store`.
const cert_store_name = "bro/known/certs" &redef;
const cert_store_name = "zeek/known/certs" &redef;
## The expiry interval of new entries in :zeek:see:`Known::cert_store`.
## This also changes the interval at which certs get logged.

68
src/Attr.cc
View file

@ -4,7 +4,6 @@
#include "Attr.h"
#include "Expr.h"
#include "Serializer.h"
#include "threading/SerialTypes.h"
const char* attr_name(attr_tag t)
@ -509,70 +508,3 @@ bool Attributes::operator==(const Attributes& other) const
return true;
}
bool Attributes::Serialize(SerialInfo* info) const
{
return SerialObj::Serialize(info);
}
Attributes* Attributes::Unserialize(UnserialInfo* info)
{
return (Attributes*) SerialObj::Unserialize(info, SER_ATTRIBUTES);
}
IMPLEMENT_SERIAL(Attributes, SER_ATTRIBUTES);
bool Attributes::DoSerialize(SerialInfo* info) const
{
DO_SERIALIZE(SER_ATTRIBUTES, BroObj);
info->s->WriteOpenTag("Attributes");
assert(type);
if ( ! (type->Serialize(info) && SERIALIZE(attrs->length())) )
return false;
loop_over_list((*attrs), i)
{
Attr* a = (*attrs)[i];
Expr* e = a->AttrExpr();
SERIALIZE_OPTIONAL(e);
if ( ! SERIALIZE(char(a->Tag())) )
return false;
}
info->s->WriteCloseTag("Attributes");
return true;
}
bool Attributes::DoUnserialize(UnserialInfo* info)
{
DO_UNSERIALIZE(BroObj);
type = BroType::Unserialize(info);
if ( ! type )
return false;
int len;
if ( ! UNSERIALIZE(&len) )
return false;
attrs = new attr_list(len);
while ( len-- )
{
Expr* e;
UNSERIALIZE_OPTIONAL(e, Expr::Unserialize(info))
char tag;
if ( ! UNSERIALIZE(&tag) )
{
delete e;
return false;
}
attrs->append(new Attr((attr_tag)tag, e));
}
return true;
}

5
src/Attr.h
View file

@ -92,17 +92,12 @@ public:
attr_list* Attrs() { return attrs; }
bool Serialize(SerialInfo* info) const;
static Attributes* Unserialize(UnserialInfo* info);
bool operator==(const Attributes& other) const;
protected:
Attributes() : type(), attrs(), in_record() { }
void CheckAttr(Attr* attr);
DECLARE_SERIAL(Attributes);
BroType* type;
attr_list* attrs;
bool in_record;

2
src/BroString.cc
View file

@ -288,7 +288,7 @@ void BroString::ToUpper()
BroString* BroString::GetSubstring(int start, int len) const
{
// This code used to live in bro.bif's sub_bytes() routine.
// This code used to live in zeek.bif's sub_bytes() routine.
if ( start < 0 || start > n )
return 0;

17
src/CMakeLists.txt
View file

@ -10,8 +10,8 @@ set(bro_ALL_GENERATED_OUTPUTS CACHE INTERNAL "automatically generated files" FO
set(bro_AUTO_BIFS CACHE INTERNAL "BIFs for automatic inclusion" FORCE)
set(bro_REGISTER_BIFS CACHE INTERNAL "BIFs for automatic registering" FORCE)
set(bro_BASE_BIF_SCRIPTS CACHE INTERNAL "Bro script stubs for BIFs in base distribution of Bro" FORCE)
set(bro_PLUGIN_BIF_SCRIPTS CACHE INTERNAL "Bro script stubs for BIFs in Bro plugins" FORCE)
set(bro_BASE_BIF_SCRIPTS CACHE INTERNAL "Zeek script stubs for BIFs in base distribution of Zeek" FORCE)
set(bro_PLUGIN_BIF_SCRIPTS CACHE INTERNAL "Zeek script stubs for BIFs in Zeek plugins" FORCE)
# If TRUE, use CMake's object libraries for sub-directories instead of
# static libraries. This requires CMake >= 2.8.8.
@ -103,7 +103,7 @@ set_property(SOURCE scan.cc APPEND_STRING PROPERTY COMPILE_FLAGS "-Wno-sign-comp
include(BifCl)
set(BIF_SRCS
bro.bif
zeek.bif
stats.bif
event.bif
const.bif
@ -247,7 +247,6 @@ set(bro_SRCS
Brofiler.cc
BroString.cc
CCL.cc
ChunkedIO.cc
CompHash.cc
Conn.cc
ConvertUTF.c
@ -302,10 +301,8 @@ set(bro_SRCS
SmithWaterman.cc
Scope.cc
SerializationFormat.cc
SerialObj.cc
Serializer.cc
Sessions.cc
StateAccess.cc
Notifier.cc
Stats.cc
Stmt.cc
Tag.cc
@ -418,7 +415,7 @@ install(CODE "
")
install(DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}/
DESTINATION include/bro
DESTINATION include/zeek
FILES_MATCHING
PATTERN "*.h"
PATTERN "*.pac"
@ -426,7 +423,7 @@ install(DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}/
)
install(DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}/
DESTINATION include/bro
DESTINATION include/zeek
FILES_MATCHING
PATTERN "*.bif.func_h"
PATTERN "*.bif.netvar_h"
@ -435,5 +432,5 @@ install(DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}/
)
install(FILES ${CMAKE_CURRENT_SOURCE_DIR}/3rdparty/sqlite3.h
DESTINATION include/bro/3rdparty
DESTINATION include/zeek/3rdparty
)

1357
src/ChunkedIO.cc
View file

File diff suppressed because it is too large Load diff

362
src/ChunkedIO.h
View file

@ -1,362 +0,0 @@
// Implements non-blocking chunk-wise I/O.
#ifndef CHUNKEDIO_H
#define CHUNKEDIO_H
#include "zeek-config.h"
#include "List.h"
#include "util.h"
#include "Flare.h"
#include "iosource/FD_Set.h"
#include <list>
#ifdef NEED_KRB5_H
# include <krb5.h>
#endif
class CompressedChunkedIO;
// #define DEBUG_COMMUNICATION 10
// Abstract base class.
class ChunkedIO {
public:
ChunkedIO();
virtual ~ChunkedIO() { }
struct Chunk {
typedef void (*FreeFunc)(char*);
static void free_func_free(char* data) { free(data); }
static void free_func_delete(char* data) { delete [] data; }
Chunk()
: data(), len(), free_func(free_func_delete)
{ }
// Takes ownership of data.
Chunk(char* arg_data, uint32 arg_len,
FreeFunc arg_ff = free_func_delete)
: data(arg_data), len(arg_len), free_func(arg_ff)
{ }
~Chunk()
{ free_func(data); }
char* data;
uint32 len;
FreeFunc free_func;
};
// Initialization before any I/O operation is performed. Returns false
// on any form of error.
virtual bool Init() { return true; }
// Tries to read the next chunk of data. If it can be read completely,
// a pointer to it is returned in 'chunk' (ownership of chunk is
// passed). If not, 'chunk' is set to nil. Returns false if any
// I/O error occurred (use Eof() to see if it's an end-of-file).
// If 'may_block' is true, we explicitly allow blocking.
virtual bool Read(Chunk** chunk, bool may_block = false) = 0;
// Puts the chunk into the write queue and writes as much data
// as possible (takes ownership of chunk).
// Returns false on any I/O error.
virtual bool Write(Chunk* chunk) = 0;
// Tries to write as much as currently possible.
// Returns false on any I/O error.
virtual bool Flush() = 0;
// If an I/O error has been encountered, returns a string describing it.
virtual const char* Error() = 0;
// Return true if there is currently at least one chunk available
// for reading.
virtual bool CanRead() = 0;
// Return true if there is currently at least one chunk waiting to be
// written.
virtual bool CanWrite() = 0;
// Returns true if source believes that there won't be much data soon.
virtual bool IsIdle() = 0;
// Returns true if internal write buffers are about to fill up.
virtual bool IsFillingUp() = 0;
// Throws away buffered data.
virtual void Clear() = 0;
// Returns true,if end-of-file has been reached.
virtual bool Eof() = 0;
// Returns underlying fd if available, -1 otherwise.
virtual int Fd() { return -1; }
// Returns supplementary file descriptors that become read-ready in order
// to signal that there is some work that can be performed.
virtual iosource::FD_Set ExtraReadFDs() const
{ return iosource::FD_Set(); }
// Makes sure that no additional protocol data is written into
// the output stream. If this is activated, the output cannot
// be read again by any of these classes!
void MakePure() { pure = true; }
bool IsPure() { return pure; }
// Writes a log message to the error_fd.
void Log(const char* str);
struct Statistics {
Statistics()
{
bytes_read = 0;
bytes_written = 0;
chunks_read = 0;
chunks_written = 0;
reads = 0;
writes = 0;
pending = 0;
}
unsigned long bytes_read;
unsigned long bytes_written;
unsigned long chunks_read;
unsigned long chunks_written;
unsigned long reads; // # calls which transferred > 0 bytes
unsigned long writes;
unsigned long pending;
};
// Returns raw statistics.
const Statistics* Stats() const { return &stats; }
// Puts a formatted string containing statistics into buffer.
virtual void Stats(char* buffer, int length);
#ifdef DEBUG_COMMUNICATION
void DumpDebugData(const char* basefnname, bool want_reads);
#endif
protected:
void InternalError(const char* msg)
// We can't use the reporter here as we might be running in a
// sub-process.
{ fprintf(stderr, "%s", msg); abort(); }
Statistics stats;
const char* tag;
#ifdef DEBUG_COMMUNICATION
void AddToBuffer(char* data, bool is_read)
{ AddToBuffer(strlen(data), data, is_read); }
void AddToBuffer(uint32 len, char* data, bool is_read);
void AddToBuffer(Chunk* chunk, bool is_read);
std::list<Chunk*> data_read;
std::list<Chunk*> data_written;
#endif
private:
bool pure;
};
// Chunked I/O using a file descriptor.
class ChunkedIOFd : public ChunkedIO {
public:
// fd is an open bidirectional file descriptor, tag is used in error
// messages, and pid gives a pid to monitor (if the process dies, we
// return EOF).
ChunkedIOFd(int fd, const char* tag, pid_t pid = 0);
~ChunkedIOFd() override;
bool Read(Chunk** chunk, bool may_block = false) override;
bool Write(Chunk* chunk) override;
bool Flush() override;
const char* Error() override;
bool CanRead() override;
bool CanWrite() override;
bool IsIdle() override;
bool IsFillingUp() override;
void Clear() override;
bool Eof() override { return eof; }
int Fd() override { return fd; }
iosource::FD_Set ExtraReadFDs() const override;
void Stats(char* buffer, int length) override;
private:
bool PutIntoWriteBuffer(Chunk* chunk);
bool FlushWriteBuffer();
Chunk* ExtractChunk();
// Returns size of next chunk in buffer or 0 if none.
uint32 ChunkAvailable();
// Flushes if it thinks it is time to.
bool OptionalFlush();
// Concatenates the the data of the two chunks forming a new one.
// The old chunkds are deleted.
Chunk* ConcatChunks(Chunk* c1, Chunk* c2);
// Reads/writes on chunk of upto BUFFER_SIZE bytes.
bool WriteChunk(Chunk* chunk, bool partial);
bool ReadChunk(Chunk** chunk, bool may_block);
int fd;
bool eof;
double last_flush;
int failed_reads;
// Optimally, this should match the file descriptor's
// buffer size (for sockets, it may be helpful to
// increase the send/receive buffers).
static const unsigned int BUFFER_SIZE = 1024 * 1024 * 1;
// We 'or' this to the length of a data chunk to mark
// that it's part of a larger one. This has to be larger
// than BUFFER_SIZE.
static const uint32 FLAG_PARTIAL = 0x80000000;
char* read_buffer;
uint32 read_len;
uint32 read_pos;
Chunk* partial; // when we read an oversized chunk, we store it here
char* write_buffer;
uint32 write_len;
uint32 write_pos;
struct ChunkQueue {
Chunk* chunk;
ChunkQueue* next;
};
// Chunks that don't fit into our write buffer.
ChunkQueue* pending_head;
ChunkQueue* pending_tail;
pid_t pid;
bro::Flare write_flare;
bro::Flare read_flare;
};
// From OpenSSL. We forward-declare these here to avoid introducing a
// dependency on OpenSSL headers just for this header file.
typedef struct ssl_ctx_st SSL_CTX;
typedef struct ssl_st SSL;
// Chunked I/O using an SSL connection.
class ChunkedIOSSL : public ChunkedIO {
public:
// Argument is an open socket and a flag indicating whether we are the
// server side of the connection.
ChunkedIOSSL(int socket, bool server);
~ChunkedIOSSL() override;
bool Init() override;
bool Read(Chunk** chunk, bool mayblock = false) override;
bool Write(Chunk* chunk) override;
bool Flush() override;
const char* Error() override;
bool CanRead() override;
bool CanWrite() override;
bool IsIdle() override;
bool IsFillingUp() override;
void Clear() override;
bool Eof() override { return eof; }
int Fd() override { return socket; }
iosource::FD_Set ExtraReadFDs() const override;
void Stats(char* buffer, int length) override;
private:
// Only returns true if all data has been read. If not, call
// it again with the same parameters as long as error is not
// set to true.
bool ReadData(char* p, uint32 len, bool* error);
// Same for writing.
bool WriteData(char* p, uint32 len, bool* error);
int socket;
int last_ret; // last error code
bool eof;
bool server; // are we the server?
bool setup; // has the connection been setup successfully?
SSL* ssl;
// Write queue.
struct Queue {
Chunk* chunk;
Queue* next;
};
// The chunk part we are reading/writing
enum State { LEN, DATA };
State write_state;
Queue* write_head;
Queue* write_tail;
State read_state;
Chunk* read_chunk;
char* read_ptr;
// One SSL for all connections.
static SSL_CTX* ctx;
bro::Flare write_flare;
};
#include <zlib.h>
// Wrapper class around a another ChunkedIO which the (un-)compresses data.
class CompressedChunkedIO : public ChunkedIO {
public:
explicit CompressedChunkedIO(ChunkedIO* arg_io) // takes ownership
: io(arg_io), zin(), zout(), error(), compress(), uncompress(),
uncompressed_bytes_read(), uncompressed_bytes_written() {}
~CompressedChunkedIO() override { delete io; }
bool Init() override; // does *not* call arg_io->Init()
bool Read(Chunk** chunk, bool may_block = false) override;
bool Write(Chunk* chunk) override;
bool Flush() override { return io->Flush(); }
const char* Error() override { return error ? error : io->Error(); }
bool CanRead() override { return io->CanRead(); }
bool CanWrite() override { return io->CanWrite(); }
bool IsIdle() override { return io->IsIdle(); }
bool IsFillingUp() override { return io->IsFillingUp(); }
void Clear() override { return io->Clear(); }
bool Eof() override { return io->Eof(); }
int Fd() override { return io->Fd(); }
iosource::FD_Set ExtraReadFDs() const override
{ return io->ExtraReadFDs(); }
void Stats(char* buffer, int length) override;
void EnableCompression(int level)
{ deflateInit(&zout, level); compress = true; }
void EnableDecompression()
{ inflateInit(&zin); uncompress = true; }
protected:
// Only compress block with size >= this.
static const unsigned int MIN_COMPRESS_SIZE = 30;
ChunkedIO* io;
z_stream zin;
z_stream zout;
const char* error;
bool compress;
bool uncompress;
// Keep some statistics.
unsigned long uncompressed_bytes_read;
unsigned long uncompressed_bytes_written;
};
#endif

193
src/Conn.cc
View file

@ -50,70 +50,10 @@ void ConnectionTimer::Dispatch(double t, int is_expire)
reporter->InternalError("reference count inconsistency in ConnectionTimer::Dispatch");
}
IMPLEMENT_SERIAL(ConnectionTimer, SER_CONNECTION_TIMER);
bool ConnectionTimer::DoSerialize(SerialInfo* info) const
{
DO_SERIALIZE(SER_CONNECTION_TIMER, Timer);
// We enumerate all the possible timer functions here ... This
// has to match the list is DoUnserialize()!
char type = 0;
if ( timer == timer_func(&Connection::DeleteTimer) )
type = 1;
else if ( timer == timer_func(&Connection::InactivityTimer) )
type = 2;
else if ( timer == timer_func(&Connection::StatusUpdateTimer) )
type = 3;
else if ( timer == timer_func(&Connection::RemoveConnectionTimer) )
type = 4;
else
reporter->InternalError("unknown function in ConnectionTimer::DoSerialize()");
return conn->Serialize(info) && SERIALIZE(type) && SERIALIZE(do_expire);
}
bool ConnectionTimer::DoUnserialize(UnserialInfo* info)
{
DO_UNSERIALIZE(Timer);
conn = Connection::Unserialize(info);
if ( ! conn )
return false;
char type;
if ( ! UNSERIALIZE(&type) || ! UNSERIALIZE(&do_expire) )
return false;
switch ( type ) {
case 1:
timer = timer_func(&Connection::DeleteTimer);
break;
case 2:
timer = timer_func(&Connection::InactivityTimer);
break;
case 3:
timer = timer_func(&Connection::StatusUpdateTimer);
break;
case 4:
timer = timer_func(&Connection::RemoveConnectionTimer);
break;
default:
info->s->Error("unknown connection timer function");
return false;
}
return true;
}
uint64 Connection::total_connections = 0;
uint64 Connection::current_connections = 0;
uint64 Connection::external_connections = 0;
IMPLEMENT_SERIAL(Connection, SER_CONNECTION);
Connection::Connection(NetSessions* s, HashKey* k, double t, const ConnID* id,
uint32 flow, const Packet* pkt,
const EncapsulationStack* arg_encap)
@ -900,139 +840,6 @@ void Connection::IDString(ODesc* d) const
d->Add(ntohs(resp_port));
}
bool Connection::Serialize(SerialInfo* info) const
{
return SerialObj::Serialize(info);
}
Connection* Connection::Unserialize(UnserialInfo* info)
{
return (Connection*) SerialObj::Unserialize(info, SER_CONNECTION);
}
bool Connection::DoSerialize(SerialInfo* info) const
{
DO_SERIALIZE(SER_CONNECTION, BroObj);
// First we write the members which are needed to
// create the HashKey.
if ( ! SERIALIZE(orig_addr) || ! SERIALIZE(resp_addr) )
return false;
if ( ! SERIALIZE(orig_port) || ! SERIALIZE(resp_port) )
return false;
if ( ! SERIALIZE(timers.length()) )
return false;
loop_over_list(timers, i)
if ( ! timers[i]->Serialize(info) )
return false;
SERIALIZE_OPTIONAL(conn_val);
// FIXME: RuleEndpointState not yet serializable.
// FIXME: Analyzers not yet serializable.
return
SERIALIZE(int(proto)) &&
SERIALIZE(history) &&
SERIALIZE(hist_seen) &&
SERIALIZE(start_time) &&
SERIALIZE(last_time) &&
SERIALIZE(inactivity_timeout) &&
SERIALIZE(suppress_event) &&
SERIALIZE(login_conn != 0) &&
SERIALIZE_BIT(installed_status_timer) &&
SERIALIZE_BIT(timers_canceled) &&
SERIALIZE_BIT(is_active) &&
SERIALIZE_BIT(skip) &&
SERIALIZE_BIT(weird) &&
SERIALIZE_BIT(finished) &&
SERIALIZE_BIT(record_packets) &&
SERIALIZE_BIT(record_contents);
}
bool Connection::DoUnserialize(UnserialInfo* info)
{
DO_UNSERIALIZE(BroObj);
// Build the hash key first. Some of the recursive *::Unserialize()
// functions may need it.
ConnID id;
if ( ! UNSERIALIZE(&orig_addr) || ! UNSERIALIZE(&resp_addr) )
goto error;
if ( ! UNSERIALIZE(&orig_port) || ! UNSERIALIZE(&resp_port) )
goto error;
id.src_addr = orig_addr;
id.dst_addr = resp_addr;
// This doesn't work for ICMP. But I guess this is not really important.
id.src_port = orig_port;
id.dst_port = resp_port;
id.is_one_way = 0; // ### incorrect for ICMP
key = BuildConnIDHashKey(id);
int len;
if ( ! UNSERIALIZE(&len) )
goto error;
while ( len-- )
{
Timer* t = Timer::Unserialize(info);
if ( ! t )
goto error;
timers.append(t);
}
UNSERIALIZE_OPTIONAL(conn_val,
(RecordVal*) Val::Unserialize(info, connection_type));
int iproto;
if ( ! (UNSERIALIZE(&iproto) &&
UNSERIALIZE(&history) &&
UNSERIALIZE(&hist_seen) &&
UNSERIALIZE(&start_time) &&
UNSERIALIZE(&last_time) &&
UNSERIALIZE(&inactivity_timeout) &&
UNSERIALIZE(&suppress_event)) )
goto error;
proto = static_cast<TransportProto>(iproto);
bool has_login_conn;
if ( ! UNSERIALIZE(&has_login_conn) )
goto error;
login_conn = has_login_conn ? (LoginConn*) this : 0;
UNSERIALIZE_BIT(installed_status_timer);
UNSERIALIZE_BIT(timers_canceled);
UNSERIALIZE_BIT(is_active);
UNSERIALIZE_BIT(skip);
UNSERIALIZE_BIT(weird);
UNSERIALIZE_BIT(finished);
UNSERIALIZE_BIT(record_packets);
UNSERIALIZE_BIT(record_contents);
// Hmm... Why does each connection store a sessions ptr?
sessions = ::sessions;
root_analyzer = 0;
primary_PIA = 0;
conn_timer_mgr = 0;
return true;
error:
abort();
CancelTimers();
return false;
}
void Connection::SetRootAnalyzer(analyzer::TransportLayerAnalyzer* analyzer, analyzer::pia::PIA* pia)
{
root_analyzer = analyzer;

8
src/Conn.h
View file

@ -11,7 +11,6 @@
#include "Dict.h"
#include "Val.h"
#include "Timer.h"
#include "Serializer.h"
#include "RuleMatcher.h"
#include "IPAddr.h"
#include "TunnelEncapsulation.h"
@ -235,11 +234,6 @@ public:
// Returns true if connection has been received externally.
bool IsExternal() const { return conn_timer_mgr != 0; }
bool Serialize(SerialInfo* info) const;
static Connection* Unserialize(UnserialInfo* info);
DECLARE_SERIAL(Connection);
// Statistics.
// Just a lower bound.
@ -385,8 +379,6 @@ protected:
void Init(Connection* conn, timer_func timer, int do_expire);
DECLARE_SERIAL(ConnectionTimer);
Connection* conn;
timer_func timer;
int do_expire;

2
src/Debug.cc
View file

@ -721,7 +721,7 @@ static char* get_prompt(bool reset_counter = false)
if ( reset_counter )
counter = 0;
safe_snprintf(prompt, sizeof(prompt), "(Bro [%d]) ", counter++);
safe_snprintf(prompt, sizeof(prompt), "(Zeek [%d]) ", counter++);
return prompt;
}

2
src/DebugCmdInfoConstants.cc
View file

@ -35,7 +35,7 @@ void init_global_dbg_constants () {
"quit"
};
info = new DebugCmdInfo (dcQuit, names, 1, false, "Exit Bro",
info = new DebugCmdInfo (dcQuit, names, 1, false, "Exit Zeek",
false);
g_DebugCmdInfos.push_back(info);
}

2
src/DebugCmdInfoConstants.in
View file

@ -13,7 +13,7 @@ help: Get help with debugger commands
cmd: dcQuit
names: quit
resume: false
help: Exit Bro
help: Exit Zeek
cmd: dcNext
names: next

5
src/DebugLogger.cc
View file

@ -12,8 +12,7 @@ DebugLogger debug_logger;
// Same order here as in DebugStream.
DebugLogger::Stream DebugLogger::streams[NUM_DBGS] = {
{ "serial", 0, false }, { "rules", 0, false },
{ "state", 0, false }, { "chunkedio", 0, false },
{"string", 0, false },
{ "string", 0, false },
{ "notifiers", 0, false }, { "main-loop", 0, false },
{ "dpd", 0, false }, { "tm", 0, false },
{ "logging", 0, false }, {"input", 0, false },
@ -72,7 +71,7 @@ void DebugLogger::ShowStreamsHelp()
fprintf(stderr," %s\n", streams[i].prefix);
fprintf(stderr, "\n");
fprintf(stderr, " plugin-<plugin-name> (replace '::' in name with '-'; e.g., '-B plugin-Bro-Netmap')\n");
fprintf(stderr, " plugin-<plugin-name> (replace '::' in name with '-'; e.g., '-B plugin-Zeek-Netmap')\n");
fprintf(stderr, "\n");
fprintf(stderr, "Pseudo streams\n");
fprintf(stderr, " verbose Increase verbosity.\n");

4
src/DebugLogger.h
View file

@ -16,10 +16,8 @@
enum DebugStream {
DBG_SERIAL, // Serialization
DBG_RULES, // Signature matching
DBG_STATE, // StateAccess logging
DBG_CHUNKEDIO, // ChunkedIO logging
DBG_STRING, // String code
DBG_NOTIFIERS, // Notifiers (see StateAccess.h)
DBG_NOTIFIERS, // Notifiers
DBG_MAINLOOP, // Main IOSource loop
DBG_ANALYZER, // Analyzer framework
DBG_TM, // Time-machine packet input via Brocolli

6
src/Event.cc
View file

@ -58,12 +58,6 @@ void Event::Dispatch(bool no_remote)
if ( src == SOURCE_BROKER )
no_remote = true;
if ( event_serializer )
{
SerialInfo info(event_serializer);
event_serializer->Serialize(&info, handler->Name(), &args);
}
if ( handler->ErrorHandler() )
reporter->BeginErrorHandler();

1
src/Event.h
View file

@ -4,7 +4,6 @@
#define event_h
#include "EventRegistry.h"
#include "Serializer.h"
#include "analyzer/Tag.h"
#include "analyzer/Analyzer.h"

20
src/EventHandler.cc
View file

@ -171,23 +171,3 @@ void EventHandler::NewEvent(val_list* vl)
mgr.Dispatch(ev);
}
bool EventHandler::Serialize(SerialInfo* info) const
{
return SERIALIZE(name);
}
EventHandler* EventHandler::Unserialize(UnserialInfo* info)
{
char* name;
if ( ! UNSERIALIZE_STR(&name, 0) )
return 0;
EventHandler* h = event_registry->Lookup(name);
if ( ! h )
{
h = new EventHandler(name);
event_registry->Register(h);
}
return h;
}

8
src/EventHandler.h
View file

@ -11,9 +11,6 @@
class Func;
class FuncType;
class Serializer;
class SerialInfo;
class UnserialInfo;
class EventHandler {
public:
@ -56,11 +53,6 @@ public:
void SetGenerateAlways() { generate_always = true; }
bool GenerateAlways() { return generate_always; }
// We don't serialize the handler(s) itself here, but
// just the reference to it.
bool Serialize(SerialInfo* info) const;
static EventHandler* Unserialize(UnserialInfo* info);
private:
void NewEvent(val_list* vl); // Raise new_event() meta event.

950
src/Expr.cc
View file

File diff suppressed because it is too large Load diff

139
src/Expr.h
View file

@ -53,7 +53,8 @@ typedef enum {
EXPR_FLATTEN,
EXPR_CAST,
EXPR_IS,
#define NUM_EXPRS (int(EXPR_IS) + 1)
EXPR_INDEX_SLICE_ASSIGN,
#define NUM_EXPRS (int(EXPR_INDEX_SLICE_ASSIGN) + 1)
} BroExprTag;
extern const char* expr_name(BroExprTag t);
@ -62,6 +63,7 @@ class Stmt;
class Frame;
class ListExpr;
class NameExpr;
class IndexExpr;
class AssignExpr;
class CallExpr;
class EventExpr;
@ -90,7 +92,7 @@ public:
const;
// Assign to the given value, if appropriate.
virtual void Assign(Frame* f, Val* v, Opcode op = OP_ASSIGN);
virtual void Assign(Frame* f, Val* v);
// Returns the type corresponding to this expression interpreted
// as an initialization. The type should be Unref()'d when done
@ -193,10 +195,19 @@ public:
return (AssignExpr*) this;
}
void Describe(ODesc* d) const override;
const IndexExpr* AsIndexExpr() const
{
CHECK_TAG(tag, EXPR_INDEX, "ExprVal::AsIndexExpr", expr_name)
return (const IndexExpr*) this;
}
bool Serialize(SerialInfo* info) const;
static Expr* Unserialize(UnserialInfo* info, BroExprTag want = EXPR_ANY);
IndexExpr* AsIndexExpr()
{
CHECK_TAG(tag, EXPR_INDEX, "ExprVal::AsIndexExpr", expr_name)
return (IndexExpr*) this;
}
void Describe(ODesc* d) const override;
virtual TraversalCode Traverse(TraversalCallback* cb) const = 0;
@ -220,8 +231,6 @@ protected:
void RuntimeErrorWithCallStack(const std::string& msg) const;
DECLARE_ABSTRACT_SERIAL(Expr);
BroExprTag tag;
BroType* type;
@ -236,7 +245,7 @@ public:
ID* Id() const { return id; }
Val* Eval(Frame* f) const override;
void Assign(Frame* f, Val* v, Opcode op = OP_ASSIGN) override;
void Assign(Frame* f, Val* v) override;
Expr* MakeLvalue() override;
int IsPure() const override;
@ -248,8 +257,6 @@ protected:
void ExprDescribe(ODesc* d) const override;
DECLARE_SERIAL(NameExpr);
ID* id;
bool in_const_init;
};
@ -270,8 +277,6 @@ protected:
ConstExpr() { val = 0; }
void ExprDescribe(ODesc* d) const override;
DECLARE_SERIAL(ConstExpr);
Val* val;
};
@ -300,8 +305,6 @@ protected:
// Returns the expression folded using the given constant.
virtual Val* Fold(Val* v) const;
DECLARE_SERIAL(UnaryExpr);
Expr* op;
};
@ -363,8 +366,6 @@ protected:
void ExprDescribe(ODesc* d) const override;
DECLARE_SERIAL(BinaryExpr);
Expr* op1;
Expr* op2;
};
@ -379,8 +380,6 @@ protected:
CloneExpr() { }
Val* Fold(Val* v) const override;
DECLARE_SERIAL(CloneExpr);
};
class IncrExpr : public UnaryExpr {
@ -394,8 +393,6 @@ public:
protected:
friend class Expr;
IncrExpr() { }
DECLARE_SERIAL(IncrExpr);
};
class ComplementExpr : public UnaryExpr {
@ -407,8 +404,6 @@ protected:
ComplementExpr() { }
Val* Fold(Val* v) const override;
DECLARE_SERIAL(ComplementExpr);
};
class NotExpr : public UnaryExpr {
@ -420,8 +415,6 @@ protected:
NotExpr() { }
Val* Fold(Val* v) const override;
DECLARE_SERIAL(NotExpr);
};
class PosExpr : public UnaryExpr {
@ -433,8 +426,6 @@ protected:
PosExpr() { }
Val* Fold(Val* v) const override;
DECLARE_SERIAL(PosExpr);
};
class NegExpr : public UnaryExpr {
@ -446,8 +437,6 @@ protected:
NegExpr() { }
Val* Fold(Val* v) const override;
DECLARE_SERIAL(NegExpr);
};
class SizeExpr : public UnaryExpr {
@ -460,7 +449,6 @@ protected:
SizeExpr() { }
Val* Fold(Val* v) const override;
DECLARE_SERIAL(SizeExpr);
};
class AddExpr : public BinaryExpr {
@ -471,9 +459,6 @@ public:
protected:
friend class Expr;
AddExpr() { }
DECLARE_SERIAL(AddExpr);
};
class AddToExpr : public BinaryExpr {
@ -484,8 +469,6 @@ public:
protected:
friend class Expr;
AddToExpr() { }
DECLARE_SERIAL(AddToExpr);
};
class RemoveFromExpr : public BinaryExpr {
@ -496,8 +479,6 @@ public:
protected:
friend class Expr;
RemoveFromExpr() { }
DECLARE_SERIAL(RemoveFromExpr);
};
class SubExpr : public BinaryExpr {
@ -507,9 +488,6 @@ public:
protected:
friend class Expr;
SubExpr() { }
DECLARE_SERIAL(SubExpr);
};
class TimesExpr : public BinaryExpr {
@ -520,9 +498,6 @@ public:
protected:
friend class Expr;
TimesExpr() { }
DECLARE_SERIAL(TimesExpr);
};
class DivideExpr : public BinaryExpr {
@ -534,9 +509,6 @@ protected:
DivideExpr() { }
Val* AddrFold(Val* v1, Val* v2) const override;
DECLARE_SERIAL(DivideExpr);
};
class ModExpr : public BinaryExpr {
@ -546,8 +518,6 @@ public:
protected:
friend class Expr;
ModExpr() { }
DECLARE_SERIAL(ModExpr);
};
class BoolExpr : public BinaryExpr {
@ -560,8 +530,6 @@ public:
protected:
friend class Expr;
BoolExpr() { }
DECLARE_SERIAL(BoolExpr);
};
class BitExpr : public BinaryExpr {
@ -571,8 +539,6 @@ public:
protected:
friend class Expr;
BitExpr() { }
DECLARE_SERIAL(BitExpr);
};
class EqExpr : public BinaryExpr {
@ -585,8 +551,6 @@ protected:
EqExpr() { }
Val* Fold(Val* v1, Val* v2) const override;
DECLARE_SERIAL(EqExpr);
};
class RelExpr : public BinaryExpr {
@ -597,8 +561,6 @@ public:
protected:
friend class Expr;
RelExpr() { }
DECLARE_SERIAL(RelExpr);
};
class CondExpr : public Expr {
@ -621,8 +583,6 @@ protected:
void ExprDescribe(ODesc* d) const override;
DECLARE_SERIAL(CondExpr);
Expr* op1;
Expr* op2;
Expr* op3;
@ -632,14 +592,12 @@ class RefExpr : public UnaryExpr {
public:
explicit RefExpr(Expr* op);
void Assign(Frame* f, Val* v, Opcode op = OP_ASSIGN) override;
void Assign(Frame* f, Val* v) override;
Expr* MakeLvalue() override;
protected:
friend class Expr;
RefExpr() { }
DECLARE_SERIAL(RefExpr);
};
class AssignExpr : public BinaryExpr {
@ -663,12 +621,20 @@ protected:
bool TypeCheck(attr_list* attrs = 0);
bool TypeCheckArithmetics(TypeTag bt1, TypeTag bt2);
DECLARE_SERIAL(AssignExpr);
int is_init;
Val* val; // optional
};
class IndexSliceAssignExpr : public AssignExpr {
public:
IndexSliceAssignExpr(Expr* op1, Expr* op2, int is_init);
Val* Eval(Frame* f) const override;
protected:
friend class Expr;
IndexSliceAssignExpr() {}
};
class IndexExpr : public BinaryExpr {
public:
IndexExpr(Expr* op1, ListExpr* op2, bool is_slice = false);
@ -679,7 +645,7 @@ public:
void Add(Frame* f) override;
void Delete(Frame* f) override;
void Assign(Frame* f, Val* v, Opcode op = OP_ASSIGN) override;
void Assign(Frame* f, Val* v) override;
Expr* MakeLvalue() override;
// Need to override Eval since it can take a vector arg but does
@ -688,6 +654,8 @@ public:
TraversalCode Traverse(TraversalCallback* cb) const override;
bool IsSlice() const { return is_slice; }
protected:
friend class Expr;
IndexExpr() { }
@ -696,7 +664,7 @@ protected:
void ExprDescribe(ODesc* d) const override;
DECLARE_SERIAL(IndexExpr);
bool is_slice;
};
class FieldExpr : public UnaryExpr {
@ -709,7 +677,7 @@ public:
int CanDel() const override;
void Assign(Frame* f, Val* v, Opcode op = OP_ASSIGN) override;
void Assign(Frame* f, Val* v) override;
void Delete(Frame* f) override;
Expr* MakeLvalue() override;
@ -722,8 +690,6 @@ protected:
void ExprDescribe(ODesc* d) const override;
DECLARE_SERIAL(FieldExpr);
const char* field_name;
const TypeDecl* td;
int field; // -1 = attributes
@ -746,8 +712,6 @@ protected:
void ExprDescribe(ODesc* d) const override;
DECLARE_SERIAL(HasFieldExpr);
const char* field_name;
int field;
};
@ -765,8 +729,6 @@ protected:
Val* Fold(Val* v) const override;
void ExprDescribe(ODesc* d) const override;
DECLARE_SERIAL(RecordConstructorExpr);
};
class TableConstructorExpr : public UnaryExpr {
@ -787,8 +749,6 @@ protected:
void ExprDescribe(ODesc* d) const override;
DECLARE_SERIAL(TableConstructorExpr);
Attributes* attrs;
};
@ -810,8 +770,6 @@ protected:
void ExprDescribe(ODesc* d) const override;
DECLARE_SERIAL(SetConstructorExpr);
Attributes* attrs;
};
@ -828,8 +786,6 @@ protected:
Val* InitVal(const BroType* t, Val* aggr) const override;
void ExprDescribe(ODesc* d) const override;
DECLARE_SERIAL(VectorConstructorExpr);
};
class FieldAssignExpr : public UnaryExpr {
@ -847,8 +803,6 @@ protected:
void ExprDescribe(ODesc* d) const override;
DECLARE_SERIAL(FieldAssignExpr);
string field_name;
};
@ -862,8 +816,6 @@ protected:
Val* FoldSingleVal(Val* v, InternalTypeTag t) const;
Val* Fold(Val* v) const override;
DECLARE_SERIAL(ArithCoerceExpr);
};
class RecordCoerceExpr : public UnaryExpr {
@ -878,8 +830,6 @@ protected:
Val* InitVal(const BroType* t, Val* aggr) const override;
Val* Fold(Val* v) const override;
DECLARE_SERIAL(RecordCoerceExpr);
// For each super-record slot, gives subrecord slot with which to
// fill it.
int* map;
@ -896,8 +846,6 @@ protected:
TableCoerceExpr() { }
Val* Fold(Val* v) const override;
DECLARE_SERIAL(TableCoerceExpr);
};
class VectorCoerceExpr : public UnaryExpr {
@ -910,8 +858,6 @@ protected:
VectorCoerceExpr() { }
Val* Fold(Val* v) const override;
DECLARE_SERIAL(VectorCoerceExpr);
};
// An internal operator for flattening array indices that are records
@ -926,8 +872,6 @@ protected:
Val* Fold(Val* v) const override;
DECLARE_SERIAL(FlattenExpr);
int num_fields;
};
@ -967,8 +911,6 @@ protected:
void ExprDescribe(ODesc* d) const override;
DECLARE_SERIAL(ScheduleExpr);
Expr* when;
EventExpr* event;
};
@ -983,8 +925,6 @@ protected:
Val* Fold(Val* v1, Val* v2) const override;
DECLARE_SERIAL(InExpr);
};
class CallExpr : public Expr {
@ -1007,8 +947,6 @@ protected:
void ExprDescribe(ODesc* d) const override;
DECLARE_SERIAL(CallExpr);
Expr* func;
ListExpr* args;
};
@ -1058,8 +996,6 @@ protected:
void ExprDescribe(ODesc* d) const override;
DECLARE_SERIAL(EventExpr);
string name;
EventHandlerPtr handler;
ListExpr* args;
@ -1087,7 +1023,7 @@ public:
BroType* InitType() const override;
Val* InitVal(const BroType* t, Val* aggr) const override;
Expr* MakeLvalue() override;
void Assign(Frame* f, Val* v, Opcode op = OP_ASSIGN) override;
void Assign(Frame* f, Val* v) override;
TraversalCode Traverse(TraversalCallback* cb) const override;
@ -1096,8 +1032,6 @@ protected:
void ExprDescribe(ODesc* d) const override;
DECLARE_SERIAL(ListExpr);
expr_list exprs;
};
@ -1111,8 +1045,6 @@ public:
protected:
friend class Expr;
RecordAssignExpr() { }
DECLARE_SERIAL(RecordAssignExpr);
};
class CastExpr : public UnaryExpr {
@ -1125,8 +1057,6 @@ protected:
Val* Eval(Frame* f) const override;
void ExprDescribe(ODesc* d) const override;
DECLARE_SERIAL(CastExpr);
};
class IsExpr : public UnaryExpr {
@ -1140,7 +1070,6 @@ protected:
Val* Fold(Val* v) const override;
void ExprDescribe(ODesc* d) const override;
DECLARE_SERIAL(IsExpr);
private:
BroType* t;

138
src/File.cc
View file

@ -25,7 +25,6 @@
#include "Expr.h"
#include "NetVar.h"
#include "Net.h"
#include "Serializer.h"
#include "Event.h"
#include "Reporter.h"
@ -518,11 +517,6 @@ double BroFile::Size()
return s.st_size;
}
bool BroFile::Serialize(SerialInfo* info) const
{
return SerialObj::Serialize(info);
}
BroFile* BroFile::GetFile(const char* name)
{
for ( BroFile* f = head; f; f = f->next )
@ -534,135 +528,3 @@ BroFile* BroFile::GetFile(const char* name)
return new BroFile(name, "w", 0);
}
BroFile* BroFile::Unserialize(UnserialInfo* info)
{
BroFile* file = (BroFile*) SerialObj::Unserialize(info, SER_BRO_FILE);
if ( ! file )
return 0;
if ( file->is_open )
return file;
// If there is already an object for this file, return it.
if ( file->name )
{
for ( BroFile* f = head; f; f = f->next )
{
if ( f->name && streq(file->name, f->name) )
{
Unref(file);
Ref(f);
return f;
}
}
}
// Otherwise, open, but don't clobber.
if ( ! file->Open(0, "a") )
{
info->s->Error(fmt("cannot open %s: %s",
file->name, strerror(errno)));
return 0;
}
// Here comes a hack. This method will return a pointer to a newly
// instantiated file object. As soon as this pointer is Unref'ed, the
// file will be closed. That means that when we unserialize the same
// file next time, we will re-open it and thereby delete the first one,
// i.e., we will be keeping to delete what we've written just before.
//
// To avoid this loop, we do an extra Ref here, i.e., this file will
// *never* be closed anymore (as long the file cache does not overflow).
Ref(file);
file->SetBuf(file->buffered);
return file;
}
IMPLEMENT_SERIAL(BroFile, SER_BRO_FILE);
bool BroFile::DoSerialize(SerialInfo* info) const
{
DO_SERIALIZE(SER_BRO_FILE, BroObj);
const char* s = name;
if ( ! okay_to_manage )
{
// We can handle stdin/stdout/stderr but no others.
if ( f == stdin )
s = "/dev/stdin";
else if ( f == stdout )
s = "/dev/stdout";
else if ( f == stderr )
s = "/dev/stderr";
else
{
// We don't manage the file, and therefore don't
// really know how to pass it on to the other side.
// However, in order to not abort communication
// when this happens, we still send the name if we
// have one; or if we don't, we create a special
// "dont-have-a-file" file to be created on the
// receiver side.
if ( ! s )
s = "unmanaged-bro-output-file.log";
}
}
if ( ! (SERIALIZE(s) && SERIALIZE(buffered)) )
return false;
SERIALIZE_OPTIONAL_STR(access);
if ( ! t->Serialize(info) )
return false;
SERIALIZE_OPTIONAL(attrs);
return true;
}
bool BroFile::DoUnserialize(UnserialInfo* info)
{
DO_UNSERIALIZE(BroObj);
if ( ! (UNSERIALIZE_STR(&name, 0) && UNSERIALIZE(&buffered)) )
return false;
UNSERIALIZE_OPTIONAL_STR(access);
t = BroType::Unserialize(info);
if ( ! t )
return false;
UNSERIALIZE_OPTIONAL(attrs, Attributes::Unserialize(info));
// Parse attributes.
SetAttrs(attrs);
// SetAttrs() has ref'ed attrs again.
Unref(attrs);
// Bind stdin/stdout/stderr.
FILE* file = 0;
is_open = false;
f = 0;
if ( streq(name, "/dev/stdin") )
file = stdin;
else if ( streq(name, "/dev/stdout") )
file = stdout;
else if ( streq(name, "/dev/stderr") )
file = stderr;
if ( file )
{
delete [] name;
name = 0;
f = file;
is_open = true;
}
return true;
}

5
src/File.h
View file

@ -66,9 +66,6 @@ public:
void EnableRawOutput() { raw_output = true; }
bool IsRawOutput() const { return raw_output; }
bool Serialize(SerialInfo* info) const;
static BroFile* Unserialize(UnserialInfo* info);
protected:
BroFile() { Init(); }
void Init();
@ -100,8 +97,6 @@ protected:
// Raises a file_opened event.
void RaiseOpenEvent();
DECLARE_SERIAL(BroFile);
FILE* f;
BroType* t;
char* name;

140
src/Func.cc
View file

@ -41,7 +41,6 @@
#include "analyzer/protocol/login/Login.h"
#include "Sessions.h"
#include "RE.h"
#include "Serializer.h"
#include "Event.h"
#include "Traverse.h"
#include "Reporter.h"
@ -127,110 +126,6 @@ void Func::AddBody(Stmt* /* new_body */, id_list* /* new_inits */,
Internal("Func::AddBody called");
}
bool Func::Serialize(SerialInfo* info) const
{
return SerialObj::Serialize(info);
}
Func* Func::Unserialize(UnserialInfo* info)
{
Func* f = (Func*) SerialObj::Unserialize(info, SER_FUNC);
// For builtins, we return a reference to the (hopefully) already
// existing function.
if ( f && f->kind == BUILTIN_FUNC )
{
const char* name = ((BuiltinFunc*) f)->Name();
ID* id = global_scope()->Lookup(name);
if ( ! id )
{
info->s->Error(fmt("can't find built-in %s", name));
return 0;
}
if ( ! (id->HasVal() && id->ID_Val()->Type()->Tag() == TYPE_FUNC) )
{
info->s->Error(fmt("ID %s is not a built-in", name));
return 0;
}
Unref(f);
f = id->ID_Val()->AsFunc();
Ref(f);
}
return f;
}
bool Func::DoSerialize(SerialInfo* info) const
{
DO_SERIALIZE(SER_FUNC, BroObj);
if ( ! SERIALIZE(int(bodies.size())) )
return false;
for ( unsigned int i = 0; i < bodies.size(); ++i )
{
if ( ! bodies[i].stmts->Serialize(info) )
return false;
if ( ! SERIALIZE(bodies[i].priority) )
return false;
}
if ( ! SERIALIZE(char(kind) ) )
return false;
if ( ! type->Serialize(info) )
return false;
if ( ! SERIALIZE(Name()) )
return false;
// We don't serialize scope as only global functions are considered here
// anyway.
return true;
}
bool Func::DoUnserialize(UnserialInfo* info)
{
DO_UNSERIALIZE(BroObj);
int len;
if ( ! UNSERIALIZE(&len) )
return false;
while ( len-- )
{
Body b;
b.stmts = Stmt::Unserialize(info);
if ( ! b.stmts )
return false;
if ( ! UNSERIALIZE(&b.priority) )
return false;
bodies.push_back(b);
}
char c;
if ( ! UNSERIALIZE(&c) )
return false;
kind = (Kind) c;
type = BroType::Unserialize(info);
if ( ! type )
return false;
const char* n;
if ( ! UNSERIALIZE_STR(&n, 0) )
return false;
name = n;
delete [] n;
return true;
}
Val* Func::DoClone()
{
@ -665,21 +560,6 @@ Stmt* BroFunc::AddInits(Stmt* body, id_list* inits)
return stmt_series;
}
IMPLEMENT_SERIAL(BroFunc, SER_BRO_FUNC);
bool BroFunc::DoSerialize(SerialInfo* info) const
{
DO_SERIALIZE(SER_BRO_FUNC, Func);
return SERIALIZE(frame_size);
}
bool BroFunc::DoUnserialize(UnserialInfo* info)
{
DO_UNSERIALIZE(Func);
return UNSERIALIZE(&frame_size);
}
BuiltinFunc::BuiltinFunc(built_in_func arg_func, const char* arg_name,
int arg_is_pure)
: Func(BUILTIN_FUNC)
@ -762,20 +642,6 @@ void BuiltinFunc::Describe(ODesc* d) const
d->AddCount(is_pure);
}
IMPLEMENT_SERIAL(BuiltinFunc, SER_BUILTIN_FUNC);
bool BuiltinFunc::DoSerialize(SerialInfo* info) const
{
DO_SERIALIZE(SER_BUILTIN_FUNC, Func);
return true;
}
bool BuiltinFunc::DoUnserialize(UnserialInfo* info)
{
DO_UNSERIALIZE(Func);
return true;
}
void builtin_error(const char* msg, BroObj* arg)
{
auto emit = [=](const CallExpr* ce)
@ -842,13 +708,13 @@ void builtin_error(const char* msg, BroObj* arg)
emit(last_call.call);
}
#include "bro.bif.func_h"
#include "zeek.bif.func_h"
#include "stats.bif.func_h"
#include "reporter.bif.func_h"
#include "strings.bif.func_h"
#include "option.bif.func_h"
#include "bro.bif.func_def"
#include "zeek.bif.func_def"
#include "stats.bif.func_def"
#include "reporter.bif.func_def"
#include "strings.bif.func_def"
@ -875,7 +741,7 @@ void init_builtin_funcs()
var_sizes = internal_type("var_sizes")->AsTableType();
#include "bro.bif.func_init"
#include "zeek.bif.func_init"
#include "stats.bif.func_init"
#include "reporter.bif.func_init"
#include "strings.bif.func_init"

9
src/Func.h
View file

@ -64,9 +64,6 @@ public:
void Describe(ODesc* d) const override = 0;
virtual void DescribeDebug(ODesc* d, const val_list* args) const;
// This (un-)serializes only a single body (as given in SerialInfo).
bool Serialize(SerialInfo* info) const;
static Func* Unserialize(UnserialInfo* info);
virtual Val* DoClone();
virtual TraversalCode Traverse(TraversalCallback* cb) const;
@ -81,8 +78,6 @@ protected:
// Helper function for handling result of plugin hook.
std::pair<bool, Val*> HandlePluginResult(std::pair<bool, Val*> plugin_result, val_list* args, function_flavor flavor) const;
DECLARE_ABSTRACT_SERIAL(Func);
vector<Body> bodies;
Scope* scope;
Kind kind;
@ -117,8 +112,6 @@ protected:
BroFunc() : Func(BRO_FUNC) {}
Stmt* AddInits(Stmt* body, id_list* inits);
DECLARE_SERIAL(BroFunc);
int frame_size;
private:
@ -155,8 +148,6 @@ public:
protected:
BuiltinFunc() { func = 0; is_pure = 0; }
DECLARE_SERIAL(BuiltinFunc);
built_in_func func;
int is_pure;
};

2
src/Hash.cc
View file

@ -26,7 +26,7 @@ void init_hash_function()
{
// Make sure we have already called init_random_seed().
if ( ! (hmac_key_set && siphash_key_set) )
reporter->InternalError("Bro's hash functions aren't fully initialized");
reporter->InternalError("Zeek's hash functions aren't fully initialized");
}
HashKey::HashKey(bro_int_t i)

249
src/ID.cc
View file

@ -9,7 +9,6 @@
#include "Func.h"
#include "Scope.h"
#include "File.h"
#include "Serializer.h"
#include "Scope.h"
#include "Traverse.h"
#include "zeekygen/Manager.h"
@ -60,34 +59,14 @@ void ID::ClearVal()
val = 0;
}
void ID::SetVal(Val* v, Opcode op, bool arg_weak_ref)
void ID::SetVal(Val* v, bool arg_weak_ref)
{
if ( op != OP_NONE )
{
MutableVal::Properties props = 0;
if ( attrs && attrs->FindAttr(ATTR_TRACKED) )
props |= MutableVal::TRACKED;
if ( props )
{
if ( v->IsMutableVal() )
v->AsMutableVal()->AddProperties(props);
}
#ifndef DEBUG
if ( props )
#else
if ( debug_logger.IsVerbose() || props )
#endif
StateAccess::Log(new StateAccess(op, this, v, val));
}
if ( ! weak_ref )
Unref(val);
val = v;
weak_ref = arg_weak_ref;
Modified();
#ifdef DEBUG
UpdateValID();
@ -176,16 +155,6 @@ void ID::UpdateValAttrs()
if ( ! attrs )
return;
MutableVal::Properties props = 0;
if ( val && val->IsMutableVal() )
{
if ( attrs->FindAttr(ATTR_TRACKED) )
props |= MutableVal::TRACKED;
val->AsMutableVal()->AddProperties(props);
}
if ( val && val->Type()->Tag() == TYPE_TABLE )
val->AsTableVal()->SetAttrs(attrs);
@ -243,16 +212,6 @@ void ID::RemoveAttr(attr_tag a)
{
if ( attrs )
attrs->RemoveAttr(a);
if ( val && val->IsMutableVal() )
{
MutableVal::Properties props = 0;
if ( a == ATTR_TRACKED )
props |= MutableVal::TRACKED;
val->AsMutableVal()->RemoveProperties(props);
}
}
void ID::SetOption()
@ -283,11 +242,6 @@ void ID::EvalFunc(Expr* ef, Expr* ev)
Unref(ce);
}
bool ID::Serialize(SerialInfo* info) const
{
return (ID*) SerialObj::Serialize(info);
}
#if 0
void ID::CopyFrom(const ID* id)
{
@ -320,205 +274,6 @@ void ID::CopyFrom(const ID* id)
#endif
#endif
ID* ID::Unserialize(UnserialInfo* info)
{
ID* id = (ID*) SerialObj::Unserialize(info, SER_ID);
if ( ! id )
return 0;
if ( ! id->IsGlobal() )
return id;
// Globals.
ID* current = global_scope()->Lookup(id->name);
if ( ! current )
{
if ( ! info->install_globals )
{
info->s->Error("undefined");
Unref(id);
return 0;
}
Ref(id);
global_scope()->Insert(id->Name(), id);
#ifdef USE_PERFTOOLS_DEBUG
heap_checker->IgnoreObject(id);
#endif
}
else
{
switch ( info->id_policy ) {
case UnserialInfo::Keep:
Unref(id);
Ref(current);
id = current;
break;
case UnserialInfo::Replace:
Unref(current);
Ref(id);
global_scope()->Insert(id->Name(), id);
break;
case UnserialInfo::CopyNewToCurrent:
if ( ! same_type(current->type, id->type) )
{
info->s->Error("type mismatch");
Unref(id);
return 0;
}
if ( ! current->weak_ref )
Unref(current->val);
current->val = id->val;
current->weak_ref = id->weak_ref;
if ( current->val && ! current->weak_ref )
Ref(current->val);
#ifdef DEBUG
current->UpdateValID();
#endif
Unref(id);
Ref(current);
id = current;
break;
case UnserialInfo::CopyCurrentToNew:
if ( ! same_type(current->type, id->type) )
{
info->s->Error("type mismatch");
return 0;
}
if ( ! id->weak_ref )
Unref(id->val);
id->val = current->val;
id->weak_ref = current->weak_ref;
if ( id->val && ! id->weak_ref )
Ref(id->val);
#ifdef DEBUG
id->UpdateValID();
#endif
Unref(current);
Ref(id);
global_scope()->Insert(id->Name(), id);
break;
case UnserialInfo::InstantiateNew:
// Do nothing.
break;
default:
reporter->InternalError("unknown type for UnserialInfo::id_policy");
}
}
return id;
}
IMPLEMENT_SERIAL(ID, SER_ID);
bool ID::DoSerialize(SerialInfo* info) const
{
DO_SERIALIZE_WITH_SUSPEND(SER_ID, BroObj);
if ( info->cont.NewInstance() )
{
DisableSuspend suspend(info);
info->s->WriteOpenTag("ID");
if ( ! (SERIALIZE(name) &&
SERIALIZE(char(scope)) &&
SERIALIZE(is_export) &&
SERIALIZE(is_const) &&
SERIALIZE(is_enum_const) &&
SERIALIZE(is_type) &&
SERIALIZE(offset) &&
SERIALIZE(infer_return_type) &&
SERIALIZE(weak_ref) &&
type->Serialize(info)) )
return false;
SERIALIZE_OPTIONAL(attrs);
}
SERIALIZE_OPTIONAL(val);
return true;
}
bool ID::DoUnserialize(UnserialInfo* info)
{
bool installed_tmp = false;
DO_UNSERIALIZE(BroObj);
char id_scope;
if ( ! (UNSERIALIZE_STR(&name, 0) &&
UNSERIALIZE(&id_scope) &&
UNSERIALIZE(&is_export) &&
UNSERIALIZE(&is_const) &&
UNSERIALIZE(&is_enum_const) &&
UNSERIALIZE(&is_type) &&
UNSERIALIZE(&offset) &&
UNSERIALIZE(&infer_return_type) &&
UNSERIALIZE(&weak_ref)
) )
return false;
scope = IDScope(id_scope);
info->s->SetErrorDescr(fmt("unserializing ID %s", name));
type = BroType::Unserialize(info);
if ( ! type )
return false;
UNSERIALIZE_OPTIONAL(attrs, Attributes::Unserialize(info));
// If it's a global function not currently known,
// we temporarily install it in global scope.
// This is necessary for recursive functions.
if ( IsGlobal() && Type()->Tag() == TYPE_FUNC )
{
ID* current = global_scope()->Lookup(name);
if ( ! current )
{
installed_tmp = true;
global_scope()->Insert(Name(), this);
}
}
UNSERIALIZE_OPTIONAL(val, Val::Unserialize(info));
#ifdef DEBUG
UpdateValID();
#endif
if ( weak_ref )
{
// At this point at least the serialization cache will hold a
// reference so this will not delete the val.
assert(val->RefCnt() > 1);
Unref(val);
}
if ( installed_tmp && ! global_scope()->Remove(name) )
reporter->InternalWarning("missing tmp ID in %s unserialization", name);
return true;
}
TraversalCode ID::Traverse(TraversalCallback* cb) const
{
TraversalCode tc = cb->PreID(this);

16
src/ID.h
View file

@ -5,18 +5,17 @@
#include "Type.h"
#include "Attr.h"
#include "StateAccess.h"
#include "Notifier.h"
#include "TraverseTypes.h"
#include <string>
class Val;
class SerialInfo;
class Func;
typedef enum { INIT_NONE, INIT_FULL, INIT_EXTRA, INIT_REMOVE, } init_class;
typedef enum { SCOPE_FUNCTION, SCOPE_MODULE, SCOPE_GLOBAL } IDScope;
class ID : public BroObj {
class ID : public BroObj, public notifier::Modifiable {
public:
ID(const char* name, IDScope arg_scope, bool arg_is_export);
~ID() override;
@ -47,7 +46,7 @@ public:
// reference to the Val, the Val will be destroyed (naturally,
// you have to take care that it will not be accessed via
// the ID afterwards).
void SetVal(Val* v, Opcode op = OP_ASSIGN, bool weak_ref = false);
void SetVal(Val* v, bool weak_ref = false);
void SetVal(Val* v, init_class c);
void SetVal(Expr* ev, init_class c);
@ -71,10 +70,6 @@ public:
bool IsRedefinable() const { return FindAttr(ATTR_REDEF) != 0; }
// Returns true if ID is one of those internal globally unique IDs
// to which MutableVals are bound (there name start with a '#').
bool IsInternalGlobal() const { return name && name[0] == '#'; }
void SetAttrs(Attributes* attr);
void AddAttrs(Attributes* attr);
void RemoveAttr(attr_tag a);
@ -98,9 +93,6 @@ public:
void DescribeReST(ODesc* d, bool roles_only = false) const;
void DescribeReSTShort(ODesc* d) const;
bool Serialize(SerialInfo* info) const;
static ID* Unserialize(UnserialInfo* info);
bool DoInferReturnType() const
{ return infer_return_type; }
void SetInferReturnType(bool infer)
@ -124,8 +116,6 @@ protected:
void UpdateValID();
#endif
DECLARE_SERIAL(ID);
const char* name;
IDScope scope;
bool is_export;

1
src/Net.cc
View file

@ -27,7 +27,6 @@
#include "Reporter.h"
#include "Net.h"
#include "Anon.h"
#include "Serializer.h"
#include "PacketDumper.h"
#include "iosource/Manager.h"
#include "iosource/PktSrc.h"

14
src/NetVar.cc
View file

@ -77,7 +77,6 @@ bool udp_content_deliver_all_orig;
bool udp_content_deliver_all_resp;
double dns_session_timeout;
double ntp_session_timeout;
double rpc_timeout;
ListVal* skip_authentication;
@ -103,8 +102,6 @@ TableType* pm_mappings;
RecordType* pm_port_request;
RecordType* pm_callit_request;
RecordType* ntp_msg;
RecordType* geo_location;
RecordType* entropy_test_result;
@ -168,10 +165,6 @@ StringVal* log_rotate_base_time;
StringVal* peer_description;
bro_uint_t chunked_io_buffer_soft_cap;
StringVal* ssl_ca_certificate;
StringVal* ssl_private_key;
StringVal* ssl_passphrase;
Val* profiling_file;
double profiling_interval;
int expensive_profiling_multiple;
@ -247,10 +240,6 @@ void init_general_global_var()
internal_val("peer_description")->AsStringVal();
chunked_io_buffer_soft_cap = opt_internal_unsigned("chunked_io_buffer_soft_cap");
ssl_ca_certificate = internal_val("ssl_ca_certificate")->AsStringVal();
ssl_private_key = internal_val("ssl_private_key")->AsStringVal();
ssl_passphrase = internal_val("ssl_passphrase")->AsStringVal();
packet_filter_default = opt_internal_int("packet_filter_default");
sig_max_group_size = opt_internal_int("sig_max_group_size");
@ -360,7 +349,6 @@ void init_net_var()
bool(internal_val("udp_content_deliver_all_resp")->AsBool());
dns_session_timeout = opt_internal_double("dns_session_timeout");
ntp_session_timeout = opt_internal_double("ntp_session_timeout");
rpc_timeout = opt_internal_double("rpc_timeout");
watchdog_interval = int(opt_internal_double("watchdog_interval"));
@ -390,8 +378,6 @@ void init_net_var()
pm_port_request = internal_type("pm_port_request")->AsRecordType();
pm_callit_request = internal_type("pm_callit_request")->AsRecordType();
ntp_msg = internal_type("ntp_msg")->AsRecordType();
geo_location = internal_type("geo_location")->AsRecordType();
entropy_test_result = internal_type("entropy_test_result")->AsRecordType();

7
src/NetVar.h
View file

@ -80,7 +80,6 @@ extern bool udp_content_deliver_all_orig;
extern bool udp_content_deliver_all_resp;
extern double dns_session_timeout;
extern double ntp_session_timeout;
extern double rpc_timeout;
extern ListVal* skip_authentication;
@ -106,8 +105,6 @@ extern TableType* pm_mappings;
extern RecordType* pm_port_request;
extern RecordType* pm_callit_request;
extern RecordType* ntp_msg;
extern RecordType* geo_location;
extern RecordType* entropy_test_result;
@ -171,10 +168,6 @@ extern StringVal* log_rotate_base_time;
extern StringVal* peer_description;
extern bro_uint_t chunked_io_buffer_soft_cap;
extern StringVal* ssl_ca_certificate;
extern StringVal* ssl_private_key;
extern StringVal* ssl_passphrase;
extern Val* profiling_file;
extern double profiling_interval;
extern int expensive_profiling_multiple;

72
src/Notifier.cc Normal file
View file

@ -0,0 +1,72 @@
// See the file "COPYING" in the main distribution directory for copyright.
#include "DebugLogger.h"
#include "Notifier.h"
notifier::Registry notifier::registry;
notifier::Receiver::Receiver()
{
DBG_LOG(DBG_NOTIFIERS, "creating receiver %p", this);
}
notifier::Receiver::~Receiver()
{
DBG_LOG(DBG_NOTIFIERS, "deleting receiver %p", this);
}
notifier::Registry::~Registry()
{
while ( registrations.begin() != registrations.end() )
Unregister(registrations.begin()->first);
}
void notifier::Registry::Register(Modifiable* m, notifier::Receiver* r)
{
DBG_LOG(DBG_NOTIFIERS, "registering object %p for receiver %p", m, r);
registrations.insert({m, r});
++m->num_receivers;
}
void notifier::Registry::Unregister(Modifiable* m, notifier::Receiver* r)
{
DBG_LOG(DBG_NOTIFIERS, "unregistering object %p from receiver %p", m, r);
auto x = registrations.equal_range(m);
for ( auto i = x.first; i != x.second; i++ )
{
if ( i->second == r )
{
--i->first->num_receivers;
registrations.erase(i);
break;
}
}
}
void notifier::Registry::Unregister(Modifiable* m)
{
DBG_LOG(DBG_NOTIFIERS, "unregistering object %p from all notifiers", m);
auto x = registrations.equal_range(m);
for ( auto i = x.first; i != x.second; i++ )
--i->first->num_receivers;
registrations.erase(x.first, x.second);
}
void notifier::Registry::Modified(Modifiable* m)
{
DBG_LOG(DBG_NOTIFIERS, "object %p has been modified", m);
auto x = registrations.equal_range(m);
for ( auto i = x.first; i != x.second; i++ )
i->second->Modified(m);
}
notifier::Modifiable::~Modifiable()
{
if ( num_receivers )
registry.Unregister(this);
}

116
src/Notifier.h Normal file
View file

@ -0,0 +1,116 @@
// See the file "COPYING" in the main distribution directory for copyright.
//
// A notification framework to inform interested parties of modifications to
// selected global objects. To get notified about a change, derive a class
// from notifier::Receiver and register the interesting objects with the
// notification::Registry.
#ifndef NOTIFIER_H
#define NOTIFIER_H
#include <set>
#include <unordered_map>
#include <string>
#include "util.h"
#include "DebugLogger.h"
namespace notifier {
class Modifiable;
/** Interface class for receivers of notifications. */
class Receiver {
public:
Receiver();
virtual ~Receiver();
/**
* Callback executed when a register object has been modified.
*
* @param m object that was modified
*/
virtual void Modified(Modifiable* m) = 0;
};
/** Singleton class tracking all notification requests globally. */
class Registry {
public:
~Registry();
/**
* Registers a receiver to be informed when a modifiable object has
* changed.
*
* @param m object to track. Does not take ownership, but the object
* will automatically unregister itself on destruction.
*
* @param r receiver to notify on changes. Does not take ownershop,
* the receiver must remain valid as long as the registration stays
* in place.
*/
void Register(Modifiable* m, Receiver* r);
/**
* Cancels a receiver's request to be informed about an object's
* modification. The arguments to the method must match what was
* originally registered.
*
* @param m object to no loger track.
*
* @param r receiver to no longer notify.
*/
void Unregister(Modifiable* m, Receiver* Receiver);
/**
* Cancels any active receiver requests to be informed about a
* partilar object's modifications.
*
* @param m object to no loger track.
*/
void Unregister(Modifiable* m);
private:
friend class Modifiable;
// Inform all registered receivers of a modification to an object.
// Will be called from the object itself.
void Modified(Modifiable* m);
typedef std::unordered_multimap<Modifiable*, Receiver*> ModifiableMap;
ModifiableMap registrations;
};
/**
* Singleton object tracking all global notification requests.
*/
extern Registry registry;
/**
* Base class for objects that can trigger notifications to receivers when
* modified.
*/
class Modifiable {
public:
/**
* Calling this method signals to all registered receivers that the
* object has been modified.
*/
void Modified()
{
if ( num_receivers )
registry.Modified(this);
}
protected:
friend class Registry;
virtual ~Modifiable();
// Number of currently registered receivers.
uint64 num_receivers = 0;
};
}
#endif

65
src/Obj.cc
View file

@ -5,7 +5,6 @@
#include <stdlib.h>
#include "Obj.h"
#include "Serializer.h"
#include "Func.h"
#include "File.h"
#include "plugin/Manager.h"
@ -14,47 +13,6 @@ Location no_location("<no location>", 0, 0, 0, 0);
Location start_location("<start uninitialized>", 0, 0, 0, 0);
Location end_location("<end uninitialized>", 0, 0, 0, 0);
bool Location::Serialize(SerialInfo* info) const
{
return SerialObj::Serialize(info);
}
Location* Location::Unserialize(UnserialInfo* info)
{
return (Location*) SerialObj::Unserialize(info, SER_LOCATION);
}
IMPLEMENT_SERIAL(Location, SER_LOCATION);
bool Location::DoSerialize(SerialInfo* info) const
{
DO_SERIALIZE(SER_LOCATION, SerialObj);
info->s->WriteOpenTag("Location");
if ( ! (SERIALIZE(filename) &&
SERIALIZE(first_line) &&
SERIALIZE(last_line) &&
SERIALIZE(first_column) &&
SERIALIZE(last_column)) )
return false;
info->s->WriteCloseTag("Location");
return true;
}
bool Location::DoUnserialize(UnserialInfo* info)
{
DO_UNSERIALIZE(SerialObj);
delete_data = true;
return UNSERIALIZE_STR(&filename, 0)
&& UNSERIALIZE(&first_line)
&& UNSERIALIZE(&last_line)
&& UNSERIALIZE(&first_column)
&& UNSERIALIZE(&last_column);
}
void Location::Describe(ODesc* d) const
{
if ( filename )
@ -230,29 +188,6 @@ void BroObj::PinPoint(ODesc* d, const BroObj* obj2, int pinpoint_only) const
d->Add(")");
}
bool BroObj::DoSerialize(SerialInfo* info) const
{
DO_SERIALIZE(SER_BRO_OBJ, SerialObj);
info->s->WriteOpenTag("Object");
Location* loc = info->include_locations ? location : 0;
SERIALIZE_OPTIONAL(loc);
info->s->WriteCloseTag("Object");
return true;
}
bool BroObj::DoUnserialize(UnserialInfo* info)
{
DO_UNSERIALIZE(SerialObj);
delete location;
UNSERIALIZE_OPTIONAL(location, Location::Unserialize(info));
return true;
}
void print(const BroObj* obj)
{
static BroFile fstderr(stderr);

21
src/Obj.h
View file

@ -7,12 +7,8 @@
#include "input.h"
#include "Desc.h"
#include "SerialObj.h"
class Serializer;
class SerialInfo;
class Location : SerialObj {
class Location {
public:
Location(const char* fname, int line_f, int line_l, int col_f, int col_l)
{
@ -36,7 +32,7 @@ public:
text = 0;
}
~Location() override
virtual ~Location()
{
if ( delete_data )
delete [] filename;
@ -44,9 +40,6 @@ public:
void Describe(ODesc* d) const;
bool Serialize(SerialInfo* info) const;
static Location* Unserialize(UnserialInfo* info);
bool operator==(const Location& l) const;
bool operator!=(const Location& l) const
{ return ! (*this == l); }
@ -59,8 +52,6 @@ public:
// Timestamp and text for compatibility with Bison's default yyltype.
int timestamp;
char* text;
protected:
DECLARE_SERIAL(Location);
};
#define YYLTYPE yyltype
@ -86,7 +77,7 @@ inline void set_location(const Location start, const Location end)
end_location = end;
}
class BroObj : public SerialObj {
class BroObj {
public:
BroObj()
{
@ -112,7 +103,7 @@ public:
SetLocationInfo(&start_location, &end_location);
}
~BroObj() override;
virtual ~BroObj();
// Report user warnings/errors. If obj2 is given, then it's
// included in the message, though if pinpoint_only is non-zero,
@ -168,10 +159,6 @@ public:
bool in_ser_cache;
protected:
friend class SerializationCache;
DECLARE_ABSTRACT_SERIAL(BroObj);
Location* location; // all that matters in real estate
private:

23
src/Op.h
View file

@ -1,23 +0,0 @@
// See the file "COPYING" in the main distribution directory for copyright.
#ifndef op_h
#define op_h
// BRO operations.
typedef enum {
OP_INCR, OP_DECR, OP_NOT, OP_NEGATE,
OP_PLUS, OP_MINUS, OP_TIMES, OP_DIVIDE, OP_MOD,
OP_AND, OP_OR,
OP_LT, OP_LE, OP_EQ, OP_NE, OP_GE, OP_GT,
OP_MATCH,
OP_ASSIGN,
OP_INDEX, OP_FIELD,
OP_IN,
OP_LIST,
OP_CALL,
OP_SCHED,
OP_NAME, OP_CONST, OP_THIS
} BroOP;
#endif

718
src/OpaqueVal.cc
View file

@ -3,10 +3,153 @@
#include "OpaqueVal.h"
#include "NetVar.h"
#include "Reporter.h"
#include "Serializer.h"
#include "probabilistic/BloomFilter.h"
#include "probabilistic/CardinalityCounter.h"
#include <broker/error.hh>
// Helper to retrieve a broker value out of a broker::vector at a specified
// index, and casted to the expected destination type.
template<typename S, typename V, typename D>
inline bool get_vector_idx(const V& v, unsigned int i, D* dst)
{
if ( i >= v.size() )
return false;
auto x = caf::get_if<S>(&v[i]);
if ( ! x )
return false;
*dst = static_cast<D>(*x);
return true;
}
OpaqueMgr* OpaqueMgr::mgr()
{
static OpaqueMgr mgr;
return &mgr;
}
OpaqueVal::OpaqueVal(OpaqueType* t) : Val(t)
{
}
OpaqueVal::~OpaqueVal()
{
}
const std::string& OpaqueMgr::TypeID(const OpaqueVal* v) const
{
auto x = _types.find(v->OpaqueName());
if ( x == _types.end() )
reporter->InternalError("OpaqueMgr::TypeID: opaque type %s not registered",
v->OpaqueName());
return x->first;
}
OpaqueVal* OpaqueMgr::Instantiate(const std::string& id) const
{
auto x = _types.find(id);
return x != _types.end() ? (*x->second)() : nullptr;
}
broker::expected<broker::data> OpaqueVal::Serialize() const
{
auto type = OpaqueMgr::mgr()->TypeID(this);
auto d = DoSerialize();
if ( ! d )
return d.error();
return {broker::vector{std::move(type), std::move(*d)}};
}
OpaqueVal* OpaqueVal::Unserialize(const broker::data& data)
{
auto v = caf::get_if<broker::vector>(&data);
if ( ! (v && v->size() == 2) )
return nullptr;
auto type = caf::get_if<std::string>(&(*v)[0]);
if ( ! type )
return nullptr;
auto val = OpaqueMgr::mgr()->Instantiate(*type);
if ( ! val )
return nullptr;
if ( ! val->DoUnserialize((*v)[1]) )
{
Unref(val);
return nullptr;
}
return val;
}
broker::expected<broker::data> OpaqueVal::SerializeType(BroType* t)
{
if ( t->InternalType() == TYPE_INTERNAL_ERROR )
return broker::ec::invalid_data;
if ( t->InternalType() == TYPE_INTERNAL_OTHER )
{
// Serialize by name.
assert(t->GetName().size());
return {broker::vector{true, t->GetName()}};
}
// A base type.
return {broker::vector{false, static_cast<uint64>(t->Tag())}};
}
BroType* OpaqueVal::UnserializeType(const broker::data& data)
{
auto v = caf::get_if<broker::vector>(&data);
if ( ! (v && v->size() == 2) )
return nullptr;
auto by_name = caf::get_if<bool>(&(*v)[0]);
if ( ! by_name )
return nullptr;
if ( *by_name )
{
auto name = caf::get_if<std::string>(&(*v)[1]);
if ( ! name )
return nullptr;
ID* id = global_scope()->Lookup(name->c_str());
if ( ! id )
return nullptr;
BroType* t = id->AsType();
if ( ! t )
return nullptr;
return t->Ref();
}
auto tag = caf::get_if<uint64>(&(*v)[1]);
if ( ! tag )
return nullptr;
return base_type(static_cast<TypeTag>(*tag));
}
Val* OpaqueVal::DoClone(CloneState* state)
{
auto d = OpaqueVal::Serialize();
if ( ! d )
return nullptr;
auto rval = OpaqueVal::Unserialize(std::move(*d));
return state->NewClone(this, rval);
}
bool HashVal::IsValid() const
{
return valid;
@ -63,20 +206,6 @@ HashVal::HashVal(OpaqueType* t) : OpaqueVal(t)
valid = false;
}
IMPLEMENT_SERIAL(HashVal, SER_HASH_VAL);
bool HashVal::DoSerialize(SerialInfo* info) const
{
DO_SERIALIZE(SER_HASH_VAL, OpaqueVal);
return SERIALIZE(valid);
}
bool HashVal::DoUnserialize(UnserialInfo* info)
{
DO_UNSERIALIZE(OpaqueVal);
return UNSERIALIZE(&valid);
}
MD5Val::MD5Val() : HashVal(md5_type)
{
}
@ -160,64 +289,72 @@ StringVal* MD5Val::DoGet()
return new StringVal(md5_digest_print(digest));
}
IMPLEMENT_SERIAL(MD5Val, SER_MD5_VAL);
IMPLEMENT_OPAQUE_VALUE(MD5Val)
bool MD5Val::DoSerialize(SerialInfo* info) const
broker::expected<broker::data> MD5Val::DoSerialize() const
{
DO_SERIALIZE(SER_MD5_VAL, HashVal);
if ( ! IsValid() )
return true;
return {broker::vector{false}};
MD5_CTX* md = (MD5_CTX*) EVP_MD_CTX_md_data(ctx);
if ( ! (SERIALIZE(md->A) &&
SERIALIZE(md->B) &&
SERIALIZE(md->C) &&
SERIALIZE(md->D) &&
SERIALIZE(md->Nl) &&
SERIALIZE(md->Nh)) )
broker::vector d = {
true,
static_cast<uint64>(md->A),
static_cast<uint64>(md->B),
static_cast<uint64>(md->C),
static_cast<uint64>(md->D),
static_cast<uint64>(md->Nl),
static_cast<uint64>(md->Nh),
static_cast<uint64>(md->num)
};
for ( int i = 0; i < MD5_LBLOCK; ++i )
d.emplace_back(static_cast<uint64>(md->data[i]));
return {std::move(d)};
}
bool MD5Val::DoUnserialize(const broker::data& data)
{
auto d = caf::get_if<broker::vector>(&data);
if ( ! d )
return false;
auto valid = caf::get_if<bool>(&(*d)[0]);
if ( ! valid )
return false;
if ( ! *valid )
{
assert(! IsValid()); // default set by ctor
return true;
}
Init();
MD5_CTX* md = (MD5_CTX*) EVP_MD_CTX_md_data(ctx);
if ( ! get_vector_idx<uint64_t>(*d, 1, &md->A) )
return false;
if ( ! get_vector_idx<uint64_t>(*d, 2, &md->B) )
return false;
if ( ! get_vector_idx<uint64_t>(*d, 3, &md->C) )
return false;
if ( ! get_vector_idx<uint64_t>(*d, 4, &md->D) )
return false;
if ( ! get_vector_idx<uint64_t>(*d, 5, &md->Nl) )
return false;
if ( ! get_vector_idx<uint64_t>(*d, 6, &md->Nh) )
return false;
if ( ! get_vector_idx<uint64_t>(*d, 7, &md->num) )
return false;
for ( int i = 0; i < MD5_LBLOCK; ++i )
{
if ( ! SERIALIZE(md->data[i]) )
if ( ! get_vector_idx<uint64_t>(*d, 8 + i, &md->data[i]) )
return false;
}
if ( ! SERIALIZE(md->num) )
return false;
return true;
}
bool MD5Val::DoUnserialize(UnserialInfo* info)
{
DO_UNSERIALIZE(HashVal);
if ( ! IsValid() )
return true;
ctx = hash_init(Hash_MD5);
MD5_CTX* md = (MD5_CTX*) EVP_MD_CTX_md_data(ctx);
if ( ! (UNSERIALIZE(&md->A) &&
UNSERIALIZE(&md->B) &&
UNSERIALIZE(&md->C) &&
UNSERIALIZE(&md->D) &&
UNSERIALIZE(&md->Nl) &&
UNSERIALIZE(&md->Nh)) )
return false;
for ( int i = 0; i < MD5_LBLOCK; ++i )
{
if ( ! UNSERIALIZE(&md->data[i]) )
return false;
}
if ( ! UNSERIALIZE(&md->num) )
return false;
return true;
}
@ -293,66 +430,75 @@ StringVal* SHA1Val::DoGet()
return new StringVal(sha1_digest_print(digest));
}
IMPLEMENT_SERIAL(SHA1Val, SER_SHA1_VAL);
IMPLEMENT_OPAQUE_VALUE(SHA1Val)
bool SHA1Val::DoSerialize(SerialInfo* info) const
broker::expected<broker::data> SHA1Val::DoSerialize() const
{
DO_SERIALIZE(SER_SHA1_VAL, HashVal);
if ( ! IsValid() )
return true;
return {broker::vector{false}};
SHA_CTX* md = (SHA_CTX*) EVP_MD_CTX_md_data(ctx);
if ( ! (SERIALIZE(md->h0) &&
SERIALIZE(md->h1) &&
SERIALIZE(md->h2) &&
SERIALIZE(md->h3) &&
SERIALIZE(md->h4) &&
SERIALIZE(md->Nl) &&
SERIALIZE(md->Nh)) )
broker::vector d = {
true,
static_cast<uint64>(md->h0),
static_cast<uint64>(md->h1),
static_cast<uint64>(md->h2),
static_cast<uint64>(md->h3),
static_cast<uint64>(md->h4),
static_cast<uint64>(md->Nl),
static_cast<uint64>(md->Nh),
static_cast<uint64>(md->num)
};
for ( int i = 0; i < SHA_LBLOCK; ++i )
d.emplace_back(static_cast<uint64>(md->data[i]));
return {std::move(d)};
}
bool SHA1Val::DoUnserialize(const broker::data& data)
{
auto d = caf::get_if<broker::vector>(&data);
if ( ! d )
return false;
auto valid = caf::get_if<bool>(&(*d)[0]);
if ( ! valid )
return false;
if ( ! *valid )
{
assert(! IsValid()); // default set by ctor
return true;
}
Init();
SHA_CTX* md = (SHA_CTX*) EVP_MD_CTX_md_data(ctx);
if ( ! get_vector_idx<uint64_t>(*d, 1, &md->h0) )
return false;
if ( ! get_vector_idx<uint64_t>(*d, 2, &md->h1) )
return false;
if ( ! get_vector_idx<uint64_t>(*d, 3, &md->h2) )
return false;
if ( ! get_vector_idx<uint64_t>(*d, 4, &md->h3) )
return false;
if ( ! get_vector_idx<uint64_t>(*d, 5, &md->h4) )
return false;
if ( ! get_vector_idx<uint64_t>(*d, 6, &md->Nl) )
return false;
if ( ! get_vector_idx<uint64_t>(*d, 7, &md->Nh) )
return false;
if ( ! get_vector_idx<uint64_t>(*d, 8, &md->num) )
return false;
for ( int i = 0; i < SHA_LBLOCK; ++i )
{
if ( ! SERIALIZE(md->data[i]) )
if ( ! get_vector_idx<uint64_t>(*d, 9 + i, &md->data[i]) )
return false;
}
if ( ! SERIALIZE(md->num) )
return false;
return true;
}
bool SHA1Val::DoUnserialize(UnserialInfo* info)
{
DO_UNSERIALIZE(HashVal);
if ( ! IsValid() )
return true;
ctx = hash_init(Hash_SHA1);
SHA_CTX* md = (SHA_CTX*) EVP_MD_CTX_md_data(ctx);
if ( ! (UNSERIALIZE(&md->h0) &&
UNSERIALIZE(&md->h1) &&
UNSERIALIZE(&md->h2) &&
UNSERIALIZE(&md->h3) &&
UNSERIALIZE(&md->h4) &&
UNSERIALIZE(&md->Nl) &&
UNSERIALIZE(&md->Nh)) )
return false;
for ( int i = 0; i < SHA_LBLOCK; ++i )
{
if ( ! UNSERIALIZE(&md->data[i]) )
return false;
}
if ( ! UNSERIALIZE(&md->num) )
return false;
return true;
}
@ -428,71 +574,72 @@ StringVal* SHA256Val::DoGet()
return new StringVal(sha256_digest_print(digest));
}
IMPLEMENT_SERIAL(SHA256Val, SER_SHA256_VAL);
IMPLEMENT_OPAQUE_VALUE(SHA256Val)
bool SHA256Val::DoSerialize(SerialInfo* info) const
broker::expected<broker::data> SHA256Val::DoSerialize() const
{
DO_SERIALIZE(SER_SHA256_VAL, HashVal);
if ( ! IsValid() )
return true;
return {broker::vector{false}};
SHA256_CTX* md = (SHA256_CTX*) EVP_MD_CTX_md_data(ctx);
for ( int i = 0; i < 8; ++i )
{
if ( ! SERIALIZE(md->h[i]) )
return false;
}
broker::vector d = {
true,
static_cast<uint64>(md->Nl),
static_cast<uint64>(md->Nh),
static_cast<uint64>(md->num),
static_cast<uint64>(md->md_len)
};
if ( ! (SERIALIZE(md->Nl) &&
SERIALIZE(md->Nh)) )
return false;
for ( int i = 0; i < 8; ++i )
d.emplace_back(static_cast<uint64>(md->h[i]));
for ( int i = 0; i < SHA_LBLOCK; ++i )
{
if ( ! SERIALIZE(md->data[i]) )
return false;
d.emplace_back(static_cast<uint64>(md->data[i]));
return {std::move(d)};
}
if ( ! (SERIALIZE(md->num) &&
SERIALIZE(md->md_len)) )
bool SHA256Val::DoUnserialize(const broker::data& data)
{
auto d = caf::get_if<broker::vector>(&data);
if ( ! d )
return false;
auto valid = caf::get_if<bool>(&(*d)[0]);
if ( ! valid )
return false;
if ( ! *valid )
{
assert(! IsValid()); // default set by ctor
return true;
}
bool SHA256Val::DoUnserialize(UnserialInfo* info)
{
DO_UNSERIALIZE(HashVal);
if ( ! IsValid() )
return true;
ctx = hash_init(Hash_SHA256);
Init();
SHA256_CTX* md = (SHA256_CTX*) EVP_MD_CTX_md_data(ctx);
if ( ! get_vector_idx<uint64_t>(*d, 1, &md->Nl) )
return false;
if ( ! get_vector_idx<uint64_t>(*d, 2, &md->Nh) )
return false;
if ( ! get_vector_idx<uint64_t>(*d, 3, &md->num) )
return false;
if ( ! get_vector_idx<uint64_t>(*d, 4, &md->md_len) )
return false;
for ( int i = 0; i < 8; ++i )
{
if ( ! UNSERIALIZE(&md->h[i]) )
if ( ! get_vector_idx<uint64_t>(*d, 5 + i, &md->h[i]) )
return false;
}
if ( ! (UNSERIALIZE(&md->Nl) &&
UNSERIALIZE(&md->Nh)) )
return false;
for ( int i = 0; i < SHA_LBLOCK; ++i )
{
if ( ! UNSERIALIZE(&md->data[i]) )
if ( ! get_vector_idx<uint64_t>(*d, 13 + i, &md->data[i]) )
return false;
}
if ( ! (UNSERIALIZE(&md->num) &&
UNSERIALIZE(&md->md_len)) )
return false;
return true;
}
@ -500,26 +647,6 @@ EntropyVal::EntropyVal() : OpaqueVal(entropy_type)
{
}
Val* EntropyVal::DoClone(CloneState* state)
{
SerializationFormat* form = new BinarySerializationFormat();
form->StartWrite();
CloneSerializer ss(form);
SerialInfo sinfo(&ss);
sinfo.cache = false;
sinfo.include_locations = false;
if ( ! this->Serialize(&sinfo) )
return nullptr;
char* data;
uint32 len = form->EndWrite(&data);
form->StartRead(data, len);
UnserialInfo uinfo(&ss);
uinfo.cache = false;
Val* clone = Unserialize(&uinfo, type);
free(data);
return state->NewClone(this, clone);
}
bool EntropyVal::Feed(const void* data, size_t size)
{
state.add(data, size);
@ -533,79 +660,86 @@ bool EntropyVal::Get(double *r_ent, double *r_chisq, double *r_mean,
return true;
}
IMPLEMENT_SERIAL(EntropyVal, SER_ENTROPY_VAL);
IMPLEMENT_OPAQUE_VALUE(EntropyVal)
bool EntropyVal::DoSerialize(SerialInfo* info) const
broker::expected<broker::data> EntropyVal::DoSerialize() const
{
DO_SERIALIZE(SER_ENTROPY_VAL, OpaqueVal);
broker::vector d =
{
static_cast<uint64>(state.totalc),
static_cast<uint64>(state.mp),
static_cast<uint64>(state.sccfirst),
static_cast<uint64>(state.inmont),
static_cast<uint64>(state.mcount),
static_cast<uint64>(state.cexp),
static_cast<uint64>(state.montex),
static_cast<uint64>(state.montey),
static_cast<uint64>(state.montepi),
static_cast<uint64>(state.sccu0),
static_cast<uint64>(state.scclast),
static_cast<uint64>(state.scct1),
static_cast<uint64>(state.scct2),
static_cast<uint64>(state.scct3),
};
d.reserve(256 + 3 + RT_MONTEN + 11);
for ( int i = 0; i < 256; ++i )
d.emplace_back(static_cast<uint64>(state.ccount[i]));
for ( int i = 0; i < RT_MONTEN; ++i )
d.emplace_back(static_cast<uint64>(state.monte[i]));
return {std::move(d)};
}
bool EntropyVal::DoUnserialize(const broker::data& data)
{
auto d = caf::get_if<broker::vector>(&data);
if ( ! d )
return false;
if ( ! get_vector_idx<uint64_t>(*d, 0, &state.totalc) )
return false;
if ( ! get_vector_idx<uint64_t>(*d, 1, &state.mp) )
return false;
if ( ! get_vector_idx<uint64_t>(*d, 2, &state.sccfirst) )
return false;
if ( ! get_vector_idx<uint64_t>(*d, 3, &state.inmont) )
return false;
if ( ! get_vector_idx<uint64_t>(*d, 4, &state.mcount) )
return false;
if ( ! get_vector_idx<uint64_t>(*d, 5, &state.cexp) )
return false;
if ( ! get_vector_idx<uint64_t>(*d, 6, &state.montex) )
return false;
if ( ! get_vector_idx<uint64_t>(*d, 7, &state.montey) )
return false;
if ( ! get_vector_idx<uint64_t>(*d, 8, &state.montepi) )
return false;
if ( ! get_vector_idx<uint64_t>(*d, 9, &state.sccu0) )
return false;
if ( ! get_vector_idx<uint64_t>(*d, 10, &state.scclast) )
return false;
if ( ! get_vector_idx<uint64_t>(*d, 11, &state.scct1) )
return false;
if ( ! get_vector_idx<uint64_t>(*d, 12, &state.scct2) )
return false;
if ( ! get_vector_idx<uint64_t>(*d, 13, &state.scct3) )
return false;
for ( int i = 0; i < 256; ++i )
{
if ( ! SERIALIZE(state.ccount[i]) )
if ( ! get_vector_idx<uint64_t>(*d, 14 + i, &state.ccount[i]) )
return false;
}
if ( ! (SERIALIZE(state.totalc) &&
SERIALIZE(state.mp) &&
SERIALIZE(state.sccfirst)) )
return false;
for ( int i = 0; i < RT_MONTEN; ++i )
{
if ( ! SERIALIZE(state.monte[i]) )
if ( ! get_vector_idx<uint64_t>(*d, 14 + 256 + i, &state.monte[i]) )
return false;
}
if ( ! (SERIALIZE(state.inmont) &&
SERIALIZE(state.mcount) &&
SERIALIZE(state.cexp) &&
SERIALIZE(state.montex) &&
SERIALIZE(state.montey) &&
SERIALIZE(state.montepi) &&
SERIALIZE(state.sccu0) &&
SERIALIZE(state.scclast) &&
SERIALIZE(state.scct1) &&
SERIALIZE(state.scct2) &&
SERIALIZE(state.scct3)) )
return false;
return true;
}
bool EntropyVal::DoUnserialize(UnserialInfo* info)
{
DO_UNSERIALIZE(OpaqueVal);
for ( int i = 0; i < 256; ++i )
{
if ( ! UNSERIALIZE(&state.ccount[i]) )
return false;
}
if ( ! (UNSERIALIZE(&state.totalc) &&
UNSERIALIZE(&state.mp) &&
UNSERIALIZE(&state.sccfirst)) )
return false;
for ( int i = 0; i < RT_MONTEN; ++i )
{
if ( ! UNSERIALIZE(&state.monte[i]) )
return false;
}
if ( ! (UNSERIALIZE(&state.inmont) &&
UNSERIALIZE(&state.mcount) &&
UNSERIALIZE(&state.cexp) &&
UNSERIALIZE(&state.montex) &&
UNSERIALIZE(&state.montey) &&
UNSERIALIZE(&state.montepi) &&
UNSERIALIZE(&state.sccu0) &&
UNSERIALIZE(&state.scclast) &&
UNSERIALIZE(&state.scct1) &&
UNSERIALIZE(&state.scct2) &&
UNSERIALIZE(&state.scct3)) )
return false;
return true;
}
@ -739,42 +873,52 @@ BloomFilterVal::~BloomFilterVal()
delete bloom_filter;
}
IMPLEMENT_SERIAL(BloomFilterVal, SER_BLOOMFILTER_VAL);
IMPLEMENT_OPAQUE_VALUE(BloomFilterVal)
bool BloomFilterVal::DoSerialize(SerialInfo* info) const
broker::expected<broker::data> BloomFilterVal::DoSerialize() const
{
DO_SERIALIZE(SER_BLOOMFILTER_VAL, OpaqueVal);
broker::vector d;
bool is_typed = (type != 0);
if ( type )
{
auto t = SerializeType(type);
if ( ! t )
return broker::ec::invalid_data;
if ( ! SERIALIZE(is_typed) )
return false;
d.emplace_back(std::move(*t));
}
else
d.emplace_back(broker::none());
if ( is_typed && ! type->Serialize(info) )
return false;
auto bf = bloom_filter->Serialize();
if ( ! bf )
return broker::ec::invalid_data; // Cannot serialize;
return bloom_filter->Serialize(info);
d.emplace_back(*bf);
return {std::move(d)};
}
bool BloomFilterVal::DoUnserialize(UnserialInfo* info)
bool BloomFilterVal::DoUnserialize(const broker::data& data)
{
DO_UNSERIALIZE(OpaqueVal);
auto v = caf::get_if<broker::vector>(&data);
bool is_typed;
if ( ! UNSERIALIZE(&is_typed) )
if ( ! (v && v->size() == 2) )
return false;
if ( is_typed )
auto no_type = caf::get_if<broker::none>(&(*v)[0]);
if ( ! no_type )
{
BroType* t = BroType::Unserialize(info);
if ( ! Typify(t) )
BroType* t = UnserializeType((*v)[0]);
if ( ! (t && Typify(t)) )
return false;
Unref(t);
}
bloom_filter = probabilistic::BloomFilter::Unserialize(info);
return bloom_filter != 0;
auto bf = probabilistic::BloomFilter::Unserialize((*v)[1]);
if ( ! bf )
return false;
bloom_filter = bf.release();
return true;
}
CardinalityVal::CardinalityVal() : OpaqueVal(cardinality_type)
@ -805,44 +949,6 @@ Val* CardinalityVal::DoClone(CloneState* state)
new CardinalityVal(new probabilistic::CardinalityCounter(*c)));
}
IMPLEMENT_SERIAL(CardinalityVal, SER_CARDINALITY_VAL);
bool CardinalityVal::DoSerialize(SerialInfo* info) const
{
DO_SERIALIZE(SER_CARDINALITY_VAL, OpaqueVal);
bool valid = true;
bool is_typed = (type != 0);
valid &= SERIALIZE(is_typed);
if ( is_typed )
valid &= type->Serialize(info);
return c->Serialize(info);
}
bool CardinalityVal::DoUnserialize(UnserialInfo* info)
{
DO_UNSERIALIZE(OpaqueVal);
bool is_typed;
if ( ! UNSERIALIZE(&is_typed) )
return false;
if ( is_typed )
{
BroType* t = BroType::Unserialize(info);
if ( ! Typify(t) )
return false;
Unref(t);
}
c = probabilistic::CardinalityCounter::Unserialize(info);
return c != 0;
}
bool CardinalityVal::Typify(BroType* arg_type)
{
if ( type )
@ -870,3 +976,51 @@ void CardinalityVal::Add(const Val* val)
c->AddElement(key->Hash());
delete key;
}
IMPLEMENT_OPAQUE_VALUE(CardinalityVal)
broker::expected<broker::data> CardinalityVal::DoSerialize() const
{
broker::vector d;
if ( type )
{
auto t = SerializeType(type);
if ( ! t )
return broker::ec::invalid_data;
d.emplace_back(std::move(*t));
}
else
d.emplace_back(broker::none());
auto cs = c->Serialize();
if ( ! cs )
return broker::ec::invalid_data;
d.emplace_back(*cs);
return {std::move(d)};
}
bool CardinalityVal::DoUnserialize(const broker::data& data)
{
auto v = caf::get_if<broker::vector>(&data);
if ( ! (v && v->size() == 2) )
return false;
auto no_type = caf::get_if<broker::none>(&(*v)[0]);
if ( ! no_type )
{
BroType* t = UnserializeType((*v)[0]);
if ( ! (t && Typify(t)) )
return false;
}
auto cu = probabilistic::CardinalityCounter::Unserialize((*v)[1]);
if ( ! cu )
return false;
c = cu.release();
return true;
}

169
src/OpaqueVal.h
View file

@ -3,12 +3,154 @@
#ifndef OPAQUEVAL_H
#define OPAQUEVAL_H
#include <typeinfo>
#include <broker/data.hh>
#include <broker/expected.hh>
#include "RandTest.h"
#include "Val.h"
#include "digest.h"
class OpaqueVal;
/**
* Singleton that registers all available all available types of opaque
* values. This faciliates their serialization into Broker values.
*/
class OpaqueMgr {
public:
using Factory = OpaqueVal* ();
/**
* Return's a unique ID for the type of an opaque value.
* @param v opaque value to return type for; its class must have been
* registered with the manager, otherwise this method will abort
* execution.
*
* @return type ID, which can used with *Instantiate()* to create a
* new instance of the same type.
*/
const std::string& TypeID(const OpaqueVal* v) const;
/**
* Instantiates a new opaque value of a specific opaque type.
*
* @param id unique type ID for the class to instantiate; this will
* normally have been returned earlier by *TypeID()*.
*
* @return A freshly instantiated value of the OpaqueVal-derived
* classes that *id* specifies, with reference count at +1. If *id*
* is unknown, this will return null.
*
*/
OpaqueVal* Instantiate(const std::string& id) const;
/** Returns the global manager singleton object. */
static OpaqueMgr* mgr();
/**
* Internal helper class to register an OpaqueVal-derived classes
* with the manager.
*/
template<class T>
class Register {
public:
Register(const char* id)
{ OpaqueMgr::mgr()->_types.emplace(id, &T::OpaqueInstantiate); }
};
private:
std::unordered_map<std::string, Factory*> _types;
};
/** Macro to insert into an OpaqueVal-derived class's declaration. */
#define DECLARE_OPAQUE_VALUE(T) \
friend class OpaqueMgr::Register<T>; \
broker::expected<broker::data> DoSerialize() const override; \
bool DoUnserialize(const broker::data& data) override; \
const char* OpaqueName() const override { return #T; } \
static OpaqueVal* OpaqueInstantiate() { return new T(); }
#define __OPAQUE_MERGE(a, b) a ## b
#define __OPAQUE_ID(x) __OPAQUE_MERGE(_opaque, x)
/** Macro to insert into an OpaqueVal-derived class's implementation file. */
#define IMPLEMENT_OPAQUE_VALUE(T) static OpaqueMgr::Register<T> __OPAQUE_ID(__LINE__)(#T);
/**
* Base class for all opaque values. Opaque values are types that are managed
* completely internally, with no further script-level operators provided
* (other than bif functions). See OpaqueVal.h for derived classes.
*/
class OpaqueVal : public Val {
public:
explicit OpaqueVal(OpaqueType* t);
~OpaqueVal() override;
/**
* Serializes the value into a Broker representation.
*
* @return the broker representation, or an error if serialization
* isn't supported or failed.
*/
broker::expected<broker::data> Serialize() const;
/**
* Reinstantiates a value from its serialized Broker representation.
*
* @param data Broker representation as returned by *Serialize()*.
* @return unserialized instances with reference count at +1
*/
static OpaqueVal* Unserialize(const broker::data& data);
protected:
friend class Val;
friend class OpaqueMgr;
OpaqueVal() { }
/**
* Must be overridden to provide a serialized version of the derived
* class' state.
*
* @return the serialized data or an error if serialization
* isn't supported or failed.
*/
virtual broker::expected<broker::data> DoSerialize() const = 0;
/**
* Must be overridden to recreate the the derived class' state from a
* serialization.
*
* @return true if successful.
*/
virtual bool DoUnserialize(const broker::data& data) = 0;
/**
* Internal helper for the serialization machinery. Automatically
* overridden by the `DECLARE_OPAQUE_VALUE` macro.
*/
virtual const char* OpaqueName() const = 0;
/**
* Provides an implementation of *Val::DoClone()* that leverages the
* serialization methods to deep-copy an instance. Derived classes
* may also override this with a more efficient custom clone
* implementation of their own.
*/
Val* DoClone(CloneState* state) override;
/**
* Helper function for derived class that need to record a type
* during serialization.
*/
static broker::expected<broker::data> SerializeType(BroType* t);
/**
* Helper function for derived class that need to restore a type
* during unserialization. Returns the type at reference count +1.
*/
static BroType* UnserializeType(const broker::data& data);
};
namespace probabilistic {
class BloomFilter;
class CardinalityCounter;
@ -22,15 +164,13 @@ public:
virtual StringVal* Get();
protected:
HashVal() { };
HashVal() { valid = false; }
explicit HashVal(OpaqueType* t);
virtual bool DoInit();
virtual bool DoFeed(const void* data, size_t size);
virtual StringVal* DoGet();
DECLARE_SERIAL(HashVal);
private:
// This flag exists because Get() can only be called once.
bool valid;
@ -56,8 +196,7 @@ protected:
bool DoFeed(const void* data, size_t size) override;
StringVal* DoGet() override;
DECLARE_SERIAL(MD5Val);
DECLARE_OPAQUE_VALUE(MD5Val)
private:
EVP_MD_CTX* ctx;
};
@ -78,8 +217,7 @@ protected:
bool DoFeed(const void* data, size_t size) override;
StringVal* DoGet() override;
DECLARE_SERIAL(SHA1Val);
DECLARE_OPAQUE_VALUE(SHA1Val)
private:
EVP_MD_CTX* ctx;
};
@ -100,8 +238,7 @@ protected:
bool DoFeed(const void* data, size_t size) override;
StringVal* DoGet() override;
DECLARE_SERIAL(SHA256Val);
DECLARE_OPAQUE_VALUE(SHA256Val)
private:
EVP_MD_CTX* ctx;
};
@ -110,8 +247,6 @@ class EntropyVal : public OpaqueVal {
public:
EntropyVal();
Val* DoClone(CloneState* state) override;
bool Feed(const void* data, size_t size);
bool Get(double *r_ent, double *r_chisq, double *r_mean,
double *r_montepicalc, double *r_scc);
@ -119,8 +254,7 @@ public:
protected:
friend class Val;
DECLARE_SERIAL(EntropyVal);
DECLARE_OPAQUE_VALUE(EntropyVal)
private:
RandTest state;
};
@ -149,8 +283,7 @@ protected:
BloomFilterVal();
explicit BloomFilterVal(OpaqueType* t);
DECLARE_SERIAL(BloomFilterVal);
DECLARE_OPAQUE_VALUE(BloomFilterVal)
private:
// Disable.
BloomFilterVal(const BloomFilterVal&);
@ -174,18 +307,16 @@ public:
BroType* Type() const;
bool Typify(BroType* type);
probabilistic::CardinalityCounter* Get() { return c; };
protected:
CardinalityVal();
DECLARE_OPAQUE_VALUE(CardinalityVal)
private:
BroType* type;
CompositeHash* hash;
probabilistic::CardinalityCounter* c;
DECLARE_SERIAL(CardinalityVal);
};
#endif

53
src/RE.cc
View file

@ -9,7 +9,7 @@
#include "DFA.h"
#include "CCL.h"
#include "EquivClass.h"
#include "Serializer.h"
#include "Reporter.h"
CCL* curr_ccl = 0;
@ -469,57 +469,6 @@ int RE_Matcher::Compile(int lazy)
return re_anywhere->Compile(lazy) && re_exact->Compile(lazy);
}
bool RE_Matcher::Serialize(SerialInfo* info) const
{
return SerialObj::Serialize(info);
}
RE_Matcher* RE_Matcher::Unserialize(UnserialInfo* info)
{
return (RE_Matcher*) SerialObj::Unserialize(info, SER_RE_MATCHER);
}
IMPLEMENT_SERIAL(RE_Matcher, SER_RE_MATCHER);
bool RE_Matcher::DoSerialize(SerialInfo* info) const
{
DO_SERIALIZE(SER_RE_MATCHER, SerialObj);
return SERIALIZE(re_anywhere->PatternText())
&& SERIALIZE(re_exact->PatternText());
}
bool RE_Matcher::DoUnserialize(UnserialInfo* info)
{
DO_UNSERIALIZE(SerialObj);
re_anywhere = new Specific_RE_Matcher(MATCH_ANYWHERE);
re_exact = new Specific_RE_Matcher(MATCH_EXACTLY);
const char* pat;
if ( ! UNSERIALIZE_STR(&pat, 0) )
return false;
re_anywhere->SetPat(pat);
if ( ! re_anywhere->Compile() )
{
info->s->Error(fmt("Can't compile regexp '%s'", pat));
return false;
}
if ( ! UNSERIALIZE_STR(&pat, 0) )
return false;
re_exact->SetPat(pat);
if ( ! re_exact->Compile() )
{
info->s->Error(fmt("Can't compile regexp '%s'", pat));
return false;
}
return true;
}
static RE_Matcher* matcher_merge(const RE_Matcher* re1, const RE_Matcher* re2,
const char* merge_op)
{

9
src/RE.h
View file

@ -171,12 +171,12 @@ protected:
int current_pos;
};
class RE_Matcher : SerialObj {
class RE_Matcher {
public:
RE_Matcher();
explicit RE_Matcher(const char* pat);
RE_Matcher(const char* exact_pat, const char* anywhere_pat);
virtual ~RE_Matcher() override;
virtual ~RE_Matcher();
void AddPat(const char* pat);
@ -212,9 +212,6 @@ public:
const char* PatternText() const { return re_exact->PatternText(); }
const char* AnywherePatternText() const { return re_anywhere->PatternText(); }
bool Serialize(SerialInfo* info) const;
static RE_Matcher* Unserialize(UnserialInfo* info);
unsigned int MemoryAllocation() const
{
return padded_sizeof(*this)
@ -223,8 +220,6 @@ public:
}
protected:
DECLARE_SERIAL(RE_Matcher);
Specific_RE_Matcher* re_anywhere;
Specific_RE_Matcher* re_exact;
};

35
src/Reassem.cc
View file

@ -6,7 +6,6 @@
#include "zeek-config.h"
#include "Reassem.h"
#include "Serializer.h"
static const bool DEBUG_reassem = false;
@ -357,37 +356,3 @@ uint64 Reassembler::MemoryAllocation(ReassemblerType rtype)
return Reassembler::sizes[rtype];
}
bool Reassembler::Serialize(SerialInfo* info) const
{
return SerialObj::Serialize(info);
}
Reassembler* Reassembler::Unserialize(UnserialInfo* info)
{
return (Reassembler*) SerialObj::Unserialize(info, SER_REASSEMBLER);
}
bool Reassembler::DoSerialize(SerialInfo* info) const
{
DO_SERIALIZE(SER_REASSEMBLER, BroObj);
// I'm not sure if it makes sense to actually save the buffered data.
// For now, we just remember the seq numbers so that we don't get
// complaints about missing content.
return SERIALIZE(trim_seq) && SERIALIZE(int(0));
}
bool Reassembler::DoUnserialize(UnserialInfo* info)
{
DO_UNSERIALIZE(BroObj);
blocks = last_block = 0;
int dummy; // For backwards compatibility.
if ( ! UNSERIALIZE(&trim_seq) || ! UNSERIALIZE(&dummy) )
return false;
last_reassem_seq = trim_seq;
return true;
}

5
src/Reassem.h
View file

@ -62,9 +62,6 @@ public:
void Describe(ODesc* d) const override;
bool Serialize(SerialInfo* info) const;
static Reassembler* Unserialize(UnserialInfo* info);
// Sum over all data buffered in some reassembler.
static uint64 TotalMemoryAllocation() { return total_size; }
@ -76,8 +73,6 @@ public:
protected:
Reassembler() { }
DECLARE_ABSTRACT_SERIAL(Reassembler);
friend class DataBlock;
virtual void Undelivered(uint64 up_to_seq);

170
src/SerialInfo.h
View file

@ -1,170 +0,0 @@
// Helper classes to pass data between serialization methods.
#ifndef serialinfo_h
#define serialinfo_h
#include "ChunkedIO.h"
class SerialInfo {
public:
SerialInfo(Serializer* arg_s)
{
chunk = 0;
s = arg_s;
may_suspend = clear_containers = false;
cache = globals_as_names = true;
type = SER_NONE;
pid_32bit = false;
include_locations = true;
new_cache_strategy = false;
}
SerialInfo(const SerialInfo& info)
{
chunk = info.chunk;
s = info.s;
may_suspend = info.may_suspend;
cache = info.cache;
type = info.type;
clear_containers = info.clear_containers;
globals_as_names = info.globals_as_names;
pid_32bit = info.pid_32bit;
include_locations = info.include_locations;
new_cache_strategy = info.new_cache_strategy;
}
// Parameters that control serialization.
Serializer* s; // serializer to use
bool cache; // true if object caching is ok
bool may_suspend; // if true, suspending serialization is ok
bool clear_containers; // if true, store container values as empty
bool include_locations; // if true, include locations in serialization
// If true, for NameExpr's serialize just the names of globals, just
// their value.
bool globals_as_names;
bool pid_32bit; // if true, use old-style 32-bit permanent IDs
// If true, we support keeping objs in cache permanently.
bool new_cache_strategy;
ChunkedIO::Chunk* chunk; // chunk written right before the serialization
// Attributes set during serialization.
SerialType type; // type of currently serialized object
// State for suspending/resuming serialization
Continuation cont;
};
class UnserialInfo {
public:
UnserialInfo(Serializer* arg_s)
{
s = arg_s;
cache = true;
type = SER_NONE;
chunk = 0;
install_globals = install_conns = true;
install_uniques = false;
ignore_callbacks = false;
id_policy = Replace;
print = 0;
pid_32bit = false;
new_cache_strategy = false;
}
UnserialInfo(const UnserialInfo& info)
{
s = info.s;
cache = info.cache;
type = info.type;
chunk = info.chunk;
install_globals = info.install_globals;
install_uniques = info.install_uniques;
install_conns = info.install_conns;
ignore_callbacks = info.ignore_callbacks;
id_policy = info.id_policy;
print = info.print;
pid_32bit = info.pid_32bit;
new_cache_strategy = info.new_cache_strategy;
}
// Parameters that control unserialization.
Serializer* s; // serializer to use
bool cache; // if true, object caching is ok
FILE* print; // print read objects to given file (human-readable)
ChunkedIO::Chunk* chunk; // chunk to parse (rather than reading one)
bool install_globals; // if true, install unknown globals
// in global scope
bool install_conns; // if true, add connections to session table
bool install_uniques; // if true, install unknown globally
// unique IDs in global scope
bool ignore_callbacks; // if true, don't call Got*() callbacks
bool pid_32bit; // if true, use old-style 32-bit permanent IDs.
// If true, we support keeping objs in cache permanently.
bool new_cache_strategy;
// If a global ID already exits, of these policies is used.
enum {
Keep, // keep the old ID and ignore the new
Replace, // install the new ID (default)
// Keep current ID instance but copy the new value into it
// (types have to match).
CopyNewToCurrent,
// Install the new ID instance but replace its value
// with that of the old one (types have to match).
CopyCurrentToNew,
// Instantiate a new ID, but do not insert it into the global
// space.
InstantiateNew,
} id_policy;
// Attributes set during unserialization.
SerialType type; // type of currently unserialized object
};
// Helper class to temporarily disable suspending for all next-level calls
// using the given SerialInfo. It saves the current value of info.may_suspend
// and then sets it to false. When it goes out of scope, the original value
// is restored.
//
// We need this because not all classes derived from SerialObj are
// suspension-aware yet, i.e., they don't work correctly if one of the
// next-level functions suspends. Eventually this may change, but actually
// it's not very important: most classes don't need to suspend anyway as
// their data volume is very small. We have to make sure though that those
// which do (e.g. TableVals) support suspension.
class DisableSuspend {
public:
DisableSuspend(SerialInfo* arg_info)
{
info = arg_info;
old_may_suspend = info->may_suspend;
info->may_suspend = false;
}
~DisableSuspend() { Restore(); }
void Release() { info = 0; }
// Restores the suspension-state to its original value.
void Restore()
{
if ( info )
info->may_suspend = old_may_suspend;
}
private:
SerialInfo* info;
bool old_may_suspend;
};
#endif

277
src/SerialObj.cc
View file

@ -1,277 +0,0 @@
#include "SerialObj.h"
#include "Serializer.h"
TransientID::ID TransientID::counter = 0;
SerialObj::FactoryMap* SerialObj::factories = 0;
SerialObj::ClassNameMap* SerialObj::names = 0;
uint64 SerialObj::time_counter = NEVER + ALWAYS + 1;
SerialObj* SerialObj::Instantiate(SerialType type)
{
FactoryMap::iterator f = factories->find(type & SER_TYPE_MASK_EXACT);
if ( f != factories->end() )
{
SerialObj* o = (SerialObj*) (*f->second)();
#ifdef DEBUG
o->serial_type = o->GetSerialType();
#endif
return o;
}
reporter->Error("Unknown object type 0x%08x", type);
return 0;
}
const char* SerialObj::ClassName(SerialType type)
{
ClassNameMap::iterator f = names->find(type);
if ( f != names->end() )
return f->second;
reporter->Error("Unknown object type 0x%08x", type);
return "<no-class-name>";
}
void SerialObj::Register(SerialType type, FactoryFunc f, const char* name)
{
if ( ! factories )
{
factories = new FactoryMap;
names = new ClassNameMap;
}
type = type & SER_TYPE_MASK_EXACT;
FactoryMap::iterator i = factories->find(type);
if ( i != factories->end() )
reporter->InternalError("SerialType 0x%08x registered twice", type);
(*factories)[type] = f;
(*names)[type] = name;
}
inline bool SerializePID(SerialInfo* info, bool full, SerializationCache::PermanentID pid)
{
if ( ! SERIALIZE(full) )
return false;
if ( ! info->pid_32bit )
return SERIALIZE(pid);
// Broccoli compatibility mode with 32bit pids.
uint32 tmp = uint32(pid);
return SERIALIZE(tmp);
}
bool SerialObj::Serialize(SerialInfo* info) const
{
assert(info);
if ( info->cont.NewInstance() )
{
SerializationCache::PermanentID pid = SerializationCache::NONE;
const TransientID* tid = GetTID();
if ( ! tid )
reporter->InternalError("no tid - missing DECLARE_SERIAL?");
if ( info->cache )
pid = info->s->Cache()->Lookup(*tid);
if ( pid != SerializationCache::NONE && info->cache )
{
DBG_LOG(DBG_SERIAL, "%s [%p, ref pid %lld, tid %lld]", __PRETTY_FUNCTION__, this, (long long) pid, tid->Value() );
DBG_LOG(DBG_SERIAL, "-- Caching");
DBG_PUSH(DBG_SERIAL);
if ( ! SerializePID(info, false, pid) )
{
DBG_POP(DBG_SERIAL);
return false;
}
DBG_POP(DBG_SERIAL);
return true;
}
if ( info->cache )
pid = info->s->Cache()->Register(this,
SerializationCache::NONE,
info->new_cache_strategy);
DBG_LOG(DBG_SERIAL, "%s [%p, new pid %lld, tid %lld]", __PRETTY_FUNCTION__, this, (long long) pid, tid->Value() );
DBG_LOG(DBG_SERIAL, "-- Caching");
DBG_PUSH(DBG_SERIAL);
if ( ! SerializePID(info, true, pid) )
{
DBG_POP(DBG_SERIAL);
return false;
}
info->type = SER_NONE;
DBG_POP(DBG_SERIAL);
}
DBG_PUSH(DBG_SERIAL);
info->cont.SaveContext();
bool ret = DoSerialize(info);
info->cont.RestoreContext();
DBG_POP(DBG_SERIAL);
if ( info->cont.ChildSuspended() )
return ret;
#ifdef DEBUG
if ( debug_logger.IsEnabled(DBG_SERIAL) && IsBroObj(serial_type) )
{
ODesc desc(DESC_READABLE);
((BroObj*)this)->Describe(&desc);
DBG_LOG(DBG_SERIAL, "-- Desc: %s", desc.Description());
}
#endif
return ret;
}
SerialObj* SerialObj::Unserialize(UnserialInfo* info, SerialType type)
{
SerializationCache::PermanentID pid = SerializationCache::NONE;
DBG_LOG(DBG_SERIAL, "%s", __PRETTY_FUNCTION__);
bool full_obj;
DBG_LOG(DBG_SERIAL, "-- Caching");
DBG_PUSH(DBG_SERIAL);
bool result;
if ( ! info->pid_32bit )
result = UNSERIALIZE(&full_obj) && UNSERIALIZE(&pid);
else
{
// Broccoli compatibility mode with 32bit pids.
uint32 tmp = 0;
result = UNSERIALIZE(&full_obj) && UNSERIALIZE(&tmp);
pid = tmp;
}
if ( ! result )
{
DBG_POP(DBG_SERIAL);
return 0;
}
DBG_POP(DBG_SERIAL);
DBG_LOG(DBG_SERIAL, "-- [%s pid %lld]", full_obj ? "obj" : "ref", (long long) pid);
if ( ! full_obj )
{
// FIXME: Yet another const_cast to check eventually...
SerialObj* obj =
const_cast<SerialObj*>(info->s->Cache()->Lookup(pid));
if ( obj )
{
if ( obj->IsBroObj() )
Ref((BroObj*) obj);
return obj;
}
// In the following we'd like the format specifier to match
// the type of pid; but pid is uint64, for which there's
// no portable format specifier. So we upcast it to long long,
// which is at least that size, and use a matching format.
info->s->Error(fmt("unknown object %lld referenced",
(long long) pid));
return 0;
}
uint16 stype;
if ( ! UNSERIALIZE(&stype) )
return 0;
SerialObj* obj = Instantiate(SerialType(stype));
if ( ! obj )
{
info->s->Error("unknown object type");
return 0;
}
#ifdef DEBUG
obj->serial_type = stype;
#endif
const TransientID* tid = obj->GetTID();
if ( ! tid )
reporter->InternalError("no tid - missing DECLARE_SERIAL?");
if ( info->cache )
info->s->Cache()->Register(obj, pid, info->new_cache_strategy);
info->type = stype;
DBG_PUSH(DBG_SERIAL);
if ( ! obj->DoUnserialize(info) )
{
DBG_POP(DBG_SERIAL);
return 0;
}
DBG_POP(DBG_SERIAL);
if ( ! SerialObj::CheckTypes(stype, type) )
{
info->s->Error("type mismatch");
return 0;
}
#ifdef DEBUG
if ( debug_logger.IsEnabled(DBG_SERIAL) && IsBroObj(stype) )
{
ODesc desc(DESC_READABLE);
((BroObj*)obj)->Describe(&desc);
DBG_LOG(DBG_SERIAL, "-- Desc: %s", desc.Description());
}
#endif
assert(obj);
return obj;
}
bool SerialObj::DoSerialize(SerialInfo* info) const
{
assert(info->type != SER_NONE);
#ifdef DEBUG
const_cast<SerialObj*>(this)->serial_type = info->type;
#endif
DBG_LOG(DBG_SERIAL, __PRETTY_FUNCTION__);
DBG_PUSH(DBG_SERIAL);
uint16 stype = uint16(info->type);
if ( ! info->new_cache_strategy )
{
// This is a bit unfortunate: to make sure we're sending
// out the same types as in the past, we need to strip out
// the new cache stable bit.
stype &= ~SER_IS_CACHE_STABLE;
}
bool ret = SERIALIZE(stype);
DBG_POP(DBG_SERIAL);
return ret;
}
bool SerialObj::DoUnserialize(UnserialInfo* info)
{
DBG_LOG(DBG_SERIAL, __PRETTY_FUNCTION__);
return true;
}

382
src/SerialObj.h
View file

@ -1,382 +0,0 @@
// Infrastructure for serializable objects.
//
// How to make objects of class Foo serializable:
//
// 1. Derive Foo (directly or indirectly) from SerialObj.
// 2. Add a SER_FOO constant to SerialTypes in SerialTypes.h.
// 3. Add DECLARE_SERIAL(Foo) into class definition.
// 4. Add a (preferably protected) default ctor if it doesn't already exist.
// 5. For non-abstract classes, add IMPLEMENT_SERIAL(Foo, SER_FOO) to *.cc
// 6. Add two methods like this to *.cc (keep names of arguments!)
//
// bool Foo::DoSerialize(SerialInfo* info) const
// {
// DO_SERIALIZE(SER_FOO, ParentClassOfFoo);
// <... serialize class members via methods in Serializer ...>
// return true if everything ok;
// }
//
// bool Foo::DoUnserialize(UnserialInfo* info)
// {
// DO_UNSERIALIZE(ParentClassOfFoo);
// <... unserialize class members via methods in Serializer ...>
// return true if everything ok;
// }
//
// (7. If no parent class of Foo already contains Serialize()/Unserialize()
// methods, these need to be added somewhere too. But most of the various
// parts of the class hierarchy already have them.)
#ifndef SERIALOBJ_H
#define SERIALOBJ_H
#include <map>
#include <util.h>
#include "DebugLogger.h"
#include "Continuation.h"
#include "SerialTypes.h"
#include "zeek-config.h"
#if SIZEOF_LONG_LONG < 8
# error "Serialization requires that sizeof(long long) is at least 8. (Remove this message only if you know what you're doing.)"
#endif
class Serializer;
class SerialInfo;
class UnserialInfo;
class SerializationCache;
// Per-process unique ID.
class TransientID {
public:
TransientID() { id = ++counter; }
typedef unsigned long long ID;
ID Value() const { return id; }
private:
ID id;
static ID counter;
};
// Abstract base class for serializable objects.
class SerialObj {
public:
virtual ~SerialObj() { }
virtual const TransientID* GetTID() const { return 0; }
virtual SerialType GetSerialType() const { return 0; }
bool IsBroObj() const { return IsBroObj(GetSerialType()); }
bool IsCacheStable() const { return IsCacheStable(GetSerialType()); }
static const uint64 NEVER = 0;
static const uint64 ALWAYS = 1;
// Returns time of last modification. This "time" is a monotonically
// increasing counter which is incremented each time a modification is
// performed (more precisely: each time an object is modified which
// returns something different than NEVER). Such times can thus be
// compared to see whether some modification took place before another.
//
// There are two special values:
// NEVER: This object will never change.
// ALWAYS: Always consider this object as changed, i.e., don't
// cache it.
virtual uint64 LastModified() const { return NEVER; }
// Instantiate an object of the given type. Return nil
// if unknown.
static SerialObj* Instantiate(SerialType type);
static const char* ClassName(SerialType type);
// Associate a "factory" function with the given type.
// A factory is a class or function that creates instances
// of a certain type.
typedef SerialObj* (*FactoryFunc)();
static void Register(SerialType type, FactoryFunc f,
const char* class_name);
static bool IsBroObj(SerialType type)
{ return type & SER_IS_BRO_OBJ; }
static bool IsCacheStable(SerialType type)
{ return type & SER_IS_CACHE_STABLE; }
static bool CheckTypes(SerialType type1, SerialType type2)
{ return (type1 & SER_TYPE_MASK_PARENT) ==
(type2 & SER_TYPE_MASK_PARENT); }
protected:
friend class SerializationCache;
SerialObj()
{
#ifdef DEBUG
serial_type = 0;
#endif
}
// Serializes this object. If info->cache is false, we can use
// DECLARE_NON_CACHEABLE_SERIAL (instead of DECLARE_SERIAL) which
// avoids storing a per-object id.
bool Serialize(SerialInfo* info) const;
// Unserializes next object.
static SerialObj* Unserialize(UnserialInfo* info,
SerialType type);
virtual bool DoSerialize(SerialInfo* info) const;
virtual bool DoUnserialize(UnserialInfo* info);
typedef std::map<SerialType, FactoryFunc> FactoryMap;
static FactoryMap* factories;
typedef std::map<SerialType, const char*> ClassNameMap;
static ClassNameMap* names;
static uint64 time_counter;
static uint64 IncreaseTimeCounter() { return ++time_counter; }
static uint64 GetTimeCounter() { return time_counter; }
#ifdef DEBUG
SerialType serial_type;
#endif
};
// A class that registers a factory function upon instantiation.
class SerialTypeRegistrator {
public:
SerialTypeRegistrator(SerialType type, SerialObj::FactoryFunc func,
const char* class_name)
{
SerialObj::Register(type, func, class_name);
}
};
// Macro helpers.
#define DECLARE_ABSTRACT_SERIAL(classname) \
bool DoSerialize(SerialInfo*) const override; \
bool DoUnserialize(UnserialInfo*) override; \
#define DECLARE_SERIAL(classname) \
static classname* Instantiate(); \
static SerialTypeRegistrator register_type; \
bool DoSerialize(SerialInfo*) const override; \
bool DoUnserialize(UnserialInfo*) override; \
const TransientID* GetTID() const override { return &tid; } \
SerialType GetSerialType() const override; \
TransientID tid;
// Only needed (and usable) for non-abstract classes.
#define IMPLEMENT_SERIAL(classname, classtype) \
SerialTypeRegistrator classname::register_type(classtype, \
FactoryFunc(&classname::Instantiate), #classname); \
SerialType classname::GetSerialType() const { return classtype; }; \
classname* classname::Instantiate() { return new classname(); } \
// Pushes debug level on instantiation and pops when it goes out of scope.
class AutoPush {
public:
AutoPush() { DBG_PUSH(DBG_SERIAL); }
~AutoPush() { DBG_POP(DBG_SERIAL); }
};
// Note that by default we disable suspending. Use DO_SERIALIZE_WITH_SUSPEND
// to enable, but be careful to make sure that whomever calls us is aware of
// the fact (or has already disabled suspension itself).
#define DO_SERIALIZE(classtype, super) \
DBG_LOG(DBG_SERIAL, __PRETTY_FUNCTION__); \
if ( info->type == SER_NONE ) \
info->type = classtype; \
DisableSuspend suspend(info); \
AutoPush auto_push; \
if ( ! super::DoSerialize(info) ) \
return false;
// Unfortunately, this is getting quite long. :-(
#define DO_SERIALIZE_WITH_SUSPEND(classtype, super) \
DBG_LOG(DBG_SERIAL, __PRETTY_FUNCTION__); \
if ( info->type == SER_NONE ) \
info->type = classtype; \
AutoPush auto_push; \
\
bool call_super = info->cont.NewInstance(); \
\
if ( info->cont.ChildSuspended() ) \
{ \
void* user_ptr = info->cont.RestoreState(); \
if ( user_ptr == &call_super ) \
call_super = true; \
} \
\
if ( call_super ) \
{ \
info->cont.SaveState(&call_super); \
info->cont.SaveContext(); \
bool result = super::DoSerialize(info); \
info->cont.RestoreContext(); \
if ( ! result ) \
return false; \
if ( info->cont.ChildSuspended() ) \
return true; \
info->cont.SaveState(0); \
} \
#define DO_UNSERIALIZE(super) \
DBG_LOG(DBG_SERIAL, __PRETTY_FUNCTION__); \
AutoPush auto_push; \
if ( ! super::DoUnserialize(info) ) \
return false;
#define SERIALIZE(x) \
info->s->Write(x, #x)
#define SERIALIZE_STR(x, y) \
info->s->Write(x, y, #x)
#define SERIALIZE_BIT(bit) \
info->s->Write(bool(bit), #bit)
#define UNSERIALIZE(x) \
info->s->Read(x, #x)
#define UNSERIALIZE_STR(x, y) \
info->s->Read(x, y, #x)
#define UNSERIALIZE_BIT(bit) \
{ \
bool tmp; \
if ( ! info->s->Read(&tmp, #bit) ) \
return false; \
bit = (unsigned int) tmp; \
}
// Some helpers for pointers which may be nil.
#define SERIALIZE_OPTIONAL(ptr) \
{ \
if ( ptr ) \
{ \
if ( ! info->cont.ChildSuspended() ) \
if ( ! info->s->Write(true, "has_" #ptr) ) \
return false; \
\
info->cont.SaveContext(); \
bool result = ptr->Serialize(info); \
info->cont.RestoreContext(); \
if ( ! result ) \
return false; \
\
if ( info->cont.ChildSuspended() ) \
return true; \
} \
\
else if ( ! info->s->Write(false, "has_" #ptr) ) \
return false; \
}
#define SERIALIZE_OPTIONAL_STR(str) \
{ \
if ( str ) \
{ \
if ( ! (info->s->Write(true, "has_" #str) && info->s->Write(str, "str")) ) \
return false; \
} \
\
else if ( ! info->s->Write(false, "has_" #str) ) \
return false; \
}
#define UNSERIALIZE_OPTIONAL(dst, unserialize) \
{ \
bool has_it; \
if ( ! info->s->Read(&has_it, "has_" #dst) ) \
return false; \
\
if ( has_it ) \
{ \
dst = unserialize; \
if ( ! dst ) \
return false; \
} \
\
else \
dst = 0; \
}
#define UNSERIALIZE_OPTIONAL_STR(dst) \
{ \
bool has_it; \
if ( ! info->s->Read(&has_it, "has_" #dst) ) \
return false; \
\
if ( has_it ) \
{ \
if ( ! info->s->Read(&dst, 0, "has_" #dst) ) \
return false; \
if ( ! dst ) \
return false; \
} \
\
else \
dst = 0; \
}
#define UNSERIALIZE_OPTIONAL_STR_DEL(dst, del) \
{ \
bool has_it; \
if ( ! info->s->Read(&has_it, "has_" #dst) ) \
{ \
delete del; \
return 0; \
} \
\
if ( has_it ) \
{ \
if ( ! info->s->Read(&dst, 0, "has_" #dst) ) \
{ \
delete del; \
return 0; \
} \
if ( ! dst ) \
{ \
delete del; \
return 0; \
} \
} \
\
else \
dst = 0; \
}
#define UNSERIALIZE_OPTIONAL_STATIC(dst, unserialize, del) \
{ \
bool has_it; \
if ( ! info->s->Read(&has_it, "has_" #dst) ) \
{ \
delete del; \
return 0; \
} \
\
if ( has_it ) \
{ \
dst = unserialize; \
if ( ! dst ) \
{ \
delete del; \
return 0; \
} \
} \
\
else \
dst = 0; \
}
#endif

235
src/SerialTypes.h
View file

@ -1,235 +0,0 @@
#ifndef serialtypes_h
#define serialtypes_h
// Each serializable class gets a type.
//
// The type enables a form of poor man's type-checking:
// Bit 0-7: Number (unique relative to main parent (see below)).
// Bit 8-12: Main parent class (SER_IS_*)
// Bit 13: unused
// Bit 14: 1 if preference is to keep in cache.
// Bit 15: 1 if derived from BroObj.
typedef uint16 SerialType;
static const SerialType SER_TYPE_MASK_EXACT = 0x1fff;
static const SerialType SER_TYPE_MASK_PARENT = 0x1f00;
static const SerialType SER_IS_CACHE_STABLE = 0x4000;
static const SerialType SER_IS_BRO_OBJ = 0x8000;
#define SERIAL_CONST(name, val, type) \
const SerialType SER_ ## name = val | SER_IS_ ## type;
#define SERIAL_CONST2(name) SERIAL_CONST(name, 1, name)
#define SERIAL_IS(name, val) \
static const SerialType SER_IS_ ## name = val;
#define SERIAL_IS_BO(name, val) \
static const SerialType SER_IS_ ## name = val | SER_IS_BRO_OBJ;
#define SERIAL_IS_BO_AND_CACHE_STABLE(name, val) \
static const SerialType SER_IS_ ## name = val | (SER_IS_BRO_OBJ | SER_IS_CACHE_STABLE);
SERIAL_IS_BO(CONNECTION, 0x0100)
SERIAL_IS(TIMER, 0x0200)
SERIAL_IS(TCP_ENDPOINT, 0x0300)
SERIAL_IS_BO(TCP_ANALYZER, 0x0400)
SERIAL_IS_BO(TCP_ENDPOINT_ANALYZER, 0x0500)
SERIAL_IS(TCP_CONTENTS, 0x0600)
SERIAL_IS(REASSEMBLER, 0x0700)
SERIAL_IS_BO(VAL, 0x0800)
SERIAL_IS_BO_AND_CACHE_STABLE(EXPR, 0x0900)
SERIAL_IS_BO_AND_CACHE_STABLE(BRO_TYPE, 0x0a00)
SERIAL_IS_BO_AND_CACHE_STABLE(STMT, 0x0b00)
SERIAL_IS_BO_AND_CACHE_STABLE(ATTRIBUTES, 0x0c00)
SERIAL_IS_BO_AND_CACHE_STABLE(EVENT_HANDLER, 0x0d00)
SERIAL_IS_BO_AND_CACHE_STABLE(BRO_FILE, 0x0e00)
SERIAL_IS_BO_AND_CACHE_STABLE(FUNC, 0x0f00)
SERIAL_IS_BO(ID, 0x1000)
SERIAL_IS(STATE_ACCESS, 0x1100)
SERIAL_IS_BO(CASE, 0x1200)
SERIAL_IS(LOCATION, 0x1300)
SERIAL_IS(RE_MATCHER, 0x1400)
SERIAL_IS(BITVECTOR, 0x1500)
SERIAL_IS(COUNTERVECTOR, 0x1600)
SERIAL_IS(BLOOMFILTER, 0x1700)
SERIAL_IS(HASHER, 0x1800)
// These are the externally visible types.
const SerialType SER_NONE = 0;
SERIAL_CONST2(BRO_OBJ)
#define SERIAL_CONN(name, val) SERIAL_CONST(name, val, CONNECTION)
SERIAL_CONN(CONNECTION, 1)
SERIAL_CONN(ICMP_ANALYZER, 2)
// We use ICMP_Echo here rather than ICMP_ECHO because the latter gets
// macro expanded :-(.
SERIAL_CONN(ICMP_Echo, 3)
SERIAL_CONN(ICMP_CONTEXT, 4)
SERIAL_CONN(TCP_CONNECTION, 5)
SERIAL_CONN(TCP_CONNECTION_CONTENTS, 6)
SERIAL_CONN(FTP_CONN, 7)
SERIAL_CONN(UDP_CONNECTION, 8)
#define SERIAL_TIMER(name, val) SERIAL_CONST(name, val, TIMER)
SERIAL_TIMER(TIMER, 1)
SERIAL_TIMER(CONNECTION_TIMER, 2)
SERIAL_CONST2(TCP_ENDPOINT)
SERIAL_CONST2(TCP_ANALYZER)
SERIAL_CONST2(TCP_ENDPOINT_ANALYZER)
#define SERIAL_TCP_CONTENTS(name, val) SERIAL_CONST(name, val, TCP_CONTENTS)
SERIAL_TCP_CONTENTS(TCP_CONTENTS, 1)
SERIAL_TCP_CONTENTS(TCP_CONTENT_LINE, 2)
SERIAL_TCP_CONTENTS(TCP_NVT, 3)
#define SERIAL_REASSEMBLER(name, val) SERIAL_CONST(name, val, REASSEMBLER)
SERIAL_REASSEMBLER(REASSEMBLER, 1)
SERIAL_REASSEMBLER(TCP_REASSEMBLER, 2)
SERIAL_REASSEMBLER(FILE_REASSEMBLER, 3)
#define SERIAL_VAL(name, val) SERIAL_CONST(name, val, VAL)
SERIAL_VAL(VAL, 1)
SERIAL_VAL(INTERVAL_VAL, 2)
SERIAL_VAL(PORT_VAL, 3)
SERIAL_VAL(ADDR_VAL, 4)
SERIAL_VAL(SUBNET_VAL, 5)
SERIAL_VAL(STRING_VAL, 6)
SERIAL_VAL(PATTERN_VAL, 7)
SERIAL_VAL(LIST_VAL, 8)
SERIAL_VAL(TABLE_VAL, 9)
SERIAL_VAL(RECORD_VAL, 10)
SERIAL_VAL(ENUM_VAL, 11)
SERIAL_VAL(VECTOR_VAL, 12)
SERIAL_VAL(MUTABLE_VAL, 13)
SERIAL_VAL(OPAQUE_VAL, 14)
SERIAL_VAL(HASH_VAL, 15)
SERIAL_VAL(MD5_VAL, 16)
SERIAL_VAL(SHA1_VAL, 17)
SERIAL_VAL(SHA256_VAL, 18)
SERIAL_VAL(ENTROPY_VAL, 19)
SERIAL_VAL(TOPK_VAL, 20)
SERIAL_VAL(BLOOMFILTER_VAL, 21)
SERIAL_VAL(CARDINALITY_VAL, 22)
SERIAL_VAL(X509_VAL, 23)
SERIAL_VAL(COMM_STORE_HANDLE_VAL, 24)
SERIAL_VAL(COMM_DATA_VAL, 25)
SERIAL_VAL(OCSP_RESP_VAL, 26)
#define SERIAL_EXPR(name, val) SERIAL_CONST(name, val, EXPR)
SERIAL_EXPR(EXPR, 1)
SERIAL_EXPR(NAME_EXPR, 2)
SERIAL_EXPR(CONST_EXPR, 3)
SERIAL_EXPR(UNARY_EXPR, 4)
SERIAL_EXPR(BINARY_EXPR, 5)
SERIAL_EXPR(INCR_EXPR, 6)
SERIAL_EXPR(NOT_EXPR, 7)
SERIAL_EXPR(POS_EXPR, 8)
SERIAL_EXPR(NEG_EXPR, 9)
SERIAL_EXPR(ADD_EXPR, 10)
SERIAL_EXPR(SUB_EXPR, 11)
SERIAL_EXPR(TIMES_EXPR, 12)
SERIAL_EXPR(DIVIDE_EXPR, 13)
SERIAL_EXPR(MOD_EXPR, 14)
SERIAL_EXPR(BOOL_EXPR, 15)
SERIAL_EXPR(EQ_EXPR, 16)
SERIAL_EXPR(REL_EXPR, 17)
SERIAL_EXPR(COND_EXPR, 18)
SERIAL_EXPR(REF_EXPR, 19)
SERIAL_EXPR(ASSIGN_EXPR, 20)
SERIAL_EXPR(INDEX_EXPR, 21)
SERIAL_EXPR(FIELD_EXPR, 22)
SERIAL_EXPR(HAS_FIELD_EXPR, 23)
SERIAL_EXPR(RECORD_CONSTRUCTOR_EXPR, 24)
SERIAL_EXPR(FIELD_ASSIGN_EXPR, 25)
// There used to be a SERIAL_EXPR(RECORD_MATCH_EXPR, 26) here
SERIAL_EXPR(ARITH_COERCE_EXPR, 27)
SERIAL_EXPR(RECORD_COERCE_EXPR, 28)
SERIAL_EXPR(FLATTEN_EXPR, 29)
SERIAL_EXPR(SCHEDULE_EXPR, 30)
SERIAL_EXPR(IN_EXPR, 31)
SERIAL_EXPR(CALL_EXPR, 32)
SERIAL_EXPR(EVENT_EXPR, 33)
SERIAL_EXPR(LIST_EXPR, 34)
SERIAL_EXPR(RECORD_ASSIGN_EXPR, 35)
SERIAL_EXPR(ADD_TO_EXPR, 36)
SERIAL_EXPR(REMOVE_FROM_EXPR, 37)
SERIAL_EXPR(SIZE_EXPR, 38)
SERIAL_EXPR(CLONE_EXPR, 39)
SERIAL_EXPR(TABLE_CONSTRUCTOR_EXPR, 40)
SERIAL_EXPR(SET_CONSTRUCTOR_EXPR, 41)
SERIAL_EXPR(VECTOR_CONSTRUCTOR_EXPR, 42)
SERIAL_EXPR(TABLE_COERCE_EXPR, 43)
SERIAL_EXPR(VECTOR_COERCE_EXPR, 44)
SERIAL_EXPR(CAST_EXPR, 45)
SERIAL_EXPR(IS_EXPR_, 46) // Name conflict with internal SER_IS_EXPR constant.
SERIAL_EXPR(BIT_EXPR, 47)
SERIAL_EXPR(COMPLEMENT_EXPR, 48)
#define SERIAL_STMT(name, val) SERIAL_CONST(name, val, STMT)
SERIAL_STMT(STMT, 1)
SERIAL_STMT(EXPR_LIST_STMT, 2)
// There used to be ALARM_STMT (3) here.
SERIAL_STMT(PRINT_STMT, 4)
SERIAL_STMT(EXPR_STMT, 5)
SERIAL_STMT(IF_STMT, 6)
SERIAL_STMT(SWITCH_STMT, 7)
SERIAL_STMT(ADD_STMT, 8)
SERIAL_STMT(DEL_STMT, 9)
SERIAL_STMT(EVENT_STMT, 10)
SERIAL_STMT(FOR_STMT, 11)
SERIAL_STMT(NEXT_STMT, 12)
SERIAL_STMT(BREAK_STMT, 13)
SERIAL_STMT(RETURN_STMT, 14)
SERIAL_STMT(STMT_LIST, 15)
SERIAL_STMT(EVENT_BODY_LIST, 16)
SERIAL_STMT(INIT_STMT, 17)
SERIAL_STMT(NULL_STMT, 18)
SERIAL_STMT(WHEN_STMT, 19)
SERIAL_STMT(FALLTHROUGH_STMT, 20)
SERIAL_STMT(WHILE_STMT, 21)
#define SERIAL_TYPE(name, val) SERIAL_CONST(name, val, BRO_TYPE)
SERIAL_TYPE(BRO_TYPE, 1)
SERIAL_TYPE(TYPE_LIST, 2)
SERIAL_TYPE(INDEX_TYPE, 3)
SERIAL_TYPE(TABLE_TYPE, 4)
SERIAL_TYPE(SET_TYPE, 5)
SERIAL_TYPE(FUNC_TYPE, 6)
SERIAL_TYPE(RECORD_TYPE, 7)
SERIAL_TYPE(SUBNET_TYPE, 8)
SERIAL_TYPE(FILE_TYPE, 9)
SERIAL_TYPE(ENUM_TYPE, 10)
SERIAL_TYPE(VECTOR_TYPE, 11)
SERIAL_TYPE(OPAQUE_TYPE, 12)
SERIAL_CONST2(ATTRIBUTES)
SERIAL_CONST2(EVENT_HANDLER)
SERIAL_CONST2(BRO_FILE)
#define SERIAL_FUNC(name, val) SERIAL_CONST(name, val, FUNC)
SERIAL_FUNC(FUNC, 1)
SERIAL_FUNC(BRO_FUNC, 2)
SERIAL_FUNC(DEBUG_FUNC, 3)
SERIAL_FUNC(BUILTIN_FUNC, 4)
#define SERIAL_BLOOMFILTER(name, val) SERIAL_CONST(name, val, BLOOMFILTER)
SERIAL_BLOOMFILTER(BLOOMFILTER, 1)
SERIAL_BLOOMFILTER(BASICBLOOMFILTER, 2)
SERIAL_BLOOMFILTER(COUNTINGBLOOMFILTER, 3)
#define SERIAL_HASHER(name, val) SERIAL_CONST(name, val, HASHER)
SERIAL_HASHER(HASHER, 1)
SERIAL_HASHER(DEFAULTHASHER, 2)
SERIAL_HASHER(DOUBLEHASHER, 3)
SERIAL_CONST2(ID)
SERIAL_CONST2(STATE_ACCESS)
SERIAL_CONST2(CASE)
SERIAL_CONST2(LOCATION)
SERIAL_CONST2(RE_MATCHER)
SERIAL_CONST2(BITVECTOR)
SERIAL_CONST2(COUNTERVECTOR)
#endif

2
src/SerializationFormat.cc
View file

@ -2,7 +2,7 @@
#include "net_util.h"
#include "SerializationFormat.h"
#include "Serializer.h"
#include "DebugLogger.h"
#include "Reporter.h"
const float SerializationFormat::GROWTH_FACTOR = 2.5;

1059
src/Serializer.cc
View file

File diff suppressed because it is too large Load diff

363
src/Serializer.h
View file

@ -1,363 +0,0 @@
#ifndef SERIALIZER_H
#define SERIALIZER_H
#include <map>
#include <list>
#include <pcap.h>
#include "ID.h"
#include "List.h"
#include "Expr.h"
#include "ChunkedIO.h"
#include "SerializationFormat.h"
#include "StateAccess.h"
#include "PriorityQueue.h"
#include "SerialInfo.h"
#include "IP.h"
#include "Timer.h"
#include "iosource/IOSource.h"
#include "Reporter.h"
class SerializationCache;
class SerialInfo;
class Connection;
class Timer;
class Packet;
class Serializer {
public:
// Currently ID serialization is the only method which may suspend.
bool Serialize(SerialInfo* info, const ID& id);
bool Serialize(SerialInfo* info, const char* func, val_list* args);
bool Serialize(SerialInfo* info, const StateAccess& s);
bool Serialize(SerialInfo* info, const Connection& c);
bool Serialize(SerialInfo* info, const Timer& t);
bool Serialize(SerialInfo* info, const Packet& p);
// Access to the current cache.
SerializationCache* Cache() { return current_cache; }
void SetCache(SerializationCache* cache)
{ current_cache = cache; }
// Input/output methods.
#define DECLARE_READ(type) \
bool Read(type* v, const char* tag) { return format->Read(v, tag); }
#define DECLARE_WRITE(type) \
bool Write(type v, const char* tag) \
{ return format->Write(v, tag); }
#define DECLARE_IO(type) \
DECLARE_READ(type) \
DECLARE_WRITE(type)
DECLARE_IO(int)
DECLARE_IO(uint16)
DECLARE_IO(uint32)
DECLARE_IO(int64)
DECLARE_IO(uint64)
DECLARE_IO(char)
DECLARE_IO(bool)
DECLARE_IO(double)
bool Read(char** str, int* len, const char* tag)
{ return format->Read(str, len, tag); }
bool Read(const char** str, int* len, const char* tag)
// This cast is ok.
{ return format->Read(const_cast<char**>(str), len, tag); }
bool Read(string* s, const char* tag);
bool Read(IPAddr* a, const char* tag) { return format->Read(a, tag); }
bool Read(IPPrefix* p, const char* tag) { return format->Read(p, tag); }
bool Write(const char* s, const char* tag)
{ return format->Write(s, tag); }
bool Write(const char* buf, int len, const char* tag)
{ return format->Write(buf, len, tag); }
bool Write(const string& s, const char* tag)
{ return format->Write(s.data(), s.size(), tag); }
bool Write(const IPAddr& a, const char* tag) { return format->Write(a, tag); }
bool Write(const IPPrefix& p, const char* tag) { return format->Write(p, tag); }
bool WriteOpenTag(const char* tag)
{ return format->WriteOpenTag(tag); }
bool WriteCloseTag(const char* tag)
{ return format->WriteCloseTag(tag); }
bool WriteSeparator() { return format->WriteSeparator(); }
void Error(const char* msg);
void Warning(const char* msg);
void SetErrorDescr(const char* descr)
{ delete [] error_descr; error_descr = copy_string(descr); }
protected:
// Format defaults to binary serialization.
explicit Serializer(SerializationFormat* format = 0);
virtual ~Serializer();
// Reads next object.
// If 'block' is true, wait until an object can be read.
// Returns 0 if no more object available, -1 on error.
int Unserialize(UnserialInfo* info, bool block = false);
// Callback for error messages.
virtual void ReportError(const char* msg) = 0;
// Callbacks for unserialized objects.
// id points to ID in global scope, val is unserialized value.
virtual void GotID(ID* id, Val* val) = 0;
virtual void GotEvent(const char* name, double time,
EventHandlerPtr event, val_list* args) = 0;
virtual void GotFunctionCall(const char* name, double time,
Func* func, val_list* args) = 0;
virtual void GotStateAccess(StateAccess* s) = 0;
virtual void GotTimer(Timer* t) = 0;
virtual void GotConnection(Connection* c) = 0;
virtual void GotPacket(Packet* packet) = 0;
// Magic to recognize state files.
static const uint32 MAGIC = 0x42525354;
// This will be increased whenever there is an incompatible change
// in the data format.
static const uint32 DATA_FORMAT_VERSION = 26;
ChunkedIO* io;
private:
bool StartSerialization(SerialInfo* info, const char* descr, char tag);
bool EndSerialization(SerialInfo* info);
bool UnserializeID(UnserialInfo* info);
bool UnserializeCall(UnserialInfo* info);
bool UnserializeStateAccess(UnserialInfo* info);
bool UnserializeTimer(UnserialInfo* info);
bool UnserializeConnection(UnserialInfo* info);
bool UnserializePacket(UnserialInfo* info);
SerializationFormat* format;
SerializationCache* current_cache;
const char* error_descr; // used in error messages
};
// We maintain an LRU-cache for some of the objects which have already been
// serialized. For the cache, we need two types of IDs: TransientIDs (defined
// in SerialObj.cc) uniquely reference an object during the lifetime of a
// process. PermanentIDs uniquely reference an object within a serialization.
class SerializationCache {
public:
typedef uint64 PermanentID;
static const PermanentID NONE = 0;
// If max_cache_size is greater than zero, we'll remove old entries
// automatically if limit is reached (LRU expiration).
explicit SerializationCache(unsigned int max_cache_size = 0);
~SerializationCache();
PermanentID Register(const SerialObj* obj, PermanentID pid,
bool new_cache_strategy);
const SerialObj* Lookup(PermanentID pid)
{
PIDMap::const_iterator i = pid_map.find(pid);
if ( i == pid_map.end() )
return 0;
assert(i->second);
MoveEntryToTail(i->second);
return i->second->obj.serial;
}
PermanentID Lookup(const TransientID& tid)
{
TIDMap::const_iterator i = tid_map.find(tid.Value());
if ( i == tid_map.end() )
return 0;
uint64 modified = i->second->obj.serial->LastModified();
if ( modified == SerialObj::ALWAYS || modified > i->second->time )
return 0;
assert(i->second);
MoveEntryToTail(i->second);
return i->second->pid;
}
unsigned int GetMaxCacheSize() const { return max_cache_size; }
void SetMaxCacheSize(unsigned int size) { max_cache_size = size; }
// These methods have to be called at the start/end of the
// serialization of an entity. The cache guarentees that objects
// registered after Begin() remain valid until End() is called.
// After End(), objects which are not derived from BroObj are
// discarded; others *may* remain valid.
void Begin(bool can_keep_in_cache) { End(can_keep_in_cache); }
void End(bool can_keep_in_cache);
void Clear();
private:
struct CacheList;
struct CacheEntry {
union {
const SerialObj* serial;
const BroObj* bro;
} obj;
bool is_bro_obj;
PermanentID pid;
TransientID::ID tid;
uint64 time;
struct CacheList* cache;
CacheEntry* prev;
CacheEntry* next;
SerialType stype; // primarily for debugging
};
// We maintain two LRU-sorted lists, one for often-changing objects and
// one for only rarely changing objects;
struct CacheList {
CacheEntry* head;
CacheEntry* tail;
unsigned int size;
};
void RemoveEntry(CacheEntry* e);
void UnlinkEntry(CacheEntry* e);
void MoveEntryToTail(CacheEntry* e);
unsigned int max_cache_size;
typedef map<PermanentID, CacheEntry*> PIDMap;
typedef map<TransientID::ID, CacheEntry*> TIDMap;
TIDMap tid_map;
PIDMap pid_map;
CacheList cache_stable;
CacheList cache_unstable;
// Objects in the cache which aren't derived from BroObj. These are
// always stored in the unstable cache.
typedef list<CacheEntry*> VolatileList;
VolatileList volatiles;
PermanentID next_id;
};
// A serializer for cloning objects. Objects can be serialized into
// the serializer and unserialized into new objects. An absolutely
// minimal implementation of Serializer!
class CloneSerializer : public Serializer {
public:
explicit CloneSerializer(SerializationFormat* format = 0) : Serializer(format) { }
~CloneSerializer() override
{ }
protected:
void ReportError(const char* msg) override { reporter->Error("%s", msg); }
void GotID(ID* id, Val* val) override { }
void GotEvent(const char* name, double time, EventHandlerPtr event, val_list* args) override { }
void GotFunctionCall(const char* name, double time,
Func* func, val_list* args) override { }
void GotStateAccess(StateAccess* s) override { delete s; }
void GotTimer(Timer* t) override { }
void GotConnection(Connection* c) override { }
void GotPacket(Packet* packet) override { }
};
// Write values/events to file or fd.
class FileSerializer : public Serializer {
public:
explicit FileSerializer(SerializationFormat* format = 0);
~FileSerializer() override;
// Opens the file for serialization.
bool Open(const char* file, bool pure = false);
bool Close();
// Reads the file.
bool Read(UnserialInfo* info, const char* file, bool header = true);
protected:
void ReportError(const char* msg) override;
void GotID(ID* id, Val* val) override;
void GotEvent(const char* name, double time,
EventHandlerPtr event, val_list* args) override;
void GotFunctionCall(const char* name, double time,
Func* func, val_list* args) override;
void GotStateAccess(StateAccess* s) override;
void GotTimer(Timer* t) override;
void GotConnection(Connection* c) override;
void GotPacket(Packet* packet) override;
bool OpenFile(const char* file, bool readonly, bool should_exist = false);
void CloseFile();
bool ReadFile(const char* file);
bool PrepareForWriting();
bool ReadHeader(UnserialInfo* info = 0);
SerializationCache cache;
const char* file;
int fd;
};
// Abstract interface class for external sources providing a stream of events.
class EventSource {
public:
virtual ~EventSource() { }
// Returns time of the oldest event (0 if none available).
virtual double NextTimestamp(double* local_network_time) = 0;
// Dispatches the oldest event and removes it.
virtual void DispatchNextEvent() = 0;
// Returns true if there are more events to expect from this source.
virtual bool IsActive() = 0;
};
// Plays a file of events back.
class EventPlayer : public FileSerializer, public iosource::IOSource {
public:
explicit EventPlayer(const char* file);
~EventPlayer() override;
void GetFds(iosource::FD_Set* read, iosource::FD_Set* write,
iosource::FD_Set* except) override;
double NextTimestamp(double* local_network_time) override;
void Process() override;
const char* Tag() override { return "EventPlayer"; }
protected:
void GotID(ID* id, Val* val) override {}
void GotEvent(const char* name, double time,
EventHandlerPtr event, val_list* args) override;
void GotFunctionCall(const char* name, double time,
Func* func, val_list* args) override;
double stream_time; // time of first captured event
double replay_time; // network time of replay start
// Next event waiting to be dispatched.
double ne_time;
EventHandlerPtr ne_handler;
val_list ne_args;
};
extern FileSerializer* event_serializer;
extern FileSerializer* state_serializer;
#endif

785
src/StateAccess.cc
View file

@ -1,785 +0,0 @@
#include "Val.h"
#include "StateAccess.h"
#include "Serializer.h"
#include "Event.h"
#include "NetVar.h"
#include "DebugLogger.h"
int StateAccess::replaying = 0;
StateAccess::StateAccess(Opcode arg_opcode,
const MutableVal* arg_target, const Val* arg_op1,
const Val* arg_op2, const Val* arg_op3)
{
opcode = arg_opcode;
target.val = const_cast<MutableVal*>(arg_target);
target_type = TYPE_MVAL;
op1.val = const_cast<Val*>(arg_op1);
op1_type = TYPE_VAL;
op2 = const_cast<Val*>(arg_op2);
op3 = const_cast<Val*>(arg_op3);
delete_op1_key = false;
RefThem();
}
StateAccess::StateAccess(Opcode arg_opcode,
const ID* arg_target, const Val* arg_op1,
const Val* arg_op2, const Val* arg_op3)
{
opcode = arg_opcode;
target.id = const_cast<ID*>(arg_target);
target_type = TYPE_ID;
op1.val = const_cast<Val*>(arg_op1);
op1_type = TYPE_VAL;
op2 = const_cast<Val*>(arg_op2);
op3 = const_cast<Val*>(arg_op3);
delete_op1_key = false;
RefThem();
}
StateAccess::StateAccess(Opcode arg_opcode,
const ID* arg_target, const HashKey* arg_op1,
const Val* arg_op2, const Val* arg_op3)
{
opcode = arg_opcode;
target.id = const_cast<ID*>(arg_target);
target_type = TYPE_ID;
op1.key = new HashKey(arg_op1->Key(), arg_op1->Size(), arg_op1->Hash());
op1_type = TYPE_KEY;
op2 = const_cast<Val*>(arg_op2);
op3 = const_cast<Val*>(arg_op3);
delete_op1_key = true;
RefThem();
}
StateAccess::StateAccess(Opcode arg_opcode,
const MutableVal* arg_target, const HashKey* arg_op1,
const Val* arg_op2, const Val* arg_op3)
{
opcode = arg_opcode;
target.val = const_cast<MutableVal*>(arg_target);
target_type = TYPE_MVAL;
op1.key = new HashKey(arg_op1->Key(), arg_op1->Size(), arg_op1->Hash());
op1_type = TYPE_KEY;
op2 = const_cast<Val*>(arg_op2);
op3 = const_cast<Val*>(arg_op3);
delete_op1_key = true;
RefThem();
}
StateAccess::StateAccess(const StateAccess& sa)
: SerialObj()
{
opcode = sa.opcode;
target_type = sa.target_type;
op1_type = sa.op1_type;
delete_op1_key = false;
if ( target_type == TYPE_ID )
target.id = sa.target.id;
else
target.val = sa.target.val;
if ( op1_type == TYPE_VAL )
op1.val = sa.op1.val;
else
{
// We need to copy the key as the pointer may not be
// valid anymore later.
op1.key = new HashKey(sa.op1.key->Key(), sa.op1.key->Size(),
sa.op1.key->Hash());
delete_op1_key = true;
}
op2 = sa.op2;
op3 = sa.op3;
RefThem();
}
StateAccess::~StateAccess()
{
if ( target_type == TYPE_ID )
Unref(target.id);
else
Unref(target.val);
if ( op1_type == TYPE_VAL )
Unref(op1.val);
else if ( delete_op1_key )
delete op1.key;
Unref(op2);
Unref(op3);
}
void StateAccess::RefThem()
{
if ( target_type == TYPE_ID )
Ref(target.id);
else
Ref(target.val);
if ( op1_type == TYPE_VAL && op1.val )
Ref(op1.val);
if ( op2 )
Ref(op2);
if ( op3 )
Ref(op3);
}
static Val* GetInteger(bro_int_t n, TypeTag t)
{
if ( t == TYPE_INT )
return val_mgr->GetInt(n);
return val_mgr->GetCount(n);
}
void StateAccess::Replay()
{
// For simplicity we assume that we only replay unserialized accesses.
assert(target_type == TYPE_ID && op1_type == TYPE_VAL);
if ( ! target.id )
return;
Val* v = target.id->ID_Val();
TypeTag t = v ? v->Type()->Tag() : TYPE_VOID;
if ( opcode != OP_ASSIGN && ! v )
{
// FIXME: I think this warrants an internal error,
// but let's check that first ...
// reporter->InternalError("replay id lacking a value");
reporter->Error("replay id lacks a value");
return;
}
++replaying;
switch ( opcode ) {
case OP_ASSIGN:
assert(op1.val);
// There mustn't be a direct assignment to a unique ID.
assert(target.id->Name()[0] != '#');
target.id->SetVal(op1.val->Ref());
break;
case OP_INCR:
if ( IsIntegral(t) )
{
assert(op1.val && op2);
// We derive the amount as difference between old
// and new value.
bro_int_t amount =
op1.val->CoerceToInt() - op2->CoerceToInt();
target.id->SetVal(GetInteger(v->CoerceToInt() + amount, t),
OP_INCR);
}
break;
case OP_ASSIGN_IDX:
assert(op1.val);
if ( t == TYPE_TABLE )
{
assert(op2);
v->AsTableVal()->Assign(op1.val, op2 ? op2->Ref() : 0);
}
else if ( t == TYPE_RECORD )
{
const char* field = op1.val->AsString()->CheckString();
int idx = v->Type()->AsRecordType()->FieldOffset(field);
if ( idx >= 0 )
v->AsRecordVal()->Assign(idx, op2 ? op2->Ref() : 0);
else
reporter->Error("access replay: unknown record field %s for assign", field);
}
else if ( t == TYPE_VECTOR )
{
assert(op2);
bro_uint_t index = op1.val->AsCount();
v->AsVectorVal()->Assign(index, op2 ? op2->Ref() : 0);
}
else
reporter->InternalError("unknown type in replaying index assign");
break;
case OP_INCR_IDX:
{
assert(op1.val && op2 && op3);
// We derive the amount as the difference between old
// and new value.
bro_int_t amount = op2->CoerceToInt() - op3->CoerceToInt();
if ( t == TYPE_TABLE )
{
t = v->Type()->AsTableType()->YieldType()->Tag();
Val* lookup_op1 = v->AsTableVal()->Lookup(op1.val);
int delta = lookup_op1->CoerceToInt() + amount;
Val* new_val = GetInteger(delta, t);
v->AsTableVal()->Assign(op1.val, new_val, OP_INCR );
}
else if ( t == TYPE_RECORD )
{
const char* field = op1.val->AsString()->CheckString();
int idx = v->Type()->AsRecordType()->FieldOffset(field);
if ( idx >= 0 )
{
t = v->Type()->AsRecordType()->FieldType(idx)->Tag();
Val* lookup_field =
v->AsRecordVal()->Lookup(idx);
bro_int_t delta =
lookup_field->CoerceToInt() + amount;
Val* new_val = GetInteger(delta, t);
v->AsRecordVal()->Assign(idx, new_val, OP_INCR);
}
else
reporter->Error("access replay: unknown record field %s for assign", field);
}
else if ( t == TYPE_VECTOR )
{
bro_uint_t index = op1.val->AsCount();
t = v->Type()->AsVectorType()->YieldType()->Tag();
Val* lookup_op1 = v->AsVectorVal()->Lookup(index);
int delta = lookup_op1->CoerceToInt() + amount;
Val* new_val = GetInteger(delta, t);
v->AsVectorVal()->Assign(index, new_val);
}
else
reporter->InternalError("unknown type in replaying index increment");
break;
}
case OP_ADD:
assert(op1.val);
if ( t == TYPE_TABLE )
{
v->AsTableVal()->Assign(op1.val, 0);
}
break;
case OP_DEL:
assert(op1.val);
if ( t == TYPE_TABLE )
{
Unref(v->AsTableVal()->Delete(op1.val));
}
break;
case OP_EXPIRE:
assert(op1.val);
if ( t == TYPE_TABLE )
{
// No old check for expire. It may have already
// been deleted by ourselves. Furthermore, we
// ignore the expire_func's return value.
TableVal* tv = v->AsTableVal();
if ( tv->Lookup(op1.val, false) )
{
// We want to propagate state updates which
// are performed in the expire_func.
StateAccess::ResumeReplay();
tv->CallExpireFunc(op1.val->Ref());
StateAccess::SuspendReplay();
Unref(tv->AsTableVal()->Delete(op1.val));
}
}
break;
case OP_PRINT:
assert(op1.val);
reporter->InternalError("access replay for print not implemented");
break;
case OP_READ_IDX:
if ( t == TYPE_TABLE )
{
assert(op1.val);
TableVal* tv = v->AsTableVal();
// Update the timestamp if we have a read_expire.
if ( tv->FindAttr(ATTR_EXPIRE_READ) )
{
tv->UpdateTimestamp(op1.val);
}
}
else
reporter->Error("read for non-table");
break;
default:
reporter->InternalError("access replay: unknown opcode for StateAccess");
break;
}
--replaying;
}
ID* StateAccess::Target() const
{
return target_type == TYPE_ID ? target.id : target.val->UniqueID();
}
bool StateAccess::Serialize(SerialInfo* info) const
{
return SerialObj::Serialize(info);
}
StateAccess* StateAccess::Unserialize(UnserialInfo* info)
{
StateAccess* sa =
(StateAccess*) SerialObj::Unserialize(info, SER_STATE_ACCESS);
return sa;
}
IMPLEMENT_SERIAL(StateAccess, SER_STATE_ACCESS);
bool StateAccess::DoSerialize(SerialInfo* info) const
{
DO_SERIALIZE(SER_STATE_ACCESS, SerialObj);
if ( ! SERIALIZE(char(opcode)) )
return false;
const ID* id =
target_type == TYPE_ID ? target.id : target.val->UniqueID();
if ( ! SERIALIZE(id->Name()) )
return false;
if ( op1_type == TYPE_KEY )
{
Val* index =
id->ID_Val()->AsTableVal()->RecoverIndex(this->op1.key);
if ( ! index )
return false;
if ( ! index->Serialize(info) )
return false;
Unref(index);
}
else if ( ! op1.val->Serialize(info) )
return false;
// Don't send the "old" operand if we don't want consistency checks.
// Unfortunately, it depends on the opcode which operand that actually
// is.
const Val* null = 0;
switch ( opcode ) {
case OP_PRINT:
case OP_EXPIRE:
case OP_READ_IDX:
// No old.
SERIALIZE_OPTIONAL(null);
SERIALIZE_OPTIONAL(null);
break;
case OP_INCR:
case OP_INCR_IDX:
// Always need old.
SERIALIZE_OPTIONAL(op2);
SERIALIZE_OPTIONAL(op3);
break;
case OP_ASSIGN:
case OP_ADD:
case OP_DEL:
// Op2 is old.
SERIALIZE_OPTIONAL(null);
SERIALIZE_OPTIONAL(null);
break;
case OP_ASSIGN_IDX:
// Op3 is old.
SERIALIZE_OPTIONAL(op2);
SERIALIZE_OPTIONAL(null);
break;
default:
reporter->InternalError("StateAccess::DoSerialize: unknown opcode");
}
return true;
}
bool StateAccess::DoUnserialize(UnserialInfo* info)
{
DO_UNSERIALIZE(SerialObj);
char c;
if ( ! UNSERIALIZE(&c) )
return false;
opcode = Opcode(c);
const char* name;
if ( ! UNSERIALIZE_STR(&name, 0) )
return false;
target_type = TYPE_ID;
target.id = global_scope()->Lookup(name);
if ( target.id )
// Otherwise, we'll delete it below.
delete [] name;
op1_type = TYPE_VAL;
op1.val = Val::Unserialize(info);
if ( ! op1.val )
return false;
UNSERIALIZE_OPTIONAL(op2, Val::Unserialize(info));
UNSERIALIZE_OPTIONAL(op3, Val::Unserialize(info));
if ( target.id )
Ref(target.id);
else
{
// This may happen as long as we haven't agreed on the
// unique name for an ID during initial synchronization, or if
// the local peer has already deleted the ID.
DBG_LOG(DBG_STATE, "state access referenced unknown id %s", name);
if ( info->install_uniques )
{
target.id = new ID(name, SCOPE_GLOBAL, true);
Ref(target.id);
global_scope()->Insert(name, target.id);
#ifdef USE_PERFTOOLS_DEBUG
heap_checker->IgnoreObject(target.id);
#endif
}
delete [] name;
}
return true;
}
void StateAccess::Describe(ODesc* d) const
{
const ID* id;
const char* id_str = "";
const char* unique_str = "";
d->SetShort();
if ( target_type == TYPE_ID )
{
id = target.id;
if ( ! id )
{
d->Add("(unknown id)");
return;
}
id_str = id->Name();
if ( id->ID_Val() && id->ID_Val()->IsMutableVal() &&
id->Name()[0] != '#' )
unique_str = fmt(" [id] (%s)", id->ID_Val()->AsMutableVal()->UniqueID()->Name());
}
else
{
id = target.val->UniqueID();
#ifdef DEBUG
if ( target.val->GetID() )
{
id_str = target.val->GetID()->Name();
unique_str = fmt(" [val] (%s)", id->Name());
}
else
#endif
id_str = id->Name();
}
const Val* op1 = op1_type == TYPE_VAL ?
this->op1.val :
id->ID_Val()->AsTableVal()->RecoverIndex(this->op1.key);
switch ( opcode ) {
case OP_ASSIGN:
assert(op1);
d->Add(id_str);
d->Add(" = ");
op1->Describe(d);
if ( op2 )
{
d->Add(" (");
op2->Describe(d);
d->Add(")");
}
d->Add(unique_str);
break;
case OP_INCR:
assert(op1 && op2);
d->Add(id_str);
d->Add(" += ");
d->Add(op1->CoerceToInt() - op2->CoerceToInt());
d->Add(unique_str);
break;
case OP_ASSIGN_IDX:
assert(op1);
d->Add(id_str);
d->Add("[");
op1->Describe(d);
d->Add("]");
d->Add(" = ");
if ( op2 )
op2->Describe(d);
else
d->Add("(null)");
if ( op3 )
{
d->Add(" (");
op3->Describe(d);
d->Add(")");
}
d->Add(unique_str);
break;
case OP_INCR_IDX:
assert(op1 && op2 && op3);
d->Add(id_str);
d->Add("[");
op1->Describe(d);
d->Add("]");
d->Add(" += ");
d->Add(op2->CoerceToInt() - op3->CoerceToInt());
d->Add(unique_str);
break;
case OP_ADD:
assert(op1);
d->Add("add ");
d->Add(id_str);
d->Add("[");
op1->Describe(d);
d->Add("]");
if ( op2 )
{
d->Add(" (");
op2->Describe(d);
d->Add(")");
}
d->Add(unique_str);
break;
case OP_DEL:
assert(op1);
d->Add("del ");
d->Add(id_str);
d->Add("[");
op1->Describe(d);
d->Add("]");
if ( op2 )
{
d->Add(" (");
op2->Describe(d);
d->Add(")");
}
d->Add(unique_str);
break;
case OP_EXPIRE:
assert(op1);
d->Add("expire ");
d->Add(id_str);
d->Add("[");
op1->Describe(d);
d->Add("]");
if ( op2 )
{
d->Add(" (");
op2->Describe(d);
d->Add(")");
}
d->Add(unique_str);
break;
case OP_PRINT:
assert(op1);
d->Add("print ");
d->Add(id_str);
op1->Describe(d);
d->Add(unique_str);
break;
case OP_READ_IDX:
assert(op1);
d->Add("read ");
d->Add(id_str);
d->Add("[");
op1->Describe(d);
d->Add("]");
break;
default:
reporter->InternalError("unknown opcode for StateAccess");
break;
}
if ( op1_type != TYPE_VAL )
Unref(const_cast<Val*>(op1));
}
void StateAccess::Log(StateAccess* access)
{
bool tracked = false;
if ( access->target_type == TYPE_ID )
{
if ( access->target.id->FindAttr(ATTR_TRACKED) )
tracked = true;
}
else
{
if ( access->target.val->GetProperties() & MutableVal::TRACKED )
tracked = true;
}
if ( tracked )
notifiers.AccessPerformed(*access);
#ifdef DEBUG
ODesc desc;
access->Describe(&desc);
DBG_LOG(DBG_STATE, "operation: %s%s",
desc.Description(), replaying > 0 ? " (replay)" : "");
#endif
delete access;
}
NotifierRegistry notifiers;
void NotifierRegistry::Register(ID* id, NotifierRegistry::Notifier* notifier)
{
DBG_LOG(DBG_NOTIFIERS, "registering ID %s for notifier %s",
id->Name(), notifier->Name());
Attr* attr = new Attr(ATTR_TRACKED);
if ( id->Attrs() )
{
if ( ! id->Attrs()->FindAttr(ATTR_TRACKED) )
id->Attrs()->AddAttr(attr);
}
else
{
attr_list* a = new attr_list{attr};
id->SetAttrs(new Attributes(a, id->Type(), false));
}
Unref(attr);
NotifierMap::iterator i = ids.find(id->Name());
if ( i != ids.end() )
i->second->insert(notifier);
else
{
NotifierSet* s = new NotifierSet;
s->insert(notifier);
ids.insert(NotifierMap::value_type(id->Name(), s));
}
Ref(id);
}
void NotifierRegistry::Register(Val* val, NotifierRegistry::Notifier* notifier)
{
if ( val->IsMutableVal() )
Register(val->AsMutableVal()->UniqueID(), notifier);
}
void NotifierRegistry::Unregister(ID* id, NotifierRegistry::Notifier* notifier)
{
DBG_LOG(DBG_NOTIFIERS, "unregistering ID %s for notifier %s",
id->Name(), notifier->Name());
NotifierMap::iterator i = ids.find(id->Name());
if ( i == ids.end() )
return;
Attr* attr = id->Attrs()->FindAttr(ATTR_TRACKED);
id->Attrs()->RemoveAttr(ATTR_TRACKED);
Unref(attr);
NotifierSet* s = i->second;
s->erase(notifier);
if ( s->size() == 0 )
{
delete s;
ids.erase(i);
}
Unref(id);
}
void NotifierRegistry::Unregister(Val* val, NotifierRegistry::Notifier* notifier)
{
if ( val->IsMutableVal() )
Unregister(val->AsMutableVal()->UniqueID(), notifier);
}
void NotifierRegistry::AccessPerformed(const StateAccess& sa)
{
ID* id = sa.Target();
NotifierMap::iterator i = ids.find(id->Name());
if ( i == ids.end() )
return;
DBG_LOG(DBG_NOTIFIERS, "modification to tracked ID %s", id->Name());
NotifierSet* s = i->second;
if ( id->IsInternalGlobal() )
for ( NotifierSet::iterator j = s->begin(); j != s->end(); j++ )
(*j)->Access(id->ID_Val(), sa);
else
for ( NotifierSet::iterator j = s->begin(); j != s->end(); j++ )
(*j)->Access(id, sa);
}
const char* NotifierRegistry::Notifier::Name() const
{
return fmt("%p", this);
}

146
src/StateAccess.h
View file

@ -1,146 +0,0 @@
// A class describing a state-modyfing access to a Value or an ID.
#ifndef STATEACESSS_H
#define STATEACESSS_H
#include <set>
#include <map>
#include <string>
#include "SerialObj.h"
class Val;
class ID;
class MutableVal;
class HashKey;
class ODesc;
class Serializer;
class TableVal;
enum Opcode { // Op1 Op2 Op3 (Vals)
OP_NONE,
OP_ASSIGN, // new old
OP_ASSIGN_IDX, // idx new old
OP_ADD, // idx old
OP_INCR, // idx new old
OP_INCR_IDX, // idx new old
OP_DEL, // idx old
OP_PRINT, // args
OP_EXPIRE, // idx
OP_READ_IDX, // idx
};
class StateAccess : public SerialObj {
public:
StateAccess(Opcode opcode, const ID* target, const Val* op1,
const Val* op2 = 0, const Val* op3 = 0);
StateAccess(Opcode opcode, const MutableVal* target, const Val* op1,
const Val* op2 = 0, const Val* op3 = 0);
// For tables, the idx operand may be given as an index HashKey.
// This is for efficiency. While we need to reconstruct the index
// if we are actually going to serialize the access, we can at
// least skip it if we don't.
StateAccess(Opcode opcode, const ID* target, const HashKey* op1,
const Val* op2 = 0, const Val* op3 = 0);
StateAccess(Opcode opcode, const MutableVal* target, const HashKey* op1,
const Val* op2 = 0, const Val* op3 = 0);
StateAccess(const StateAccess& sa);
~StateAccess() override;
// Replays this access in the our environment.
void Replay();
// Returns target ID which may be an internal one for unbound vals.
ID* Target() const;
void Describe(ODesc* d) const;
bool Serialize(SerialInfo* info) const;
static StateAccess* Unserialize(UnserialInfo* info);
// Main entry point when StateAcesses are performed.
// For every state-changing operation, this has to be called.
static void Log(StateAccess* access);
// If we're going to make additional non-replaying accesses during a
// Replay(), we have to call these.
static void SuspendReplay() { --replaying; }
static void ResumeReplay() { ++replaying; }
private:
StateAccess() { target.id = 0; op1.val = op2 = op3 = 0; }
void RefThem();
DECLARE_SERIAL(StateAccess);
Opcode opcode;
union {
ID* id;
MutableVal* val;
} target;
union {
Val* val;
const HashKey* key;
} op1;
Val* op2;
Val* op3;
enum Type { TYPE_ID, TYPE_VAL, TYPE_MVAL, TYPE_KEY };
Type target_type;
Type op1_type;
bool delete_op1_key;
static int replaying;
};
// We provide a notifier framework to inform interested parties of
// modifications to selected global IDs/Vals. To get notified about a change,
// derive a class from Notifier and register the interesting IDs/Vals with
// the NotifierRegistry.
//
// Note: For containers (e.g., tables), notifications are only issued if the
// container itself is modified, *not* for changes to the values contained
// therein.
class NotifierRegistry {
public:
class Notifier {
public:
virtual ~Notifier() { }
// Called when a change is being performed. Note that when these
// methods are called, it is undefined whether the change has
// already been done or is just going to be performed soon.
virtual void Access(ID* id, const StateAccess& sa) = 0;
virtual void Access(Val* val, const StateAccess& sa) = 0;
virtual const char* Name() const; // for debugging
};
NotifierRegistry() { }
~NotifierRegistry() { }
// Inform the given notifier if ID/Val changes.
void Register(ID* id, Notifier* notifier);
void Register(Val* val, Notifier* notifier);
// Cancel notification for this ID/Val.
void Unregister(ID* id, Notifier* notifier);
void Unregister(Val* val, Notifier* notifier);
private:
friend class StateAccess;
void AccessPerformed(const StateAccess& sa);
typedef std::set<Notifier*> NotifierSet;
typedef std::map<std::string, NotifierSet*> NotifierMap;
NotifierMap ids;
};
extern NotifierRegistry notifiers;
#endif

2
src/Stats.cc
View file

@ -255,7 +255,7 @@ void ProfileLogger::Log()
while ( (id = globals->NextEntry(c)) )
// We don't show/count internal globals as they are always
// contained in some other global user-visible container.
if ( id->HasVal() && ! id->IsInternalGlobal() )
if ( id->HasVal() )
{
Val* v = id->ID_Val();

525
src/Stmt.cc
View file

@ -117,47 +117,6 @@ void Stmt::AccessStats(ODesc* d) const
}
}
bool Stmt::Serialize(SerialInfo* info) const
{
return SerialObj::Serialize(info);
}
Stmt* Stmt::Unserialize(UnserialInfo* info, BroStmtTag want)
{
Stmt* stmt = (Stmt*) SerialObj::Unserialize(info, SER_STMT);
if ( want != STMT_ANY && stmt->tag != want )
{
info->s->Error("wrong stmt type");
Unref(stmt);
return 0;
}
return stmt;
}
bool Stmt::DoSerialize(SerialInfo* info) const
{
DO_SERIALIZE(SER_STMT, BroObj);
return SERIALIZE(char(tag)) && SERIALIZE(last_access)
&& SERIALIZE(access_count);
}
bool Stmt::DoUnserialize(UnserialInfo* info)
{
DO_UNSERIALIZE(BroObj);
char c;
if ( ! UNSERIALIZE(&c) )
return 0;
tag = BroStmtTag(c);
return UNSERIALIZE(&last_access) && UNSERIALIZE(&access_count);
}
ExprListStmt::ExprListStmt(BroStmtTag t, ListExpr* arg_l)
: Stmt(t)
{
@ -207,19 +166,6 @@ void ExprListStmt::PrintVals(ODesc* d, val_list* vals, int offset) const
describe_vals(vals, d, offset);
}
bool ExprListStmt::DoSerialize(SerialInfo* info) const
{
DO_SERIALIZE(SER_EXPR_LIST_STMT, Stmt);
return l->Serialize(info);
}
bool ExprListStmt::DoUnserialize(UnserialInfo* info)
{
DO_UNSERIALIZE(Stmt);
l = (ListExpr*) Expr::Unserialize(info, EXPR_LIST);
return l != 0;
}
TraversalCode ExprListStmt::Traverse(TraversalCallback* cb) const
{
TraversalCode tc = cb->PreStmt(this);
@ -305,20 +251,6 @@ Val* PrintStmt::DoExec(val_list* vals, stmt_flow_type& /* flow */) const
return 0;
}
IMPLEMENT_SERIAL(PrintStmt, SER_PRINT_STMT);
bool PrintStmt::DoSerialize(SerialInfo* info) const
{
DO_SERIALIZE(SER_PRINT_STMT, ExprListStmt);
return true;
}
bool PrintStmt::DoUnserialize(UnserialInfo* info)
{
DO_UNSERIALIZE(ExprListStmt);
return true;
}
ExprStmt::ExprStmt(Expr* arg_e) : Stmt(STMT_EXPR)
{
e = arg_e;
@ -404,22 +336,6 @@ TraversalCode ExprStmt::Traverse(TraversalCallback* cb) const
HANDLE_TC_STMT_POST(tc);
}
IMPLEMENT_SERIAL(ExprStmt, SER_EXPR_STMT);
bool ExprStmt::DoSerialize(SerialInfo* info) const
{
DO_SERIALIZE(SER_EXPR_STMT, Stmt);
SERIALIZE_OPTIONAL(e);
return true;
}
bool ExprStmt::DoUnserialize(UnserialInfo* info)
{
DO_UNSERIALIZE(Stmt);
UNSERIALIZE_OPTIONAL(e, Expr::Unserialize(info));
return true;
}
IfStmt::IfStmt(Expr* test, Stmt* arg_s1, Stmt* arg_s2) : ExprStmt(STMT_IF, test)
{
s1 = arg_s1;
@ -507,25 +423,6 @@ TraversalCode IfStmt::Traverse(TraversalCallback* cb) const
HANDLE_TC_STMT_POST(tc);
}
IMPLEMENT_SERIAL(IfStmt, SER_IF_STMT);
bool IfStmt::DoSerialize(SerialInfo* info) const
{
DO_SERIALIZE(SER_IF_STMT, ExprStmt);
return s1->Serialize(info) && s2->Serialize(info);
}
bool IfStmt::DoUnserialize(UnserialInfo* info)
{
DO_UNSERIALIZE(ExprStmt);
s1 = Stmt::Unserialize(info);
if ( ! s1 )
return false;
s2 = Stmt::Unserialize(info);
return s2 != 0;
}
static BroStmtTag get_last_stmt_tag(const Stmt* stmt)
{
if ( ! stmt )
@ -655,67 +552,6 @@ TraversalCode Case::Traverse(TraversalCallback* cb) const
return TC_CONTINUE;
}
bool Case::Serialize(SerialInfo* info) const
{
return SerialObj::Serialize(info);
}
Case* Case::Unserialize(UnserialInfo* info)
{
return (Case*) SerialObj::Unserialize(info, SER_CASE);
}
IMPLEMENT_SERIAL(Case, SER_CASE);
bool Case::DoSerialize(SerialInfo* info) const
{
DO_SERIALIZE(SER_CASE, BroObj);
if ( ! expr_cases->Serialize(info) )
return false;
id_list empty;
id_list* types = (type_cases ? type_cases : &empty);
if ( ! SERIALIZE(types->length()) )
return false;
loop_over_list((*types), i)
{
if ( ! (*types)[i]->Serialize(info) )
return false;
}
return this->s->Serialize(info);
}
bool Case::DoUnserialize(UnserialInfo* info)
{
DO_UNSERIALIZE(BroObj);
expr_cases = (ListExpr*) Expr::Unserialize(info, EXPR_LIST);
if ( ! expr_cases )
return false;
int len;
if ( ! UNSERIALIZE(&len) )
return false;
type_cases = new id_list(len);
while ( len-- )
{
ID* id = ID::Unserialize(info);
if ( ! id )
return false;
type_cases->append(id);
}
this->s = Stmt::Unserialize(info);
return this->s != 0;
}
static void int_del_func(void* v)
{
delete (int*) v;
@ -1028,66 +864,6 @@ TraversalCode SwitchStmt::Traverse(TraversalCallback* cb) const
HANDLE_TC_STMT_POST(tc);
}
IMPLEMENT_SERIAL(SwitchStmt, SER_SWITCH_STMT);
bool SwitchStmt::DoSerialize(SerialInfo* info) const
{
DO_SERIALIZE(SER_SWITCH_STMT, ExprStmt);
if ( ! SERIALIZE(cases->length()) )
return false;
loop_over_list((*cases), i)
if ( ! (*cases)[i]->Serialize(info) )
return false;
if ( ! SERIALIZE(default_case_idx) )
return false;
return true;
}
bool SwitchStmt::DoUnserialize(UnserialInfo* info)
{
DO_UNSERIALIZE(ExprStmt);
Init();
int len;
if ( ! UNSERIALIZE(&len) )
return false;
while ( len-- )
{
Case* c = Case::Unserialize(info);
if ( ! c )
return false;
cases->append(c);
}
if ( ! UNSERIALIZE(&default_case_idx) )
return false;
loop_over_list(*cases, i)
{
const ListExpr* le = (*cases)[i]->ExprCases();
if ( ! le )
continue;
const expr_list& exprs = le->Exprs();
loop_over_list(exprs, j)
{
if ( ! AddCaseLabelValueMapping(exprs[j]->ExprVal(), i) )
return false;
}
}
return true;
}
AddStmt::AddStmt(Expr* arg_e) : ExprStmt(STMT_ADD, arg_e)
{
if ( ! e->CanAdd() )
@ -1121,20 +897,6 @@ TraversalCode AddStmt::Traverse(TraversalCallback* cb) const
HANDLE_TC_STMT_POST(tc);
}
IMPLEMENT_SERIAL(AddStmt, SER_ADD_STMT);
bool AddStmt::DoSerialize(SerialInfo* info) const
{
DO_SERIALIZE(SER_ADD_STMT, ExprStmt);
return true;
}
bool AddStmt::DoUnserialize(UnserialInfo* info)
{
DO_UNSERIALIZE(ExprStmt);
return true;
}
DelStmt::DelStmt(Expr* arg_e) : ExprStmt(STMT_DELETE, arg_e)
{
if ( e->IsError() )
@ -1170,20 +932,6 @@ TraversalCode DelStmt::Traverse(TraversalCallback* cb) const
HANDLE_TC_STMT_POST(tc);
}
IMPLEMENT_SERIAL(DelStmt, SER_DEL_STMT);
bool DelStmt::DoSerialize(SerialInfo* info) const
{
DO_SERIALIZE(SER_DEL_STMT, ExprStmt);
return true;
}
bool DelStmt::DoUnserialize(UnserialInfo* info)
{
DO_UNSERIALIZE(ExprStmt);
return true;
}
EventStmt::EventStmt(EventExpr* arg_e) : ExprStmt(STMT_EVENT, arg_e)
{
event_expr = arg_e;
@ -1218,22 +966,6 @@ TraversalCode EventStmt::Traverse(TraversalCallback* cb) const
HANDLE_TC_STMT_POST(tc);
}
IMPLEMENT_SERIAL(EventStmt, SER_EVENT_STMT);
bool EventStmt::DoSerialize(SerialInfo* info) const
{
DO_SERIALIZE(SER_EVENT_STMT, ExprStmt);
return event_expr->Serialize(info);
}
bool EventStmt::DoUnserialize(UnserialInfo* info)
{
DO_UNSERIALIZE(ExprStmt);
event_expr = (EventExpr*) Expr::Unserialize(info, EXPR_EVENT);
return event_expr != 0;
}
WhileStmt::WhileStmt(Expr* arg_loop_condition, Stmt* arg_body)
: loop_condition(arg_loop_condition), body(arg_body)
{
@ -1319,30 +1051,6 @@ Val* WhileStmt::Exec(Frame* f, stmt_flow_type& flow) const
return rval;
}
IMPLEMENT_SERIAL(WhileStmt, SER_WHILE_STMT);
bool WhileStmt::DoSerialize(SerialInfo* info) const
{
DO_SERIALIZE(SER_WHILE_STMT, Stmt);
if ( ! loop_condition->Serialize(info) )
return false;
return body->Serialize(info);
}
bool WhileStmt::DoUnserialize(UnserialInfo* info)
{
DO_UNSERIALIZE(Stmt);
loop_condition = Expr::Unserialize(info);
if ( ! loop_condition )
return false;
body = Stmt::Unserialize(info);
return body != 0;
}
ForStmt::ForStmt(id_list* arg_loop_vars, Expr* loop_expr)
: ExprStmt(STMT_FOR, loop_expr)
{
@ -1607,47 +1315,6 @@ TraversalCode ForStmt::Traverse(TraversalCallback* cb) const
HANDLE_TC_STMT_POST(tc);
}
IMPLEMENT_SERIAL(ForStmt, SER_FOR_STMT);
bool ForStmt::DoSerialize(SerialInfo* info) const
{
DO_SERIALIZE(SER_FOR_STMT, ExprStmt);
if ( ! SERIALIZE(loop_vars->length()) )
return false;
loop_over_list((*loop_vars), i)
{
if ( ! (*loop_vars)[i]->Serialize(info) )
return false;
}
return body->Serialize(info);
}
bool ForStmt::DoUnserialize(UnserialInfo* info)
{
DO_UNSERIALIZE(ExprStmt);
int len;
if ( ! UNSERIALIZE(&len) )
return false;
loop_vars = new id_list(len);
while ( len-- )
{
ID* id = ID::Unserialize(info);
if ( ! id )
return false;
loop_vars->append(id);
}
body = Stmt::Unserialize(info);
return body != 0;
}
Val* NextStmt::Exec(Frame* /* f */, stmt_flow_type& flow) const
{
RegisterAccess();
@ -1675,20 +1342,6 @@ TraversalCode NextStmt::Traverse(TraversalCallback* cb) const
HANDLE_TC_STMT_POST(tc);
}
IMPLEMENT_SERIAL(NextStmt, SER_NEXT_STMT);
bool NextStmt::DoSerialize(SerialInfo* info) const
{
DO_SERIALIZE(SER_NEXT_STMT, Stmt);
return true;
}
bool NextStmt::DoUnserialize(UnserialInfo* info)
{
DO_UNSERIALIZE(Stmt);
return true;
}
Val* BreakStmt::Exec(Frame* /* f */, stmt_flow_type& flow) const
{
RegisterAccess();
@ -1716,20 +1369,6 @@ TraversalCode BreakStmt::Traverse(TraversalCallback* cb) const
HANDLE_TC_STMT_POST(tc);
}
IMPLEMENT_SERIAL(BreakStmt, SER_BREAK_STMT);
bool BreakStmt::DoSerialize(SerialInfo* info) const
{
DO_SERIALIZE(SER_BREAK_STMT, Stmt);
return true;
}
bool BreakStmt::DoUnserialize(UnserialInfo* info)
{
DO_UNSERIALIZE(Stmt);
return true;
}
Val* FallthroughStmt::Exec(Frame* /* f */, stmt_flow_type& flow) const
{
RegisterAccess();
@ -1757,20 +1396,6 @@ TraversalCode FallthroughStmt::Traverse(TraversalCallback* cb) const
HANDLE_TC_STMT_POST(tc);
}
IMPLEMENT_SERIAL(FallthroughStmt, SER_FALLTHROUGH_STMT);
bool FallthroughStmt::DoSerialize(SerialInfo* info) const
{
DO_SERIALIZE(SER_FALLTHROUGH_STMT, Stmt);
return true;
}
bool FallthroughStmt::DoUnserialize(UnserialInfo* info)
{
DO_UNSERIALIZE(Stmt);
return true;
}
ReturnStmt::ReturnStmt(Expr* arg_e) : ExprStmt(STMT_RETURN, arg_e)
{
Scope* s = current_scope();
@ -1838,20 +1463,6 @@ void ReturnStmt::Describe(ODesc* d) const
DescribeDone(d);
}
IMPLEMENT_SERIAL(ReturnStmt, SER_RETURN_STMT);
bool ReturnStmt::DoSerialize(SerialInfo* info) const
{
DO_SERIALIZE(SER_RETURN_STMT, ExprStmt);
return true;
}
bool ReturnStmt::DoUnserialize(UnserialInfo* info)
{
DO_UNSERIALIZE(ExprStmt);
return true;
}
StmtList::StmtList() : Stmt(STMT_LIST)
{
}
@ -1941,43 +1552,6 @@ TraversalCode StmtList::Traverse(TraversalCallback* cb) const
HANDLE_TC_STMT_POST(tc);
}
IMPLEMENT_SERIAL(StmtList, SER_STMT_LIST);
bool StmtList::DoSerialize(SerialInfo* info) const
{
DO_SERIALIZE(SER_STMT_LIST, Stmt);
if ( ! SERIALIZE(stmts.length()) )
return false;
loop_over_list(stmts, i)
if ( ! stmts[i]->Serialize(info) )
return false;
return true;
}
bool StmtList::DoUnserialize(UnserialInfo* info)
{
DO_UNSERIALIZE(Stmt);
int len;
if ( ! UNSERIALIZE(&len) )
return false;
while ( len-- )
{
Stmt* stmt = Stmt::Unserialize(info);
if ( ! stmt )
return false;
stmts.append(stmt);
}
return true;
}
Val* EventBodyList::Exec(Frame* f, stmt_flow_type& flow) const
{
RegisterAccess();
@ -2036,20 +1610,6 @@ void EventBodyList::Describe(ODesc* d) const
StmtList::Describe(d);
}
IMPLEMENT_SERIAL(EventBodyList, SER_EVENT_BODY_LIST);
bool EventBodyList::DoSerialize(SerialInfo* info) const
{
DO_SERIALIZE(SER_EVENT_BODY_LIST, StmtList);
return SERIALIZE(topmost);
}
bool EventBodyList::DoUnserialize(UnserialInfo* info)
{
DO_UNSERIALIZE(StmtList);
return UNSERIALIZE(&topmost);
}
InitStmt::~InitStmt()
{
loop_over_list(*inits, i)
@ -2123,45 +1683,6 @@ TraversalCode InitStmt::Traverse(TraversalCallback* cb) const
HANDLE_TC_STMT_POST(tc);
}
IMPLEMENT_SERIAL(InitStmt, SER_INIT_STMT);
bool InitStmt::DoSerialize(SerialInfo* info) const
{
DO_SERIALIZE(SER_INIT_STMT, Stmt);
if ( ! SERIALIZE(inits->length()) )
return false;
loop_over_list((*inits), i)
{
if ( ! (*inits)[i]->Serialize(info) )
return false;
}
return true;
}
bool InitStmt::DoUnserialize(UnserialInfo* info)
{
DO_UNSERIALIZE(Stmt);
int len;
if ( ! UNSERIALIZE(&len) )
return false;
inits = new id_list(len);
while ( len-- )
{
ID* id = ID::Unserialize(info);
if ( ! id )
return false;
inits->append(id);
}
return true;
}
Val* NullStmt::Exec(Frame* /* f */, stmt_flow_type& flow) const
{
RegisterAccess();
@ -2191,20 +1712,6 @@ TraversalCode NullStmt::Traverse(TraversalCallback* cb) const
HANDLE_TC_STMT_POST(tc);
}
IMPLEMENT_SERIAL(NullStmt, SER_NULL_STMT);
bool NullStmt::DoSerialize(SerialInfo* info) const
{
DO_SERIALIZE(SER_NULL_STMT, Stmt);
return true;
}
bool NullStmt::DoUnserialize(UnserialInfo* info)
{
DO_UNSERIALIZE(Stmt);
return true;
}
WhenStmt::WhenStmt(Expr* arg_cond, Stmt* arg_s1, Stmt* arg_s2,
Expr* arg_timeout, bool arg_is_return)
: Stmt(STMT_WHEN)
@ -2320,35 +1827,3 @@ TraversalCode WhenStmt::Traverse(TraversalCallback* cb) const
HANDLE_TC_STMT_POST(tc);
}
IMPLEMENT_SERIAL(WhenStmt, SER_WHEN_STMT);
bool WhenStmt::DoSerialize(SerialInfo* info) const
{
DO_SERIALIZE(SER_WHEN_STMT, Stmt);
if ( cond->Serialize(info) && s1->Serialize(info) )
return false;
SERIALIZE_OPTIONAL(s2);
SERIALIZE_OPTIONAL(timeout);
return true;
}
bool WhenStmt::DoUnserialize(UnserialInfo* info)
{
DO_UNSERIALIZE(Stmt);
cond = Expr::Unserialize(info);
if ( ! cond )
return false;
s1 = Stmt::Unserialize(info);
if ( ! s1 )
return false;
UNSERIALIZE_OPTIONAL(s2, Stmt::Unserialize(info));
UNSERIALIZE_OPTIONAL(timeout, Expr::Unserialize(info));
return true;
}

47
src/Stmt.h
View file

@ -71,9 +71,6 @@ public:
virtual unsigned int BPCount() const { return breakpoint_count; }
bool Serialize(SerialInfo* info) const;
static Stmt* Unserialize(UnserialInfo* info, BroStmtTag want = STMT_ANY);
virtual TraversalCode Traverse(TraversalCallback* cb) const = 0;
protected:
@ -83,8 +80,6 @@ protected:
void AddTag(ODesc* d) const;
void DescribeDone(ODesc* d) const;
DECLARE_ABSTRACT_SERIAL(Stmt);
BroStmtTag tag;
int breakpoint_count; // how many breakpoints on this statement
@ -111,8 +106,6 @@ protected:
void Describe(ODesc* d) const override;
void PrintVals(ODesc* d, val_list* vals, int offset) const;
DECLARE_ABSTRACT_SERIAL(ExprListStmt);
ListExpr* l;
};
@ -125,8 +118,6 @@ protected:
PrintStmt() {}
Val* DoExec(val_list* vals, stmt_flow_type& flow) const override;
DECLARE_SERIAL(PrintStmt);
};
class ExprStmt : public Stmt {
@ -151,8 +142,6 @@ protected:
int IsPure() const override;
DECLARE_SERIAL(ExprStmt);
Expr* e;
};
@ -175,8 +164,6 @@ protected:
Val* DoExec(Frame* f, Val* v, stmt_flow_type& flow) const override;
int IsPure() const override;
DECLARE_SERIAL(IfStmt);
Stmt* s1;
Stmt* s2;
};
@ -197,17 +184,12 @@ public:
void Describe(ODesc* d) const override;
bool Serialize(SerialInfo* info) const;
static Case* Unserialize(UnserialInfo* info);
TraversalCode Traverse(TraversalCallback* cb) const;
protected:
friend class Stmt;
Case() { expr_cases = 0; type_cases = 0; s = 0; }
DECLARE_SERIAL(Case);
ListExpr* expr_cases;
id_list* type_cases;
Stmt* s;
@ -234,8 +216,6 @@ protected:
Val* DoExec(Frame* f, Val* v, stmt_flow_type& flow) const override;
int IsPure() const override;
DECLARE_SERIAL(SwitchStmt);
// Initialize composite hash and case label map.
void Init();
@ -274,8 +254,6 @@ public:
protected:
friend class Stmt;
AddStmt() {}
DECLARE_SERIAL(AddStmt);
};
class DelStmt : public ExprStmt {
@ -290,8 +268,6 @@ public:
protected:
friend class Stmt;
DelStmt() {}
DECLARE_SERIAL(DelStmt);
};
class EventStmt : public ExprStmt {
@ -306,8 +282,6 @@ protected:
friend class Stmt;
EventStmt() { event_expr = 0; }
DECLARE_SERIAL(EventStmt);
EventExpr* event_expr;
};
@ -331,8 +305,6 @@ protected:
Val* Exec(Frame* f, stmt_flow_type& flow) const override;
DECLARE_SERIAL(WhileStmt);
Expr* loop_condition;
Stmt* body;
};
@ -362,8 +334,6 @@ protected:
Val* DoExec(Frame* f, Val* v, stmt_flow_type& flow) const override;
DECLARE_SERIAL(ForStmt);
id_list* loop_vars;
Stmt* body;
// Stores the value variable being used for a key value for loop.
@ -383,7 +353,6 @@ public:
TraversalCode Traverse(TraversalCallback* cb) const override;
protected:
DECLARE_SERIAL(NextStmt);
};
class BreakStmt : public Stmt {
@ -398,7 +367,6 @@ public:
TraversalCode Traverse(TraversalCallback* cb) const override;
protected:
DECLARE_SERIAL(BreakStmt);
};
class FallthroughStmt : public Stmt {
@ -413,7 +381,6 @@ public:
TraversalCode Traverse(TraversalCallback* cb) const override;
protected:
DECLARE_SERIAL(FallthroughStmt);
};
class ReturnStmt : public ExprStmt {
@ -427,8 +394,6 @@ public:
protected:
friend class Stmt;
ReturnStmt() {}
DECLARE_SERIAL(ReturnStmt);
};
class StmtList : public Stmt {
@ -448,8 +413,6 @@ public:
protected:
int IsPure() const override;
DECLARE_SERIAL(StmtList);
stmt_list stmts;
};
@ -467,9 +430,6 @@ public:
// bool IsTopmost() { return topmost; }
protected:
DECLARE_SERIAL(EventBodyList);
bool topmost;
};
@ -496,8 +456,6 @@ protected:
friend class Stmt;
InitStmt() { inits = 0; }
DECLARE_SERIAL(InitStmt);
id_list* inits;
};
@ -511,9 +469,6 @@ public:
void Describe(ODesc* d) const override;
TraversalCode Traverse(TraversalCallback* cb) const override;
protected:
DECLARE_SERIAL(NullStmt);
};
class WhenStmt : public Stmt {
@ -537,8 +492,6 @@ public:
protected:
WhenStmt() { cond = 0; s1 = s2 = 0; timeout = 0; is_return = 0; }
DECLARE_SERIAL(WhenStmt);
Expr* cond;
Stmt* s1;
Stmt* s2;

36
src/Timer.cc
View file

@ -5,7 +5,6 @@
#include "util.h"
#include "Timer.h"
#include "Desc.h"
#include "Serializer.h"
#include "broker/Manager.h"
// Names of timers in same order than in TimerType.
@ -53,41 +52,6 @@ void Timer::Describe(ODesc* d) const
d->Add(Time());
}
bool Timer::Serialize(SerialInfo* info) const
{
return SerialObj::Serialize(info);
}
Timer* Timer::Unserialize(UnserialInfo* info)
{
Timer* timer = (Timer*) SerialObj::Unserialize(info, SER_TIMER);
if ( ! timer )
return 0;
timer_mgr->Add(timer);
return timer;
}
bool Timer::DoSerialize(SerialInfo* info) const
{
DO_SERIALIZE(SER_TIMER, SerialObj);
char tmp = type;
return SERIALIZE(tmp) && SERIALIZE(time);
}
bool Timer::DoUnserialize(UnserialInfo* info)
{
DO_UNSERIALIZE(SerialObj);
char tmp;
if ( ! UNSERIALIZE(&tmp) )
return false;
type = tmp;
return UNSERIALIZE(&time);
}
unsigned int TimerMgr::current_timers[NUM_TIMER_TYPES];
TimerMgr::~TimerMgr()

9
src/Timer.h
View file

@ -6,7 +6,6 @@
#include <string>
#include <string>
#include "SerialObj.h"
#include "PriorityQueue.h"
extern "C" {
@ -49,10 +48,9 @@ const int NUM_TIMER_TYPES = int(TIMER_TIMERMGR_EXPIRE) + 1;
extern const char* timer_type_to_string(TimerType type);
class Serializer;
class ODesc;
class Timer : public SerialObj, public PQ_Element {
class Timer : public PQ_Element {
public:
Timer(double t, TimerType arg_type) : PQ_Element(t)
{ type = (char) arg_type; }
@ -67,14 +65,9 @@ public:
void Describe(ODesc* d) const;
bool Serialize(SerialInfo* info) const;
static Timer* Unserialize(UnserialInfo* info);
protected:
Timer() {}
DECLARE_ABSTRACT_SERIAL(Timer);
unsigned int type:8;
};

31
src/Trigger.cc
View file

@ -33,7 +33,7 @@ TraversalCode TriggerTraversalCallback::PreExpr(const Expr* expr)
trigger->Register(e->Id());
Val* v = e->Id()->ID_Val();
if ( v && v->IsMutableVal() )
if ( v && v->Modifiable() )
trigger->Register(v);
break;
};
@ -382,38 +382,35 @@ void Trigger::Timeout()
void Trigger::Register(ID* id)
{
assert(! disabled);
notifiers.Register(id, this);
notifier::registry.Register(id, this);
Ref(id);
ids.insert(id);
objs.push_back({id, id});
}
void Trigger::Register(Val* val)
{
if ( ! val->Modifiable() )
return;
assert(! disabled);
notifiers.Register(val, this);
notifier::registry.Register(val->Modifiable(), this);
Ref(val);
vals.insert(val);
objs.emplace_back(val, val->Modifiable());
}
void Trigger::UnregisterAll()
{
loop_over_list(ids, i)
DBG_LOG(DBG_NOTIFIERS, "%s: unregistering all", Name());
for ( const auto& o : objs )
{
notifiers.Unregister(ids[i], this);
Unref(ids[i]);
notifier::registry.Unregister(o.second, this);
Unref(o.first);
}
ids.clear();
loop_over_list(vals, j)
{
notifiers.Unregister(vals[j], this);
Unref(vals[j]);
}
vals.clear();
objs.clear();
}
void Trigger::Attach(Trigger *trigger)

13
src/Trigger.h
View file

@ -4,7 +4,7 @@
#include <list>
#include <map>
#include "StateAccess.h"
#include "Notifier.h"
#include "Traverse.h"
// Triggers are the heart of "when" statements: expressions that when
@ -13,7 +13,7 @@
class TriggerTimer;
class TriggerTraversalCallback;
class Trigger : public NotifierRegistry::Notifier, public BroObj {
class Trigger : public BroObj, public notifier::Receiver {
public:
// Don't access Trigger objects; they take care of themselves after
// instantiation. Note that if the condition is already true, the
@ -61,12 +61,10 @@ public:
{ d->Add("<trigger>"); }
// Overidden from Notifier. We queue the trigger and evaluate it
// later to avoid race conditions.
void Access(ID* id, const StateAccess& sa) override
{ QueueTrigger(this); }
void Access(Val* val, const StateAccess& sa) override
void Modified(notifier::Modifiable* m) override
{ QueueTrigger(this); }
const char* Name() const override;
const char* Name() const;
static void QueueTrigger(Trigger* trigger);
@ -104,8 +102,7 @@ private:
bool delayed; // true if a function call is currently being delayed
bool disabled;
val_list vals;
id_list ids;
std::vector<std::pair<BroObj *, notifier::Modifiable*>> objs;
typedef map<const CallExpr*, Val*> ValCache;
ValCache cache;

594
src/Type.cc
View file

@ -6,7 +6,6 @@
#include "Attr.h"
#include "Expr.h"
#include "Scope.h"
#include "Serializer.h"
#include "Reporter.h"
#include "zeekygen/Manager.h"
#include "zeekygen/utils.h"
@ -122,27 +121,30 @@ BroType::BroType(TypeTag t, bool arg_base_type)
}
BroType* BroType::Clone() const
BroType* BroType::ShallowClone()
{
SerializationFormat* form = new BinarySerializationFormat();
form->StartWrite();
CloneSerializer ss(form);
SerialInfo sinfo(&ss);
sinfo.cache = false;
switch ( tag ) {
case TYPE_VOID:
case TYPE_BOOL:
case TYPE_INT:
case TYPE_COUNT:
case TYPE_COUNTER:
case TYPE_DOUBLE:
case TYPE_TIME:
case TYPE_INTERVAL:
case TYPE_STRING:
case TYPE_PATTERN:
case TYPE_TIMER:
case TYPE_PORT:
case TYPE_ADDR:
case TYPE_SUBNET:
case TYPE_ANY:
return new BroType(tag, base_type);
this->Serialize(&sinfo);
char* data;
uint32 len = form->EndWrite(&data);
form->StartRead(data, len);
UnserialInfo uinfo(&ss);
uinfo.cache = false;
BroType* rval = this->Unserialize(&uinfo, false);
assert(rval != this);
free(data);
return rval;
default:
reporter->InternalError("cloning illegal base BroType");
}
return nullptr;
}
int BroType::MatchesIndex(ListExpr*& index) const
@ -203,124 +205,6 @@ unsigned int BroType::MemoryAllocation() const
return padded_sizeof(*this);
}
bool BroType::Serialize(SerialInfo* info) const
{
// We always send full types (see below).
if ( ! SERIALIZE(true) )
return false;
bool ret = SerialObj::Serialize(info);
return ret;
}
BroType* BroType::Unserialize(UnserialInfo* info, bool use_existing)
{
// To avoid external Broccoli clients needing to always send full type
// objects, we allow them to give us only the name of a type. To
// differentiate between the two cases, we exchange a flag first.
bool full_type = true;;
if ( ! UNSERIALIZE(&full_type) )
return 0;
if ( ! full_type )
{
const char* name;
if ( ! UNSERIALIZE_STR(&name, 0) )
return 0;
ID* id = global_scope()->Lookup(name);
if ( ! id )
{
info->s->Error(fmt("unknown type %s", name));
return 0;
}
BroType* t = id->AsType();
if ( ! t )
{
info->s->Error(fmt("%s is not a type", name));
return 0;
}
return t->Ref();
}
BroType* t = (BroType*) SerialObj::Unserialize(info, SER_BRO_TYPE);
if ( ! t || ! use_existing )
return t;
if ( ! t->name.empty() )
{
// Avoid creating a new type if it's known by name.
// Also avoids loss of base type name alias (from condition below).
ID* id = global_scope()->Lookup(t->name.c_str());
BroType* t2 = id ? id->AsType() : 0;
if ( t2 )
{
Unref(t);
return t2->Ref();
}
}
if ( t->base_type )
{
BroType* t2 = ::base_type(TypeTag(t->tag));
Unref(t);
assert(t2);
return t2;
}
assert(t);
return t;
}
IMPLEMENT_SERIAL(BroType, SER_BRO_TYPE)
bool BroType::DoSerialize(SerialInfo* info) const
{
DO_SERIALIZE(SER_BRO_TYPE, BroObj);
info->s->WriteOpenTag("Type");
if ( ! (SERIALIZE(char(tag)) && SERIALIZE(char(internal_tag))) )
return false;
if ( ! (SERIALIZE(is_network_order) && SERIALIZE(base_type)) )
return false;
SERIALIZE_STR(name.c_str(), name.size());
info->s->WriteCloseTag("Type");
return true;
}
bool BroType::DoUnserialize(UnserialInfo* info)
{
DO_UNSERIALIZE(BroObj);
char c1, c2;
if ( ! (UNSERIALIZE(&c1) && UNSERIALIZE(&c2) ) )
return 0;
tag = (TypeTag) c1;
internal_tag = (InternalTypeTag) c2;
if ( ! (UNSERIALIZE(&is_network_order) && UNSERIALIZE(&base_type)) )
return 0;
const char* n;
if ( ! UNSERIALIZE_STR(&n, 0) )
return false;
name = n;
delete [] n;
return true;
}
TypeList::~TypeList()
{
loop_over_list(types, i)
@ -383,47 +267,6 @@ void TypeList::Describe(ODesc* d) const
}
}
IMPLEMENT_SERIAL(TypeList, SER_TYPE_LIST);
bool TypeList::DoSerialize(SerialInfo* info) const
{
DO_SERIALIZE(SER_TYPE_LIST, BroType);
SERIALIZE_OPTIONAL(pure_type);
if ( ! SERIALIZE(types.length()) )
return false;
loop_over_list(types, j)
{
if ( ! types[j]->Serialize(info) )
return false;
}
return true;
}
bool TypeList::DoUnserialize(UnserialInfo* info)
{
DO_UNSERIALIZE(BroType);
UNSERIALIZE_OPTIONAL(pure_type, BroType::Unserialize(info));
int len;
if ( ! UNSERIALIZE(&len) )
return false;
while ( len-- )
{
BroType* t = BroType::Unserialize(info);
if ( ! t )
return false;
types.append(t);
}
return true;
}
IndexType::~IndexType()
{
Unref(indices);
@ -530,25 +373,6 @@ bool IndexType::IsSubNetIndex() const
return false;
}
IMPLEMENT_SERIAL(IndexType, SER_INDEX_TYPE);
bool IndexType::DoSerialize(SerialInfo* info) const
{
DO_SERIALIZE(SER_INDEX_TYPE, BroType);
SERIALIZE_OPTIONAL(yield_type);
return indices->Serialize(info);
}
bool IndexType::DoUnserialize(UnserialInfo* info)
{
DO_UNSERIALIZE(BroType);
UNSERIALIZE_OPTIONAL(yield_type, BroType::Unserialize(info));
indices = (TypeList*) BroType::Unserialize(info);
return indices != 0;
}
TableType::TableType(TypeList* ind, BroType* yield)
: IndexType(TYPE_TABLE, ind, yield)
{
@ -577,6 +401,16 @@ TableType::TableType(TypeList* ind, BroType* yield)
}
}
TableType* TableType::ShallowClone()
{
if ( indices )
indices->Ref();
if ( yield_type )
yield_type->Ref();
return new TableType(indices, yield_type);
}
bool TableType::IsUnspecifiedTable() const
{
// Unspecified types have an empty list of indices.
@ -650,18 +484,17 @@ SetType::SetType(TypeList* ind, ListExpr* arg_elements) : TableType(ind, 0)
}
}
IMPLEMENT_SERIAL(TableType, SER_TABLE_TYPE);
bool TableType::DoSerialize(SerialInfo* info) const
SetType* SetType::ShallowClone()
{
DO_SERIALIZE(SER_TABLE_TYPE, IndexType);
return true;
// constructor only consumes indices when elements
// is set
if ( elements && indices )
{
elements->Ref();
indices->Ref();
}
bool TableType::DoUnserialize(UnserialInfo* info)
{
DO_UNSERIALIZE(IndexType);
return true;
return new SetType(indices, elements);
}
SetType::~SetType()
@ -669,24 +502,6 @@ SetType::~SetType()
Unref(elements);
}
IMPLEMENT_SERIAL(SetType, SER_SET_TYPE);
bool SetType::DoSerialize(SerialInfo* info) const
{
DO_SERIALIZE(SER_SET_TYPE, TableType);
SERIALIZE_OPTIONAL(elements);
return true;
}
bool SetType::DoUnserialize(UnserialInfo* info)
{
DO_UNSERIALIZE(TableType);
UNSERIALIZE_OPTIONAL(elements, (ListExpr*) Expr::Unserialize(info, EXPR_LIST));
return true;
}
FuncType::FuncType(RecordType* arg_args, BroType* arg_yield, function_flavor arg_flavor)
: BroType(TYPE_FUNC)
{
@ -716,6 +531,16 @@ FuncType::FuncType(RecordType* arg_args, BroType* arg_yield, function_flavor arg
}
}
FuncType* FuncType::ShallowClone()
{
auto f = new FuncType();
f->args = args->Ref()->AsRecordType();
f->arg_types = arg_types->Ref()->AsTypeList();
f->yield = yield->Ref();
f->flavor = flavor;
return f;
}
string FuncType::FlavorString() const
{
switch ( flavor ) {
@ -822,80 +647,6 @@ void FuncType::DescribeReST(ODesc* d, bool roles_only) const
}
}
IMPLEMENT_SERIAL(FuncType, SER_FUNC_TYPE);
bool FuncType::DoSerialize(SerialInfo* info) const
{
DO_SERIALIZE(SER_FUNC_TYPE, BroType);
assert(args);
assert(arg_types);
SERIALIZE_OPTIONAL(yield);
int ser_flavor = 0;
switch ( flavor ) {
case FUNC_FLAVOR_FUNCTION:
ser_flavor = 0;
break;
case FUNC_FLAVOR_EVENT:
ser_flavor = 1;
break;
case FUNC_FLAVOR_HOOK:
ser_flavor = 2;
break;
default:
reporter->InternalError("Invalid function flavor serialization");
break;
}
return args->Serialize(info) &&
arg_types->Serialize(info) &&
SERIALIZE(ser_flavor);
}
bool FuncType::DoUnserialize(UnserialInfo* info)
{
DO_UNSERIALIZE(BroType);
UNSERIALIZE_OPTIONAL(yield, BroType::Unserialize(info));
args = (RecordType*) BroType::Unserialize(info);
if ( ! args )
return false;
arg_types = (TypeList*) BroType::Unserialize(info);
if ( ! arg_types )
return false;
int ser_flavor = 0;
if ( ! UNSERIALIZE(&ser_flavor) )
return false;
switch ( ser_flavor ) {
case 0:
flavor = FUNC_FLAVOR_FUNCTION;
break;
case 1:
flavor = FUNC_FLAVOR_EVENT;
break;
case 2:
flavor = FUNC_FLAVOR_HOOK;
break;
default:
reporter->InternalError("Invalid function flavor unserialization");
break;
}
return true;
}
TypeDecl::TypeDecl(BroType* t, const char* i, attr_list* arg_attrs, bool in_record)
{
type = t;
@ -921,35 +672,6 @@ TypeDecl::~TypeDecl()
delete [] id;
}
bool TypeDecl::Serialize(SerialInfo* info) const
{
assert(type);
assert(id);
SERIALIZE_OPTIONAL(attrs);
if ( ! (type->Serialize(info) && SERIALIZE(id)) )
return false;
return true;
}
TypeDecl* TypeDecl::Unserialize(UnserialInfo* info)
{
TypeDecl* t = new TypeDecl(0, 0, 0);
UNSERIALIZE_OPTIONAL_STATIC(t->attrs, Attributes::Unserialize(info), t);
t->type = BroType::Unserialize(info);
if ( ! (t->type && UNSERIALIZE_STR(&t->id, 0)) )
{
delete t;
return 0;
}
return t;
}
void TypeDecl::DescribeReST(ODesc* d, bool roles_only) const
{
d->Add(id);
@ -977,6 +699,16 @@ RecordType::RecordType(type_decl_list* arg_types) : BroType(TYPE_RECORD)
num_fields = types ? types->length() : 0;
}
// in this case the clone is actually not so shallow, since
// it gets modified by everyone.
RecordType* RecordType::ShallowClone()
{
auto pass = new type_decl_list();
loop_over_list(*types, i)
pass->append(new TypeDecl(*(*types)[i]));
return new RecordType(pass);
}
RecordType::~RecordType()
{
if ( types )
@ -1253,67 +985,6 @@ void RecordType::DescribeFieldsReST(ODesc* d, bool func_args) const
d->PopIndentNoNL();
}
IMPLEMENT_SERIAL(RecordType, SER_RECORD_TYPE)
bool RecordType::DoSerialize(SerialInfo* info) const
{
DO_SERIALIZE(SER_RECORD_TYPE, BroType);
if ( ! SERIALIZE(num_fields) )
return false;
if ( types )
{
if ( ! (SERIALIZE(true) && SERIALIZE(types->length())) )
return false;
loop_over_list(*types, i)
{
if ( ! (*types)[i]->Serialize(info) )
return false;
}
}
else if ( ! SERIALIZE(false) )
return false;
return true;
}
bool RecordType::DoUnserialize(UnserialInfo* info)
{
DO_UNSERIALIZE(BroType);
if ( ! UNSERIALIZE(&num_fields) )
return false;
bool has_it;
if ( ! UNSERIALIZE(&has_it) )
return false;
if ( has_it )
{
int len;
if ( ! UNSERIALIZE(&len) )
return false;
types = new type_decl_list(len);
while ( len-- )
{
TypeDecl* t = TypeDecl::Unserialize(info);
if ( ! t )
return false;
types->append(t);
}
}
else
types = 0;
return true;
}
SubNetType::SubNetType() : BroType(TYPE_SUBNET)
{
}
@ -1326,20 +997,6 @@ void SubNetType::Describe(ODesc* d) const
d->Add(int(Tag()));
}
IMPLEMENT_SERIAL(SubNetType, SER_SUBNET_TYPE);
bool SubNetType::DoSerialize(SerialInfo* info) const
{
DO_SERIALIZE(SER_SUBNET_TYPE, BroType);
return true;
}
bool SubNetType::DoUnserialize(UnserialInfo* info)
{
DO_UNSERIALIZE(BroType);
return true;
}
FileType::FileType(BroType* yield_type)
: BroType(TYPE_FILE)
{
@ -1370,24 +1027,6 @@ void FileType::Describe(ODesc* d) const
}
}
IMPLEMENT_SERIAL(FileType, SER_FILE_TYPE);
bool FileType::DoSerialize(SerialInfo* info) const
{
DO_SERIALIZE(SER_FILE_TYPE, BroType);
assert(yield);
return yield->Serialize(info);
}
bool FileType::DoUnserialize(UnserialInfo* info)
{
DO_UNSERIALIZE(BroType);
yield = BroType::Unserialize(info);
return yield != 0;
}
OpaqueType::OpaqueType(const string& arg_name) : BroType(TYPE_OPAQUE)
{
name = arg_name;
@ -1408,28 +1047,6 @@ void OpaqueType::DescribeReST(ODesc* d, bool roles_only) const
d->Add(fmt(":zeek:type:`%s` of %s", type_name(Tag()), name.c_str()));
}
IMPLEMENT_SERIAL(OpaqueType, SER_OPAQUE_TYPE);
bool OpaqueType::DoSerialize(SerialInfo* info) const
{
DO_SERIALIZE(SER_OPAQUE_TYPE, BroType);
return SERIALIZE_STR(name.c_str(), name.size());
}
bool OpaqueType::DoUnserialize(UnserialInfo* info)
{
DO_UNSERIALIZE(BroType);
const char* n;
if ( ! UNSERIALIZE_STR(&n, 0) )
return false;
name = n;
delete [] n;
return true;
}
EnumType::EnumType(const string& name)
: BroType(TYPE_ENUM)
{
@ -1437,18 +1054,26 @@ EnumType::EnumType(const string& name)
SetName(name);
}
EnumType::EnumType(EnumType* e)
EnumType::EnumType(const EnumType* e)
: BroType(TYPE_ENUM)
{
counter = e->counter;
SetName(e->GetName());
for ( NameMap::iterator it = e->names.begin(); it != e->names.end(); ++it )
for ( auto it = e->names.begin(); it != e->names.end(); ++it )
names[it->first] = it->second;
vals = e->vals;
}
EnumType* EnumType::ShallowClone()
{
if ( counter == 0 )
return new EnumType(GetName());
return new EnumType(this);
}
EnumType::~EnumType()
{
for ( auto& kv : vals )
@ -1672,64 +1297,16 @@ void EnumType::DescribeReST(ODesc* d, bool roles_only) const
}
}
IMPLEMENT_SERIAL(EnumType, SER_ENUM_TYPE);
bool EnumType::DoSerialize(SerialInfo* info) const
{
DO_SERIALIZE(SER_ENUM_TYPE, BroType);
if ( ! (SERIALIZE(counter) && SERIALIZE((unsigned int) names.size()) &&
// Dummy boolean for backwards compatibility.
SERIALIZE(false)) )
return false;
for ( NameMap::const_iterator iter = names.begin();
iter != names.end(); ++iter )
{
if ( ! SERIALIZE(iter->first) || ! SERIALIZE(iter->second) )
return false;
}
return true;
}
bool EnumType::DoUnserialize(UnserialInfo* info)
{
DO_UNSERIALIZE(BroType);
unsigned int len;
bool dummy;
if ( ! UNSERIALIZE(&counter) ||
! UNSERIALIZE(&len) ||
// Dummy boolean for backwards compatibility.
! UNSERIALIZE(&dummy) )
return false;
while ( len-- )
{
const char* name;
bro_int_t val;
if ( ! (UNSERIALIZE_STR(&name, 0) && UNSERIALIZE(&val)) )
return false;
names[name] = val;
delete [] name; // names[name] converts to std::string
// note: the 'vals' map gets populated lazily, which works fine and
// also happens to avoid a leak due to circular reference between the
// types and vals (there's a special case for unserializing a known
// type that will unserialze and then immediately want to unref the
// type if we already have it, except that won't delete it as intended
// if we've already created circular references to it here).
}
return true;
}
VectorType::VectorType(BroType* element_type)
: BroType(TYPE_VECTOR), yield_type(element_type)
{
}
VectorType* VectorType::ShallowClone()
{
return new VectorType(yield_type);
}
VectorType::~VectorType()
{
Unref(yield_type);
@ -1773,10 +1350,12 @@ int VectorType::MatchesIndex(ListExpr*& index) const
{
expr_list& el = index->Exprs();
if ( el.length() != 1 )
if ( el.length() != 1 && el.length() != 2)
return DOES_NOT_MATCH_INDEX;
if ( el[0]->Type()->Tag() == TYPE_VECTOR )
if ( el.length() == 2 )
return MATCHES_INDEX_VECTOR;
else if ( el[0]->Type()->Tag() == TYPE_VECTOR )
return (IsIntegral(el[0]->Type()->YieldType()->Tag()) ||
IsBool(el[0]->Type()->YieldType()->Tag())) ?
MATCHES_INDEX_VECTOR : DOES_NOT_MATCH_INDEX;
@ -1791,21 +1370,6 @@ bool VectorType::IsUnspecifiedVector() const
return yield_type->Tag() == TYPE_VOID;
}
IMPLEMENT_SERIAL(VectorType, SER_VECTOR_TYPE);
bool VectorType::DoSerialize(SerialInfo* info) const
{
DO_SERIALIZE(SER_VECTOR_TYPE, BroType);
return yield_type->Serialize(info);
}
bool VectorType::DoUnserialize(UnserialInfo* info)
{
DO_UNSERIALIZE(BroType);
yield_type = BroType::Unserialize(info);
return yield_type != 0;
}
void VectorType::Describe(ODesc* d) const
{
if ( d->IsReadable() )

68
src/Type.h
View file

@ -72,7 +72,6 @@ class SubNetType;
class FuncType;
class ListExpr;
class EnumType;
class Serializer;
class VectorType;
class TypeType;
class OpaqueType;
@ -87,7 +86,15 @@ public:
explicit BroType(TypeTag tag, bool base_type = false);
~BroType() override { }
BroType* Clone() const;
// Performs a shallow clone operation of the Bro type.
// This especially means that especially for tables the types
// are not recursively cloned; altering one type will in this case
// alter one of them.
// The main use for this is alias tracking.
// Clone operations will mostly be implemented in the derived classes;
// in addition cloning will be limited to classes that can be reached by
// the script-level.
virtual BroType* ShallowClone();
TypeTag Tag() const { return tag; }
InternalTypeTag InternalType() const { return internal_tag; }
@ -108,7 +115,7 @@ public:
// this type is a table[string] of port, then returns the "port"
// type. Returns nil if this is not an index type.
virtual BroType* YieldType();
const BroType* YieldType() const
virtual const BroType* YieldType() const
{ return ((BroType*) this)->YieldType(); }
// Returns true if this type is a record and contains the
@ -256,9 +263,6 @@ public:
virtual unsigned MemoryAllocation() const;
bool Serialize(SerialInfo* info) const;
static BroType* Unserialize(UnserialInfo* info, bool use_existing = true);
void SetName(const string& arg_name) { name = arg_name; }
string GetName() const { return name; }
@ -275,8 +279,6 @@ protected:
void SetError();
DECLARE_SERIAL(BroType)
private:
TypeTag tag;
InternalTypeTag internal_tag;
@ -325,8 +327,6 @@ public:
}
protected:
DECLARE_SERIAL(TypeList)
BroType* pure_type;
type_list types;
};
@ -338,7 +338,7 @@ public:
TypeList* Indices() const { return indices; }
const type_list* IndexTypes() const { return indices->Types(); }
BroType* YieldType() override;
const BroType* YieldType() const;
const BroType* YieldType() const override;
void Describe(ODesc* d) const override;
void DescribeReST(ODesc* d, bool roles_only = false) const override;
@ -356,8 +356,6 @@ protected:
}
~IndexType() override;
DECLARE_SERIAL(IndexType)
TypeList* indices;
BroType* yield_type;
};
@ -366,6 +364,8 @@ class TableType : public IndexType {
public:
TableType(TypeList* ind, BroType* yield);
TableType* ShallowClone() override;
// Returns true if this table type is "unspecified", which is
// what one gets using an empty "set()" or "table()" constructor.
bool IsUnspecifiedTable() const;
@ -374,8 +374,6 @@ protected:
TableType() {}
TypeList* ExpandRecordIndex(RecordType* rt) const;
DECLARE_SERIAL(TableType)
};
class SetType : public TableType {
@ -383,25 +381,26 @@ public:
SetType(TypeList* ind, ListExpr* arg_elements);
~SetType() override;
SetType* ShallowClone() override;
ListExpr* SetElements() const { return elements; }
protected:
SetType() {}
ListExpr* elements;
DECLARE_SERIAL(SetType)
};
class FuncType : public BroType {
public:
FuncType(RecordType* args, BroType* yield, function_flavor f);
FuncType* ShallowClone() override;
~FuncType() override;
RecordType* Args() const { return args; }
BroType* YieldType() override;
const BroType* YieldType() const;
const BroType* YieldType() const override;
void SetYieldType(BroType* arg_yield) { yield = arg_yield; }
function_flavor Flavor() const { return flavor; }
string FlavorString() const;
@ -419,9 +418,7 @@ public:
void DescribeReST(ODesc* d, bool roles_only = false) const override;
protected:
FuncType() { args = 0; arg_types = 0; yield = 0; flavor = FUNC_FLAVOR_FUNCTION; }
DECLARE_SERIAL(FuncType)
FuncType() : BroType(TYPE_FUNC) { args = 0; arg_types = 0; yield = 0; flavor = FUNC_FLAVOR_FUNCTION; }
RecordType* args;
TypeList* arg_types;
BroType* yield;
@ -431,6 +428,7 @@ protected:
class TypeType : public BroType {
public:
explicit TypeType(BroType* t) : BroType(TYPE_TYPE) { type = t->Ref(); }
TypeType* ShallowClone() override { return new TypeType(type); }
~TypeType() override { Unref(type); }
BroType* Type() { return type; }
@ -450,9 +448,6 @@ public:
const Attr* FindAttr(attr_tag a) const
{ return attrs ? attrs->FindAttr(a) : 0; }
bool Serialize(SerialInfo* info) const;
static TypeDecl* Unserialize(UnserialInfo* info);
virtual void DescribeReST(ODesc* d, bool roles_only = false) const;
BroType* type;
@ -466,6 +461,7 @@ typedef PList(TypeDecl) type_decl_list;
class RecordType : public BroType {
public:
explicit RecordType(type_decl_list* types);
RecordType* ShallowClone() override;
~RecordType() override;
@ -501,8 +497,6 @@ public:
protected:
RecordType() { types = 0; }
DECLARE_SERIAL(RecordType)
int num_fields;
type_decl_list* types;
};
@ -511,13 +505,12 @@ class SubNetType : public BroType {
public:
SubNetType();
void Describe(ODesc* d) const override;
protected:
DECLARE_SERIAL(SubNetType)
};
class FileType : public BroType {
public:
explicit FileType(BroType* yield_type);
FileType* ShallowClone() override { return new FileType(yield->Ref()); }
~FileType() override;
BroType* YieldType() override;
@ -527,14 +520,13 @@ public:
protected:
FileType() { yield = 0; }
DECLARE_SERIAL(FileType)
BroType* yield;
};
class OpaqueType : public BroType {
public:
explicit OpaqueType(const string& name);
OpaqueType* ShallowClone() override { return new OpaqueType(name); }
~OpaqueType() override { };
const string& Name() const { return name; }
@ -545,8 +537,6 @@ public:
protected:
OpaqueType() { }
DECLARE_SERIAL(OpaqueType)
string name;
};
@ -554,8 +544,9 @@ class EnumType : public BroType {
public:
typedef std::list<std::pair<string, bro_int_t> > enum_name_list;
explicit EnumType(EnumType* e);
explicit EnumType(const EnumType* e);
explicit EnumType(const string& arg_name);
EnumType* ShallowClone() override;
~EnumType() override;
// The value of this name is next internal counter value, starting
@ -582,8 +573,6 @@ public:
protected:
EnumType() { counter = 0; }
DECLARE_SERIAL(EnumType)
void AddNameInternal(const string& module_name,
const char* name, bro_int_t val, bool is_export);
@ -609,9 +598,10 @@ protected:
class VectorType : public BroType {
public:
explicit VectorType(BroType* t);
VectorType* ShallowClone() override;
~VectorType() override;
BroType* YieldType() override;
const BroType* YieldType() const;
const BroType* YieldType() const override;
int MatchesIndex(ListExpr*& index) const override;
@ -625,8 +615,6 @@ public:
protected:
VectorType() { yield_type = 0; }
DECLARE_SERIAL(VectorType)
BroType* yield_type;
};
@ -707,10 +695,6 @@ bool is_atomic_type(const BroType* t);
// True if the given type tag corresponds to a function type.
#define IsFunc(t) (t == TYPE_FUNC)
// True if the given type tag corresponds to mutable type.
#define IsMutable(t) \
(t == TYPE_RECORD || t == TYPE_TABLE || t == TYPE_VECTOR)
// True if the given type type is a vector.
#define IsVector(t) (t == TYPE_VECTOR)

1218
src/Val.cc
View file

File diff suppressed because it is too large Load diff

223
src/Val.h
View file

@ -3,11 +3,10 @@
#ifndef val_h
#define val_h
// BRO values.
#include <vector>
#include <list>
#include <array>
#include <unordered_map>
#include "net_util.h"
#include "Type.h"
@ -18,8 +17,9 @@
#include "Timer.h"
#include "ID.h"
#include "Scope.h"
#include "StateAccess.h"
#include "Notifier.h"
#include "IPAddr.h"
#include "DebugLogger.h"
// We have four different port name spaces: TCP, UDP, ICMP, and UNKNOWN.
// We distinguish between them based on the bits specified in the *_PORT_MASK
@ -37,7 +37,6 @@ class Func;
class BroFile;
class RE_Matcher;
class PrefixTable;
class SerialInfo;
class PortVal;
class AddrVal;
@ -50,7 +49,7 @@ class RecordVal;
class ListVal;
class StringVal;
class EnumVal;
class MutableVal;
class OpaqueVal;
class StateAccess;
@ -305,6 +304,7 @@ public:
CONVERTER(TYPE_STRING, StringVal*, AsStringVal)
CONVERTER(TYPE_VECTOR, VectorVal*, AsVectorVal)
CONVERTER(TYPE_ENUM, EnumVal*, AsEnumVal)
CONVERTER(TYPE_OPAQUE, OpaqueVal*, AsOpaqueVal)
#define CONST_CONVERTER(tag, ctype, name) \
const ctype name() const \
@ -322,36 +322,14 @@ public:
CONST_CONVERTER(TYPE_LIST, ListVal*, AsListVal)
CONST_CONVERTER(TYPE_STRING, StringVal*, AsStringVal)
CONST_CONVERTER(TYPE_VECTOR, VectorVal*, AsVectorVal)
bool IsMutableVal() const
{
return IsMutable(type->Tag());
}
const MutableVal* AsMutableVal() const
{
if ( ! IsMutableVal() )
BadTag("Val::AsMutableVal", type_name(type->Tag()));
return (MutableVal*) this;
}
MutableVal* AsMutableVal()
{
if ( ! IsMutableVal() )
BadTag("Val::AsMutableVal", type_name(type->Tag()));
return (MutableVal*) this;
}
CONST_CONVERTER(TYPE_OPAQUE, OpaqueVal*, AsOpaqueVal)
void Describe(ODesc* d) const override;
virtual void DescribeReST(ODesc* d) const;
bool Serialize(SerialInfo* info) const;
static Val* Unserialize(UnserialInfo* info, TypeTag type = TYPE_ANY)
{ return Unserialize(info, type, 0); }
static Val* Unserialize(UnserialInfo* info, const BroType* exact_type)
{ return Unserialize(info, exact_type->Tag(), exact_type); }
DECLARE_SERIAL(Val);
// To be overridden by mutable derived class to enable change
// notification.
virtual notifier::Modifiable* Modifiable() { return 0; }
#ifdef DEBUG
// For debugging, we keep a reference to the global ID to which a
@ -377,6 +355,7 @@ protected:
friend class RecordVal;
friend class VectorVal;
friend class ValManager;
friend class TableEntryVal;
virtual void ValDescribe(ODesc* d) const;
virtual void ValDescribeReST(ODesc* d) const;
@ -421,10 +400,6 @@ protected:
ACCESSOR(TYPE_TABLE, PDict(TableEntryVal)*, table_val, AsNonConstTable)
ACCESSOR(TYPE_RECORD, val_list*, val_list_val, AsNonConstRecord)
// Just an internal helper.
static Val* Unserialize(UnserialInfo* info, TypeTag type,
const BroType* exact_type);
// For internal use by the Val::Clone() methods.
struct CloneState {
// Caches a cloned value for later reuse during the same
@ -524,79 +499,6 @@ private:
extern ValManager* val_mgr;
class MutableVal : public Val {
public:
// Each MutableVal gets a globally unique ID that can be used to
// reference it no matter if it's directly bound to any user-visible
// ID. This ID is inserted into the global namespace.
ID* UniqueID() const { return id ? id : Bind(); }
// Returns true if we've already generated a unique ID.
bool HasUniqueID() const { return id; }
// Transfers the unique ID of the given value to this value. We keep our
// old ID as an alias.
void TransferUniqueID(MutableVal* mv);
// MutableVals can have properties (let's refrain from calling them
// attributes!). Most properties are recursive. If a derived object
// can contain MutableVals itself, the object has to override
// {Add,Remove}Properties(). RecursiveProp(state) masks out all non-
// recursive properties. If this is non-null, an overriden method must
// call itself with RecursiveProp(state) as argument for all contained
// values. (In any case, don't forget to call the parent's method.)
typedef char Properties;
// Tracked by NotifierRegistry, not recursive.
static const int TRACKED = 0x04;
int RecursiveProps(int prop) const { return prop & ~TRACKED; }
Properties GetProperties() const { return props; }
virtual bool AddProperties(Properties state);
virtual bool RemoveProperties(Properties state);
// Whether StateAccess:LogAccess needs to be called.
bool LoggingAccess() const
{
#ifndef DEBUG
return props & TRACKED;
#else
return debug_logger.IsVerbose() ||
(props & TRACKED);
#endif
}
uint64 LastModified() const override { return last_modified; }
// Mark value as changed.
void Modified()
{
last_modified = IncreaseTimeCounter();
}
protected:
explicit MutableVal(BroType* t) : Val(t)
{ props = 0; id = 0; last_modified = SerialObj::ALWAYS; }
MutableVal() { props = 0; id = 0; last_modified = SerialObj::ALWAYS; }
~MutableVal() override;
friend class ID;
friend class Val;
void SetID(ID* arg_id) { Unref(id); id = arg_id; }
DECLARE_SERIAL(MutableVal);
private:
ID* Bind() const;
mutable ID* id;
list<ID*> aliases;
Properties props;
uint64 last_modified;
};
#define Microseconds 1e-6
#define Milliseconds 1e-3
#define Seconds 1.0
@ -612,8 +514,6 @@ protected:
IntervalVal() {}
void ValDescribe(ODesc* d) const override;
DECLARE_SERIAL(IntervalVal);
};
@ -660,8 +560,6 @@ protected:
void ValDescribe(ODesc* d) const override;
Val* DoClone(CloneState* state) override;
DECLARE_SERIAL(PortVal);
};
class AddrVal : public Val {
@ -686,8 +584,6 @@ protected:
explicit AddrVal(BroType* t) : Val(t) { }
Val* DoClone(CloneState* state) override;
DECLARE_SERIAL(AddrVal);
};
class SubNetVal : public Val {
@ -716,8 +612,6 @@ protected:
void ValDescribe(ODesc* d) const override;
Val* DoClone(CloneState* state) override;
DECLARE_SERIAL(SubNetVal);
};
class StringVal : public Val {
@ -749,8 +643,6 @@ protected:
void ValDescribe(ODesc* d) const override;
Val* DoClone(CloneState* state) override;
DECLARE_SERIAL(StringVal);
};
class PatternVal : public Val {
@ -770,8 +662,6 @@ protected:
void ValDescribe(ODesc* d) const override;
Val* DoClone(CloneState* state) override;
DECLARE_SERIAL(PatternVal);
};
// ListVals are mainly used to index tables that have more than one
@ -817,8 +707,6 @@ protected:
Val* DoClone(CloneState* state) override;
DECLARE_SERIAL(ListVal);
val_list vals;
TypeTag tag;
};
@ -831,16 +719,15 @@ public:
{
val = v;
last_access_time = network_time;
expire_access_time = last_read_update =
expire_access_time =
int(network_time - bro_start_network_time);
}
TableEntryVal* Clone()
TableEntryVal* Clone(Val::CloneState* state)
{
auto rval = new TableEntryVal(val ? val->Clone() : nullptr);
auto rval = new TableEntryVal(val ? val->Clone(state) : nullptr);
rval->last_access_time = last_access_time;
rval->expire_access_time = expire_access_time;
rval->last_read_update = last_read_update;
return rval;
}
@ -856,24 +743,16 @@ public:
void SetExpireAccess(double time)
{ expire_access_time = int(time - bro_start_network_time); }
// Returns/sets time of when we propagated the last OP_READ_IDX
// for this item.
double LastReadUpdate() const
{ return bro_start_network_time + last_read_update; }
void SetLastReadUpdate(double time)
{ last_read_update = int(time - bro_start_network_time); }
protected:
friend class TableVal;
Val* val;
double last_access_time;
// The next two entries store seconds since Bro's start. We use
// ints here to save a few bytes, as we do not need a high resolution
// for these anyway.
// The next entry stores seconds since Bro's start. We use ints here
// to save a few bytes, as we do not need a high resolution for these
// anyway.
int expire_access_time;
int last_read_update;
};
class TableValTimer : public Timer {
@ -890,7 +769,7 @@ protected:
};
class CompositeHash;
class TableVal : public MutableVal {
class TableVal : public Val, public notifier::Modifiable {
public:
explicit TableVal(TableType* t, Attributes* attrs = 0);
~TableVal() override;
@ -900,8 +779,8 @@ public:
// version takes a HashKey and Unref()'s it when done. If we're a
// set, new_val has to be nil. If we aren't a set, index may be nil
// in the second version.
int Assign(Val* index, Val* new_val, Opcode op = OP_ASSIGN);
int Assign(Val* index, HashKey* k, Val* new_val, Opcode op = OP_ASSIGN);
int Assign(Val* index, Val* new_val);
int Assign(Val* index, HashKey* k, Val* new_val);
Val* SizeVal() const override { return val_mgr->GetCount(Size()); }
@ -1003,19 +882,17 @@ public:
HashKey* ComputeHash(const Val* index) const
{ return table_hash->ComputeHash(index, 1); }
notifier::Modifiable* Modifiable() override { return this; }
protected:
friend class Val;
friend class StateAccess;
TableVal() {}
void Init(TableType* t);
void CheckExpireAttr(attr_tag at);
int ExpandCompoundAndInit(val_list* vl, int k, Val* new_val);
int CheckAndAssign(Val* index, Val* new_val, Opcode op = OP_ASSIGN);
bool AddProperties(Properties arg_state) override;
bool RemoveProperties(Properties arg_state) override;
int CheckAndAssign(Val* index, Val* new_val);
// Calculates default value for index. Returns 0 if none.
Val* Default(Val* index);
@ -1032,13 +909,8 @@ protected:
// takes ownership of the reference.
double CallExpireFunc(Val *idx);
// Propagates a read operation if necessary.
void ReadOperation(Val* index, TableEntryVal *v);
Val* DoClone(CloneState* state) override;
DECLARE_SERIAL(TableVal);
TableType* table_type;
CompositeHash* table_hash;
Attributes* attrs;
@ -1050,7 +922,7 @@ protected:
Val* def_val;
};
class RecordVal : public MutableVal {
class RecordVal : public Val, public notifier::Modifiable {
public:
explicit RecordVal(RecordType* t, bool init_fields = true);
~RecordVal() override;
@ -1058,7 +930,7 @@ public:
Val* SizeVal() const override
{ return val_mgr->GetCount(Type()->AsRecordType()->NumFields()); }
void Assign(int field, Val* new_val, Opcode op = OP_ASSIGN);
void Assign(int field, Val* new_val);
Val* Lookup(int field) const; // Does not Ref() value.
Val* LookupWithDefault(int field) const; // Does Ref() value.
@ -1097,6 +969,8 @@ public:
unsigned int MemoryAllocation() const override;
void DescribeReST(ODesc* d) const override;
notifier::Modifiable* Modifiable() override { return this; }
// Extend the underlying arrays of record instances created during
// parsing to match the number of fields in the record type (they may
// mismatch as a result of parse-time record type redefinitions.
@ -1106,13 +980,9 @@ protected:
friend class Val;
RecordVal() {}
bool AddProperties(Properties arg_state) override;
bool RemoveProperties(Properties arg_state) override;
Val* DoClone(CloneState* state) override;
DECLARE_SERIAL(RecordVal);
RecordType* record_type;
BroObj* origin;
static vector<RecordVal*> parse_time_records;
@ -1142,12 +1012,10 @@ protected:
void ValDescribe(ODesc* d) const override;
Val* DoClone(CloneState* state) override;
DECLARE_SERIAL(EnumVal);
};
class VectorVal : public MutableVal {
class VectorVal : public Val, public notifier::Modifiable {
public:
explicit VectorVal(VectorType* t);
~VectorVal() override;
@ -1161,11 +1029,11 @@ public:
// Note: does NOT Ref() the element! Remember to do so unless
// the element was just created and thus has refcount 1.
//
bool Assign(unsigned int index, Val* element, Opcode op = OP_ASSIGN);
bool Assign(Val* index, Val* element, Opcode op = OP_ASSIGN)
bool Assign(unsigned int index, Val* element);
bool Assign(Val* index, Val* element)
{
return Assign(index->AsListVal()->Index(0)->CoerceToUnsigned(),
element, op);
element);
}
// Assigns the value to how_many locations starting at index.
@ -1195,37 +1063,24 @@ public:
// Won't shrink size.
unsigned int ResizeAtLeast(unsigned int new_num_elements);
notifier::Modifiable* Modifiable() override { return this; }
// Insert an element at a specific position into the underlying vector.
bool Insert(unsigned int index, Val* element);
// Removes an element at a specific position.
bool Remove(unsigned int index);
protected:
friend class Val;
VectorVal() { }
bool AddProperties(Properties arg_state) override;
bool RemoveProperties(Properties arg_state) override;
void ValDescribe(ODesc* d) const override;
Val* DoClone(CloneState* state) override;
DECLARE_SERIAL(VectorVal);
VectorType* vector_type;
};
// Base class for values with types that are managed completely internally,
// with no further script-level operators provided (other than bif
// functions). See OpaqueVal.h for derived classes.
class OpaqueVal : public Val {
public:
explicit OpaqueVal(OpaqueType* t);
~OpaqueVal() override;
protected:
friend class Val;
OpaqueVal() { }
Val* DoClone(CloneState* state) override;
DECLARE_SERIAL(OpaqueVal);
};
// Checks the given value for consistency with the given type. If an
// exact match, returns it. If promotable, returns the promoted version,
// Unref()'ing the original. If not a match, generates an error message

3
src/Var.cc
View file

@ -6,7 +6,6 @@
#include "Func.h"
#include "Stmt.h"
#include "Scope.h"
#include "Serializer.h"
#include "EventRegistry.h"
#include "Traverse.h"
@ -274,7 +273,7 @@ void add_type(ID* id, BroType* t, attr_list* attr)
tnew = t;
else
// Clone the type to preserve type name aliasing.
tnew = t->Clone();
tnew = t->ShallowClone();
BroType::AddAlias(new_type_name, tnew);

2
src/analyzer/protocol/CMakeLists.txt
View file

@ -36,9 +36,9 @@ add_subdirectory(rdp)
add_subdirectory(rfb)
add_subdirectory(rpc)
add_subdirectory(sip)
add_subdirectory(snmp)
add_subdirectory(smb)
add_subdirectory(smtp)
add_subdirectory(snmp)
add_subdirectory(socks)
add_subdirectory(ssh)
add_subdirectory(ssl)

10
src/analyzer/protocol/arp/CMakeLists.txt
View file

@ -4,12 +4,12 @@
# it's also parsing a protocol just like them. The current structure
# is merely a left-over from when this code was written.
include(BroPlugin)
include(ZeekPlugin)
include_directories(BEFORE ${CMAKE_CURRENT_SOURCE_DIR} ${CMAKE_CURRENT_BINARY_DIR})
bro_plugin_begin(Bro ARP)
bro_plugin_cc(ARP.cc Plugin.cc)
bro_plugin_bif(events.bif)
bro_plugin_end()
zeek_plugin_begin(Zeek ARP)
zeek_plugin_cc(ARP.cc Plugin.cc)
zeek_plugin_bif(events.bif)
zeek_plugin_end()

4
src/analyzer/protocol/arp/Plugin.cc
View file

@ -4,14 +4,14 @@
#include "plugin/Plugin.h"
namespace plugin {
namespace Bro_ARP {
namespace Zeek_ARP {
class Plugin : public plugin::Plugin {
public:
plugin::Configuration Configure()
{
plugin::Configuration config;
config.name = "Bro::ARP";
config.name = "Zeek::ARP";
config.description = "ARP Parsing";
return config;
}

6
src/analyzer/protocol/arp/events.bif
View file

@ -40,7 +40,7 @@ event arp_request%(mac_src: string, mac_dst: string, SPA: addr, SHA: string,
event arp_reply%(mac_src: string, mac_dst: string, SPA: addr, SHA: string,
TPA: addr, THA: string%);
## Generated for ARP packets that Bro cannot interpret. Examples are packets
## Generated for ARP packets that Zeek cannot interpret. Examples are packets
## with non-standard hardware address formats or hardware addresses that do not
## match the originator of the packet.
##
@ -56,8 +56,8 @@ event arp_reply%(mac_src: string, mac_dst: string, SPA: addr, SHA: string,
##
## .. zeek:see:: arp_reply arp_request
##
## .. todo:: Bro's current default configuration does not activate the protocol
## .. todo:: Zeek's current default configuration does not activate the protocol
## analyzer that generates this event; the corresponding script has not yet
## been ported to Bro 2.x. To still enable this event, one needs to
## been ported. To still enable this event, one needs to
## register a port for it or add a DPD payload signature.
event bad_arp%(SPA: addr, SHA: string, TPA: addr, THA: string, explanation: string%);

10
src/analyzer/protocol/ayiya/CMakeLists.txt
View file

@ -1,9 +1,9 @@
include(BroPlugin)
include(ZeekPlugin)
include_directories(BEFORE ${CMAKE_CURRENT_SOURCE_DIR} ${CMAKE_CURRENT_BINARY_DIR})
bro_plugin_begin(Bro AYIYA)
bro_plugin_cc(AYIYA.cc Plugin.cc)
bro_plugin_pac(ayiya.pac ayiya-protocol.pac ayiya-analyzer.pac)
bro_plugin_end()
zeek_plugin_begin(Zeek AYIYA)
zeek_plugin_cc(AYIYA.cc Plugin.cc)
zeek_plugin_pac(ayiya.pac ayiya-protocol.pac ayiya-analyzer.pac)
zeek_plugin_end()

4
src/analyzer/protocol/ayiya/Plugin.cc
View file

@ -6,7 +6,7 @@
#include "AYIYA.h"
namespace plugin {
namespace Bro_AYIYA {
namespace Zeek_AYIYA {
class Plugin : public plugin::Plugin {
public:
@ -15,7 +15,7 @@ public:
AddComponent(new ::analyzer::Component("AYIYA", ::analyzer::ayiya::AYIYA_Analyzer::Instantiate));
plugin::Configuration config;
config.name = "Bro::AYIYA";
config.name = "Zeek::AYIYA";
config.description = "AYIYA Analyzer";
return config;
}

10
src/analyzer/protocol/backdoor/CMakeLists.txt
View file

@ -1,9 +1,9 @@
include(BroPlugin)
include(ZeekPlugin)
include_directories(BEFORE ${CMAKE_CURRENT_SOURCE_DIR} ${CMAKE_CURRENT_BINARY_DIR})
bro_plugin_begin(Bro BackDoor)
bro_plugin_cc(BackDoor.cc Plugin.cc)
bro_plugin_bif(events.bif)
bro_plugin_end()
zeek_plugin_begin(Zeek BackDoor)
zeek_plugin_cc(BackDoor.cc Plugin.cc)
zeek_plugin_bif(events.bif)
zeek_plugin_end()

4
src/analyzer/protocol/backdoor/Plugin.cc
View file

@ -6,7 +6,7 @@
#include "BackDoor.h"
namespace plugin {
namespace Bro_BackDoor {
namespace Zeek_BackDoor {
class Plugin : public plugin::Plugin {
public:
@ -15,7 +15,7 @@ public:
AddComponent(new ::analyzer::Component("BackDoor", ::analyzer::backdoor::BackDoor_Analyzer::Instantiate));
plugin::Configuration config;
config.name = "Bro::BackDoor";
config.name = "Zeek::BackDoor";
config.description = "Backdoor Analyzer deprecated";
return config;
}

12
src/analyzer/protocol/bittorrent/CMakeLists.txt
View file

@ -1,10 +1,10 @@
include(BroPlugin)
include(ZeekPlugin)
include_directories(BEFORE ${CMAKE_CURRENT_SOURCE_DIR} ${CMAKE_CURRENT_BINARY_DIR})
bro_plugin_begin(Bro BitTorrent)
bro_plugin_cc(BitTorrent.cc BitTorrentTracker.cc Plugin.cc)
bro_plugin_bif(events.bif)
bro_plugin_pac(bittorrent.pac bittorrent-analyzer.pac bittorrent-protocol.pac)
bro_plugin_end()
zeek_plugin_begin(Zeek BitTorrent)
zeek_plugin_cc(BitTorrent.cc BitTorrentTracker.cc Plugin.cc)
zeek_plugin_bif(events.bif)
zeek_plugin_pac(bittorrent.pac bittorrent-analyzer.pac bittorrent-protocol.pac)
zeek_plugin_end()

4
src/analyzer/protocol/bittorrent/Plugin.cc
View file

@ -7,7 +7,7 @@
#include "BitTorrentTracker.h"
namespace plugin {
namespace Bro_BitTorrent {
namespace Zeek_BitTorrent {
class Plugin : public plugin::Plugin {
public:
@ -17,7 +17,7 @@ public:
AddComponent(new ::analyzer::Component("BitTorrentTracker", ::analyzer::bittorrent::BitTorrentTracker_Analyzer::Instantiate));
plugin::Configuration config;
config.name = "Bro::BitTorrent";
config.name = "Zeek::BitTorrent";
config.description = "BitTorrent Analyzer";
return config;
}

12
src/analyzer/protocol/conn-size/CMakeLists.txt
View file

@ -1,10 +1,10 @@
include(BroPlugin)
include(ZeekPlugin)
include_directories(BEFORE ${CMAKE_CURRENT_SOURCE_DIR} ${CMAKE_CURRENT_BINARY_DIR})
bro_plugin_begin(Bro ConnSize)
bro_plugin_cc(ConnSize.cc Plugin.cc)
bro_plugin_bif(events.bif)
bro_plugin_bif(functions.bif)
bro_plugin_end()
zeek_plugin_begin(Zeek ConnSize)
zeek_plugin_cc(ConnSize.cc Plugin.cc)
zeek_plugin_bif(events.bif)
zeek_plugin_bif(functions.bif)
zeek_plugin_end()

4
src/analyzer/protocol/conn-size/Plugin.cc
View file

@ -6,7 +6,7 @@
#include "ConnSize.h"
namespace plugin {
namespace Bro_ConnSize {
namespace Zeek_ConnSize {
class Plugin : public plugin::Plugin {
public:
@ -15,7 +15,7 @@ public:
AddComponent(new ::analyzer::Component("ConnSize", ::analyzer::conn_size::ConnSize_Analyzer::Instantiate));
plugin::Configuration config;
config.name = "Bro::ConnSize";
config.name = "Zeek::ConnSize";
config.description = "Connection size analyzer";
return config;
}

12
src/analyzer/protocol/dce-rpc/CMakeLists.txt
View file

@ -1,12 +1,12 @@
include(BroPlugin)
include(ZeekPlugin)
include_directories(BEFORE ${CMAKE_CURRENT_SOURCE_DIR} ${CMAKE_CURRENT_BINARY_DIR})
bro_plugin_begin(Bro DCE_RPC)
bro_plugin_cc(DCE_RPC.cc Plugin.cc)
bro_plugin_bif(consts.bif types.bif events.bif)
bro_plugin_pac(
zeek_plugin_begin(Zeek DCE_RPC)
zeek_plugin_cc(DCE_RPC.cc Plugin.cc)
zeek_plugin_bif(consts.bif types.bif events.bif)
zeek_plugin_pac(
dce_rpc.pac
dce_rpc-protocol.pac
dce_rpc-analyzer.pac
@ -14,5 +14,5 @@ bro_plugin_pac(
endpoint-atsvc.pac
endpoint-epmapper.pac
)
bro_plugin_end()
zeek_plugin_end()

4
src/analyzer/protocol/dce-rpc/Plugin.cc
View file

@ -6,7 +6,7 @@
#include "DCE_RPC.h"
namespace plugin {
namespace Bro_DCE_RPC {
namespace Zeek_DCE_RPC {
class Plugin : public plugin::Plugin {
public:
@ -15,7 +15,7 @@ public:
AddComponent(new ::analyzer::Component("DCE_RPC", ::analyzer::dce_rpc::DCE_RPC_Analyzer::Instantiate));
plugin::Configuration config;
config.name = "Bro::DCE_RPC";
config.name = "Zeek::DCE_RPC";
config.description = "DCE-RPC analyzer";
return config;
}

14
src/analyzer/protocol/dhcp/CMakeLists.txt
View file

@ -1,11 +1,11 @@
include(BroPlugin)
include(ZeekPlugin)
include_directories(BEFORE ${CMAKE_CURRENT_SOURCE_DIR} ${CMAKE_CURRENT_BINARY_DIR})
bro_plugin_begin(Bro DHCP)
bro_plugin_cc(DHCP.cc Plugin.cc)
bro_plugin_bif(events.bif)
bro_plugin_bif(types.bif)
bro_plugin_pac(dhcp.pac dhcp-protocol.pac dhcp-analyzer.pac dhcp-options.pac)
bro_plugin_end()
zeek_plugin_begin(Zeek DHCP)
zeek_plugin_cc(DHCP.cc Plugin.cc)
zeek_plugin_bif(events.bif)
zeek_plugin_bif(types.bif)
zeek_plugin_pac(dhcp.pac dhcp-protocol.pac dhcp-analyzer.pac dhcp-options.pac)
zeek_plugin_end()

4
src/analyzer/protocol/dhcp/Plugin.cc
View file

@ -6,7 +6,7 @@
#include "DHCP.h"
namespace plugin {
namespace Bro_DHCP {
namespace Zeek_DHCP {
class Plugin : public plugin::Plugin {
public:
@ -15,7 +15,7 @@ public:
AddComponent(new ::analyzer::Component("DHCP", ::analyzer::dhcp::DHCP_Analyzer::Instantiate));
plugin::Configuration config;
config.name = "Bro::DHCP";
config.name = "Zeek::DHCP";
config.description = "DHCP analyzer";
return config;
}

12
src/analyzer/protocol/dnp3/CMakeLists.txt
View file

@ -1,10 +1,10 @@
include(BroPlugin)
include(ZeekPlugin)
include_directories(BEFORE ${CMAKE_CURRENT_SOURCE_DIR} ${CMAKE_CURRENT_BINARY_DIR})
bro_plugin_begin(Bro DNP3)
bro_plugin_cc(DNP3.cc Plugin.cc)
bro_plugin_bif(events.bif)
bro_plugin_pac(dnp3.pac dnp3-analyzer.pac dnp3-protocol.pac dnp3-objects.pac)
bro_plugin_end()
zeek_plugin_begin(Zeek DNP3)
zeek_plugin_cc(DNP3.cc Plugin.cc)
zeek_plugin_bif(events.bif)
zeek_plugin_pac(dnp3.pac dnp3-analyzer.pac dnp3-protocol.pac dnp3-objects.pac)
zeek_plugin_end()

4
src/analyzer/protocol/dnp3/Plugin.cc
View file

@ -6,7 +6,7 @@
#include "DNP3.h"
namespace plugin {
namespace Bro_DNP3 {
namespace Zeek_DNP3 {
class Plugin : public plugin::Plugin {
public:
@ -16,7 +16,7 @@ public:
AddComponent(new ::analyzer::Component("DNP3_UDP", ::analyzer::dnp3::DNP3_UDP_Analyzer::Instantiate));
plugin::Configuration config;
config.name = "Bro::DNP3";
config.name = "Zeek::DNP3";
config.description = "DNP3 UDP/TCP analyzers";
return config;
}

10
src/analyzer/protocol/dns/CMakeLists.txt
View file

@ -1,9 +1,9 @@
include(BroPlugin)
include(ZeekPlugin)
include_directories(BEFORE ${CMAKE_CURRENT_SOURCE_DIR} ${CMAKE_CURRENT_BINARY_DIR})
bro_plugin_begin(Bro DNS)
bro_plugin_cc(DNS.cc Plugin.cc)
bro_plugin_bif(events.bif)
bro_plugin_end()
zeek_plugin_begin(Zeek DNS)
zeek_plugin_cc(DNS.cc Plugin.cc)
zeek_plugin_bif(events.bif)
zeek_plugin_end()

34
src/analyzer/protocol/dns/DNS.cc
View file

@ -281,6 +281,10 @@ int DNS_Interpreter::ParseAnswer(DNS_MsgInfo* msg,
status = ParseRR_TXT(msg, data, len, rdlength, msg_start);
break;
case TYPE_SPF:
status = ParseRR_SPF(msg, data, len, rdlength, msg_start);
break;
case TYPE_CAA:
status = ParseRR_CAA(msg, data, len, rdlength, msg_start);
break;
@ -1321,6 +1325,36 @@ int DNS_Interpreter::ParseRR_TXT(DNS_MsgInfo* msg,
return rdlength == 0;
}
int DNS_Interpreter::ParseRR_SPF(DNS_MsgInfo* msg,
const u_char*& data, int& len, int rdlength,
const u_char* msg_start)
{
if ( ! dns_SPF_reply || msg->skip_event )
{
data += rdlength;
len -= rdlength;
return 1;
}
VectorVal* char_strings = new VectorVal(string_vec);
StringVal* char_string;
while ( (char_string = extract_char_string(analyzer, data, len, rdlength)) )
char_strings->Assign(char_strings->Size(), char_string);
if ( dns_SPF_reply )
analyzer->ConnectionEventFast(dns_SPF_reply, {
analyzer->BuildConnVal(),
msg->BuildHdrVal(),
msg->BuildAnswerVal(),
char_strings,
});
else
Unref(char_strings);
return rdlength == 0;
}
int DNS_Interpreter::ParseRR_CAA(DNS_MsgInfo* msg,
const u_char*& data, int& len, int rdlength,
const u_char* msg_start)

5
src/analyzer/protocol/dns/DNS.h
View file

@ -63,6 +63,8 @@ typedef enum {
TYPE_DNSKEY = 48, ///< DNS Key record (RFC 4034)
TYPE_DS = 43, ///< Delegation signer (RFC 4034)
TYPE_NSEC3 = 50,
// Obsoleted
TYPE_SPF = 99, ///< Alternative: storing SPF data in TXT records, using the same format (RFC 4408). Support for it was discontinued in RFC 7208
// The following are only valid in queries.
TYPE_AXFR = 252,
TYPE_ALL = 255,
@ -282,6 +284,9 @@ protected:
int ParseRR_TXT(DNS_MsgInfo* msg,
const u_char*& data, int& len, int rdlength,
const u_char* msg_start);
int ParseRR_SPF(DNS_MsgInfo* msg,
const u_char*& data, int& len, int rdlength,
const u_char* msg_start);
int ParseRR_CAA(DNS_MsgInfo* msg,
const u_char*& data, int& len, int rdlength,
const u_char* msg_start);

Some files were not shown because too many files have changed in this diff Show more