Mirror/zeek
1
0
Fork 0
mirror of https://github.com/zeek/zeek.git synced 2025-10-02 06:38:20 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 1

Merge remote-tracking branch 'origin/topic/johanna/243'

Browse source 
* origin/topic/johanna/243:
  Fix missing include file on Linux
  Deprecations: Update NEWS, and tie up a few loose ends.
  Remove support for using &&/|| with patterns.
  Remove RemoteSerializer and related code/types.
  Remove PersistenceSerializer.
  Remove synchrnized and persistent attributes.
This commit is contained in:
Jon Siwek 2019-05-06 18:43:13 -07:00
commit 9029acd7e5
72 changed files with 151 additions and 6770 deletions
Download patch file Download diff file Expand all files Collapse all files

3
.gitmodules vendored
View file

@ -4,9 +4,6 @@
[submodule "aux/binpac"]
path = aux/binpac
url = https://github.com/zeek/binpac
[submodule "aux/broccoli"]
path = aux/broccoli
url = https://github.com/zeek/broccoli
[submodule "aux/broctl"]
path = aux/broctl
url = https://github.com/zeek/broctl

14
CHANGES
View file

@ -1,4 +1,18 @@
2.6-272 | 2019-05-06 18:43:13 -0700
* Remove support for using && and || with patterns. (Johanna Amann, Corelight)
This was never documented and previously deprecated.
* Remove RemoteSerializer and related code/types. (Johanna Amann, Corelight)
Also removes broccoli from the source tree.
* Remove PersistenceSerializer. (Johanna Amann, Corelight)
* Remove &synchronized and &persistent attributes. (Johanna Amann, Corelight)
2.6-264 | 2019-05-03 11:16:38 -0700
* Fix sporadic openflow/broker test failure (Jon Siwek, Corelight)

2
CMakeLists.txt
View file

@ -326,7 +326,6 @@ include(CheckOptionalBuildSources)
CheckOptionalBuildSources(aux/broctl Broctl INSTALL_BROCTL)
CheckOptionalBuildSources(aux/bro-aux Bro-Aux INSTALL_AUX_TOOLS)
CheckOptionalBuildSources(aux/broccoli Broccoli INSTALL_BROCCOLI)
########################################################################
## Packaging Setup
@ -366,7 +365,6 @@ message(
"\nCXXFLAGS: ${CMAKE_CXX_FLAGS} ${CMAKE_CXX_FLAGS_${BuildType}}"
"\nCPP: ${CMAKE_CXX_COMPILER}"
"\n"
"\nBroccoli: ${INSTALL_BROCCOLI}"
"\nBroctl: ${INSTALL_BROCTL}"
"\nAux. Tools: ${INSTALL_AUX_TOOLS}"
"\n"

49
NEWS
View file

@ -244,11 +244,55 @@ Removed Functionality
- ``dhcp_offer``
- ``dhcp_release``
- ``dhcp_request``
- ``remote_state_access_performed``
- ``remote_state_inconsistency``
- ``remote_connection_established``
- ``remote_connection_closed``
- ``remote_connection_handshake_done``
- ``remote_event_registered``
- ``remote_connection_error``
- ``remote_capture_filter``
- ``remote_log_peer``
- ``remote_log``
- ``finished_send_state``
- ``remote_pong``
- The following types/records were deprecated in version 2.6 or below and are
removed from this release:
- ``peer_id``
- ``event_peer``
- The following configuration options were deprecated in version 2.6 or below and are
removed from this release:
- ``max_remote_events_processed``
- ``forward_remote_events``
- ``forward_remote_state_changes``
- ``enable_syslog``
- ``remote_trace_sync_interval``
- ``remote_trace_sync_peers``
- ``remote_check_sync_consistency``
- The following constants were used as part of deprecated functionality in version 2.6
or below and are removed from this release:
- ``PEER_ID_NONE``
- ``REMOTE_LOG_INFO``
- ``REMOTE_SRC_CHILD``
- ``REMOTE_SRC_PARENT``
- ``REMOTE_SRC_SCRIPT``
- The deprecated script ``policy/protocols/smb/__load__.bro`` was removed.
Instead of ``@load policy/protocols/smb`` use ``@load base/protocols/smb``.
- Broccoli, which had been deprecated in version 2.6 and was no longer built by default
was removed from the source tree.
- Support for the &persistent and the &synchronized attributes, which were deprecated
in Bro 2.6, was removed. The ``-g`` command-line option (dump-config) which relied on
this functionality was also removed.
Deprecated Functionality
------------------------
@ -263,6 +307,11 @@ Deprecated Functionality
such that existing code will not break, but will emit a deprecation
warning.
- The ``rotate_file``, ``rotate_file_by_name`` and ``calc_next_rotate`` functions
were marked as deprecated. These functions were used with the old pre-2.0 logging
framework and are no longer used. They also were marked as deprecated in their
documentation, however the functions themselves did not carry the deprecation marker.
Bro 2.6
=======

2
VERSION
View file

@ -1 +1 @@
2.6-264
2.6-272

1
aux/broccoli

@ -1 +0,0 @@
Subproject commit 8668422406cb74f4f0c574a0c9b6365a21f3e81a

6
configure vendored
View file

@ -51,7 +51,6 @@ Usage: $0 [OPTION]... [VAR=VALUE]...
(automatically on when perftools is present on Linux)
--enable-perftools-debug use Google's perftools for debugging
--enable-jemalloc link against jemalloc
--enable-broccoli build or install the Broccoli library (deprecated)
--enable-static-broker build broker statically (ignored if --with-broker is specified)
--enable-static-binpac build binpac statically (ignored if --with-binpac is specified)
--disable-broctl don't install Broctl
@ -140,7 +139,6 @@ append_cache_entry ENABLE_PERFTOOLS BOOL false
append_cache_entry ENABLE_PERFTOOLS_DEBUG BOOL false
append_cache_entry ENABLE_JEMALLOC BOOL false
append_cache_entry BUILD_SHARED_LIBS BOOL true
append_cache_entry INSTALL_BROCCOLI BOOL false
append_cache_entry INSTALL_AUX_TOOLS BOOL true
append_cache_entry INSTALL_BROCTL BOOL true
append_cache_entry CPACK_SOURCE_IGNORE_FILES STRING
@ -221,10 +219,6 @@ while [ $# -ne 0 ]; do
--enable-jemalloc)
append_cache_entry ENABLE_JEMALLOC BOOL true
;;
--enable-broccoli)
append_cache_entry DISABLE_RUBY_BINDINGS BOOL true
append_cache_entry INSTALL_BROCCOLI BOOL yes
;;
--enable-static-broker)
append_cache_entry BUILD_STATIC_BROKER BOOL true
;;

2
doc

@ -1 +1 @@
Subproject commit 8aa690e20d19f79805d7f680e454e4ea10231add
Subproject commit 736323fe8a78fd8478325c134afe38c075e297a7

3
man/bro.8
View file

@ -36,9 +36,6 @@ augment loaded policies by given code
\fB\-f\fR,\ \-\-filter <filter>
tcpdump filter
.TP
\fB\-g\fR,\ \-\-dump\-config
dump current config into .state dir
.TP
\fB\-h\fR,\ \-\-help|\-?
command line help
.TP

3
scripts/base/frameworks/cluster/nodes/logger.zeek
View file

@ -24,6 +24,3 @@ redef Log::default_mail_alarms_interval = 24 hrs;
## Use the cluster's archive logging script.
redef Log::default_rotation_postprocessor_cmd = "archive-log";
## We're processing essentially *only* remote events.
redef max_remote_events_processed = 10000;

3
scripts/base/frameworks/cluster/nodes/manager.zeek
View file

@ -21,6 +21,3 @@ redef Log::default_rotation_interval = 24 hrs;
## Use the cluster's delete-log script.
redef Log::default_rotation_postprocessor_cmd = "delete-log";
## We're processing essentially *only* remote events.
redef max_remote_events_processed = 10000;

4
scripts/base/frameworks/cluster/nodes/proxy.zeek
View file

@ -5,10 +5,6 @@
@prefixes += cluster-proxy
## The proxy only syncs state; does not forward events.
redef forward_remote_events = F;
redef forward_remote_state_changes = T;
## Don't do any local logging.
redef Log::enable_local_logging = F;

6
scripts/base/frameworks/packet-filter/cluster.zeek
View file

@ -4,11 +4,11 @@
module PacketFilter;
event remote_connection_handshake_done(p: event_peer) &priority=3
event Cluster::hello(name: string, id: string) &priority=-3
{
if ( Cluster::local_node_type() == Cluster::WORKER &&
p$descr in Cluster::nodes &&
Cluster::nodes[p$descr]$node_type == Cluster::MANAGER )
name in Cluster::nodes &&
Cluster::nodes[name]$node_type == Cluster::MANAGER )
{
# This ensures that a packet filter is installed and logged
# after the manager connects to us.

83
scripts/base/init-bare.zeek
View file

@ -775,29 +775,6 @@ type IPAddrAnonymizationClass: enum {
OTHER_ADDR,
};
## A locally unique ID identifying a communication peer.
##
type peer_id: count;
## A communication peer.
##
## .. zeek:see:: remote_capture_filter
## remote_connection_closed remote_connection_error
## remote_connection_established remote_connection_handshake_done
## remote_event_registered remote_log_peer remote_pong
##
## .. todo::The type's name is too narrow these days, should rename.
type event_peer: record {
id: peer_id; ##< Locally unique ID of peer
host: addr; ##< The IP address of the peer.
## Either the port we connected to at the peer; or our port the peer
## connected to if the session is remotely initiated.
p: port;
is_local: bool; ##< True if this record describes the local process.
descr: string; ##< The peer's :zeek:see:`peer_description`.
class: string &optional; ##< The self-assigned *class* of the peer.
};
## Deprecated.
##
## .. zeek:see:: rotate_file rotate_file_by_name rotate_interval
@ -1967,10 +1944,6 @@ const watchdog_interval = 10 sec &redef;
## "process all expired timers with each new packet".
const max_timer_expires = 300 &redef;
## With a similar trade-off, this gives the number of remote events
## to process in a batch before interleaving other activity.
const max_remote_events_processed = 10 &redef;
# These need to match the definitions in Login.h.
#
# .. zeek:see:: get_login_state
@ -4737,70 +4710,14 @@ const packet_filter_default = F &redef;
## Maximum size of regular expression groups for signature matching.
const sig_max_group_size = 50 &redef;
## Deprecated. No longer functional.
const enable_syslog = F &redef;
## Description transmitted to remote communication peers for identification.
const peer_description = "bro" &redef;
## If true, broadcast events received from one peer to all other peers.
##
## .. zeek:see:: forward_remote_state_changes
##
## .. note:: This option is only temporary and will disappear once we get a
## more sophisticated script-level communication framework.
const forward_remote_events = F &redef;
## If true, broadcast state updates received from one peer to all other peers.
##
## .. zeek:see:: forward_remote_events
##
## .. note:: This option is only temporary and will disappear once we get a
## more sophisticated script-level communication framework.
const forward_remote_state_changes = F &redef;
## The number of IO chunks allowed to be buffered between the child
## and parent process of remote communication before Bro starts dropping
## connections to remote peers in an attempt to catch up.
const chunked_io_buffer_soft_cap = 800000 &redef;
## Place-holder constant indicating "no peer".
const PEER_ID_NONE = 0;
# Signature payload pattern types.
# todo:: use enum to help autodoc
# todo:: Still used?
#const SIG_PATTERN_PAYLOAD = 0;
#const SIG_PATTERN_HTTP = 1;
#const SIG_PATTERN_FTP = 2;
#const SIG_PATTERN_FINGER = 3;
# Deprecated.
# todo::Should use the new logging framework directly.
const REMOTE_LOG_INFO = 1; ##< Deprecated.
const REMOTE_LOG_ERROR = 2; ##< Deprecated.
# Source of logging messages from the communication framework.
# todo:: these should go into an enum to make them autodoc'able.
const REMOTE_SRC_CHILD = 1; ##< Message from the child process.
const REMOTE_SRC_PARENT = 2; ##< Message from the parent process.
const REMOTE_SRC_SCRIPT = 3; ##< Message from a policy script.
## Synchronize trace processing at a regular basis in pseudo-realtime mode.
##
## .. zeek:see:: remote_trace_sync_peers
const remote_trace_sync_interval = 0 secs &redef;
## Number of peers across which to synchronize trace processing in
## pseudo-realtime mode.
##
## .. zeek:see:: remote_trace_sync_interval
const remote_trace_sync_peers = 0 &redef;
## Whether for :zeek:attr:`&synchronized` state to send the old value as a
## consistency check.
const remote_check_sync_consistency = F &redef;
## Reassemble the beginning of all TCP connections before doing
## signature matching. Enabling this provides more accurate matching at the
## expense of CPU cycles.

6
src/Attr.cc
View file

@ -14,7 +14,6 @@ const char* attr_name(attr_tag t)
"&rotate_interval", "&rotate_size",
"&add_func", "&delete_func", "&expire_func",
"&read_expire", "&write_expire", "&create_expire",
"&persistent", "&synchronized",
"&encrypt",
"&raw_output", "&mergeable", "&priority",
"&group", "&log", "&error_handler", "&type_column",
@ -438,8 +437,6 @@ void Attributes::CheckAttr(Attr* a)
}
break;
case ATTR_PERSISTENT:
case ATTR_SYNCHRONIZED:
case ATTR_TRACKED:
// FIXME: Check here for global ID?
break;
@ -559,8 +556,7 @@ bool Attributes::DoSerialize(SerialInfo* info) const
{
Attr* a = (*attrs)[i];
// Broccoli doesn't support expressions.
Expr* e = (! info->broccoli_peer) ? a->AttrExpr() : 0;
Expr* e = a->AttrExpr();
SERIALIZE_OPTIONAL(e);
if ( ! SERIALIZE(char(a->Tag())) )

2
src/Attr.h
View file

@ -23,8 +23,6 @@ typedef enum {
ATTR_EXPIRE_READ,
ATTR_EXPIRE_WRITE,
ATTR_EXPIRE_CREATE,
ATTR_PERSISTENT,
ATTR_SYNCHRONIZED,
ATTR_ENCRYPT,
ATTR_RAW_OUTPUT,
ATTR_MERGEABLE,

2
src/CMakeLists.txt
View file

@ -287,7 +287,6 @@ set(bro_SRCS
OpaqueVal.cc
OSFinger.cc
PacketFilter.cc
PersistenceSerializer.cc
Pipe.cc
PolicyFile.cc
PrefixTable.cc
@ -296,7 +295,6 @@ set(bro_SRCS
RandTest.cc
RE.cc
Reassem.cc
RemoteSerializer.cc
Rule.cc
RuleAction.cc
RuleCondition.cc

3
src/ChunkedIO.cc
View file

@ -14,7 +14,6 @@
#include "bro-config.h"
#include "ChunkedIO.h"
#include "NetVar.h"
#include "RemoteSerializer.h"
ChunkedIO::ChunkedIO() : stats(), tag(), pure()
{
@ -377,7 +376,7 @@ ChunkedIO::Chunk* ChunkedIOFd::ConcatChunks(Chunk* c1, Chunk* c2)
void ChunkedIO::Log(const char* str)
{
RemoteSerializer::Log(RemoteSerializer::LogError, str);
//RemoteSerializer::Log(RemoteSerializer::LogError, str);
}
bool ChunkedIOFd::Read(Chunk** chunk, bool may_block)

8
src/Conn.cc
View file

@ -151,7 +151,6 @@ Connection::Connection(NetSessions* s, HashKey* k, double t, const ConnID* id,
is_active = 1;
skip = 0;
weird = 0;
persistent = 0;
suppress_event = 0;
@ -951,15 +950,11 @@ bool Connection::DoSerialize(SerialInfo* info) const
SERIALIZE_BIT(weird) &&
SERIALIZE_BIT(finished) &&
SERIALIZE_BIT(record_packets) &&
SERIALIZE_BIT(record_contents) &&
SERIALIZE_BIT(persistent);
SERIALIZE_BIT(record_contents);
}
bool Connection::DoUnserialize(UnserialInfo* info)
{
// Make sure this is initialized for the condition in Unserialize().
persistent = 0;
DO_UNSERIALIZE(BroObj);
// Build the hash key first. Some of the recursive *::Unserialize()
@ -1022,7 +1017,6 @@ bool Connection::DoUnserialize(UnserialInfo* info)
UNSERIALIZE_BIT(finished);
UNSERIALIZE_BIT(record_packets);
UNSERIALIZE_BIT(record_contents);
UNSERIALIZE_BIT(persistent);
// Hmm... Why does each connection store a sessions ptr?
sessions = ::sessions;

12
src/Conn.h
View file

@ -12,7 +12,6 @@
#include "Val.h"
#include "Timer.h"
#include "Serializer.h"
#include "PersistenceSerializer.h"
#include "RuleMatcher.h"
#include "IPAddr.h"
#include "TunnelEncapsulation.h"
@ -228,14 +227,6 @@ public:
return 1;
}
void MakePersistent()
{
persistent = 1;
persistence_serializer->Register(this);
}
bool IsPersistent() { return persistent; }
void Describe(ODesc* d) const override;
void IDString(ODesc* d) const;
@ -315,7 +306,7 @@ public:
protected:
Connection() { persistent = 0; }
Connection() { }
// Add the given timer to expire at time t. If do_expire
// is true, then the timer is also evaluated when Bro terminates,
@ -361,7 +352,6 @@ protected:
unsigned int weird:1;
unsigned int finished:1;
unsigned int record_packets:1, record_contents:1;
unsigned int persistent:1;
unsigned int record_current_packet:1, record_current_content:1;
unsigned int saw_first_orig_packet:1, saw_first_resp_packet:1;

4
src/DebugLogger.cc
View file

@ -11,9 +11,9 @@ DebugLogger debug_logger;
// Same order here as in DebugStream.
DebugLogger::Stream DebugLogger::streams[NUM_DBGS] = {
{ "serial", 0, false }, { "rules", 0, false }, { "comm", 0, false },
{ "serial", 0, false }, { "rules", 0, false },
{ "state", 0, false }, { "chunkedio", 0, false },
{ "compressor", 0, false }, {"string", 0, false },
{"string", 0, false },
{ "notifiers", 0, false }, { "main-loop", 0, false },
{ "dpd", 0, false }, { "tm", 0, false },
{ "logging", 0, false }, {"input", 0, false },

2
src/DebugLogger.h
View file

@ -16,10 +16,8 @@
enum DebugStream {
DBG_SERIAL, // Serialization
DBG_RULES, // Signature matching
DBG_COMM, // Remote communication
DBG_STATE, // StateAccess logging
DBG_CHUNKEDIO, // ChunkedIO logging
DBG_COMPRESSOR, // Connection compressor
DBG_STRING, // String code
DBG_NOTIFIERS, // Notifiers (see StateAccess.h)
DBG_MAINLOOP, // Main IOSource loop

18
src/Event.cc
View file

@ -189,21 +189,3 @@ void EventMgr::Describe(ODesc* d) const
d->NL();
}
}
RecordVal* EventMgr::GetLocalPeerVal()
{
if ( ! src_val )
{
src_val = new RecordVal(peer);
src_val->Assign(0, val_mgr->GetCount(0));
src_val->Assign(1, new AddrVal("127.0.0.1"));
src_val->Assign(2, val_mgr->GetPort(0));
src_val->Assign(3, val_mgr->GetTrue());
Ref(peer_description);
src_val->Assign(4, peer_description);
src_val->Assign(5, 0); // class (optional).
}
return src_val;
}

3
src/Event.h
View file

@ -129,9 +129,6 @@ public:
int Size() const
{ return num_events_queued - num_events_dispatched; }
// Returns a peer record describing the local Bro.
RecordVal* GetLocalPeerVal();
void Describe(ODesc* d) const override;
protected:

18
src/EventHandler.cc
View file

@ -2,7 +2,6 @@
#include "EventHandler.h"
#include "Func.h"
#include "Scope.h"
#include "RemoteSerializer.h"
#include "NetVar.h"
#include "broker/Manager.h"
@ -28,7 +27,6 @@ EventHandler::~EventHandler()
EventHandler::operator bool() const
{
return enabled && ((local && local->HasBodies())
|| receivers.length()
|| generate_always
|| ! auto_publish.empty());
}
@ -73,12 +71,6 @@ void EventHandler::Call(val_list* vl, bool no_remote)
if ( ! no_remote )
{
loop_over_list(receivers, i)
{
SerialInfo info(remote_serializer);
remote_serializer->SendCall(&info, receivers[i], name, vl);
}
if ( ! auto_publish.empty() )
{
// Send event in form [name, xs...] where xs represent the arguments.
@ -179,16 +171,6 @@ void EventHandler::NewEvent(val_list* vl)
mgr.Dispatch(ev);
}
void EventHandler::AddRemoteHandler(SourceID peer)
{
receivers.append(peer);
}
void EventHandler::RemoveRemoteHandler(SourceID peer)
{
receivers.remove(peer);
}
bool EventHandler::Serialize(SerialInfo* info) const
{
return SERIALIZE(name);

7
src/EventHandler.h
View file

@ -26,9 +26,6 @@ public:
void SetLocalHandler(Func* f);
void AddRemoteHandler(SourceID peer);
void RemoveRemoteHandler(SourceID peer);
void AutoPublish(std::string topic)
{
auto_publish.insert(std::move(topic));
@ -75,10 +72,6 @@ private:
bool error_handler; // this handler reports error messages.
bool generate_always;
declare(List, SourceID);
typedef List(SourceID) receiver_list;
receiver_list receivers;
std::unordered_set<std::string> auto_publish;
};

2
src/EventRegistry.cc
View file

@ -1,6 +1,6 @@
#include "EventRegistry.h"
#include "RE.h"
#include "RemoteSerializer.h"
#include "Reporter.h"
void EventRegistry::Register(EventHandlerPtr handler)
{

8
src/Expr.cc
View file

@ -10,7 +10,6 @@
#include "Scope.h"
#include "Stmt.h"
#include "EventRegistry.h"
#include "RemoteSerializer.h"
#include "Net.h"
#include "Traverse.h"
#include "Trigger.h"
@ -1884,13 +1883,6 @@ BoolExpr::BoolExpr(BroExprTag arg_tag, Expr* arg_op1, Expr* arg_op2)
else
SetType(base_type(TYPE_BOOL));
}
else if ( bt1 == TYPE_PATTERN && bt2 == bt1 )
{
reporter->Warning("&& and || operators deprecated for pattern operands");
SetType(base_type(TYPE_PATTERN));
}
else
ExprError("requires boolean operands");
}

1
src/Func.cc
View file

@ -42,7 +42,6 @@
#include "Sessions.h"
#include "RE.h"
#include "Serializer.h"
#include "RemoteSerializer.h"
#include "Event.h"
#include "Traverse.h"
#include "Reporter.h"

48
src/ID.cc
View file

@ -10,8 +10,6 @@
#include "Scope.h"
#include "File.h"
#include "Serializer.h"
#include "RemoteSerializer.h"
#include "PersistenceSerializer.h"
#include "Scope.h"
#include "Traverse.h"
#include "zeekygen/Manager.h"
@ -78,12 +76,6 @@ void ID::SetVal(Val* v, Opcode op, bool arg_weak_ref)
MutableVal::Properties props = 0;
if ( attrs && attrs->FindAttr(ATTR_SYNCHRONIZED) )
props |= MutableVal::SYNCHRONIZED;
if ( attrs && attrs->FindAttr(ATTR_PERSISTENT) )
props |= MutableVal::PERSISTENT;
if ( attrs && attrs->FindAttr(ATTR_TRACKED) )
props |= MutableVal::TRACKED;
@ -198,27 +190,12 @@ void ID::UpdateValAttrs()
if ( val && val->IsMutableVal() )
{
if ( attrs->FindAttr(ATTR_SYNCHRONIZED) )
props |= MutableVal::SYNCHRONIZED;
if ( attrs->FindAttr(ATTR_PERSISTENT) )
props |= MutableVal::PERSISTENT;
if ( attrs->FindAttr(ATTR_TRACKED) )
props |= MutableVal::TRACKED;
val->AsMutableVal()->AddProperties(props);
}
if ( ! IsInternalGlobal() )
{
if ( attrs->FindAttr(ATTR_SYNCHRONIZED) )
remote_serializer->Register(this);
if ( attrs->FindAttr(ATTR_PERSISTENT) )
persistence_serializer->Register(this);
}
if ( val && val->Type()->Tag() == TYPE_TABLE )
val->AsTableVal()->SetAttrs(attrs);
@ -281,12 +258,6 @@ void ID::RemoveAttr(attr_tag a)
{
MutableVal::Properties props = 0;
if ( a == ATTR_SYNCHRONIZED )
props |= MutableVal::SYNCHRONIZED;
if ( a == ATTR_PERSISTENT )
props |= MutableVal::PERSISTENT;
if ( a == ATTR_TRACKED )
props |= MutableVal::TRACKED;
@ -337,9 +308,6 @@ void ID::CopyFrom(const ID* id)
offset = id->offset ;
infer_return_type = id->infer_return_type;
if ( FindAttr(ATTR_PERSISTENT) )
persistence_serializer->Unregister(this);
if ( id->type )
Ref(id->type);
if ( id->val && ! id->weak_ref )
@ -360,10 +328,6 @@ void ID::CopyFrom(const ID* id)
#ifdef DEBUG
UpdateValID();
#endif
if ( FindAttr(ATTR_PERSISTENT) )
persistence_serializer->Unregister(this);
}
#endif
ID* ID::Unserialize(UnserialInfo* info)
@ -396,12 +360,6 @@ ID* ID::Unserialize(UnserialInfo* info)
else
{
if ( info->id_policy != UnserialInfo::InstantiateNew )
{
persistence_serializer->Unregister(current);
remote_serializer->Unregister(current);
}
switch ( info->id_policy ) {
case UnserialInfo::Keep:
@ -473,12 +431,6 @@ ID* ID::Unserialize(UnserialInfo* info)
}
}
if ( id->FindAttr(ATTR_PERSISTENT) )
persistence_serializer->Register(id);
if ( id->FindAttr(ATTR_SYNCHRONIZED) )
remote_serializer->Register(id);
return id;
}

9
src/Net.cc
View file

@ -49,8 +49,6 @@ int reading_live = 0;
int reading_traces = 0;
int have_pending_timers = 0;
double pseudo_realtime = 0.0;
bool using_communication = false;
double network_time = 0.0; // time according to last packet timestamp
// (or current time)
double processing_start_time = 0.0; // time started working on current pkt
@ -309,7 +307,7 @@ void net_run()
}
#endif
current_iosrc = src;
auto communication_enabled = using_communication || broker_mgr->Active();
auto communication_enabled = broker_mgr->Active();
if ( src )
src->Process(); // which will call net_packet_dispatch()
@ -372,11 +370,6 @@ void net_run()
// current packet and its related events.
termination_signal();
#ifdef DEBUG_COMMUNICATION
if ( signal_val == SIGPROF && remote_serializer )
remote_serializer->DumpDebugData();
#endif
if ( ! reading_traces )
// Check whether we have timers scheduled for
// the future on which we need to wait.

4
src/Net.h
View file

@ -7,7 +7,6 @@
#include "util.h"
#include "List.h"
#include "Func.h"
#include "RemoteSerializer.h"
#include "iosource/IOSource.h"
#include "iosource/PktSrc.h"
#include "iosource/PktDumper.h"
@ -67,9 +66,6 @@ extern double bro_start_network_time;
// True if we're a in the process of cleaning-up just before termination.
extern bool terminating;
// True if the remote serializer is to be activated.
extern bool using_communication;
// True if Bro is currently parsing scripts.
extern bool is_parsing;

23
src/NetVar.cc
View file

@ -30,7 +30,6 @@ RecordType* mime_match;
int watchdog_interval;
int max_timer_expires;
int max_remote_events_processed;
int ignore_checksums;
int partial_connection_ok;
@ -173,10 +172,6 @@ StringVal* log_encryption_key;
StringVal* log_rotate_base_time;
StringVal* peer_description;
RecordType* peer;
int forward_remote_state_changes;
int forward_remote_events;
int remote_check_sync_consistency;
bro_uint_t chunked_io_buffer_soft_cap;
StringVal* ssl_ca_certificate;
@ -199,8 +194,6 @@ int packet_filter_default;
int sig_max_group_size;
int enable_syslog;
TableType* irc_join_list;
RecordType* irc_join_info;
TableVal* irc_servers;
@ -212,9 +205,6 @@ int dpd_ignore_ports;
TableVal* likely_server_ports;
double remote_trace_sync_interval;
int remote_trace_sync_peers;
int check_for_unused_event_handlers;
int dump_used_event_handlers;
@ -267,12 +257,6 @@ void init_general_global_var()
peer_description =
internal_val("peer_description")->AsStringVal();
peer = internal_type("event_peer")->AsRecordType();
forward_remote_state_changes =
opt_internal_int("forward_remote_state_changes");
forward_remote_events = opt_internal_int("forward_remote_events");
remote_check_sync_consistency =
opt_internal_int("remote_check_sync_consistency");
chunked_io_buffer_soft_cap = opt_internal_unsigned("chunked_io_buffer_soft_cap");
ssl_ca_certificate = internal_val("ssl_ca_certificate")->AsStringVal();
@ -282,7 +266,6 @@ void init_general_global_var()
packet_filter_default = opt_internal_int("packet_filter_default");
sig_max_group_size = opt_internal_int("sig_max_group_size");
enable_syslog = opt_internal_int("enable_syslog");
check_for_unused_event_handlers =
opt_internal_int("check_for_unused_event_handlers");
@ -395,8 +378,6 @@ void init_net_var()
watchdog_interval = int(opt_internal_double("watchdog_interval"));
max_timer_expires = opt_internal_int("max_timer_expires");
max_remote_events_processed =
opt_internal_int("max_remote_events_processed");
skip_authentication = internal_list_val("skip_authentication");
direct_login_prompts = internal_list_val("direct_login_prompts");
@ -498,10 +479,6 @@ void init_net_var()
irc_join_list = internal_type("irc_join_list")->AsTableType();
irc_servers = internal_val("irc_servers")->AsTableVal();
remote_trace_sync_interval =
opt_internal_double("remote_trace_sync_interval");
remote_trace_sync_peers = opt_internal_int("remote_trace_sync_peers");
dpd_reassemble_first_packets =
opt_internal_int("dpd_reassemble_first_packets");
dpd_buffer_size = opt_internal_int("dpd_buffer_size");

10
src/NetVar.h
View file

@ -33,7 +33,6 @@ extern RecordType* mime_match;
extern int watchdog_interval;
extern int max_timer_expires;
extern int max_remote_events_processed;
extern int ignore_checksums;
extern int partial_connection_ok;
@ -176,10 +175,6 @@ extern StringVal* log_encryption_key;
extern StringVal* log_rotate_base_time;
extern StringVal* peer_description;
extern RecordType* peer;
extern int forward_remote_state_changes;
extern int forward_remote_events;
extern int remote_check_sync_consistency;
extern bro_uint_t chunked_io_buffer_soft_cap;
extern StringVal* ssl_ca_certificate;
@ -201,8 +196,6 @@ extern int packet_filter_default;
extern int sig_max_group_size;
extern int enable_syslog;
extern TableType* irc_join_list;
extern RecordType* irc_join_info;
extern TableVal* irc_servers;
@ -214,9 +207,6 @@ extern int dpd_ignore_ports;
extern TableVal* likely_server_ports;
extern double remote_trace_sync_interval;
extern int remote_trace_sync_peers;
extern int check_for_unused_event_handlers;
extern int dump_used_event_handlers;

577
src/PersistenceSerializer.cc
View file

@ -1,577 +0,0 @@
#include <fcntl.h>
#include <unistd.h>
#include <errno.h>
#include <time.h>
#include <dirent.h>
#include <libgen.h>
#include <sys/time.h>
#include <sys/stat.h>
#include "PersistenceSerializer.h"
#include "RemoteSerializer.h"
#include "Conn.h"
#include "Event.h"
#include "Reporter.h"
#include "Net.h"
static void persistence_serializer_delete_func(void* val)
{
time_t* t = reinterpret_cast<time_t*>(val);
free(t);
}
class IncrementalWriteTimer : public Timer {
public:
IncrementalWriteTimer(double t, PersistenceSerializer::SerialStatus* s)
: Timer(t, TIMER_INCREMENTAL_WRITE), status(s) {}
void Dispatch(double t, int is_expire);
PersistenceSerializer::SerialStatus* status;
};
void IncrementalWriteTimer::Dispatch(double t, int is_expire)
{
// Never suspend when we're finishing up.
if ( terminating )
status->info.may_suspend = false;
persistence_serializer->RunSerialization(status);
}
PersistenceSerializer::PersistenceSerializer()
{
dir = 0;
files.SetDeleteFunc(persistence_serializer_delete_func);
}
PersistenceSerializer::~PersistenceSerializer()
{
}
void PersistenceSerializer::Register(ID* id)
{
if ( id->Type()->Tag() == TYPE_FUNC )
{
Error("can't register functions as persistent ID");
return;
}
DBG_LOG(DBG_STATE, "&persistent %s", id->Name());
HashKey key(id->Name());
if ( persistent_ids.Lookup(&key) )
return;
Ref(id);
persistent_ids.Insert(&key, id);
}
void PersistenceSerializer::Unregister(ID* id)
{
HashKey key(id->Name());
Unref((ID*) persistent_ids.Remove(&key));
}
void PersistenceSerializer::Register(Connection* conn)
{
if ( persistent_conns.Lookup(conn->Key()) )
return;
Ref(conn);
HashKey* k = conn->Key();
HashKey* new_key = new HashKey(k->Key(), k->Size(), k->Hash());
persistent_conns.Insert(new_key, conn);
delete new_key;
}
void PersistenceSerializer::Unregister(Connection* conn)
{
Unref(persistent_conns.RemoveEntry(conn->Key()));
}
bool PersistenceSerializer::CheckTimestamp(const char* file)
{
struct stat s;
if ( stat(file, &s) < 0 )
return false;
if ( ! S_ISREG(s.st_mode) )
return false;
bool changed = true;
HashKey* key = new HashKey(file, strlen(file));
time_t* t = files.Lookup(key);
if ( ! t )
{
t = (time_t*) malloc(sizeof(time_t));
if ( ! t )
out_of_memory("saving file timestamp");
files.Insert(key, t);
}
else if ( *t >= s.st_mtime )
changed = false;
*t = s.st_mtime;
delete key;
return changed;
}
bool PersistenceSerializer::CheckForFile(UnserialInfo* info, const char* file,
bool delete_file)
{
bool ret = true;
if ( CheckTimestamp(file) )
{
// Need to copy the filename here, as it may be passed
// in via fmt().
const char* f = copy_string(file);
bool ret = Read(info, f);
if ( delete_file && unlink(f) < 0 )
Error(fmt("can't delete file %s: %s", f, strerror(errno)));
delete [] f;
}
return ret;
}
bool PersistenceSerializer::ReadAll(bool is_init, bool delete_files)
{
#ifdef USE_PERFTOOLS_DEBUG
HeapLeakChecker::Disabler disabler;
#endif
assert(dir);
UnserialInfo config_info(this);
config_info.id_policy = is_init ?
UnserialInfo::Replace : UnserialInfo::CopyCurrentToNew;
if ( ! CheckForFile(&config_info, fmt("%s/config.bst", dir),
delete_files) )
return false;
UnserialInfo state_info(this);
state_info.id_policy = UnserialInfo::CopyNewToCurrent;
if ( ! CheckForFile(&state_info, fmt("%s/state.bst", dir),
delete_files) )
return false;
return true;
}
bool PersistenceSerializer::MoveFileUp(const char* dir, const char* file)
{
char oldname[PATH_MAX];
char newname[PATH_MAX];
safe_snprintf(oldname, PATH_MAX, "%s/.tmp/%s", dir, file );
safe_snprintf(newname, PATH_MAX, "%s/%s", dir, file );
if ( rename(oldname, newname) < 0 )
{
Error(fmt("can't move %s to %s: %s", oldname, newname,
strerror(errno)));
return false;
}
CheckTimestamp(newname);
return true;
}
#if 0
void PersistenceSerializer::RaiseFinishedSendState()
{
val_list* vl = new val_list;
vl->append(new AddrVal(htonl(remote_host)));
vl->append(val_mgr->GetPort(remote_port));
mgr.QueueEvent(finished_send_state, vl);
reporter->Log("Serialization done.");
}
#endif
void PersistenceSerializer::GotEvent(const char* name, double time,
EventHandlerPtr event, val_list* args)
{
mgr.QueueEvent(event, std::move(*args));
delete args;
}
void PersistenceSerializer::GotFunctionCall(const char* name, double time,
Func* func, val_list* args)
{
try
{
func->Call(args);
}
catch ( InterpreterException& e )
{ /* Already reported. */ }
}
void PersistenceSerializer::GotStateAccess(StateAccess* s)
{
s->Replay();
delete s;
}
void PersistenceSerializer::GotTimer(Timer* s)
{
reporter->Error("PersistenceSerializer::GotTimer not implemented");
}
void PersistenceSerializer::GotConnection(Connection* c)
{
Unref(c);
}
void PersistenceSerializer::GotID(ID* id, Val* /* val */)
{
Unref(id);
}
void PersistenceSerializer::GotPacket(Packet* p)
{
reporter->Error("PersistenceSerializer::GotPacket not implemented");
}
bool PersistenceSerializer::LogAccess(const StateAccess& s)
{
if ( ! IsSerializationRunning() )
return true;
loop_over_list(running, i)
{
running[i]->accesses.append(new StateAccess(s));
}
return true;
}
bool PersistenceSerializer::WriteState(bool may_suspend)
{
SerialStatus* status =
new SerialStatus(this, SerialStatus::WritingState);
status->info.may_suspend = may_suspend;
status->ids = &persistent_ids;
status->conns = &persistent_conns;
status->filename = "state.bst";
return RunSerialization(status);
}
bool PersistenceSerializer::WriteConfig(bool may_suspend)
{
if ( mgr.IsDraining() && may_suspend )
// Events which trigger checkpoint are flushed. Ignore; we'll
// checkpoint at termination in any case.
return true;
SerialStatus* status =
new SerialStatus(this, SerialStatus::WritingConfig);
status->info.may_suspend = may_suspend;
status->info.clear_containers = true;
status->ids = global_scope()->GetIDs();
status->filename = "config.bst";
return RunSerialization(status);
}
bool PersistenceSerializer::SendState(SourceID peer, bool may_suspend)
{
SerialStatus* status =
new SerialStatus(remote_serializer, SerialStatus::SendingState);
status->info.may_suspend = may_suspend;
status->ids = &persistent_ids;
status->conns = &persistent_conns;
status->peer = peer;
reporter->Info("Sending state...");
return RunSerialization(status);
}
bool PersistenceSerializer::SendConfig(SourceID peer, bool may_suspend)
{
SerialStatus* status =
new SerialStatus(remote_serializer, SerialStatus::SendingConfig);
status->info.may_suspend = may_suspend;
status->info.clear_containers = true;
status->ids = global_scope()->GetIDs();
status->peer = peer;
reporter->Info("Sending config...");
return RunSerialization(status);
}
bool PersistenceSerializer::RunSerialization(SerialStatus* status)
{
Continuation* cont = &status->info.cont;
if ( cont->NewInstance() )
{
// Serialization is starting. Initialize.
// See if there is already a serialization of this type running.
loop_over_list(running, i)
{
if ( running[i]->type == status->type )
{
reporter->Warning("Serialization of type %d already running.", status->type);
return false;
}
}
running.append(status);
// Initialize.
if ( ! (ensure_dir(dir) && ensure_dir(fmt("%s/.tmp", dir))) )
return false;
if ( ! OpenFile(fmt("%s/.tmp/%s", dir, status->filename), false) )
return false;
if ( ! PrepareForWriting() )
return false;
if ( status->ids )
{
status->id_cookie = status->ids->InitForIteration();
status->ids->MakeRobustCookie(status->id_cookie);
}
if ( status->conns )
{
status->conn_cookie = status->conns->InitForIteration();
status->conns->MakeRobustCookie(status->conn_cookie);
}
}
else if ( cont->ChildSuspended() )
{
// One of our former Serialize() calls suspended itself.
// We have to call it again.
if ( status->id_cookie )
{
if ( ! DoIDSerialization(status, status->current.id) )
return false;
if ( cont->ChildSuspended() )
{
// Oops, it did it again.
timer_mgr->Add(new IncrementalWriteTimer(network_time + state_write_delay, status));
return true;
}
}
else if ( status->conn_cookie )
{
if ( ! DoConnSerialization(status, status->current.conn) )
return false;
if ( cont->ChildSuspended() )
{
// Oops, it did it again.
timer_mgr->Add(new IncrementalWriteTimer(network_time + state_write_delay, status));
return true;
}
}
else
reporter->InternalError("unknown suspend state");
}
else if ( cont->Resuming() )
cont->Resume();
else
reporter->InternalError("unknown continuation state");
if ( status->id_cookie )
{
ID* id;
while ( (id = status->ids->NextEntry(status->id_cookie)) )
{
if ( ! DoIDSerialization(status, id) )
return false;
if ( cont->ChildSuspended() )
{
timer_mgr->Add(new IncrementalWriteTimer(network_time + state_write_delay, status));
return true;
}
if ( status->info.may_suspend )
{
timer_mgr->Add(new IncrementalWriteTimer(network_time + state_write_delay, status));
cont->Suspend();
return true;
}
}
// Cookie has been set to 0 by NextEntry().
}
if ( status->conn_cookie )
{
Connection* conn;
while ( (conn = status->conns->NextEntry(status->conn_cookie)) )
{
if ( ! DoConnSerialization(status, conn) )
return false;
if ( cont->ChildSuspended() )
{
timer_mgr->Add(new IncrementalWriteTimer(network_time + state_write_delay, status));
return true;
}
if ( status->info.may_suspend )
{
timer_mgr->Add(new IncrementalWriteTimer(network_time + state_write_delay, status));
cont->Suspend();
return true;
}
}
// Cookie has been set to 0 by NextEntry().
}
DBG_LOG(DBG_STATE, "finished serialization; %d accesses pending",
status->accesses.length());
if ( status->accesses.length() )
{
// Serialize pending state accesses.
// FIXME: Does this need to suspend?
StateAccess* access;
loop_over_list(status->accesses, i)
{
// Serializing a StateAccess will not suspend.
if ( ! DoAccessSerialization(status, status->accesses[i]) )
return false;
delete status->accesses[i];
}
}
// Finalize.
CloseFile();
bool ret = MoveFileUp(dir, status->filename);
loop_over_list(running, i)
{
if ( running[i]->type == status->type )
{
running.remove_nth(i);
break;
}
}
delete status;
return ret;
}
bool PersistenceSerializer::DoIDSerialization(SerialStatus* status, ID* id)
{
bool success = false;
Continuation* cont = &status->info.cont;
status->current.id = id;
switch ( status->type ) {
case SerialStatus::WritingState:
case SerialStatus::WritingConfig:
cont->SaveContext();
success = Serialize(&status->info, *id);
cont->RestoreContext();
break;
case SerialStatus::SendingState:
case SerialStatus::SendingConfig:
cont->SaveContext();
success = remote_serializer->SendID(&status->info,
status->peer, *id);
cont->RestoreContext();
break;
default:
reporter->InternalError("unknown serialization type");
}
return success;
}
bool PersistenceSerializer::DoConnSerialization(SerialStatus* status,
Connection* conn)
{
bool success = false;
Continuation* cont = &status->info.cont;
status->current.conn = conn;
switch ( status->type ) {
case SerialStatus::WritingState:
case SerialStatus::WritingConfig:
cont->SaveContext();
success = Serialize(&status->info, *conn);
cont->RestoreContext();
break;
case SerialStatus::SendingState:
case SerialStatus::SendingConfig:
cont->SaveContext();
success = remote_serializer->SendConnection(&status->info,
status->peer, *conn);
cont->RestoreContext();
break;
default:
reporter->InternalError("unknown serialization type");
}
return success;
}
bool PersistenceSerializer::DoAccessSerialization(SerialStatus* status,
StateAccess* access)
{
bool success = false;
DisableSuspend suspend(&status->info);
switch ( status->type ) {
case SerialStatus::WritingState:
case SerialStatus::WritingConfig:
success = Serialize(&status->info, *access);
break;
case SerialStatus::SendingState:
case SerialStatus::SendingConfig:
success = remote_serializer->SendAccess(&status->info,
status->peer, *access);
break;
default:
reporter->InternalError("unknown serialization type");
}
return success;
}

165
src/PersistenceSerializer.h
View file

@ -1,165 +0,0 @@
// Implements persistance for Bro's data structures.
#ifndef persistence_serializer_h
#define persistence_serializer_h
#include "Serializer.h"
#include "List.h"
class StateAccess;
class PersistenceSerializer : public FileSerializer {
public:
PersistenceSerializer();
~PersistenceSerializer() override;
// Define the directory where to store the data.
void SetDir(const char* arg_dir) { dir = copy_string(arg_dir); }
// Register/unregister the ID/connection to be saved by WriteAll().
void Register(ID* id);
void Unregister(ID* id);
void Register(Connection* conn);
void Unregister(Connection* conn);
// Read all data that has been changed since last scan of directory.
// is_init should be true for the first read upon start-up. All existing
// state will be cleared. If delete_files is true, file which have been
// read are removed (even if the read was unsuccessful!).
bool ReadAll(bool is_init, bool delete_files);
// Each of the following four methods may suspend operation.
// If they do, they install a Timer which resumes after some
// amount of time. If a function is called again before it
// has completely finished its task, it will do nothing and
// return false.
bool WriteState(bool may_suspend);
// Writes Bro's configuration (w/o dynamic state).
bool WriteConfig(bool may_suspend);
// Sends all registered state to remote host
// (by leveraging the remote_serializer).
bool SendState(SourceID peer, bool may_suspend);
// Sends Bro's config to remote host
// (by leveraging the remote_serializer).
bool SendConfig(SourceID peer, bool may_suspend);
// Returns true if a serialization is currently running.
bool IsSerializationRunning() const { return running.length(); }
// Tells the serializer that this access was performed. If a
// serialization is going on, it may store it. (Need only be called if
// IsSerializationRunning() returns true.)
bool LogAccess(const StateAccess& s);
protected:
friend class RemoteSerializer;
friend class IncrementalWriteTimer;
void GotID(ID* id, Val* val) override;
void GotEvent(const char* name, double time,
EventHandlerPtr event, val_list* args) override;
void GotFunctionCall(const char* name, double time,
Func* func, val_list* args) override;
void GotStateAccess(StateAccess* s) override;
void GotTimer(Timer* t) override;
void GotConnection(Connection* c) override;
void GotPacket(Packet* packet) override;
// If file has changed since last check, read it.
bool CheckForFile(UnserialInfo* info, const char* file,
bool delete_file);
// Returns true if it's a regular file and has a more recent timestamp
// than last time we checked it.
bool CheckTimestamp(const char* file);
// Move file from <dir>/tmp/<file> to <dir>/<file>. Afterwards, call
// CheckTimestamp() with <dir>/<file>.
bool MoveFileUp(const char* dir, const char* file);
// Generates an error message, terminates current serialization,
// and returns false.
bool SerialError(const char* msg);
// Start a new serialization.
struct SerialStatus;
bool RunSerialization(SerialStatus* status);
// Helpers for RunSerialization.
bool DoIDSerialization(SerialStatus* status, ID* id);
bool DoConnSerialization(SerialStatus* status, Connection* conn);
bool DoAccessSerialization(SerialStatus* status, StateAccess* access);
typedef PDict(ID) id_map;
declare(PDict, Connection);
typedef PDict(Connection) conn_map;
struct SerialStatus {
enum Type {
WritingState, WritingConfig,
SendingState, SendingConfig,
};
SerialStatus(Serializer* s, Type arg_type) : info(s)
{
type = arg_type;
ids = 0;
id_cookie = 0;
conns = 0;
conn_cookie = 0;
peer = SOURCE_LOCAL;
filename = 0;
}
Type type;
SerialInfo info;
// IDs to serialize.
id_map* ids;
IterCookie* id_cookie;
// Connections to serialize.
conn_map* conns;
IterCookie* conn_cookie;
// Accesses performed while we're serializing.
declare(PList,StateAccess);
typedef PList(StateAccess) state_access_list;
state_access_list accesses;
// The ID/Conn we're currently serializing.
union {
ID* id;
Connection* conn;
} current;
// Only set if type is Writing{State,Config}.
const char* filename;
// Only set if type is Sending{State,Config}.
SourceID peer;
};
const char* dir;
declare(PList, SerialStatus);
PList(SerialStatus) running;
id_map persistent_ids;
conn_map persistent_conns;
// To keep track of files' modification times.
declare(PDict, time_t);
typedef PDict(time_t) file_map;
file_map files;
};
extern PersistenceSerializer* persistence_serializer;
#endif

4585
src/RemoteSerializer.cc
View file

File diff suppressed because it is too large Load diff

525
src/RemoteSerializer.h
View file

@ -1,525 +0,0 @@
// Communication between two Bro's.
#ifndef REMOTE_SERIALIZER
#define REMOTE_SERIALIZER
#include "Dict.h"
#include "List.h"
#include "Serializer.h"
#include "iosource/IOSource.h"
#include "Stats.h"
#include "File.h"
#include "logging/WriterBackend.h"
#include <vector>
#include <string>
class IncrementalSendTimer;
namespace threading {
struct Field;
struct Value;
}
// This class handles the communication done in Bro's main loop.
class RemoteSerializer : public Serializer, public iosource::IOSource {
public:
RemoteSerializer();
~RemoteSerializer() override;
// Initialize the remote serializer (calling this will fork).
void Enable();
// FIXME: Use SourceID directly (or rename everything to Peer*).
typedef SourceID PeerID;
static const PeerID PEER_LOCAL = SOURCE_LOCAL;
static const PeerID PEER_NONE = SOURCE_LOCAL;
// Connect to host (returns PEER_NONE on error).
PeerID Connect(const IPAddr& ip, const string& zone_id, uint16 port,
const char* our_class, double retry, bool use_ssl);
// Close connection to host.
bool CloseConnection(PeerID peer);
// Request all events matching pattern from remote side.
bool RequestEvents(PeerID peer, RE_Matcher* pattern);
// Request synchronization of IDs with remote side. If auth is true,
// we consider our current state to authoritative and send it to
// the peer right after the handshake.
bool RequestSync(PeerID peer, bool auth);
// Requests logs from the remote side.
bool RequestLogs(PeerID id);
// Sets flag whether we're accepting state from this peer
// (default: yes).
bool SetAcceptState(PeerID peer, bool accept);
// Sets compression level (0-9, 0 is defaults and means no compression)
bool SetCompressionLevel(PeerID peer, int level);
// Signal the other side that we have finished our part of
// the initial handshake.
bool CompleteHandshake(PeerID peer);
// Start to listen.
bool Listen(const IPAddr& ip, uint16 port, bool expect_ssl, bool ipv6,
const string& zone_id, double retry);
// Stop it.
bool StopListening();
// Broadcast the event/function call.
bool SendCall(SerialInfo* info, const char* name, val_list* vl);
// Send the event/function call (only if handshake completed).
bool SendCall(SerialInfo* info, PeerID peer, const char* name, val_list* vl);
// Broadcasts the access (only if handshake completed).
bool SendAccess(SerialInfo* info, const StateAccess& access);
// Send the access.
bool SendAccess(SerialInfo* info, PeerID pid, const StateAccess& access);
// Sends ID.
bool SendID(SerialInfo* info, PeerID peer, const ID& id);
// Sends the internal connection state.
bool SendConnection(SerialInfo* info, PeerID peer, const Connection& c);
// Send capture filter.
bool SendCaptureFilter(PeerID peer, const char* filter);
// Send packet.
bool SendPacket(SerialInfo* info, PeerID peer, const Packet& p);
// Broadcast packet.
bool SendPacket(SerialInfo* info, const Packet& p);
// Broadcast ping.
bool SendPing(PeerID peer, uint32 seq);
// Broadcast remote print.
bool SendPrintHookEvent(BroFile* f, const char* txt, size_t len);
// Send a request to create a writer on a remote side.
bool SendLogCreateWriter(PeerID peer, EnumVal* id, EnumVal* writer, const logging::WriterBackend::WriterInfo& info, int num_fields, const threading::Field* const * fields);
// Broadcasts a request to create a writer.
bool SendLogCreateWriter(EnumVal* id, EnumVal* writer, const logging::WriterBackend::WriterInfo& info, int num_fields, const threading::Field* const * fields);
// Broadcast a log entry to everybody interested.
bool SendLogWrite(EnumVal* id, EnumVal* writer, string path, int num_fields, const threading::Value* const * vals);
// Synchronzizes time with all connected peers. Returns number of
// current sync-point, or -1 on error.
uint32 SendSyncPoint();
void SendFinalSyncPoint();
// Registers the ID to be &synchronized.
void Register(ID* id);
void Unregister(ID* id);
// Stop/restart propagating state updates.
void SuspendStateUpdates() { --propagate_accesses; }
void ResumeStateUpdates() { ++propagate_accesses; }
// Check for incoming events and queue them.
bool Poll(bool may_block);
// Returns the corresponding record (already ref'ed).
RecordVal* GetPeerVal(PeerID id);
// Log some statistics.
void LogStats();
// Tries to sent out all remaining data.
// FIXME: Do we still need this?
void Finish();
// Overidden from IOSource:
void GetFds(iosource::FD_Set* read, iosource::FD_Set* write,
iosource::FD_Set* except) override;
double NextTimestamp(double* local_network_time) override;
void Process() override;
TimerMgr::Tag* GetCurrentTag() override;
const char* Tag() override { return "RemoteSerializer"; }
// Gracefully finishes communication by first making sure that all
// remaining data (parent & child) has been sent out.
virtual bool Terminate();
#ifdef DEBUG_COMMUNICATION
// Dump data recently read/written into files.
void DumpDebugData();
// Read dump file and interpret as message block.
void ReadDumpAsMessageType(const char* file);
// Read dump file and interpret as serialization.
void ReadDumpAsSerialization(const char* file);
#endif
enum LogLevel { LogInfo = 1, LogError = 2, };
static void Log(LogLevel level, const char* msg);
protected:
friend class PersistenceSerializer;
friend class IncrementalSendTimer;
// Maximum size of serialization caches.
static const unsigned int MAX_CACHE_SIZE = 3000;
// When syncing traces in pseudo-realtime mode, we wait this many
// seconds after the final sync-point to make sure that all
// remaining I/O gets propagated.
static const unsigned int FINAL_SYNC_POINT_DELAY = 5;
declare(PList, EventHandler);
typedef PList(EventHandler) handler_list;
struct Peer {
PeerID id; // Unique ID (non-zero) per peer.
IPAddr ip;
uint16 port;
handler_list handlers;
RecordVal* val; // Record of type event_source.
SerializationCache* cache_in; // One cache for each direction.
SerializationCache* cache_out;
// TCP-level state of the connection to the peer.
// State of the connection to the peer.
enum { INIT, PENDING, CONNECTED, CLOSING, CLOSED } state;
// Current protocol phase of the connection (see RemoteSerializer.cc)
enum { UNKNOWN, SETUP, HANDSHAKE, SYNC, RUNNING } phase;
// Capabilities.
static const int COMPRESSION = 1;
static const int NO_CACHING = 2;
static const int PID_64BIT = 4;
static const int NEW_CACHE_STRATEGY = 8;
static const int BROCCOLI_PEER = 16;
// Constants to remember to who did something.
static const int NONE = 0;
static const int WE = 1;
static const int PEER = 2;
static const int BOTH = WE | PEER;
static const int AUTH_WE = 4;
static const int AUTH_PEER = 8;
int sent_version; // Who has sent the VERSION.
int handshake_done; // Who finished its handshake phase.
int sync_requested; // Who requested sync'ed state.
bool orig; // True if we connected to the peer.
bool accept_state; // True if we accept state from peer.
bool send_state; // True if we're supposed to initially sent our state.
int comp_level; // Compression level.
bool logs_requested; // True if the peer has requested logs.
// True if this peer triggered a net_suspend_processing().
bool suspended_processing;
uint32 caps; // Capabilities announced by peer.
int runtime; // Runtime we got from the peer.
int our_runtime; // Our runtime as we told it to this peer.
string peer_class; // Class from peer ("" = no class).
string our_class; // Class we send the peer.
uint32 sync_point; // Highest sync-point received so far
char* print_buffer; // Buffer for remote print or null.
int print_buffer_used; // Number of bytes used in buffer.
char* log_buffer; // Buffer for remote log or null.
int log_buffer_used; // Number of bytes used in buffer.
};
// Shuts down remote serializer.
void FatalError(const char* msg);
enum LogSrc { LogChild = 1, LogParent = 2, LogScript = 3, };
static void Log(LogLevel level, const char* msg, Peer* peer, LogSrc src = LogParent);
void ReportError(const char* msg) override;
void GotEvent(const char* name, double time,
EventHandlerPtr event, val_list* args) override;
void GotFunctionCall(const char* name, double time,
Func* func, val_list* args) override;
void GotID(ID* id, Val* val) override;
void GotStateAccess(StateAccess* s) override;
void GotTimer(Timer* t) override;
void GotConnection(Connection* c) override;
void GotPacket(Packet* packet) override;
void Fork();
bool DoMessage();
bool ProcessConnected();
bool ProcessSerialization();
bool ProcessRequestEventsMsg();
bool ProcessRequestSyncMsg();
bool ProcessVersionMsg();
bool ProcessLogMsg(bool is_error);
bool ProcessStatsMsg();
bool ProcessCaptureFilterMsg();
bool ProcessPhaseDone();
bool ProcessPingMsg();
bool ProcessPongMsg();
bool ProcessCapsMsg();
bool ProcessSyncPointMsg();
bool ProcessRemotePrint();
bool ProcessLogCreateWriter();
bool ProcessLogWrite();
bool ProcessRequestLogs();
Peer* AddPeer(const IPAddr& ip, uint16 port, PeerID id = PEER_NONE);
Peer* LookupPeer(PeerID id, bool only_if_connected);
void RemovePeer(Peer* peer);
bool IsConnectedPeer(PeerID id);
void PeerDisconnected(Peer* peer);
void PeerConnected(Peer* peer);
RecordVal* MakePeerVal(Peer* peer);
bool HandshakeDone(Peer* peer);
bool IsActive();
void SetupSerialInfo(SerialInfo* info, Peer* peer);
bool CheckSyncPoints();
void SendSyncPoint(uint32 syncpoint);
bool PropagateAccesses()
{
return ignore_accesses ?
propagate_accesses > 1 : propagate_accesses > 0;
}
bool CloseConnection(Peer* peer);
bool SendAllSynchronized(Peer* peer, SerialInfo* info);
bool SendCall(SerialInfo* info, Peer* peer, const char* name, val_list* vl);
bool SendAccess(SerialInfo* info, Peer* peer, const StateAccess& access);
bool SendID(SerialInfo* info, Peer* peer, const ID& id);
bool SendCapabilities(Peer* peer);
bool SendPacket(SerialInfo* info, Peer* peer, const Packet& p);
bool SendLogWrite(Peer* peer, EnumVal* id, EnumVal* writer, string path, int num_fields, const threading::Value* const * vals);
void UnregisterHandlers(Peer* peer);
void RaiseEvent(EventHandlerPtr event, Peer* peer, const char* arg = 0);
bool EnterPhaseRunning(Peer* peer);
bool FlushPrintBuffer(Peer* p);
bool FlushLogBuffer(Peer* p);
void ChildDied();
void InternalCommError(const char* msg);
// Communication helpers
bool SendCMsgToChild(char msg_type, Peer* peer);
bool SendToChild(char type, Peer* peer, char* str, int len = -1,
bool delete_with_free = false);
bool SendToChild(char type, Peer* peer, int nargs, ...); // can send uints32 only
bool SendToChild(ChunkedIO::Chunk* c);
void SetSocketBufferSize(int fd, int opt, const char *what, int size, int verbose);
private:
enum { TYPE, ARGS } msgstate; // current state of reading comm.
Peer* current_peer;
PeerID current_id;
char current_msgtype;
ChunkedIO::Chunk* current_args;
double last_flush;
id_list sync_ids;
// FIXME: Check which of these are necessary...
bool initialized;
bool listening;
int propagate_accesses;
bool ignore_accesses;
bool terminating;
int received_logs;
Peer* source_peer;
PeerID id_counter; // Keeps track of assigned IDs.
uint32 current_sync_point;
bool syncing_times;
declare(PList, Peer);
typedef PList(Peer) peer_list;
peer_list peers;
Peer* in_sync; // Peer we're currently syncing state with.
peer_list sync_pending; // List of peers waiting to sync state.
// Event buffer
struct BufferedEvent {
time_t time;
PeerID src;
EventHandlerPtr handler;
val_list* args;
};
declare(PList, BufferedEvent);
typedef PList(BufferedEvent) EventQueue;
EventQueue events;
// Packet buffer
struct BufferedPacket {
time_t time;
Packet* p;
};
declare(PList, BufferedPacket);
typedef PList(BufferedPacket) PacketQueue;
PacketQueue packets;
// Some stats
struct Statistics {
struct Pair {
Pair() : in(0), out(0) {}
unsigned long in;
unsigned long out;
};
Pair events; // actually events and function calls
Pair accesses;
Pair conns;
Pair packets;
Pair ids;
} stats;
};
// This class handles the communication done in the forked child.
class SocketComm {
public:
SocketComm();
~SocketComm();
void SetParentIO(ChunkedIO* arg_io) { io = arg_io; }
void Run(); // does not return
// Log some statistics (via pipe to parent).
bool LogStats();
// Log CPU usage (again via pipe to parent).
bool LogProf();
protected:
struct Peer {
Peer()
{
id = 0;
io = 0;
port = 0;
state = 0;
connected = false;
ssl = false;
retry = 0;
next_try = 0;
compressor = false;
}
RemoteSerializer::PeerID id;
ChunkedIO* io;
IPAddr ip;
string zone_id;
uint16 port;
char state;
bool connected;
bool ssl;
// If we get disconnected, reconnect after this many seconds.
int retry;
// Time of next connection attempt (0 if none).
time_t next_try;
// True if io is a CompressedChunkedIO.
bool compressor;
};
bool Listen();
bool AcceptConnection(int listen_fd);
bool Connect(Peer* peer);
bool CloseConnection(Peer* peer, bool reconnect);
Peer* LookupPeer(RemoteSerializer::PeerID id, bool only_if_connected);
bool ProcessRemoteMessage(Peer* peer);
bool ProcessParentMessage();
bool DoParentMessage();
bool ProcessListen();
bool ProcessConnectTo();
bool ProcessCompress();
void Log(const char* msg, Peer* peer = 0);
// The connection to the peer will be closed.
bool Error(const char* msg, Peer* peer);
// If kill is true, this is a fatal error and we kill ourselves.
void Error(const char* msg, bool kill = false);
// Kill the current process.
void Kill();
// Check whether everything has been sent out.
void CheckFinished();
// Reports the error and terminates the process.
void InternalError(const char* msg);
// Communication helpers.
bool SendToParent(char type, Peer* peer, const char* str, int len = -1);
bool SendToParent(char type, Peer* peer, int nargs, ...); // can send uints32 only
bool SendToParent(ChunkedIO::Chunk* c);
bool SendToPeer(Peer* peer, char type, const char* str, int len = -1);
bool SendToPeer(Peer* peer, char type, int nargs, ...); // can send uints32 only
bool SendToPeer(Peer* peer, ChunkedIO::Chunk* c);
bool ProcessParentCompress();
bool ProcessPeerCompress(Peer* peer);
bool ForwardChunkToParent(Peer* p, ChunkedIO::Chunk* c);
bool ForwardChunkToPeer();
const char* MakeLogString(const char* msg, Peer *peer);
// Closes all file descriptors associated with listening sockets.
void CloseListenFDs();
// Peers we are communicating with:
declare(PList, Peer);
typedef PList(Peer) peer_list;
RemoteSerializer::PeerID id_counter;
peer_list peers;
ChunkedIO* io; // I/O to parent
// Current state of reading from parent.
enum { TYPE, ARGS } parent_msgstate;
Peer* parent_peer;
RemoteSerializer::PeerID parent_id;
char parent_msgtype;
ChunkedIO::Chunk* parent_args;
vector<int> listen_fds;
// If the port we're trying to bind to is already in use, we will retry
// it regularly.
string listen_if;
string listen_zone_id; // RFC 4007 IPv6 zone_id
uint16 listen_port;
bool listen_ssl; // use SSL for IO
bool enable_ipv6; // allow IPv6 listen sockets
uint32 bind_retry_interval; // retry interval for already-in-use sockets
time_t listen_next_try; // time at which to try another bind
bool shutting_conns_down;
bool terminating;
bool killing;
};
extern RemoteSerializer* remote_serializer;
#endif

16
src/SerialInfo.h
View file

@ -3,6 +3,8 @@
#ifndef serialinfo_h
#define serialinfo_h
#include "ChunkedIO.h"
class SerialInfo {
public:
SerialInfo(Serializer* arg_s)
@ -15,7 +17,6 @@ public:
pid_32bit = false;
include_locations = true;
new_cache_strategy = false;
broccoli_peer = false;
}
SerialInfo(const SerialInfo& info)
@ -30,7 +31,6 @@ public:
pid_32bit = info.pid_32bit;
include_locations = info.include_locations;
new_cache_strategy = info.new_cache_strategy;
broccoli_peer = info.broccoli_peer;
}
// Parameters that control serialization.
@ -49,11 +49,6 @@ public:
// If true, we support keeping objs in cache permanently.
bool new_cache_strategy;
// If true, we're connecting to a Broccoli. If so, serialization
// specifics may be adapted for functionality Broccoli does not
// support.
bool broccoli_peer;
ChunkedIO::Chunk* chunk; // chunk written right before the serialization
// Attributes set during serialization.
@ -78,7 +73,6 @@ public:
print = 0;
pid_32bit = false;
new_cache_strategy = false;
broccoli_peer = false;
}
UnserialInfo(const UnserialInfo& info)
@ -95,7 +89,6 @@ public:
print = info.print;
pid_32bit = info.pid_32bit;
new_cache_strategy = info.new_cache_strategy;
broccoli_peer = info.broccoli_peer;
}
// Parameters that control unserialization.
@ -116,11 +109,6 @@ public:
// If true, we support keeping objs in cache permanently.
bool new_cache_strategy;
// If true, we're connecting to a Broccoli. If so, serialization
// specifics may be adapted for functionality Broccoli does not
// support.
bool broccoli_peer;
// If a global ID already exits, of these policies is used.
enum {
Keep, // keep the old ID and ignore the new

3
src/Serializer.cc
View file

@ -18,7 +18,6 @@
#include "NetVar.h"
#include "Conn.h"
#include "Timer.h"
#include "RemoteSerializer.h"
#include "iosource/Manager.h"
Serializer::Serializer(SerializationFormat* arg_format)
@ -508,8 +507,6 @@ bool Serializer::UnserializeConnection(UnserialInfo* info)
if ( info->install_conns )
{
if ( c->IsPersistent() && c->Key() )
persistence_serializer->Register(c);
Ref(c);
sessions->Insert(c);
}

5
src/Sessions.cc
View file

@ -1101,9 +1101,6 @@ void NetSessions::Remove(Connection* c)
tcp_stats.StateLeft(to->state, tr->state);
}
if ( c->IsPersistent() )
persistence_serializer->Unregister(c);
c->Done();
if ( connection_state_remove )
@ -1194,8 +1191,6 @@ void NetSessions::Insert(Connection* c)
// Some clean-ups similar to those in Remove() (but invisible
// to the script layer).
old->CancelTimers();
if ( old->IsPersistent() )
persistence_serializer->Unregister(old);
delete old->Key();
old->ClearKey();
Unref(old);

1
src/Sessions.h
View file

@ -180,7 +180,6 @@ public:
analyzer::tcp::TCPStateStats tcp_stats; // keeps statistics on TCP states
protected:
friend class RemoteSerializer;
friend class ConnCompressor;
friend class TimerMgrExpireTimer;
friend class IPTunnelTimer;

185
src/StateAccess.cc
View file

@ -4,8 +4,6 @@
#include "Event.h"
#include "NetVar.h"
#include "DebugLogger.h"
#include "RemoteSerializer.h"
#include "PersistenceSerializer.h"
int StateAccess::replaying = 0;
@ -135,100 +133,6 @@ void StateAccess::RefThem()
Ref(op3);
}
bool StateAccess::CheckOld(const char* op, ID* id, Val* index,
Val* should, Val* is)
{
if ( ! remote_check_sync_consistency )
return true;
if ( ! should && ! is )
return true;
// 'should == index' means that 'is' should be non-nil.
if ( should == index && is )
return true;
if ( should && is )
{
// There's no general comparison for non-atomic vals currently.
if ( ! (is_atomic_val(is) && is_atomic_val(should)) )
return true;
if ( same_atomic_val(should, is) )
return true;
}
Val* arg1;
Val* arg2;
Val* arg3;
if ( index )
{
ODesc d;
d.SetShort();
index->Describe(&d);
arg1 = new StringVal(fmt("%s[%s]", id->Name(), d.Description()));
}
else
arg1 = new StringVal(id->Name());
if ( should )
{
ODesc d;
d.SetShort();
should->Describe(&d);
arg2 = new StringVal(d.Description());
}
else
arg2 = new StringVal("<none>");
if ( is )
{
ODesc d;
d.SetShort();
is->Describe(&d);
arg3 = new StringVal(d.Description());
}
else
arg3 = new StringVal("<none>");
mgr.QueueEvent(remote_state_inconsistency, {
new StringVal(op),
arg1,
arg2,
arg3,
});
return false;
}
bool StateAccess::CheckOldSet(const char* op, ID* id, Val* index,
bool should, bool is)
{
if ( ! remote_check_sync_consistency )
return true;
if ( should == is )
return true;
ODesc d;
d.SetShort();
index->Describe(&d);
Val* arg1 = new StringVal(fmt("%s[%s]", id->Name(), d.Description()));
Val* arg2 = new StringVal(should ? "set" : "not set");
Val* arg3 = new StringVal(is ? "set" : "not set");
mgr.QueueEvent(remote_state_inconsistency, {
new StringVal(op),
arg1,
arg2,
arg3,
});
return false;
}
bool StateAccess::MergeTables(TableVal* dst, Val* src)
{
if ( src->Type()->Tag() != TYPE_TABLE )
@ -287,7 +191,6 @@ void StateAccess::Replay()
assert(op1.val);
// There mustn't be a direct assignment to a unique ID.
assert(target.id->Name()[0] != '#');
CheckOld("assign", target.id, 0, op2, v);
if ( t == TYPE_TABLE && v &&
v->AsTableVal()->FindAttr(ATTR_MERGEABLE) )
@ -329,9 +232,6 @@ void StateAccess::Replay()
break;
}
CheckOld("index assign", target.id, op1.val, op3,
v->AsTableVal()->Lookup(op1.val));
v->AsTableVal()->Assign(op1.val, op2 ? op2->Ref() : 0);
}
@ -353,8 +253,6 @@ void StateAccess::Replay()
break;
}
CheckOld("index assign", target.id, op1.val, op3,
v->AsRecordVal()->Lookup(idx));
v->AsRecordVal()->Assign(idx, op2 ? op2->Ref() : 0);
}
else
@ -377,8 +275,6 @@ void StateAccess::Replay()
break;
}
CheckOld("index assign", target.id, op1.val, op3,
v->AsVectorVal()->Lookup(index));
v->AsVectorVal()->Assign(index, op2 ? op2->Ref() : 0);
}
@ -442,8 +338,6 @@ void StateAccess::Replay()
assert(op1.val);
if ( t == TYPE_TABLE )
{
CheckOldSet("add", target.id, op1.val, op2 != 0,
v->AsTableVal()->Lookup(op1.val) != 0);
v->AsTableVal()->Assign(op1.val, 0);
}
break;
@ -452,13 +346,6 @@ void StateAccess::Replay()
assert(op1.val);
if ( t == TYPE_TABLE )
{
if ( v->Type()->AsTableType()->IsSet() )
CheckOldSet("delete", target.id, op1.val, op2 != 0,
v->AsTableVal()->Lookup(op1.val) != 0);
else
CheckOld("delete", target.id, op1.val, op2,
v->AsTableVal()->Lookup(op1.val));
Unref(v->AsTableVal()->Delete(op1.val));
}
break;
@ -477,14 +364,8 @@ void StateAccess::Replay()
// are performed in the expire_func.
StateAccess::ResumeReplay();
if ( remote_serializer )
remote_serializer->ResumeStateUpdates();
tv->CallExpireFunc(op1.val->Ref());
if ( remote_serializer )
remote_serializer->SuspendStateUpdates();
StateAccess::SuspendReplay();
Unref(tv->AsTableVal()->Delete(op1.val));
@ -507,20 +388,7 @@ void StateAccess::Replay()
// Update the timestamp if we have a read_expire.
if ( tv->FindAttr(ATTR_EXPIRE_READ) )
{
if ( ! tv->UpdateTimestamp(op1.val) &&
remote_check_sync_consistency )
{
ODesc d;
d.SetShort();
op1.val->Describe(&d);
mgr.QueueEvent(remote_state_inconsistency, {
new StringVal("read"),
new StringVal(fmt("%s[%s]", target.id->Name(), d.Description())),
new StringVal("existent"),
new StringVal("not existent"),
});
}
tv->UpdateTimestamp(op1.val);
}
}
else
@ -533,14 +401,6 @@ void StateAccess::Replay()
}
--replaying;
if ( remote_state_access_performed )
{
mgr.QueueEventFast(remote_state_access_performed, {
new StringVal(target.id->Name()),
target.id->ID_Val()->Ref(),
});
}
}
ID* StateAccess::Target() const
@ -597,14 +457,6 @@ bool StateAccess::DoSerialize(SerialInfo* info) const
const Val* null = 0;
if ( remote_check_sync_consistency )
{
SERIALIZE_OPTIONAL(op2);
SERIALIZE_OPTIONAL(op3);
}
else
{
switch ( opcode ) {
case OP_PRINT:
case OP_EXPIRE:
@ -638,7 +490,6 @@ bool StateAccess::DoSerialize(SerialInfo* info) const
default:
reporter->InternalError("StateAccess::DoSerialize: unknown opcode");
}
}
return true;
}
@ -870,57 +721,27 @@ void StateAccess::Describe(ODesc* d) const
void StateAccess::Log(StateAccess* access)
{
bool synchronized = false;
bool persistent = false;
bool tracked = false;
if ( access->target_type == TYPE_ID )
{
if ( access->target.id->FindAttr(ATTR_SYNCHRONIZED) )
synchronized = true;
if ( access->target.id->FindAttr(ATTR_PERSISTENT) )
persistent = true;
if ( access->target.id->FindAttr(ATTR_TRACKED) )
tracked = true;
}
else
{
if ( access->target.val->GetProperties() & MutableVal::SYNCHRONIZED )
synchronized = true;
if ( access->target.val->GetProperties() & MutableVal::PERSISTENT )
persistent = true;
if ( access->target.val->GetProperties() & MutableVal::TRACKED )
tracked = true;
}
if ( synchronized )
{
if ( state_serializer )
{
SerialInfo info(state_serializer);
state_serializer->Serialize(&info, *access);
}
SerialInfo info(remote_serializer);
remote_serializer->SendAccess(&info, *access);
}
if ( persistent && persistence_serializer->IsSerializationRunning() )
persistence_serializer->LogAccess(*access);
if ( tracked )
notifiers.AccessPerformed(*access);
#ifdef DEBUG
ODesc desc;
access->Describe(&desc);
DBG_LOG(DBG_STATE, "operation: %s%s [%s%s]",
desc.Description(), replaying > 0 ? " (replay)" : "",
persistent ? "P" : "", synchronized ? "S" : "");
DBG_LOG(DBG_STATE, "operation: %s%s",
desc.Description(), replaying > 0 ? " (replay)" : "");
#endif
delete access;

2
src/StateAccess.h
View file

@ -74,8 +74,6 @@ private:
StateAccess() { target.id = 0; op1.val = op2 = op3 = 0; }
void RefThem();
bool CheckOld(const char* op, ID* id, Val* index, Val* should, Val* is);
bool CheckOldSet(const char* op, ID* id, Val* index, bool should, bool is);
bool MergeTables(TableVal* dst, Val* src);
DECLARE_SERIAL(StateAccess);

4
src/Stmt.cc
View file

@ -14,7 +14,6 @@
#include "Debug.h"
#include "Traverse.h"
#include "Trigger.h"
#include "RemoteSerializer.h"
const char* stmt_name(BroStmtTag t)
{
@ -301,9 +300,6 @@ Val* PrintStmt::DoExec(val_list* vals, stmt_flow_type& /* flow */) const
{new Val(f), new StringVal(d.Len(), d.Description())}),
true);
}
if ( remote_serializer )
remote_serializer->SendPrintHookEvent(f, d.Description(), d.Len());
}
return 0;

2
src/Timer.cc
View file

@ -20,8 +20,6 @@ const char* TimerNames[] = {
"FileAnalysisInactivityTimer",
"FlowWeirdTimer",
"FragTimer",
"IncrementalSendTimer",
"IncrementalWriteTimer",
"InterconnTimer",
"IPTunnelInactivityTimer",
"NetbiosExpireTimer",

2
src/Timer.h
View file

@ -25,8 +25,6 @@ enum TimerType {
TIMER_FILE_ANALYSIS_INACTIVITY,
TIMER_FLOW_WEIRD_EXPIRE,
TIMER_FRAG,
TIMER_INCREMENTAL_SEND,
TIMER_INCREMENTAL_WRITE,
TIMER_INTERCONN,
TIMER_IP_TUNNEL_INACTIVITY,
TIMER_NB_EXPIRE,

18
src/Val.cc
View file

@ -21,7 +21,6 @@
#include "NetVar.h"
#include "Expr.h"
#include "Serializer.h"
#include "RemoteSerializer.h"
#include "PrefixTable.h"
#include "Conn.h"
#include "Reporter.h"
@ -1562,15 +1561,6 @@ int TableVal::Assign(Val* index, HashKey* k, Val* new_val, Opcode op)
else
{
// A set.
if ( old_entry_val && remote_check_sync_consistency )
{
Val* has_old_val = val_mgr->GetInt(1);
StateAccess::Log(
new StateAccess(OP_ADD, this, index,
has_old_val));
Unref(has_old_val);
}
else
StateAccess::Log(
new StateAccess(OP_ADD, this,
index, 0, 0));
@ -2056,13 +2046,6 @@ Val* TableVal::Delete(const Val* index)
if ( LoggingAccess() )
{
if ( v )
{
if ( v->Value() && remote_check_sync_consistency )
// A table.
StateAccess::Log(
new StateAccess(OP_DEL, this,
index, v->Value()));
else
{
// A set.
Val* has_old_val = val_mgr->GetInt(1);
@ -2071,7 +2054,6 @@ Val* TableVal::Delete(const Val* index)
has_old_val));
Unref(has_old_val);
}
}
else
StateAccess::Log(
new StateAccess(OP_DEL, this, index, 0));

7
src/Val.h
View file

@ -524,9 +524,6 @@ public:
// values. (In any case, don't forget to call the parent's method.)
typedef char Properties;
static const int PERSISTENT = 0x01;
static const int SYNCHRONIZED = 0x02;
// Tracked by NotifierRegistry, not recursive.
static const int TRACKED = 0x04;
@ -540,10 +537,10 @@ public:
bool LoggingAccess() const
{
#ifndef DEBUG
return props & (SYNCHRONIZED|PERSISTENT|TRACKED);
return props & TRACKED;
#else
return debug_logger.IsVerbose() ||
(props & (SYNCHRONIZED|PERSISTENT|TRACKED));
(props & TRACKED);
#endif
}

21
src/Var.cc
View file

@ -7,7 +7,6 @@
#include "Stmt.h"
#include "Scope.h"
#include "Serializer.h"
#include "RemoteSerializer.h"
#include "EventRegistry.h"
#include "Traverse.h"
@ -142,26 +141,6 @@ static void make_var(ID* id, BroType* t, init_class c, Expr* init,
}
}
if ( id->FindAttr(ATTR_PERSISTENT) || id->FindAttr(ATTR_SYNCHRONIZED) )
{
if ( dt == VAR_CONST )
{
id->Error("&persistent/synchronized with constant");
return;
}
else if ( dt == VAR_OPTION )
{
id->Error("&persistent/synchronized with option");
return;
}
if ( ! id->IsGlobal() )
{
id->Error("&persistant/synchronized with non-global");
return;
}
}
if ( do_init )
{
if ( c == INIT_NONE && dt == VAR_REDEF && t->IsTable() &&

2
src/analyzer/protocol/krb/KRB.h
View file

@ -9,6 +9,8 @@
#include <krb5.h>
#endif
#include <mutex>
namespace analyzer { namespace krb {
class KRB_Analyzer : public analyzer::Analyzer {

1
src/analyzer/protocol/tcp/TCP_Reassembler.cc
View file

@ -1,5 +1,6 @@
#include <algorithm>
#include "File.h"
#include "analyzer/Analyzer.h"
#include "TCP_Reassembler.h"
#include "analyzer/protocol/tcp/TCP.h"

1
src/analyzer/protocol/tcp/functions.bif
View file

@ -1,5 +1,6 @@
%%{
#include "File.h"
#include "analyzer/protocol/tcp/TCP.h"
%%}

6
src/bro.bif
View file

@ -4540,7 +4540,7 @@ function get_file_name%(f: file%): string
## after the rotation, and the time when *f* was opened/closed.
##
## .. zeek:see:: rotate_file_by_name calc_next_rotate
function rotate_file%(f: file%): rotate_info
function rotate_file%(f: file%): rotate_info &deprecated
%{
RecordVal* info = f->Rotate();
if ( info )
@ -4564,7 +4564,7 @@ function rotate_file%(f: file%): rotate_info
## after the rotation, and the time when *f* was opened/closed.
##
## .. zeek:see:: rotate_file calc_next_rotate
function rotate_file_by_name%(f: string%): rotate_info
function rotate_file_by_name%(f: string%): rotate_info &deprecated
%{
RecordVal* info = new RecordVal(rotate_info);
@ -4618,7 +4618,7 @@ function rotate_file_by_name%(f: string%): rotate_info
## Returns: The duration until the next file rotation time.
##
## .. zeek:see:: rotate_file rotate_file_by_name
function calc_next_rotate%(i: interval%) : interval
function calc_next_rotate%(i: interval%) : interval &deprecated
%{
const char* base_time = log_rotate_base_time ?
log_rotate_base_time->AsString()->CheckString() : 0;

1
src/broker/Data.cc
View file

@ -1,4 +1,5 @@
#include "Data.h"
#include "File.h"
#include "broker/data.bif.h"
#include <caf/stream_serializer.hpp>
#include <caf/stream_deserializer.hpp>

1
src/broker/Manager.h
View file

@ -13,6 +13,7 @@
#include "Reporter.h"
#include "iosource/IOSource.h"
#include "Val.h"
#include "logging/WriterBackend.h"
namespace bro_broker {

182
src/event.bif
View file

@ -600,188 +600,6 @@ event software_unparsed_version_found%(c: connection, host: addr, str: string%);
## generate_OS_version_event
event OS_version_found%(c: connection, host: addr, OS: OS_version%);
## Generated when a connection to a remote Bro has been established. This event
## is intended primarily for use by Bro's communication framework, but it can
## also trigger additional code if helpful.
##
## p: A record describing the peer.
##
## .. zeek:see:: remote_capture_filter remote_connection_closed remote_connection_error
## remote_connection_handshake_done remote_event_registered remote_log remote_pong
## remote_state_access_performed remote_state_inconsistency print_hook
event remote_connection_established%(p: event_peer%);
## Generated when a connection to a remote Bro has been closed. This event is
## intended primarily for use by Bro's communication framework, but it can
## also trigger additional code if helpful.
##
## p: A record describing the peer.
##
## .. zeek:see:: remote_capture_filter remote_connection_error
## remote_connection_established remote_connection_handshake_done
## remote_event_registered remote_log remote_pong remote_state_access_performed
## remote_state_inconsistency print_hook
event remote_connection_closed%(p: event_peer%);
## Generated when a remote connection's initial handshake has been completed.
## This event is intended primarily for use by Bro's communication framework,
## but it can also trigger additional code if helpful.
##
## p: A record describing the peer.
##
## .. zeek:see:: remote_capture_filter remote_connection_closed remote_connection_error
## remote_connection_established remote_event_registered remote_log remote_pong
## remote_state_access_performed remote_state_inconsistency print_hook
event remote_connection_handshake_done%(p: event_peer%);
## Generated for each event registered by a remote peer. This event is intended
## primarily for use by Bro's communication framework, but it can also trigger
## additional code if helpful.
##
## p: A record describing the peer.
##
## name: TODO.
##
## .. zeek:see:: remote_capture_filter remote_connection_closed
## remote_connection_error remote_connection_established
## remote_connection_handshake_done remote_log remote_pong
## remote_state_access_performed remote_state_inconsistency print_hook
event remote_event_registered%(p: event_peer, name: string%);
## Generated when a connection to a remote Bro encountered an error. This event
## is intended primarily for use by Bro's communication framework, but it can
## also trigger additional code if helpful.
##
## p: A record describing the peer.
##
## reason: A textual description of the error.
##
## .. zeek:see:: remote_capture_filter remote_connection_closed
## remote_connection_established remote_connection_handshake_done
## remote_event_registered remote_log remote_pong remote_state_access_performed
## remote_state_inconsistency print_hook
event remote_connection_error%(p: event_peer, reason: string%);
## Generated when a remote peer sent us a capture filter. While this event is
## intended primarily for use by Bro's communication framework, it can also
## trigger additional code if helpful.
##
## p: A record describing the peer.
##
## filter: The filter string sent by the peer.
##
## .. zeek:see:: remote_connection_closed remote_connection_error
## remote_connection_established remote_connection_handshake_done
## remote_event_registered remote_log remote_pong remote_state_access_performed
## remote_state_inconsistency print_hook
event remote_capture_filter%(p: event_peer, filter: string%);
## Generated if state synchronization detects an inconsistency. While this
## event is intended primarily for use by Bro's communication framework, it can
## also trigger additional code if helpful. This event is only raised if
## :zeek:id:`remote_check_sync_consistency` is false.
##
## operation: The textual description of the state operation performed.
##
## id: The name of the Bro script identifier that was operated on.
##
## expected_old: A textual representation of the value of *id* that was
## expected to be found before the operation was carried out.
##
## real_old: A textual representation of the value of *id* that was actually
## found before the operation was carried out. The difference between
## *real_old* and *expected_old* is the inconsistency being reported.
##
## .. zeek:see:: remote_capture_filter remote_connection_closed
## remote_connection_error remote_connection_established
## remote_connection_handshake_done remote_event_registered remote_log remote_pong
## remote_state_access_performed print_hook remote_check_sync_consistency
event remote_state_inconsistency%(operation: string, id: string,
expected_old: string, real_old: string%);
## Generated for communication log messages. While this event is
## intended primarily for use by Bro's communication framework, it can also
## trigger additional code if helpful.
##
## level: The log level, which is either :zeek:id:`REMOTE_LOG_INFO` or
## :zeek:id:`REMOTE_LOG_ERROR`.
##
## src: The component of the communication system that logged the message.
## Currently, this will be one of :zeek:id:`REMOTE_SRC_CHILD` (Bro's
## child process), :zeek:id:`REMOTE_SRC_PARENT` (Bro's main process), or
## :zeek:id:`REMOTE_SRC_SCRIPT` (the script level).
##
## msg: The message logged.
##
## .. zeek:see:: remote_capture_filter remote_connection_closed remote_connection_error
## remote_connection_established remote_connection_handshake_done
## remote_event_registered remote_pong remote_state_access_performed
## remote_state_inconsistency print_hook remote_log_peer
event remote_log%(level: count, src: count, msg: string%);
## Generated for communication log messages. While this event is
## intended primarily for use by Bro's communication framework, it can also
## trigger additional code if helpful. This event is equivalent to
## :zeek:see:`remote_log` except the message is with respect to a certain peer.
##
## p: A record describing the remote peer.
##
## level: The log level, which is either :zeek:id:`REMOTE_LOG_INFO` or
## :zeek:id:`REMOTE_LOG_ERROR`.
##
## src: The component of the communication system that logged the message.
## Currently, this will be one of :zeek:id:`REMOTE_SRC_CHILD` (Bro's
## child process), :zeek:id:`REMOTE_SRC_PARENT` (Bro's main process), or
## :zeek:id:`REMOTE_SRC_SCRIPT` (the script level).
##
## msg: The message logged.
##
## .. zeek:see:: remote_capture_filter remote_connection_closed remote_connection_error
## remote_connection_established remote_connection_handshake_done
## remote_event_registered remote_pong remote_state_access_performed
## remote_state_inconsistency print_hook remote_log
event remote_log_peer%(p: event_peer, level: count, src: count, msg: string%);
## Generated when a remote peer has answered to our ping. This event is part of
## Bro's infrastructure for measuring communication latency. One can send a ping
## by calling ``send_ping`` and when a corresponding reply is received,
## this event will be raised.
##
## p: The peer sending us the pong.
##
## seq: The sequence number passed to the original ``send_ping`` call.
## The number is sent back by the peer in its response.
##
## d1: The time interval between sending the ping and receiving the pong. This
## is the latency of the complete path.
##
## d2: The time interval between sending out the ping to the network and its
## reception at the peer. This is the network latency.
##
## d3: The time interval between when the peer's child process received the
## ping and when its parent process sent the pong. This is the
## processing latency at the peer.
##
## .. zeek:see:: remote_capture_filter remote_connection_closed remote_connection_error
## remote_connection_established remote_connection_handshake_done
## remote_event_registered remote_log remote_state_access_performed
## remote_state_inconsistency print_hook
event remote_pong%(p: event_peer, seq: count,
d1: interval, d2: interval, d3: interval%);
## Generated each time a remote state access has been replayed locally. This
## event is primarily intended for debugging.
##
## id: The name of the Bro script variable that's being operated on.
##
## v: The new value of the variable.
##
## .. zeek:see:: remote_capture_filter remote_connection_closed remote_connection_error
## remote_connection_established remote_connection_handshake_done
## remote_event_registered remote_log remote_pong remote_state_inconsistency
## print_hook
event remote_state_access_performed%(id: string, v: any%);
## Generated each time Bro's internal profiling log is updated. The file is
## defined by :zeek:id:`profiling_file`, and its update frequency by
## :zeek:id:`profiling_interval` and :zeek:id:`expensive_profiling_multiple`.

1
src/file_analysis/analyzer/extract/Extract.cc
View file

@ -1,6 +1,7 @@
// See the file "COPYING" in the main distribution directory for copyright.
#include <string>
#include <fcntl.h>
#include "Extract.h"
#include "util.h"

1
src/input/Manager.h
View file

@ -7,7 +7,6 @@
#include "BroString.h"
#include "EventHandler.h"
#include "RemoteSerializer.h"
#include "Val.h"
#include "Component.h"

2
src/iosource/Packet.cc
View file

@ -2,6 +2,8 @@
#include "Packet.h"
#include "Sessions.h"
#include "iosource/Manager.h"
#include "SerialInfo.h"
#include "Serializer.h"
extern "C" {
#ifdef HAVE_NET_ETHERNET_H

24
src/iosource/PktSrc.cc
View file

@ -160,21 +160,6 @@ double PktSrc::CheckPseudoTime()
if ( ! ExtractNextPacketInternal() )
return 0;
if ( remote_trace_sync_interval )
{
if ( next_sync_point == 0 || current_packet.time >= next_sync_point )
{
int n = remote_serializer->SendSyncPoint();
next_sync_point = first_timestamp +
n * remote_trace_sync_interval;
remote_serializer->Log(RemoteSerializer::LogInfo,
fmt("stopping at packet %.6f, next sync-point at %.6f",
current_packet.time, next_sync_point));
return 0;
}
}
double pseudo_time = current_packet.time - first_timestamp;
double ct = (current_time(true) - first_wallclock) * pseudo_realtime;
@ -308,15 +293,6 @@ bool PktSrc::ExtractNextPacketInternal()
if ( pseudo_realtime && ! IsOpen() )
{
if ( using_communication )
{
// Source has gone dry, we're done.
if ( remote_trace_sync_interval )
remote_serializer->SendFinalSyncPoint();
else
remote_serializer->Terminate();
}
if ( broker_mgr->Active() )
iosource_mgr->Terminate();
}

41
src/logging/Manager.cc
View file

@ -2,11 +2,12 @@
#include <algorithm>
#include "../Event.h"
#include "../EventHandler.h"
#include "../NetVar.h"
#include "../Net.h"
#include "../Type.h"
#include "Event.h"
#include "EventHandler.h"
#include "NetVar.h"
#include "Net.h"
#include "Type.h"
#include "File.h"
#include "broker/Manager.h"
#include "threading/Manager.h"
@ -16,8 +17,8 @@
#include "WriterFrontend.h"
#include "WriterBackend.h"
#include "logging.bif.h"
#include "../plugin/Plugin.h"
#include "../plugin/Manager.h"
#include "plugin/Plugin.h"
#include "plugin/Manager.h"
using namespace logging;
@ -1300,32 +1301,6 @@ bool Manager::WriteFromRemote(EnumVal* id, EnumVal* writer, string path, int num
return true;
}
void Manager::SendAllWritersTo(RemoteSerializer::PeerID peer)
{
auto et = internal_type("Log::Writer")->AsEnumType();
for ( vector<Stream *>::iterator s = streams.begin(); s != streams.end(); ++s )
{
Stream* stream = (*s);
if ( ! (stream && stream->enable_remote) )
continue;
for ( Stream::WriterMap::iterator i = stream->writers.begin();
i != stream->writers.end(); i++ )
{
WriterFrontend* writer = i->second->writer;
auto writer_val = et->GetVal(i->first.first);
remote_serializer->SendLogCreateWriter(peer, (*s)->id,
writer_val,
*i->second->info,
writer->NumFields(),
writer->Fields());
Unref(writer_val);
}
}
}
void Manager::SendAllWritersTo(const broker::endpoint_info& ei)
{
auto et = internal_type("Log::Writer")->AsEnumType();

6
src/logging/Manager.h
View file

@ -10,14 +10,12 @@
#include "../Val.h"
#include "../Tag.h"
#include "../EventHandler.h"
#include "../RemoteSerializer.h"
#include "../plugin/ComponentManager.h"
#include "Component.h"
#include "WriterBackend.h"
class SerializationFormat;
class RemoteSerializer;
class RotationTimer;
namespace logging {
@ -234,7 +232,6 @@ protected:
friend class WriterFrontend;
friend class RotationFinishedMessage;
friend class RotationFailedMessage;
friend class ::RemoteSerializer;
friend class ::RotationTimer;
// Instantiates a new WriterBackend of the given type (note that
@ -248,9 +245,6 @@ protected:
int num_fields, const threading::Field* const* fields,
bool local, bool remote, bool from_remote, const string& instantiating_filter="");
// Announces all instantiated writers to peer.
void SendAllWritersTo(RemoteSerializer::PeerID peer);
// Signals that a file has been rotated.
bool FinishedRotation(WriterFrontend* writer, const char* new_name, const char* old_name,
double open, double close, bool success, bool terminating);

1
src/logging/WriterBackend.cc
View file

@ -4,6 +4,7 @@
#include "util.h"
#include "threading/SerialTypes.h"
#include "SerializationFormat.h"
#include "Manager.h"
#include "WriterBackend.h"

2
src/logging/WriterBackend.h
View file

@ -9,8 +9,6 @@
#include "Component.h"
class RemoteSerializer;
namespace broker { class data; }
namespace logging {

12
src/logging/WriterFrontend.cc
View file

@ -169,12 +169,6 @@ void WriterFrontend::Init(int arg_num_fields, const Field* const * arg_fields)
if ( remote )
{
remote_serializer->SendLogCreateWriter(stream,
writer,
*info,
arg_num_fields,
arg_fields);
broker_mgr->PublishLogCreate(stream,
writer,
*info,
@ -201,12 +195,6 @@ void WriterFrontend::Write(int arg_num_fields, Value** vals)
if ( remote )
{
remote_serializer->SendLogWrite(stream,
writer,
info->path,
num_fields,
vals);
broker_mgr->PublishLogWrite(stream,
writer,
info->path,

40
src/main.cc
View file

@ -39,8 +39,6 @@ extern "C" {
#include "RuleMatcher.h"
#include "Anon.h"
#include "Serializer.h"
#include "RemoteSerializer.h"
#include "PersistenceSerializer.h"
#include "EventRegistry.h"
#include "Stats.h"
#include "Brofiler.h"
@ -101,10 +99,8 @@ name_list prefixes;
Stmt* stmts;
EventHandlerPtr net_done = 0;
RuleMatcher* rule_matcher = 0;
PersistenceSerializer* persistence_serializer = 0;
FileSerializer* event_serializer = 0;
FileSerializer* state_serializer = 0;
RemoteSerializer* remote_serializer = 0;
EventPlayer* event_player = 0;
EventRegistry* event_registry = 0;
ProfileLogger* profiling_logger = 0;
@ -167,7 +163,6 @@ void usage(int code = 1)
fprintf(stderr, " -d|--debug-policy | activate policy file debugging\n");
fprintf(stderr, " -e|--exec <bro code> | augment loaded policies by given code\n");
fprintf(stderr, " -f|--filter <filter> | tcpdump filter\n");
fprintf(stderr, " -g|--dump-config | dump current config into .state dir\n");
fprintf(stderr, " -h|--help | command line help\n");
fprintf(stderr, " -i|--iface <interface> | read from given interface\n");
fprintf(stderr, " -p|--prefix <prefix> | add given prefix to policy file resolution\n");
@ -275,10 +270,6 @@ void done_with_network()
{
set_processing_status("TERMINATING", "done_with_network");
// Release the port, which is important for checkpointing Bro.
if ( remote_serializer )
remote_serializer->StopListening();
// Cancel any pending alarms (watchdog, in particular).
(void) alarm(0);
@ -291,9 +282,6 @@ void done_with_network()
true);
}
// Save state before expiring the remaining events/timers.
persistence_serializer->WriteState(false);
if ( profiling_logger )
profiling_logger->Log();
@ -305,9 +293,6 @@ void done_with_network()
mgr.Drain();
mgr.Drain();
if ( remote_serializer )
remote_serializer->Finish();
net_finish(1);
#ifdef USE_PERFTOOLS_DEBUG
@ -355,9 +340,6 @@ void terminate_bro()
delete profiling_logger;
}
if ( remote_serializer )
remote_serializer->LogStats();
mgr.Drain();
log_mgr->Terminate();
@ -371,7 +353,6 @@ void terminate_bro()
delete zeekygen_mgr;
delete timer_mgr;
delete persistence_serializer;
delete event_serializer;
delete state_serializer;
delete event_registry;
@ -452,7 +433,6 @@ int main(int argc, char** argv)
char* debug_streams = 0;
int parse_only = false;
int bare_mode = false;
int dump_cfg = false;
int do_watchdog = 0;
int override_ignore_checksums = 0;
int rule_debug = 0;
@ -464,7 +444,6 @@ int main(int argc, char** argv)
{"parse-only", no_argument, 0, 'a'},
{"bare-mode", no_argument, 0, 'b'},
{"debug-policy", no_argument, 0, 'd'},
{"dump-config", no_argument, 0, 'g'},
{"exec", required_argument, 0, 'e'},
{"filter", required_argument, 0, 'f'},
{"help", no_argument, 0, 'h'},
@ -565,10 +544,6 @@ int main(int argc, char** argv)
user_pcap_filter = optarg;
break;
case 'g':
dump_cfg = true;
break;
case 'h':
usage(0);
break;
@ -795,8 +770,6 @@ int main(int argc, char** argv)
dns_mgr->SetDir(".state");
iosource_mgr = new iosource::Manager();
persistence_serializer = new PersistenceSerializer();
remote_serializer = new RemoteSerializer();
event_registry = new EventRegistry();
analyzer_mgr = new analyzer::Manager();
log_mgr = new logging::Manager();
@ -1012,13 +985,9 @@ int main(int argc, char** argv)
exit(0);
}
persistence_serializer->SetDir((const char *)state_dir->AsString()->CheckString());
// Print the ID.
if ( id_name )
{
persistence_serializer->ReadAll(true, false);
ID* id = global_scope()->Lookup(id_name);
if ( ! id )
reporter->FatalError("No such ID: %s\n", id_name);
@ -1032,14 +1001,6 @@ int main(int argc, char** argv)
exit(0);
}
persistence_serializer->ReadAll(true, true);
if ( dump_cfg )
{
persistence_serializer->WriteConfig(false);
exit(0);
}
if ( profiling_interval > 0 )
{
profiling_logger = new ProfileLogger(profiling_file->AsFile(),
@ -1205,7 +1166,6 @@ int main(int argc, char** argv)
}
else
{
persistence_serializer->WriteState(false);
terminate_bro();
}

7
src/parse.y
View file

@ -5,7 +5,7 @@
// Switching parser table type fixes ambiguity problems.
%define lr.type ielr
%expect 141
%expect 129
%token TOK_ADD TOK_ADD_TO TOK_ADDR TOK_ANY
%token TOK_ATENDIF TOK_ATELSE TOK_ATIF TOK_ATIFDEF TOK_ATIFNDEF
@ -25,7 +25,6 @@
%token TOK_ATTR_OPTIONAL TOK_ATTR_REDEF TOK_ATTR_ROTATE_INTERVAL
%token TOK_ATTR_ROTATE_SIZE TOK_ATTR_DEL_FUNC TOK_ATTR_EXPIRE_FUNC
%token TOK_ATTR_EXPIRE_CREATE TOK_ATTR_EXPIRE_READ TOK_ATTR_EXPIRE_WRITE
%token TOK_ATTR_PERSISTENT TOK_ATTR_SYNCHRONIZED
%token TOK_ATTR_RAW_OUTPUT TOK_ATTR_MERGEABLE
%token TOK_ATTR_PRIORITY TOK_ATTR_LOG TOK_ATTR_ERROR_HANDLER
%token TOK_ATTR_TYPE_COLUMN TOK_ATTR_DEPRECATED
@ -1308,10 +1307,6 @@ attr:
{ $$ = new Attr(ATTR_EXPIRE_READ, $3); }
| TOK_ATTR_EXPIRE_WRITE '=' expr
{ $$ = new Attr(ATTR_EXPIRE_WRITE, $3); }
| TOK_ATTR_PERSISTENT
{ $$ = new Attr(ATTR_PERSISTENT); }
| TOK_ATTR_SYNCHRONIZED
{ $$ = new Attr(ATTR_SYNCHRONIZED); }
| TOK_ATTR_ENCRYPT
{ $$ = new Attr(ATTR_ENCRYPT); }
| TOK_ATTR_ENCRYPT '=' expr

10
src/scan.l
View file

@ -310,11 +310,6 @@ when return TOK_WHEN;
return TOK_ATTR_MERGEABLE;
}
&persistent {
deprecated_attr(yytext);
return TOK_ATTR_PERSISTENT;
}
&rotate_interval {
deprecated_attr(yytext);
return TOK_ATTR_ROTATE_INTERVAL;
@ -325,11 +320,6 @@ when return TOK_WHEN;
return TOK_ATTR_ROTATE_SIZE;
}
&synchronized {
deprecated_attr(yytext);
return TOK_ATTR_SYNCHRONIZED;
}
@deprecated.* {
auto num_files = file_stack.length();
auto comment = skip_whitespace(yytext + 11);

4
src/threading/SerialTypes.cc
View file

@ -2,8 +2,8 @@
#include "SerialTypes.h"
#include "../RemoteSerializer.h"
#include "SerializationFormat.h"
#include "Reporter.h"
using namespace threading;

3
src/threading/SerialTypes.h
View file

@ -13,7 +13,6 @@
using namespace std;
class SerializationFormat;
class RemoteSerializer;
namespace threading {
@ -78,8 +77,6 @@ struct Field {
string TypeName() const;
private:
friend class ::RemoteSerializer;
// Force usage of constructor above.
Field() {}
};

4
testing/btest/Baseline/coverage.bare-mode-errors/errors
View file

@ -0,0 +1,4 @@
warning in /Users/johanna/bro/master/scripts/policy/misc/trim-trace-file.zeek, line 25: deprecated (rotate_file_by_name)
warning in /Users/johanna/bro/master/scripts/policy/misc/trim-trace-file.zeek, line 25: deprecated (rotate_file_by_name)
warning in /Users/johanna/bro/master/scripts/policy/misc/trim-trace-file.zeek, line 25: deprecated (rotate_file_by_name)
warning in /Users/johanna/bro/master/testing/btest/../../scripts//policy/misc/trim-trace-file.zeek, line 25: deprecated (rotate_file_by_name)