Mirror/zeek
1
0
Fork 0
mirror of https://github.com/zeek/zeek.git synced 2025-10-11 02:58:20 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 2

Changing semantics of Broker's remote logging to match old communication framework.

Browse source 
Broker had changed the semantics of remote logging: it sent over the
original Bro record containing the values to be logged, which on the
receiving side would then pass through the logging framework normally,
including triggering filters and events. The old communication system
however special-cases logs: it sends already processed log entries,
just as they go into the log files, and without any receiver-side
filtering etc. This more efficient as it short-cuts the processing
path, and also avoids the more expensive Val serialization. It also
lets the sender determine the specifics of what gets logged (and how).

This commit changes Broker over to now use the same semantics as the
old communication system.

TODOs:
     - The new Broker code doesn't have consistent #ifdefs yet.

     - Right now, when a new log receiver connects, all existing logs
     are broadcasted out again to all current clients. That doesn't so
     any harm, but is unncessary. Need to add a way to send the
     existing logs to just the new client.
This commit is contained in:
Robin Sommer 2017-01-31 21:51:31 -08:00
parent 0dd0bfb5bb
commit a5e9a535a5
25 changed files with 1178 additions and 159 deletions
Download patch file Download diff file Expand all files Collapse all files

34
src/broker/Manager.h
View file

@ -156,15 +156,34 @@ public:
/**
* Send a log entry to any interested peers. The topic name used is
* implicitly "bro/log/<stream-name>".
* @param stream_id the stream to which the log entry belongs.
* @param columns the data which comprises the log entry.
* @param info the record type corresponding to the log's columns.
* @param stream the stream to which the log entry belongs.
* @param writer the writer to use for outputting this log entry.
* @param path the log path to output the log entry to.
* @param num_vals the number of fields to log.
* @param vals the log values to log, of size num_vals.
* @param flags tune the behavior of how the message is send.
* See the Broker::SendFlags record type.
* @return true if the message is sent successfully.
*/
bool Log(EnumVal* stream_id, RecordVal* columns, RecordType* info,
int flags);
bool Log(EnumVal* stream, EnumVal* writer, string path, int num_vals,
const threading::Value* const * vals, int flags);
/**
* Send a message to create a log stream to any interested peers.
* The log stream may or may not already exist on the receiving side.
* The topic name used is implicitly "bro/log/<stream-name>".
* @param stream the stream to which the log entry belongs.
* @param writer the writer to use for outputting this log entry.
* @param info backend initialization information for the writer.
* @param num_fields the number of fields the log has.
* @param fields the log's fields of size num_fields.
* @param flags tune the behavior of how the message is send.
* See the Broker::SendFlags record type.
* @param peer If given, send the message only to this peer.
* @return true if the message is sent successfully.
*/
bool CreateLog(EnumVal* id, EnumVal* writer, const logging::WriterBackend::WriterInfo& info,
int num_fields, const threading::Field* const * fields, int flags, const string& peer = "");
/**
* Automatically send an event to any interested peers whenever it is
@ -325,7 +344,6 @@ public:
static int send_flags_to_int(Val* flags);
private:
// IOSource interface overrides:
void GetFds(iosource::FD_Set* read, iosource::FD_Set* write,
iosource::FD_Set* except) override;
@ -340,6 +358,9 @@ private:
broker::endpoint& Endpoint()
{ return *endpoint; }
bool ProcessCreateLog(broker::message msg);
bool ProcessWriteLog(broker::message msg);
struct QueueWithStats {
broker::message_queue q;
size_t received = 0;
@ -360,6 +381,7 @@ private:
static VectorType* vector_of_data_type;
static EnumType* log_id_type;
static EnumType* writer_id_type;
static int send_flags_self_idx;
static int send_flags_peers_idx;
static int send_flags_unsolicited_idx;