Commit graph

123 commits

Author SHA1 Message Date
Johanna Amann
67917b83aa Merge remote-tracking branch 'origin/master' into topic/johanna/table-changes 2020-07-09 17:02:57 -07:00
Jon Siwek
7669f560d1 Integrate Supervisor code review suggestions 2020-07-09 13:56:11 -07:00
Johanna Amann
3eac12b40d BrokerStore<->Zeek Tables Fix a few small test failures. 2020-07-09 19:43:45 +00:00
Jon Siwek
a06ef66edc Add Log::rotation_format_func and Log::default_rotation_dir options
These may be redefined to customize log rotation path prefixes,
including use of a directory.  File extensions are still up to
individual log writers to add themselves during the actual rotation.

These new also allow for some simplication to the default
ASCII postprocessor function: it eliminates the need for it doing an
extra/awkward rename() operation that only changes the timestamp format.

This also teaches the supervisor framework to use these new options
to rotate ascii logs into a log-queue/ directory with a specific
file name format (intended for an external archiver process to
monitor separately).
2020-07-07 18:42:37 -07:00
Johanna Amann
a220b02722 BrokerStore<->Zeek tables: &backend works for in-memory stores.
Currently this requires using this with a normal cluster - or sending
messages by yourself.

It, in principle, should also work with SQLITE - but that is a bit
nonsensical without being able to change the storage location.
2020-07-01 16:38:10 -07:00
Johanna Amann
318a72c303 BrokerStore<->Zeek table - introdude &backend attribute
The &backend attribute allows for a much more convenient way of
interacting with brokerstores. One does not need to create a broker
store anymore - instead all of this is done internally.

The current state of this partially works. This should work fine for
persistence - but clones are currently not yet correctly attached.
2020-06-30 16:33:52 -07:00
Vern Paxson
fe46ef06a0 unused variables found via use-def analysis (plus an indentation micro-nit) 2020-04-25 18:06:47 -07:00
Jon Siwek
ce9183a2ed Fix Broker topics used to uniquely identify cluster nodes
Node-specific topic prefix subscriptions/publications now add a trailing
slash like "zeek/cluster/node/<name>/".  Without the trailing slash,
messages attempting to target "proxy-10" may also be sent to "proxy-1"
since subscription matching is prefix-based.
2020-04-10 14:36:00 -07:00
Jon Siwek
b114766205 Add Supervisor::is_supervised()
And use to to avoid redef'ing the log rotation postprocessor to
"archive-log" by default since it's unlikely PATH is configured to find
that script.
2020-01-07 10:27:07 -08:00
Jon Siwek
29f386e388 Implement minimal supervised cluster configuration
More aspects of the cluster configuration to get fleshed out later,
but a basic cluster like one would use for a live deployment
can now be instantiated and run under supervision.  The new
clusterized-pcap-processing supervisor mode is also not done yet.
2019-10-23 17:37:53 -07:00
Jon Siwek
1ce0fcce49 GH-387: update Broker topic names to use "zeek/" prefix 2019-05-29 15:56:37 -07:00
Daniel Thayer
1a74516db1 Rename all BRO-prefixed environment variables
For backward compatibility when reading values, we first check
the ZEEK-prefixed value, and if not set, then check the corresponding
BRO-prefixed value.
2019-05-22 00:12:31 -05:00
Daniel Thayer
be182aac83 More bro-to-zeek renaming in scripts and other files 2019-05-16 02:36:41 -05:00
Jon Siwek
6ad7099f7e Merge remote-tracking branch 'origin/topic/robin/gh-239'
* origin/topic/robin/gh-239:
  Undo a change to btest.cfg from a recent commit
  Updating submodule.
  Fix zeek-wrapper
  Update for renaming BroControl to ZeekControl.
  Updating submodule.
  GH-239: Rename bro to zeek, bro-config to zeek-config, and bro-path-dev to zeek-path-dev.
2019-05-14 13:27:40 -07:00
Robin Sommer
89b8d6e7ba Update for renaming BroControl to ZeekControl. 2019-05-06 21:22:19 +00:00
Johanna Amann
dcd6454530 Remove RemoteSerializer and related code/types.
Also removes broccoli from the source tree.
2019-05-03 15:00:13 -07:00
Jon Siwek
aebcb1415d GH-234: rename Broxygen to Zeexygen along with roles/directives
* All "Broxygen" usages have been replaced in
  code, documentation, filenames, etc.

* Sphinx roles/directives like ":bro:see" are now ":zeek:see"

* The "--broxygen" command-line option is now "--zeexygen"
2019-04-22 19:45:50 -07:00
Jon Siwek
a994be9eeb Merge remote-tracking branch 'origin/topic/seth/zeek_init'
* origin/topic/seth/zeek_init:
  Some more testing fixes.
  Update docs and tests for bro_(init|done) -> zeek_(init|done)
  Implement the zeek_init handler.
2019-04-19 11:24:29 -07:00
Seth Hall
8cefb9be42 Implement the zeek_init handler.
Implements the change and a test.
2019-04-14 08:37:35 -04:00
Daniel Thayer
18bd74454b Rename all scripts to have ".zeek" file extension 2019-04-11 21:12:40 -05:00
Jon Siwek
01d303b480 Migrate table-based for-loops to key-value iteration 2019-03-15 19:54:44 -07:00
Jon Siwek
e0b8b4b6b1 Replace some bro.org usages with zeek.org 2019-01-04 17:51:25 -06:00
Jon Siwek
73c8cf733a Give Cluster::rr_topic "key" argument a default value 2018-09-07 09:58:57 -05:00
Jon Siwek
8db042a8c2 Remove Cluster::broadcast_topic
As enabling Broker forwarding would cause routing loops with messages
sent to such a topic (one subscribed to on all nodes).
2018-08-28 16:40:48 -05:00
Jon Siwek
da9f91fc19 Add env. variables to override Broker listen/connect retry intervals
And use them to default retries to 1sec for all unit tests.
2018-08-16 12:16:03 -05:00
Vern Paxson
88fd7510c6 reap the fruits of v += e 2018-07-26 12:51:36 -07:00
Daniel Thayer
ceefb6edaf Fix minor typos in broker reference documentation 2018-06-27 13:00:09 -05:00
Jon Siwek
8ee92c70da Add ability for BroControl to skip cluster setup
It does this by setting the BROCTL_CHECK_CONFIG env. variable.
Related to BIT-1938.
2018-06-21 11:57:18 -05:00
Jon Siwek
45178f3051 Add a counter for number of alive nodes within a given cluster pool 2018-05-24 14:33:35 -05:00
Jon Siwek
95ea84e60e Fix how cluster framework tracks worker count 2018-05-24 14:32:45 -05:00
Robin Sommer
fe7e1ee7f0 Merge topic/actor-system throug a squashed commit. 2018-05-18 22:39:23 +00:00
Daniel Thayer
59f0ea1a0c Add a test for starting a cluster with a logger node
Also modified cluster framework scripts so that any node can raise an
event to be handled on the logger node.
2016-07-15 15:23:49 -05:00
Daniel Thayer
b80298a9ad Change how logger node is detected in cluster framework
Instead of assuming the logger node always has the name "logger", now
broctl will set a boolean which the cluster framework scripts can use
to determine if there is a logger node or not.

Also removed one line from the manager node script, because it has to do
with logging, which the logger.bro script handles.
2016-07-05 10:36:03 -05:00
Daniel Thayer
ce0b9d42d3 Update docs for the new logger node type 2016-06-30 15:45:36 -05:00
Daniel Thayer
f45a3e8878 Add a new node type for logging
Changed the cluster framework scripts by adding a new Bro node type
for doing logging (this is intended to reduce the load on the manager).
If a user chooses not to specify a logger node in the cluster
configuration, then the manager will write logs locally as usual.
2016-06-29 17:55:49 -05:00
Daniel Thayer
2d9127888f Add some missing Bro script documentation
Also fixed a few reST formatting issues.
2016-05-05 16:35:31 -05:00
Robin Sommer
17bc615467 Making cluster reconnect timeout configurable. 2015-11-17 13:43:58 -08:00
Jon Siwek
186e67ec1d Allow logging filters to inherit default path from stream.
This allows the path for the default filter to be specified explicitly
when creating a stream and reduces the need to rely on the default path
function to magically supply the path.

The default path function is now only used if, when a filter is added to
a stream, it has neither a path nor a path function already.

Adapted the existing Log::create_stream calls to explicitly specify a
path value.

Addresses BIT-1324
2015-03-19 14:49:55 -05:00
Daniel Thayer
4b8171f74a Document which Bro script vars are set by BroControl 2013-10-22 16:40:29 -05:00
Daniel Thayer
60b2c5f1fe Add README files for most Bro frameworks
The text from these README files appears on the "Bro Script Packages"
page after building the documentation.  The text for these was mostly just
copied from the existing docs.
2013-10-11 00:19:37 -05:00
Daniel Thayer
f21f45ac1a Fix typos and formatting in the cluster framework docs
Changed a Time Machine link to point to something useful.
2013-10-10 14:39:44 -05:00
Robin Sommer
f830ed3edf s/bro-ids.org/bro.org/g 2013-03-07 19:33:04 -08:00
Daniel Thayer
c46029d828 Rename the alarm summary mail variable 2012-10-30 16:31:18 -05:00
Daniel Thayer
5f8cbf8a51 Change alarm summary email interval to 24hrs 2012-05-31 17:30:48 -05:00
Robin Sommer
f7261a7851 Merge remote-tracking branch 'origin/topic/jsiwek/ipv6-comm'
* origin/topic/jsiwek/ipv6-comm:
  Enable Bro to communicate with peers over non-global IPv6 addresses.
  Add unit tests for Broccoli SSL and Broccoli IPv6 connectivity.
  Remove AI_ADDRCONFIG getaddrinfo hints flag for listening sockets.
  Undo communication protocol version bump.
  Add support to Bro for connecting with peers over IPv6.

Closes #820.

Conflicts:
	src/bro.bif
2012-05-24 17:01:34 -07:00
Jon Siwek
74f3a32321 Enable Bro to communicate with peers over non-global IPv6 addresses.
This usually requires specifying an additional zone identifier
(see RFC 4007).  The connect() and listen() BIFs have been
changed to accept this zone identifier as an argument.
2012-05-17 12:59:20 -05:00
Jon Siwek
a4117016e9 Merge branch 'master' into topic/script-reference
Conflicts:
	aux/broccoli
	aux/broctl
	scripts/base/frameworks/notice/main.bro
	src/event.bif
2011-12-19 16:17:58 -06:00
Robin Sommer
8c53446292 Merge remote branch 'origin/fastpath'
* origin/fastpath:
  Fixed major bug with cluster synchronization (it was broken!)
2011-12-16 02:37:56 -08:00
Seth Hall
0b8b14a0ed Fixed major bug with cluster synchronization (it was broken!) 2011-12-15 15:59:51 -05:00
Jon Siwek
8e89d78788 Add more cluster and communication framework documentation. 2011-12-09 17:31:47 -06:00