Commit graph

68 commits

Author SHA1 Message Date
Jon Siwek
a06ef66edc Add Log::rotation_format_func and Log::default_rotation_dir options
These may be redefined to customize log rotation path prefixes,
including use of a directory.  File extensions are still up to
individual log writers to add themselves during the actual rotation.

These new also allow for some simplication to the default
ASCII postprocessor function: it eliminates the need for it doing an
extra/awkward rename() operation that only changes the timestamp format.

This also teaches the supervisor framework to use these new options
to rotate ascii logs into a log-queue/ directory with a specific
file name format (intended for an external archiver process to
monitor separately).
2020-07-07 18:42:37 -07:00
Vern Paxson
fe46ef06a0 unused variables found via use-def analysis (plus an indentation micro-nit) 2020-04-25 18:06:47 -07:00
Jon Siwek
ce9183a2ed Fix Broker topics used to uniquely identify cluster nodes
Node-specific topic prefix subscriptions/publications now add a trailing
slash like "zeek/cluster/node/<name>/".  Without the trailing slash,
messages attempting to target "proxy-10" may also be sent to "proxy-1"
since subscription matching is prefix-based.
2020-04-10 14:36:00 -07:00
Jon Siwek
b114766205 Add Supervisor::is_supervised()
And use to to avoid redef'ing the log rotation postprocessor to
"archive-log" by default since it's unlikely PATH is configured to find
that script.
2020-01-07 10:27:07 -08:00
Jon Siwek
29f386e388 Implement minimal supervised cluster configuration
More aspects of the cluster configuration to get fleshed out later,
but a basic cluster like one would use for a live deployment
can now be instantiated and run under supervision.  The new
clusterized-pcap-processing supervisor mode is also not done yet.
2019-10-23 17:37:53 -07:00
Jon Siwek
1ce0fcce49 GH-387: update Broker topic names to use "zeek/" prefix 2019-05-29 15:56:37 -07:00
Daniel Thayer
1a74516db1 Rename all BRO-prefixed environment variables
For backward compatibility when reading values, we first check
the ZEEK-prefixed value, and if not set, then check the corresponding
BRO-prefixed value.
2019-05-22 00:12:31 -05:00
Daniel Thayer
be182aac83 More bro-to-zeek renaming in scripts and other files 2019-05-16 02:36:41 -05:00
Jon Siwek
6ad7099f7e Merge remote-tracking branch 'origin/topic/robin/gh-239'
* origin/topic/robin/gh-239:
  Undo a change to btest.cfg from a recent commit
  Updating submodule.
  Fix zeek-wrapper
  Update for renaming BroControl to ZeekControl.
  Updating submodule.
  GH-239: Rename bro to zeek, bro-config to zeek-config, and bro-path-dev to zeek-path-dev.
2019-05-14 13:27:40 -07:00
Robin Sommer
89b8d6e7ba Update for renaming BroControl to ZeekControl. 2019-05-06 21:22:19 +00:00
Johanna Amann
dcd6454530 Remove RemoteSerializer and related code/types.
Also removes broccoli from the source tree.
2019-05-03 15:00:13 -07:00
Jon Siwek
aebcb1415d GH-234: rename Broxygen to Zeexygen along with roles/directives
* All "Broxygen" usages have been replaced in
  code, documentation, filenames, etc.

* Sphinx roles/directives like ":bro:see" are now ":zeek:see"

* The "--broxygen" command-line option is now "--zeexygen"
2019-04-22 19:45:50 -07:00
Jon Siwek
a994be9eeb Merge remote-tracking branch 'origin/topic/seth/zeek_init'
* origin/topic/seth/zeek_init:
  Some more testing fixes.
  Update docs and tests for bro_(init|done) -> zeek_(init|done)
  Implement the zeek_init handler.
2019-04-19 11:24:29 -07:00
Seth Hall
8cefb9be42 Implement the zeek_init handler.
Implements the change and a test.
2019-04-14 08:37:35 -04:00
Daniel Thayer
18bd74454b Rename all scripts to have ".zeek" file extension 2019-04-11 21:12:40 -05:00
Jon Siwek
01d303b480 Migrate table-based for-loops to key-value iteration 2019-03-15 19:54:44 -07:00
Jon Siwek
e0b8b4b6b1 Replace some bro.org usages with zeek.org 2019-01-04 17:51:25 -06:00
Jon Siwek
73c8cf733a Give Cluster::rr_topic "key" argument a default value 2018-09-07 09:58:57 -05:00
Jon Siwek
8db042a8c2 Remove Cluster::broadcast_topic
As enabling Broker forwarding would cause routing loops with messages
sent to such a topic (one subscribed to on all nodes).
2018-08-28 16:40:48 -05:00
Jon Siwek
da9f91fc19 Add env. variables to override Broker listen/connect retry intervals
And use them to default retries to 1sec for all unit tests.
2018-08-16 12:16:03 -05:00
Vern Paxson
88fd7510c6 reap the fruits of v += e 2018-07-26 12:51:36 -07:00
Daniel Thayer
ceefb6edaf Fix minor typos in broker reference documentation 2018-06-27 13:00:09 -05:00
Jon Siwek
8ee92c70da Add ability for BroControl to skip cluster setup
It does this by setting the BROCTL_CHECK_CONFIG env. variable.
Related to BIT-1938.
2018-06-21 11:57:18 -05:00
Jon Siwek
45178f3051 Add a counter for number of alive nodes within a given cluster pool 2018-05-24 14:33:35 -05:00
Jon Siwek
95ea84e60e Fix how cluster framework tracks worker count 2018-05-24 14:32:45 -05:00
Robin Sommer
fe7e1ee7f0 Merge topic/actor-system throug a squashed commit. 2018-05-18 22:39:23 +00:00
Daniel Thayer
59f0ea1a0c Add a test for starting a cluster with a logger node
Also modified cluster framework scripts so that any node can raise an
event to be handled on the logger node.
2016-07-15 15:23:49 -05:00
Daniel Thayer
b80298a9ad Change how logger node is detected in cluster framework
Instead of assuming the logger node always has the name "logger", now
broctl will set a boolean which the cluster framework scripts can use
to determine if there is a logger node or not.

Also removed one line from the manager node script, because it has to do
with logging, which the logger.bro script handles.
2016-07-05 10:36:03 -05:00
Daniel Thayer
ce0b9d42d3 Update docs for the new logger node type 2016-06-30 15:45:36 -05:00
Daniel Thayer
f45a3e8878 Add a new node type for logging
Changed the cluster framework scripts by adding a new Bro node type
for doing logging (this is intended to reduce the load on the manager).
If a user chooses not to specify a logger node in the cluster
configuration, then the manager will write logs locally as usual.
2016-06-29 17:55:49 -05:00
Daniel Thayer
2d9127888f Add some missing Bro script documentation
Also fixed a few reST formatting issues.
2016-05-05 16:35:31 -05:00
Robin Sommer
17bc615467 Making cluster reconnect timeout configurable. 2015-11-17 13:43:58 -08:00
Jon Siwek
186e67ec1d Allow logging filters to inherit default path from stream.
This allows the path for the default filter to be specified explicitly
when creating a stream and reduces the need to rely on the default path
function to magically supply the path.

The default path function is now only used if, when a filter is added to
a stream, it has neither a path nor a path function already.

Adapted the existing Log::create_stream calls to explicitly specify a
path value.

Addresses BIT-1324
2015-03-19 14:49:55 -05:00
Daniel Thayer
4b8171f74a Document which Bro script vars are set by BroControl 2013-10-22 16:40:29 -05:00
Daniel Thayer
60b2c5f1fe Add README files for most Bro frameworks
The text from these README files appears on the "Bro Script Packages"
page after building the documentation.  The text for these was mostly just
copied from the existing docs.
2013-10-11 00:19:37 -05:00
Daniel Thayer
f21f45ac1a Fix typos and formatting in the cluster framework docs
Changed a Time Machine link to point to something useful.
2013-10-10 14:39:44 -05:00
Robin Sommer
f830ed3edf s/bro-ids.org/bro.org/g 2013-03-07 19:33:04 -08:00
Daniel Thayer
c46029d828 Rename the alarm summary mail variable 2012-10-30 16:31:18 -05:00
Daniel Thayer
5f8cbf8a51 Change alarm summary email interval to 24hrs 2012-05-31 17:30:48 -05:00
Robin Sommer
f7261a7851 Merge remote-tracking branch 'origin/topic/jsiwek/ipv6-comm'
* origin/topic/jsiwek/ipv6-comm:
  Enable Bro to communicate with peers over non-global IPv6 addresses.
  Add unit tests for Broccoli SSL and Broccoli IPv6 connectivity.
  Remove AI_ADDRCONFIG getaddrinfo hints flag for listening sockets.
  Undo communication protocol version bump.
  Add support to Bro for connecting with peers over IPv6.

Closes #820.

Conflicts:
	src/bro.bif
2012-05-24 17:01:34 -07:00
Jon Siwek
74f3a32321 Enable Bro to communicate with peers over non-global IPv6 addresses.
This usually requires specifying an additional zone identifier
(see RFC 4007).  The connect() and listen() BIFs have been
changed to accept this zone identifier as an argument.
2012-05-17 12:59:20 -05:00
Jon Siwek
a4117016e9 Merge branch 'master' into topic/script-reference
Conflicts:
	aux/broccoli
	aux/broctl
	scripts/base/frameworks/notice/main.bro
	src/event.bif
2011-12-19 16:17:58 -06:00
Robin Sommer
8c53446292 Merge remote branch 'origin/fastpath'
* origin/fastpath:
  Fixed major bug with cluster synchronization (it was broken!)
2011-12-16 02:37:56 -08:00
Seth Hall
0b8b14a0ed Fixed major bug with cluster synchronization (it was broken!) 2011-12-15 15:59:51 -05:00
Jon Siwek
8e89d78788 Add more cluster and communication framework documentation. 2011-12-09 17:31:47 -06:00
Jon Siwek
506a42638a Omit loading local-<node>.bro scripts from base cluster framework.
The loading of these is better handled by BroControl and it seems
odd to load them from a base/ script anyway since they'll contain
site/policy specific code.

Addresses #663
2011-12-05 13:02:39 -06:00
Seth Hall
da9b8cc283 Modification to the Communication framework API.
- Simplified the communication API and made it easier to change
  to encrypted connections by not having separate variables to
  define encrypted and unencrypted ports.

- Now, to enable listening without configuring nodes just
  load the frameworks/communication/listen script.

- If encrypted listening is desired set the following:
	redef Communication::listen_encrypted=T;

- Accompanying test updates.
2011-10-07 13:29:26 -04:00
Jon Siwek
88e089864b Consolidating some node-specific functionality from scripts in broctl repo. 2011-10-05 16:33:40 -05:00
Seth Hall
aa9fdf38bb Clean up to cluster framework to make event handling clearer.
- Fixed a bug where notices were being passed to proxies.
  This was a mistake and should greatly reduce load on
  many clusters.

- Cluster event regex variables renamed to:
  - Notice::manager2worker_events
  - Notice::manager2proxy_events
  - Notice::worker2manager_events
  - Notice::worker2proxy_events
  - Notice::proxy2manager_events
  - Notice::proxy2worker_events

- The default Notice::policy set is cleared for all cluster
  nodes except for managers to cause all default notice
  processing to occur on managers.  This should reduce load
  on workers slightly.
2011-10-04 11:57:50 -04:00
Robin Sommer
c9b9bab473 Merge remote-tracking branch 'origin/topic/seth/notice-suppression'
* origin/topic/seth/notice-suppression:
  Updated a notice related baseline and added a necessary @load line.
  Notice suppression clean up and notice/cluster integrtion fixes.
  Updates for notice suppression to use the &create_expire attribute
  Small, mostly cosmetic updates and fixing a test.
  Fix crash on exit (addresses #607).
  Duplicate notice suppression.

Closes #623.
2011-09-22 19:20:34 -07:00