diff --git a/doc/cluster/index.rst b/doc/cluster/index.rst index 6e426c005e..5601463144 100644 --- a/doc/cluster/index.rst +++ b/doc/cluster/index.rst @@ -39,9 +39,11 @@ Manager ******* The manager is a Bro process that has two primary jobs. It receives log messages and notices from the rest of the nodes in the cluster using the Bro -communications protocol. The result is a single log instead of many -discrete logs that you have to combine in some manner with post-processing. -The manager also takes the opportunity to de-duplicate notices, and it has the +communications protocol (note that if you are using a logger, then the +logger receives all logs instead of the manager). The result +is a single log instead of many discrete logs that you have to +combine in some manner with post-processing. The manager also takes +the opportunity to de-duplicate notices, and it has the ability to do so since it's acting as the choke point for notices and how notices might be processed into actions (e.g., emailing, paging, or blocking). @@ -51,6 +53,20 @@ connections to the rest of the cluster. Once the workers are started and connect to the manager, logs and notices will start arriving to the manager process from the workers. +Logger +****** +The logger is an optional Bro process that receives log messages from the +rest of the nodes in the cluster using the Bro communications protocol. +The purpose of having a logger receive logs instead of the manager is +to reduce the load on the manager. If no logger is needed, then the +manager will receive logs instead. + +The logger process is started first by BroControl and it only opens its +designated port and waits for connections, it doesn't initiate any +connections to the rest of the cluster. Once the rest of the cluster is +started and connect to the logger, logs will start arriving to the logger +process. + Proxy ***** The proxy is a Bro process that manages synchronized state. Variables can diff --git a/doc/configuration/index.rst b/doc/configuration/index.rst index 800d746e72..fa49c3736e 100644 --- a/doc/configuration/index.rst +++ b/doc/configuration/index.rst @@ -44,7 +44,10 @@ workers can consume a lot of CPU resources. The maximum recommended number of workers to run on a machine should be one or two less than the number of CPU cores available on that machine. Using a load-balancing method (such as PF_RING) along with CPU pinning can decrease the load on -the worker machines. +the worker machines. Also, in order to reduce the load on the manager +process, it is recommended to have a logger in your configuration. If a +logger is defined in your cluster configuration, then it will receive logs +instead of the manager process. Basic Cluster Configuration @@ -61,13 +64,17 @@ a Bro cluster (do this as the Bro user on the manager host only): :doc:`BroControl <../components/broctl/README>` documentation. - Edit the BroControl node configuration file, ``/etc/node.cfg`` - to define where manager, proxies, and workers are to run. For a cluster - configuration, you must comment-out (or remove) the standalone node + to define where logger, manager, proxies, and workers are to run. For a + cluster configuration, you must comment-out (or remove) the standalone node in that file, and either uncomment or add node entries for each node - in your cluster (manager, proxy, and workers). For example, if you wanted - to run four Bro nodes (two workers, one proxy, and a manager) on a cluster - consisting of three machines, your cluster configuration would look like - this:: + in your cluster (logger, manager, proxy, and workers). For example, if you + wanted to run five Bro nodes (two workers, one proxy, a logger, and a + manager) on a cluster consisting of three machines, your cluster + configuration would look like this:: + + [logger] + type=logger + host=10.0.0.10 [manager] type=manager @@ -94,7 +101,7 @@ a Bro cluster (do this as the Bro user on the manager host only): file lists all of the networks which the cluster should consider as local to the monitored environment. -- Install workers and proxies using BroControl:: +- Install Bro on all machines in the cluster using BroControl:: > broctl install @@ -174,7 +181,7 @@ Installing PF_RING 5. Configure BroControl to use PF_RING (explained below). 6. Run "broctl install" on the manager. This command will install Bro and - all required scripts to the other machines in your cluster. + required scripts to all machines in your cluster. Using PF_RING ^^^^^^^^^^^^^ diff --git a/scripts/base/frameworks/cluster/__load__.bro b/scripts/base/frameworks/cluster/__load__.bro index 56d51cbc3d..f89df8eb63 100644 --- a/scripts/base/frameworks/cluster/__load__.bro +++ b/scripts/base/frameworks/cluster/__load__.bro @@ -32,7 +32,7 @@ redef Communication::listen_port = Cluster::nodes[Cluster::node]$p; @if ( Cluster::local_node_type() == Cluster::MANAGER ) @load ./nodes/manager -# If no logger is defined, then the manager writes logs. +# If no logger is defined, then the manager receives logs. @if ( "logger" !in Cluster::nodes ) @load ./nodes/logger @endif diff --git a/scripts/base/frameworks/cluster/setup-connections.bro b/scripts/base/frameworks/cluster/setup-connections.bro index db7f2c3160..bed1b3d28c 100644 --- a/scripts/base/frameworks/cluster/setup-connections.bro +++ b/scripts/base/frameworks/cluster/setup-connections.bro @@ -33,7 +33,7 @@ event bro_init() &priority=9 } else if ( me$node_type == MANAGER ) { - # If no logger is defined, then the manager writes logs. + # If no logger is defined, then the manager receives logs. local managerlogs = "logger" !in Cluster::nodes; if ( n$node_type == LOGGER && me$logger == i )