diff --git a/doc/logging-elasticsearch.rst b/doc/logging-elasticsearch.rst
index b6d22cf5fa..7571c68219 100644
--- a/doc/logging-elasticsearch.rst
+++ b/doc/logging-elasticsearch.rst
@@ -1,28 +1,38 @@
-========================================
+=========================================
Indexed Logging Output with ElasticSearch
-========================================
+=========================================
.. rst-class:: opening
Bro's default ASCII log format is not exactly the most efficient
- way for storing and searching large volumes of data. ElasticSearch
- is a new and exciting technology for dealing with tons of data.
- ElasticSearch is a search engine built on top of Apache's Lucene
+ way for searching large volumes of data. ElasticSearch
+ is a new data storage technology for dealing with tons of data.
+ It's also a search engine built on top of Apache's Lucene
project. It scales very well, both for distributed indexing and
distributed searching.
.. contents::
+Warning
+-------
+
+This writer plugin is still in testing and is not yet recommended for
+production use! The approach to how logs are handled in the plugin is "fire
+and forget" at this time, there is no error handling if the server fails to
+respond successfully to the insertion request.
+
Installing ElasticSearch
------------------------
-ElasticSearch requires a JRE to run. Please download the latest version
-from: . Once extracted, start
-ElasticSearch with::
+Download the latest version from: .
+Once extracted, start ElasticSearch with::
# ./bin/elasticsearch
+For more detailed information, refer to the ElasticSearch installation
+documentation: http://www.elasticsearch.org/guide/reference/setup/installation.html
+
Compiling Bro with ElasticSearch Support
----------------------------------------
@@ -41,49 +51,32 @@ First, ensure that you have libcurl installed the run configure.::
Activating ElasticSearch
------------------------
-The direct way to use ElasticSearch is to switch *all* log files over to
-ElasticSearch. To do that, just add ``redef
-Log::default_writer=Log::WRITER_ELASTICSEARCH;`` to your ``local.bro``.
-For testing, you can also just pass that on the command line::
+The easiest way to enable ElasticSearch output is to load the tuning/logs-to-
+elasticsearch.bro script. If you are using BroControl, the following line in
+local.bro will enable it.
- bro -r trace.pcap Log::default_writer=Log::WRITER_ELASTICSEARCH
+.. console::
-With that, Bro will now write all its output into ElasticSearch. You can
-inspect these using ElasticSearch's REST-ful interface. For more
-information, see: .
+ @load tuning/logs-to-elasticsearch
-There is also a rudimentary web interface to ElasticSearch, available at:
-.
+With that, Bro will now write most of its logs into ElasticSearch in addition
+to maintaining the Ascii logs like it would do by default. That script has
+some tunable options for choosing which logs to send to ElasticSearch, refer
+to the autogenerated script documentation for those options.
-You can also switch only individual files over to ElasticSearch by adding
-code like this to your ``local.bro``::
+There is an interface being written specifically to integrate with the data
+that Bro outputs into ElasticSearch named Brownian. It can be found here::
-.. code::bro
+ https://github.com/grigorescu/Brownian
- event bro_init()
- {
- local f = Log::get_filter(Conn::LOG, "default"); # Get default filter for connection log.
- f$writer = Log::WRITER_ELASTICSEARCH; # Change writer type.
- Log::add_filter(Conn::LOG, f); # Replace filter with adapted version.
- }
+Tuning
+------
-Configuring ElasticSearch
--------------------------
+A common problem encountered with ElasticSearch is too many files being held
+open. The ElasticSearch website has some suggestions on how to increase the
+open file limit.
-Bro's ElasticSearch writer comes with a few configuration options::
-
-- cluster_name: Currently unused.
-
-- server_host: Where to send the data. Default localhost.
-
-- server_port: What port to send the data to. Default 9200.
-
-- index_prefix: ElasticSearch indexes are like databases in a standard DB model.
- This is the name of the index to which to send the data. Default bro.
-
-- type_prefix: ElasticSearch types are like tables in a standard DB model. This is a prefix that gets prepended to Bro log names. Example: type_prefix = "bro_" would create types "bro_dns", "bro_http", etc. Default: none.
-
-- batch_size: How many messages to buffer before sending to ElasticSearch. This is mainly a memory optimization - changing this doesn't seem to affect indexing performance that much. Default: 10,000.
+ - http://www.elasticsearch.org/tutorials/2011/04/06/too-many-open-files.html
TODO
----
@@ -93,3 +86,4 @@ Lots.
- Perform multicast discovery for server.
- Better error detection.
- Better defaults (don't index loaded-plugins, for instance).
+-