Updates to httpmonitor and mimestats documentation.

This commit is contained in:
Jeannette Dopheide 2014-01-30 13:23:58 -06:00
parent 2e2cb0ffc9
commit 121db68c30
2 changed files with 26 additions and 27 deletions

View file

@ -10,7 +10,7 @@ http.log file. This file can then be used for analysis and auditing
purposes. purposes.
In the sections below we briefly explain the structure of the http.log In the sections below we briefly explain the structure of the http.log
file. Then, we show you how to perform basic HTTP traffic monitoring and file, then we show you how to perform basic HTTP traffic monitoring and
analysis tasks with Bro. Some of these ideas and techniques can later be analysis tasks with Bro. Some of these ideas and techniques can later be
applied to monitor different protocols in a similar way. applied to monitor different protocols in a similar way.
@ -40,11 +40,10 @@ request to the root of Bro website::
Network administrators and security engineers, for instance, can use the Network administrators and security engineers, for instance, can use the
information in this log to understand the HTTP activity on the network information in this log to understand the HTTP activity on the network
and troubleshoot network problems or search for anomalous activities. At and troubleshoot network problems or search for anomalous activities. We must
this point, we would like to stress out the fact that there is no just stress that there is no single right way to perform an analysis. It will
one right way to perform analysis; it will depend on the expertise of depend on the expertise of the person performing the analysis and the
the person doing the analysis and the specific details of the task to specific details of the task.
accomplish.
For more information about how to handle the HTTP protocol in Bro, For more information about how to handle the HTTP protocol in Bro,
including a complete list of the fields available in http.log, go to including a complete list of the fields available in http.log, go to
@ -58,15 +57,15 @@ Detecting a Proxy Server
A proxy server is a device on your network configured to request a A proxy server is a device on your network configured to request a
service on behalf of a third system; one of the most common examples is service on behalf of a third system; one of the most common examples is
a Web proxy server. A client without Internet access connects to the a Web proxy server. A client without Internet access connects to the
proxy and requests a Web page; the proxy then sends the request to the proxy and requests a web page, the proxy sends the request to the web
actual Web server, receives the response and passes it to the original server, which receives the response, and passes it to the original
client. client.
Proxies were conceived to help manage a network and provide better Proxies were conceived to help manage a network and provide better
encapsulation. By themselves, proxies are not a security threat, but a encapsulation. Proxies by themselves are not a security threat, but a
misconfigured or unauthorized proxy can allow others, either inside or misconfigured or unauthorized proxy can allow others, either inside or
outside the network, to access any Web site and even conduct malicious outside the network, to access any web site and even conduct malicious
activities anonymously using the network resources. activities anonymously using the network's resources.
What Proxy Server traffic looks like What Proxy Server traffic looks like
------------------------------------- -------------------------------------

View file

@ -6,19 +6,19 @@ MIME Type Statistics
==================== ====================
Files are constantly transmitted over HTTP on regular networks. These Files are constantly transmitted over HTTP on regular networks. These
files belong to a specific category (i.e., executable, text, image, files belong to a specific category (e.g., executable, text, image)
etc.) identified by a `Multipurpose Internet Mail Extension (MIME) identified by a `Multipurpose Internet Mail Extension (MIME)
<http://en.wikipedia.org/wiki/MIME>`_. Although MIME was originally <http://en.wikipedia.org/wiki/MIME>`_. Although MIME was originally
developed to identify the type of non-text attachments on email, it is developed to identify the type of non-text attachments on email, it is
also used by Web browser to identify the type of files transmitted and also used by a web browser to identify the type of files transmitted and
present them accordingly. present them accordingly.
In this tutorial, we will show how to use the Sumstats Framework to In this tutorial, we will demonstrate how to use the Sumstats Framework
collect some statistics information based on MIME types, specifically to collect statistical information based on MIME types; specifically,
the total number of occurrences, size in bytes, and number of unique the total number of occurrences, size in bytes, and number of unique
hosts transmitting files over HTTP per each type. For instructions about hosts transmitting files over HTTP per each type. For instructions on
extracting and creating a local copy of these files, visit :ref:`this extracting and creating a local copy of these files, visit :ref:`this
<http-monitor>` tutorial instead. tutorial <http-monitor>`.
------------------------------------------------ ------------------------------------------------
MIME Statistics with Sumstats MIME Statistics with Sumstats
@ -30,31 +30,31 @@ Observations, where the event is observed and fed into the framework.
(ii) Reducers, where observations are collected and measured. (iii) (ii) Reducers, where observations are collected and measured. (iii)
Sumstats, where the main functionality is implemented. Sumstats, where the main functionality is implemented.
So, we start by defining our observation along with a record to store We start by defining our observation along with a record to store
all statistics values and an observation interval. We are conducting our all statistical values and an observation interval. We are conducting our
observation on the :bro:see:`HTTP::log_http` event and we are interested observation on the :bro:see:`HTTP::log_http` event and are interested
in the MIME type, size of the file ("response_body_len") and the in the MIME type, size of the file ("response_body_len"), and the
originator host ("orig_h"). We use the MIME type as our key and create originator host ("orig_h"). We use the MIME type as our key and create
observers for the other two values. observers for the other two values.
.. btest-include:: ${DOC_ROOT}/mimestats/mimestats.bro .. btest-include:: ${DOC_ROOT}/mimestats/mimestats.bro
:lines: 6-29, 54-64 :lines: 6-29, 54-64
Next, we create the reducers. The first one will accumulate file sizes Next, we create the reducers. The first will accumulate file sizes
and the second one will make sure we only store a host ID once. Below is and the second will make sure we only store a host ID once. Below is
the partial code from a :bro:see:`bro_init` handler. the partial code from a :bro:see:`bro_init` handler.
.. btest-include:: ${DOC_ROOT}/mimestats/mimestats.bro .. btest-include:: ${DOC_ROOT}/mimestats/mimestats.bro
:lines: 34-37 :lines: 34-37
In our final step, we create the SumStats where we check for the In our final step, we create the SumStats where we check for the
observation interval and once it expires, we populate the record observation interval. Once it expires, we populate the record
(defined above) with all the relevant data and write it to a log. (defined above) with all the relevant data and write it to a log.
.. btest-include:: ${DOC_ROOT}/mimestats/mimestats.bro .. btest-include:: ${DOC_ROOT}/mimestats/mimestats.bro
:lines: 38-51 :lines: 38-51
Putting everything together we end up with the following final code for After putting the three pieces together we end up with the following final code for
our script. our script.
.. btest-include:: ${DOC_ROOT}/mimestats/mimestats.bro .. btest-include:: ${DOC_ROOT}/mimestats/mimestats.bro