mirror of
https://github.com/zeek/zeek.git
synced 2025-10-02 14:48:21 +00:00
Updates to httpmonitor and mimestats documentation.
This commit is contained in:
parent
2e2cb0ffc9
commit
121db68c30
2 changed files with 26 additions and 27 deletions
|
@ -10,7 +10,7 @@ http.log file. This file can then be used for analysis and auditing
|
||||||
purposes.
|
purposes.
|
||||||
|
|
||||||
In the sections below we briefly explain the structure of the http.log
|
In the sections below we briefly explain the structure of the http.log
|
||||||
file. Then, we show you how to perform basic HTTP traffic monitoring and
|
file, then we show you how to perform basic HTTP traffic monitoring and
|
||||||
analysis tasks with Bro. Some of these ideas and techniques can later be
|
analysis tasks with Bro. Some of these ideas and techniques can later be
|
||||||
applied to monitor different protocols in a similar way.
|
applied to monitor different protocols in a similar way.
|
||||||
|
|
||||||
|
@ -40,11 +40,10 @@ request to the root of Bro website::
|
||||||
|
|
||||||
Network administrators and security engineers, for instance, can use the
|
Network administrators and security engineers, for instance, can use the
|
||||||
information in this log to understand the HTTP activity on the network
|
information in this log to understand the HTTP activity on the network
|
||||||
and troubleshoot network problems or search for anomalous activities. At
|
and troubleshoot network problems or search for anomalous activities. We must
|
||||||
this point, we would like to stress out the fact that there is no just
|
stress that there is no single right way to perform an analysis. It will
|
||||||
one right way to perform analysis; it will depend on the expertise of
|
depend on the expertise of the person performing the analysis and the
|
||||||
the person doing the analysis and the specific details of the task to
|
specific details of the task.
|
||||||
accomplish.
|
|
||||||
|
|
||||||
For more information about how to handle the HTTP protocol in Bro,
|
For more information about how to handle the HTTP protocol in Bro,
|
||||||
including a complete list of the fields available in http.log, go to
|
including a complete list of the fields available in http.log, go to
|
||||||
|
@ -58,15 +57,15 @@ Detecting a Proxy Server
|
||||||
A proxy server is a device on your network configured to request a
|
A proxy server is a device on your network configured to request a
|
||||||
service on behalf of a third system; one of the most common examples is
|
service on behalf of a third system; one of the most common examples is
|
||||||
a Web proxy server. A client without Internet access connects to the
|
a Web proxy server. A client without Internet access connects to the
|
||||||
proxy and requests a Web page; the proxy then sends the request to the
|
proxy and requests a web page, the proxy sends the request to the web
|
||||||
actual Web server, receives the response and passes it to the original
|
server, which receives the response, and passes it to the original
|
||||||
client.
|
client.
|
||||||
|
|
||||||
Proxies were conceived to help manage a network and provide better
|
Proxies were conceived to help manage a network and provide better
|
||||||
encapsulation. By themselves, proxies are not a security threat, but a
|
encapsulation. Proxies by themselves are not a security threat, but a
|
||||||
misconfigured or unauthorized proxy can allow others, either inside or
|
misconfigured or unauthorized proxy can allow others, either inside or
|
||||||
outside the network, to access any Web site and even conduct malicious
|
outside the network, to access any web site and even conduct malicious
|
||||||
activities anonymously using the network resources.
|
activities anonymously using the network's resources.
|
||||||
|
|
||||||
What Proxy Server traffic looks like
|
What Proxy Server traffic looks like
|
||||||
-------------------------------------
|
-------------------------------------
|
||||||
|
|
|
@ -6,19 +6,19 @@ MIME Type Statistics
|
||||||
====================
|
====================
|
||||||
|
|
||||||
Files are constantly transmitted over HTTP on regular networks. These
|
Files are constantly transmitted over HTTP on regular networks. These
|
||||||
files belong to a specific category (i.e., executable, text, image,
|
files belong to a specific category (e.g., executable, text, image)
|
||||||
etc.) identified by a `Multipurpose Internet Mail Extension (MIME)
|
identified by a `Multipurpose Internet Mail Extension (MIME)
|
||||||
<http://en.wikipedia.org/wiki/MIME>`_. Although MIME was originally
|
<http://en.wikipedia.org/wiki/MIME>`_. Although MIME was originally
|
||||||
developed to identify the type of non-text attachments on email, it is
|
developed to identify the type of non-text attachments on email, it is
|
||||||
also used by Web browser to identify the type of files transmitted and
|
also used by a web browser to identify the type of files transmitted and
|
||||||
present them accordingly.
|
present them accordingly.
|
||||||
|
|
||||||
In this tutorial, we will show how to use the Sumstats Framework to
|
In this tutorial, we will demonstrate how to use the Sumstats Framework
|
||||||
collect some statistics information based on MIME types, specifically
|
to collect statistical information based on MIME types; specifically,
|
||||||
the total number of occurrences, size in bytes, and number of unique
|
the total number of occurrences, size in bytes, and number of unique
|
||||||
hosts transmitting files over HTTP per each type. For instructions about
|
hosts transmitting files over HTTP per each type. For instructions on
|
||||||
extracting and creating a local copy of these files, visit :ref:`this
|
extracting and creating a local copy of these files, visit :ref:`this
|
||||||
<http-monitor>` tutorial instead.
|
tutorial <http-monitor>`.
|
||||||
|
|
||||||
------------------------------------------------
|
------------------------------------------------
|
||||||
MIME Statistics with Sumstats
|
MIME Statistics with Sumstats
|
||||||
|
@ -30,31 +30,31 @@ Observations, where the event is observed and fed into the framework.
|
||||||
(ii) Reducers, where observations are collected and measured. (iii)
|
(ii) Reducers, where observations are collected and measured. (iii)
|
||||||
Sumstats, where the main functionality is implemented.
|
Sumstats, where the main functionality is implemented.
|
||||||
|
|
||||||
So, we start by defining our observation along with a record to store
|
We start by defining our observation along with a record to store
|
||||||
all statistics values and an observation interval. We are conducting our
|
all statistical values and an observation interval. We are conducting our
|
||||||
observation on the :bro:see:`HTTP::log_http` event and we are interested
|
observation on the :bro:see:`HTTP::log_http` event and are interested
|
||||||
in the MIME type, size of the file ("response_body_len") and the
|
in the MIME type, size of the file ("response_body_len"), and the
|
||||||
originator host ("orig_h"). We use the MIME type as our key and create
|
originator host ("orig_h"). We use the MIME type as our key and create
|
||||||
observers for the other two values.
|
observers for the other two values.
|
||||||
|
|
||||||
.. btest-include:: ${DOC_ROOT}/mimestats/mimestats.bro
|
.. btest-include:: ${DOC_ROOT}/mimestats/mimestats.bro
|
||||||
:lines: 6-29, 54-64
|
:lines: 6-29, 54-64
|
||||||
|
|
||||||
Next, we create the reducers. The first one will accumulate file sizes
|
Next, we create the reducers. The first will accumulate file sizes
|
||||||
and the second one will make sure we only store a host ID once. Below is
|
and the second will make sure we only store a host ID once. Below is
|
||||||
the partial code from a :bro:see:`bro_init` handler.
|
the partial code from a :bro:see:`bro_init` handler.
|
||||||
|
|
||||||
.. btest-include:: ${DOC_ROOT}/mimestats/mimestats.bro
|
.. btest-include:: ${DOC_ROOT}/mimestats/mimestats.bro
|
||||||
:lines: 34-37
|
:lines: 34-37
|
||||||
|
|
||||||
In our final step, we create the SumStats where we check for the
|
In our final step, we create the SumStats where we check for the
|
||||||
observation interval and once it expires, we populate the record
|
observation interval. Once it expires, we populate the record
|
||||||
(defined above) with all the relevant data and write it to a log.
|
(defined above) with all the relevant data and write it to a log.
|
||||||
|
|
||||||
.. btest-include:: ${DOC_ROOT}/mimestats/mimestats.bro
|
.. btest-include:: ${DOC_ROOT}/mimestats/mimestats.bro
|
||||||
:lines: 38-51
|
:lines: 38-51
|
||||||
|
|
||||||
Putting everything together we end up with the following final code for
|
After putting the three pieces together we end up with the following final code for
|
||||||
our script.
|
our script.
|
||||||
|
|
||||||
.. btest-include:: ${DOC_ROOT}/mimestats/mimestats.bro
|
.. btest-include:: ${DOC_ROOT}/mimestats/mimestats.bro
|
||||||
|
|
Loading…
Add table
Add a link
Reference in a new issue