Mirror/zeek
1
0
Fork 0
mirror of https://github.com/zeek/zeek.git synced 2025-10-17 14:08:20 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 3

Reformat line width of some docs (i.e. fmt -72).

Browse source
This commit is contained in:
Jon Siwek 2014-01-21 11:43:35 -06:00
parent e88ac7221d
commit c5ab33d88f
3 changed files with 149 additions and 80 deletions
Download patch file Download diff file Expand all files Collapse all files

120
doc/httpmonitor/index.rst
View file

@ -5,71 +5,85 @@
Monitoring HTTP Traffic with Bro
================================
Bro can be used to log the entire HTTP traffic from your network to the http.log file.
This file can then be used for analysis and auditing purposes.
Bro can be used to log the entire HTTP traffic from your network to the
http.log file. This file can then be used for analysis and auditing
purposes.
In the sections below we briefly explain the structure of the http.log file. Then, we
show you how to perform basic HTTP traffic monitoring and analysis tasks with Bro. Some
of these ideas and techniques can later be applied to monitor different protocols in a
similar way.
In the sections below we briefly explain the structure of the http.log
file. Then, we show you how to perform basic HTTP traffic monitoring and
analysis tasks with Bro. Some of these ideas and techniques can later be
applied to monitor different protocols in a similar way.
----------------------------
Introduction to the HTTP log
----------------------------
The http.log file contains a summary of all HTTP requests and responses sent over a Bro-monitored
network. Here are the first few columns of
The http.log file contains a summary of all HTTP requests and responses
sent over a Bro-monitored network. Here are the first few columns of
``http.log``::
# ts uid orig_h orig_p resp_h resp_p
1311627961.8 HSH4uV8KVJg 192.168.1.100 52303 192.150.187.43 80
Every single line in this log starts with a timestamp, a unique connection identifier (UID), and a
connection 4-tuple (originator host/port and responder host/port). The UID can be used to
identify all logged activity (possibly across multiple log files) associated
with a given connection 4-tuple over its lifetime.
Every single line in this log starts with a timestamp, a unique
connection identifier (UID), and a connection 4-tuple (originator
host/port and responder host/port). The UID can be used to identify all
logged activity (possibly across multiple log files) associated with a
given connection 4-tuple over its lifetime.
The remaining columns detail the activity that's occurring. For example, the columns on the line below
(shortened for brevity) show a request to the root of Bro website::
The remaining columns detail the activity that's occurring. For
example, the columns on the line below (shortened for brevity) show a
request to the root of Bro website::
# method host uri referrer user_agent
GET bro.org / - <...>Chrome/12.0.742.122<...>
Network administrators and security engineers, for instance, can use the information in this log to understand
the HTTP activity on the network and troubleshoot network problems or search for anomalous activities. At this
point, we would like to stress out the fact that there is no just one right way to perform analysis; it will
depend on the expertise of the person doing the analysis and the specific details of the task to accomplish.
Network administrators and security engineers, for instance, can use the
information in this log to understand the HTTP activity on the network
and troubleshoot network problems or search for anomalous activities. At
this point, we would like to stress out the fact that there is no just
one right way to perform analysis; it will depend on the expertise of
the person doing the analysis and the specific details of the task to
accomplish.
For more information about how to handle the HTTP protocol in Bro, including a complete list
of the fields available in http.log, go to Bro's
:doc:`HTTP script reference </scripts/base/protocols/http/main.bro>`.
For more information about how to handle the HTTP protocol in Bro,
including a complete list of the fields available in http.log, go to
Bro's :doc:`HTTP script reference
</scripts/base/protocols/http/main.bro>`.
------------------------
Detecting a Proxy Server
------------------------
A proxy server is a device on your network configured to request a service on behalf of a third system; one of the
most common examples is a Web proxy server. A client without Internet access connects to the proxy and requests
a Web page; the proxy then sends the request to the actual Web server, receives the response and passes it to the original
A proxy server is a device on your network configured to request a
service on behalf of a third system; one of the most common examples is
a Web proxy server. A client without Internet access connects to the
proxy and requests a Web page; the proxy then sends the request to the
actual Web server, receives the response and passes it to the original
client.
Proxies were conceived to help manage a network and provide better encapsulation. By themselves, proxies are not a security
threat, but a misconfigured or unauthorized proxy can allow others, either inside or outside the network, to access any
Web site and even conduct malicious activities anonymously using the network resources.
Proxies were conceived to help manage a network and provide better
encapsulation. By themselves, proxies are not a security threat, but a
misconfigured or unauthorized proxy can allow others, either inside or
outside the network, to access any Web site and even conduct malicious
activities anonymously using the network resources.
What Proxy Server traffic looks like
-------------------------------------
In general, when a client starts talking with a proxy server, the traffic consists of two parts: (i) a GET request, and
(ii) an HTTP/ reply::
In general, when a client starts talking with a proxy server, the
traffic consists of two parts: (i) a GET request, and (ii) an HTTP/
reply::
Request: GET http://www.bro.org/ HTTP/1.1
Reply: HTTP/1.0 200 OK
This will differ from traffic between a client and a normal Web server because GET requests should not include "http" on
the string. So we can use this to identify a proxy server.
This will differ from traffic between a client and a normal Web server
because GET requests should not include "http" on the string. So we can
use this to identify a proxy server.
We can write a basic script in Bro to handle the http_reply event and detect a reply for a ``GET http://`` request.
We can write a basic script in Bro to handle the http_reply event and
detect a reply for a ``GET http://`` request.
.. code:: bro
@ -81,8 +95,10 @@ We can write a basic script in Bro to handle the http_reply event and detect a r
}
}
Basically, the script is checking for a "200 OK" status code on a reply for a request that includes "http:". In reality, the HTTP
protocol defines several success status codes other than 200, so we will extend our basic script to also consider the additional codes.
Basically, the script is checking for a "200 OK" status code on a reply
for a request that includes "http:". In reality, the HTTP protocol
defines several success status codes other than 200, so we will extend
our basic script to also consider the additional codes.
.. code:: bro
@ -112,7 +128,8 @@ protocol defines several success status codes other than 200, so we will extend
}
}
Next, we will make sure that the responding proxy is part of our local network.
Next, we will make sure that the responding proxy is part of our local
network.
.. code:: bro
@ -142,9 +159,12 @@ Next, we will make sure that the responding proxy is part of our local network.
}
}
Finally, our goal should be to generate an alert when a proxy has been detected instead of printing a message on the console output.
For that, we will tag the traffic accordingly and define a new ``Open_Proxy`` ``Notice`` type to alert of all tagged communications. Once a
notification has been fired, we will further suppress it for one day. Below is the complete script.
Finally, our goal should be to generate an alert when a proxy has been
detected instead of printing a message on the console output. For that,
we will tag the traffic accordingly and define a new ``Open_Proxy``
``Notice`` type to alert of all tagged communications. Once a
notification has been fired, we will further suppress it for one day.
Below is the complete script.
.. code:: bro
@ -216,11 +236,14 @@ notification has been fired, we will further suppress it for one day. Below is t
Inspecting Files
----------------
Files are often transmitted on regular HTTP conversations between a client and a server. Most of the time these files are harmless,
just images and some other multimedia content, but there are also types of files, specially executable files, that can damage
your system. We can instruct Bro to create a copy of all executable files that it sees for later analysis using the
:ref:`File Analysis Framework <file-analysis-framework>`
(introduced with Bro 2.2) as shown in the following script.
Files are often transmitted on regular HTTP conversations between a
client and a server. Most of the time these files are harmless, just
images and some other multimedia content, but there are also types of
files, specially executable files, that can damage your system. We can
instruct Bro to create a copy of all executable files that it sees for
later analysis using the :ref:`File Analysis Framework
<file-analysis-framework>` (introduced with Bro 2.2) as shown in the
following script.
.. code:: bro
@ -239,9 +262,11 @@ your system. We can instruct Bro to create a copy of all executable files that i
Files::add_analyzer(f, Files::ANALYZER_EXTRACT, [$extract_filename=fname]);
}
Bro will extract all files from the traffic and write them on a new ``extract_files/`` subdirectory and change the file name with the right
suffix (extension) based on the content of the ext_map table. So, if you want to do the same for other extracted files besides executables
you just need to add those types to the ``ext_map`` table like this.
Bro will extract all files from the traffic and write them on a new
``extract_files/`` subdirectory and change the file name with the right
suffix (extension) based on the content of the ext_map table. So, if you
want to do the same for other extracted files besides executables you
just need to add those types to the ``ext_map`` table like this.
.. code:: bro
@ -253,4 +278,5 @@ you just need to add those types to the ``ext_map`` table like this.
["text/html"] = "html",
} &default ="";
Bro will now write the appropriate suffix for text, JPEG, PNG, and HTML files stored in the ``extract_files/`` subdirectory.
Bro will now write the appropriate suffix for text, JPEG, PNG, and HTML
files stored in the ``extract_files/`` subdirectory.