mirror of
https://github.com/zeek/zeek.git
synced 2025-10-09 10:08:20 +00:00
Reformat line width of some docs (i.e. fmt -72).
This commit is contained in:
Jon Siwek
parent
e88ac7221d
commit
c5ab33d88f
3 changed files with 149 additions and 80 deletions
|
|
@ -5,17 +5,24 @@
|
|||
Bro IDS
|
||||
=======
|
||||
|
||||
An Intrusion Detection System (IDS) allows you to detect suspicious activities happening on your network as a result of a past or active
|
||||
attack. Because of its programming capabilities, Bro can easily be configured to behave like traditional IDSs and detect common attacks
|
||||
with well known patterns, or you can create your own scripts to detect conditions specific to your particular case.
|
||||
An Intrusion Detection System (IDS) allows you to detect suspicious
|
||||
activities happening on your network as a result of a past or active
|
||||
attack. Because of its programming capabilities, Bro can easily be
|
||||
configured to behave like traditional IDSs and detect common attacks
|
||||
with well known patterns, or you can create your own scripts to detect
|
||||
conditions specific to your particular case.
|
||||
|
||||
In the following sections, we present a few examples of common uses of Bro as an IDS.
|
||||
In the following sections, we present a few examples of common uses of
|
||||
Bro as an IDS.
|
||||
|
||||
------------------------------------------------
|
||||
Detecting an FTP Bruteforce attack and notifying
|
||||
------------------------------------------------
|
||||
For the purpose of this exercise, we define FTP bruteforcing as too many rejected usernames and passwords occurring from a single address.
|
||||
We start by defining a threshold for the number of attempts and a monitoring interval in minutes.
|
||||
|
||||
For the purpose of this exercise, we define FTP bruteforcing as too many
|
||||
rejected usernames and passwords occurring from a single address. We
|
||||
start by defining a threshold for the number of attempts and a
|
||||
monitoring interval in minutes.
|
||||
|
||||
.. code:: bro
|
||||
|
||||
|
|
@ -29,8 +36,13 @@ We start by defining a threshold for the number of attempts and a monitoring int
|
|||
const bruteforce_measurement_interval = 15mins &redef;
|
||||
}
|
||||
|
||||
Now, using the ftp_reply event, we check for error codes from the `500 series <http://en.wikipedia.org/wiki/List_of_FTP_server_return_codes>`_ for the "USER" and "PASS" commands, representing rejected usernames or passwords. For this, we can use the :bro:see:`FTP::parse_ftp_reply_code` function to break down the reply code and check if the first digit is a "5" or not. If true, we then use the
|
||||
:ref:`Summary Statistics Framework <sumstats-framework>` to keep track of the number of failed attempts.
|
||||
Now, using the ftp_reply event, we check for error codes from the `500
|
||||
series <http://en.wikipedia.org/wiki/List_of_FTP_server_return_codes>`_
|
||||
for the "USER" and "PASS" commands, representing rejected usernames or
|
||||
passwords. For this, we can use the :bro:see:`FTP::parse_ftp_reply_code`
|
||||
function to break down the reply code and check if the first digit is a
|
||||
"5" or not. If true, we then use the :ref:`Summary Statistics Framework
|
||||
<sumstats-framework>` to keep track of the number of failed attempts.
|
||||
|
||||
.. code:: bro
|
||||
|
||||
|
|
@ -44,7 +56,8 @@ Now, using the ftp_reply event, we check for error codes from the `500 series <h
|
|||
}
|
||||
}
|
||||
|
||||
Next, we use the SumStats framework to automatically print a message on the console alerting of the attack when the number of failed attempts
|
||||
Next, we use the SumStats framework to automatically print a message on
|
||||
the console alerting of the attack when the number of failed attempts
|
||||
exceeds the specified threshold during the measuring interval.
|
||||
|
||||
.. code:: bro
|
||||
|
|
@ -69,8 +82,11 @@ exceeds the specified threshold during the measuring interval.
|
|||
}]);
|
||||
}
|
||||
|
||||
Printing a message on the console is a good start but it will be better if we raise an alarm instead using the :ref:`Notice Framework <notice-framework>`. For this, we need to define a new Notice type and trigger the alarm under the right
|
||||
conditions. Below is the final code for our script.
|
||||
Printing a message on the console is a good start but it will be better
|
||||
if we raise an alarm instead using the :ref:`Notice Framework
|
||||
<notice-framework>`. For this, we need to define a new Notice type and
|
||||
trigger the alarm under the right conditions. Below is the final code
|
||||
for our script.
|
||||
|
||||
.. code:: bro
|
||||
|
||||
|
|
@ -135,17 +151,27 @@ conditions. Below is the final code for our script.
|
|||
}
|
||||
}
|
||||
|
||||
As a final note, the :doc:`detect-bruteforcing.bro </scripts/policy/protocols/ftp/detect-bruteforcing.bro>` script above is include with Bro out of the box, so you only need to load it at startup to instruct Bro to detect and notify of FTP bruteforce attacks.
|
||||
As a final note, the :doc:`detect-bruteforcing.bro
|
||||
</scripts/policy/protocols/ftp/detect-bruteforcing.bro>` script above is
|
||||
include with Bro out of the box, so you only need to load it at startup
|
||||
to instruct Bro to detect and notify of FTP bruteforce attacks.
|
||||
|
||||
-------------
|
||||
Other Attacks
|
||||
-------------
|
||||
|
||||
Detecting SQL Injection attacks
|
||||
-------------------------------
|
||||
|
||||
Checking files against known malware hashes
|
||||
-------------------------------------------
|
||||
Files transmitted on your network could either be completely harmless or contain viruses and other threats. One possible action against
|
||||
this threat is to compute the hashes of the files and compare them against a list of known malware hashes. Bro simplifies this task
|
||||
by offering a :doc:`detect-MHR.bro </scripts/policy/frameworks/files/detect-MHR.bro>` script that creates and compares
|
||||
hashes against the `Malware Hash Registry <https://www.team-cymru.org/Services/MHR/>`_ maintained by Team Cymru. You only need to load this
|
||||
script along with your other scripts at startup time.
|
||||
|
||||
Files transmitted on your network could either be completely harmless or
|
||||
contain viruses and other threats. One possible action against this
|
||||
threat is to compute the hashes of the files and compare them against a
|
||||
list of known malware hashes. Bro simplifies this task by offering a
|
||||
:doc:`detect-MHR.bro </scripts/policy/frameworks/files/detect-MHR.bro>`
|
||||
script that creates and compares hashes against the `Malware Hash
|
||||
Registry <https://www.team-cymru.org/Services/MHR/>`_ maintained by Team
|
||||
Cymru. You only need to load this script along with your other scripts
|
||||
at startup time.
|
||||
|
|
|
|||
|
|
@ -5,71 +5,85 @@
|
|||
Monitoring HTTP Traffic with Bro
|
||||
================================
|
||||
|
||||
Bro can be used to log the entire HTTP traffic from your network to the http.log file.
|
||||
This file can then be used for analysis and auditing purposes.
|
||||
Bro can be used to log the entire HTTP traffic from your network to the
|
||||
http.log file. This file can then be used for analysis and auditing
|
||||
purposes.
|
||||
|
||||
In the sections below we briefly explain the structure of the http.log file. Then, we
|
||||
show you how to perform basic HTTP traffic monitoring and analysis tasks with Bro. Some
|
||||
of these ideas and techniques can later be applied to monitor different protocols in a
|
||||
similar way.
|
||||
In the sections below we briefly explain the structure of the http.log
|
||||
file. Then, we show you how to perform basic HTTP traffic monitoring and
|
||||
analysis tasks with Bro. Some of these ideas and techniques can later be
|
||||
applied to monitor different protocols in a similar way.
|
||||
|
||||
----------------------------
|
||||
Introduction to the HTTP log
|
||||
----------------------------
|
||||
|
||||
The http.log file contains a summary of all HTTP requests and responses sent over a Bro-monitored
|
||||
network. Here are the first few columns of
|
||||
The http.log file contains a summary of all HTTP requests and responses
|
||||
sent over a Bro-monitored network. Here are the first few columns of
|
||||
``http.log``::
|
||||
|
||||
# ts uid orig_h orig_p resp_h resp_p
|
||||
1311627961.8 HSH4uV8KVJg 192.168.1.100 52303 192.150.187.43 80
|
||||
|
||||
Every single line in this log starts with a timestamp, a unique connection identifier (UID), and a
|
||||
connection 4-tuple (originator host/port and responder host/port). The UID can be used to
|
||||
identify all logged activity (possibly across multiple log files) associated
|
||||
with a given connection 4-tuple over its lifetime.
|
||||
Every single line in this log starts with a timestamp, a unique
|
||||
connection identifier (UID), and a connection 4-tuple (originator
|
||||
host/port and responder host/port). The UID can be used to identify all
|
||||
logged activity (possibly across multiple log files) associated with a
|
||||
given connection 4-tuple over its lifetime.
|
||||
|
||||
The remaining columns detail the activity that's occurring. For example, the columns on the line below
|
||||
(shortened for brevity) show a request to the root of Bro website::
|
||||
The remaining columns detail the activity that's occurring. For
|
||||
example, the columns on the line below (shortened for brevity) show a
|
||||
request to the root of Bro website::
|
||||
|
||||
# method host uri referrer user_agent
|
||||
GET bro.org / - <...>Chrome/12.0.742.122<...>
|
||||
|
||||
Network administrators and security engineers, for instance, can use the information in this log to understand
|
||||
the HTTP activity on the network and troubleshoot network problems or search for anomalous activities. At this
|
||||
point, we would like to stress out the fact that there is no just one right way to perform analysis; it will
|
||||
depend on the expertise of the person doing the analysis and the specific details of the task to accomplish.
|
||||
Network administrators and security engineers, for instance, can use the
|
||||
information in this log to understand the HTTP activity on the network
|
||||
and troubleshoot network problems or search for anomalous activities. At
|
||||
this point, we would like to stress out the fact that there is no just
|
||||
one right way to perform analysis; it will depend on the expertise of
|
||||
the person doing the analysis and the specific details of the task to
|
||||
accomplish.
|
||||
|
||||
For more information about how to handle the HTTP protocol in Bro, including a complete list
|
||||
of the fields available in http.log, go to Bro's
|
||||
:doc:`HTTP script reference </scripts/base/protocols/http/main.bro>`.
|
||||
For more information about how to handle the HTTP protocol in Bro,
|
||||
including a complete list of the fields available in http.log, go to
|
||||
Bro's :doc:`HTTP script reference
|
||||
</scripts/base/protocols/http/main.bro>`.
|
||||
|
||||
------------------------
|
||||
Detecting a Proxy Server
|
||||
------------------------
|
||||
|
||||
A proxy server is a device on your network configured to request a service on behalf of a third system; one of the
|
||||
most common examples is a Web proxy server. A client without Internet access connects to the proxy and requests
|
||||
a Web page; the proxy then sends the request to the actual Web server, receives the response and passes it to the original
|
||||
A proxy server is a device on your network configured to request a
|
||||
service on behalf of a third system; one of the most common examples is
|
||||
a Web proxy server. A client without Internet access connects to the
|
||||
proxy and requests a Web page; the proxy then sends the request to the
|
||||
actual Web server, receives the response and passes it to the original
|
||||
client.
|
||||
|
||||
Proxies were conceived to help manage a network and provide better encapsulation. By themselves, proxies are not a security
|
||||
threat, but a misconfigured or unauthorized proxy can allow others, either inside or outside the network, to access any
|
||||
Web site and even conduct malicious activities anonymously using the network resources.
|
||||
Proxies were conceived to help manage a network and provide better
|
||||
encapsulation. By themselves, proxies are not a security threat, but a
|
||||
misconfigured or unauthorized proxy can allow others, either inside or
|
||||
outside the network, to access any Web site and even conduct malicious
|
||||
activities anonymously using the network resources.
|
||||
|
||||
What Proxy Server traffic looks like
|
||||
-------------------------------------
|
||||
|
||||
In general, when a client starts talking with a proxy server, the traffic consists of two parts: (i) a GET request, and
|
||||
(ii) an HTTP/ reply::
|
||||
In general, when a client starts talking with a proxy server, the
|
||||
traffic consists of two parts: (i) a GET request, and (ii) an HTTP/
|
||||
reply::
|
||||
|
||||
Request: GET http://www.bro.org/ HTTP/1.1
|
||||
Reply: HTTP/1.0 200 OK
|
||||
|
||||
This will differ from traffic between a client and a normal Web server because GET requests should not include "http" on
|
||||
the string. So we can use this to identify a proxy server.
|
||||
This will differ from traffic between a client and a normal Web server
|
||||
because GET requests should not include "http" on the string. So we can
|
||||
use this to identify a proxy server.
|
||||
|
||||
We can write a basic script in Bro to handle the http_reply event and detect a reply for a ``GET http://`` request.
|
||||
We can write a basic script in Bro to handle the http_reply event and
|
||||
detect a reply for a ``GET http://`` request.
|
||||
|
||||
.. code:: bro
|
||||
|
||||
|
|
@ -81,8 +95,10 @@ We can write a basic script in Bro to handle the http_reply event and detect a r
|
|||
}
|
||||
}
|
||||
|
||||
Basically, the script is checking for a "200 OK" status code on a reply for a request that includes "http:". In reality, the HTTP
|
||||
protocol defines several success status codes other than 200, so we will extend our basic script to also consider the additional codes.
|
||||
Basically, the script is checking for a "200 OK" status code on a reply
|
||||
for a request that includes "http:". In reality, the HTTP protocol
|
||||
defines several success status codes other than 200, so we will extend
|
||||
our basic script to also consider the additional codes.
|
||||
|
||||
.. code:: bro
|
||||
|
||||
|
|
@ -112,7 +128,8 @@ protocol defines several success status codes other than 200, so we will extend
|
|||
}
|
||||
}
|
||||
|
||||
Next, we will make sure that the responding proxy is part of our local network.
|
||||
Next, we will make sure that the responding proxy is part of our local
|
||||
network.
|
||||
|
||||
.. code:: bro
|
||||
|
||||
|
|
@ -142,9 +159,12 @@ Next, we will make sure that the responding proxy is part of our local network.
|
|||
}
|
||||
}
|
||||
|
||||
Finally, our goal should be to generate an alert when a proxy has been detected instead of printing a message on the console output.
|
||||
For that, we will tag the traffic accordingly and define a new ``Open_Proxy`` ``Notice`` type to alert of all tagged communications. Once a
|
||||
notification has been fired, we will further suppress it for one day. Below is the complete script.
|
||||
Finally, our goal should be to generate an alert when a proxy has been
|
||||
detected instead of printing a message on the console output. For that,
|
||||
we will tag the traffic accordingly and define a new ``Open_Proxy``
|
||||
``Notice`` type to alert of all tagged communications. Once a
|
||||
notification has been fired, we will further suppress it for one day.
|
||||
Below is the complete script.
|
||||
|
||||
.. code:: bro
|
||||
|
||||
|
|
@ -216,11 +236,14 @@ notification has been fired, we will further suppress it for one day. Below is t
|
|||
Inspecting Files
|
||||
----------------
|
||||
|
||||
Files are often transmitted on regular HTTP conversations between a client and a server. Most of the time these files are harmless,
|
||||
just images and some other multimedia content, but there are also types of files, specially executable files, that can damage
|
||||
your system. We can instruct Bro to create a copy of all executable files that it sees for later analysis using the
|
||||
:ref:`File Analysis Framework <file-analysis-framework>`
|
||||
(introduced with Bro 2.2) as shown in the following script.
|
||||
Files are often transmitted on regular HTTP conversations between a
|
||||
client and a server. Most of the time these files are harmless, just
|
||||
images and some other multimedia content, but there are also types of
|
||||
files, specially executable files, that can damage your system. We can
|
||||
instruct Bro to create a copy of all executable files that it sees for
|
||||
later analysis using the :ref:`File Analysis Framework
|
||||
<file-analysis-framework>` (introduced with Bro 2.2) as shown in the
|
||||
following script.
|
||||
|
||||
.. code:: bro
|
||||
|
||||
|
|
@ -239,9 +262,11 @@ your system. We can instruct Bro to create a copy of all executable files that i
|
|||
Files::add_analyzer(f, Files::ANALYZER_EXTRACT, [$extract_filename=fname]);
|
||||
}
|
||||
|
||||
Bro will extract all files from the traffic and write them on a new ``extract_files/`` subdirectory and change the file name with the right
|
||||
suffix (extension) based on the content of the ext_map table. So, if you want to do the same for other extracted files besides executables
|
||||
you just need to add those types to the ``ext_map`` table like this.
|
||||
Bro will extract all files from the traffic and write them on a new
|
||||
``extract_files/`` subdirectory and change the file name with the right
|
||||
suffix (extension) based on the content of the ext_map table. So, if you
|
||||
want to do the same for other extracted files besides executables you
|
||||
just need to add those types to the ``ext_map`` table like this.
|
||||
|
||||
.. code:: bro
|
||||
|
||||
|
|
@ -253,4 +278,5 @@ you just need to add those types to the ``ext_map`` table like this.
|
|||
["text/html"] = "html",
|
||||
} &default ="";
|
||||
|
||||
Bro will now write the appropriate suffix for text, JPEG, PNG, and HTML files stored in the ``extract_files/`` subdirectory.
|
||||
Bro will now write the appropriate suffix for text, JPEG, PNG, and HTML
|
||||
files stored in the ``extract_files/`` subdirectory.
|
||||
|
|
|
|||
|
|
@ -5,24 +5,37 @@
|
|||
MIME Type Statistics
|
||||
====================
|
||||
|
||||
Files are constantly transmitted over HTTP on regular networks. These files belong to a specific category (i.e., executable, text, image, etc.) identified
|
||||
by a `Multipurpose Internet Mail Extension (MIME) <http://en.wikipedia.org/wiki/MIME>`_. Although MIME was originally developed to identify the type of
|
||||
non-text attachments on email, it is also used by Web browser to identify the type of files transmitted and present them accordingly.
|
||||
Files are constantly transmitted over HTTP on regular networks. These
|
||||
files belong to a specific category (i.e., executable, text, image,
|
||||
etc.) identified by a `Multipurpose Internet Mail Extension (MIME)
|
||||
<http://en.wikipedia.org/wiki/MIME>`_. Although MIME was originally
|
||||
developed to identify the type of non-text attachments on email, it is
|
||||
also used by Web browser to identify the type of files transmitted and
|
||||
present them accordingly.
|
||||
|
||||
In this tutorial, we will show how to use the Sumstats Framework to collect some statistics information based on MIME types, specifically the total number of
|
||||
occurrences, size in bytes, and number of unique hosts transmitting files over HTTP per each type. For instructions about extracting and creating a local copy
|
||||
of these files, visit :ref:`this <http-monitor>` tutorial instead.
|
||||
In this tutorial, we will show how to use the Sumstats Framework to
|
||||
collect some statistics information based on MIME types, specifically
|
||||
the total number of occurrences, size in bytes, and number of unique
|
||||
hosts transmitting files over HTTP per each type. For instructions about
|
||||
extracting and creating a local copy of these files, visit :ref:`this
|
||||
<http-monitor>` tutorial instead.
|
||||
|
||||
------------------------------------------------
|
||||
MIME Statistics with Sumstats
|
||||
------------------------------------------------
|
||||
When working with the :ref:`Summary Statistics Framework <sumstats-framework>`, you need to define three different pieces: (i) Observations, where
|
||||
the event is observed and fed into the framework. (ii) Reducers, where observations are collected and measured. (iii) Sumstats, where the main functionality
|
||||
is implemented.
|
||||
|
||||
So, we start by defining our observation along with a record to store all statistics values and an observation interval. We are conducting our observation on
|
||||
the :bro:see:`HTTP::log_http` event and we are interested in the MIME type, size of the file ("response_body_len") and the originator host ("orig_h"). We use the MIME
|
||||
type as our key and create observers for the other two values.
|
||||
When working with the :ref:`Summary Statistics Framework
|
||||
<sumstats-framework>`, you need to define three different pieces: (i)
|
||||
Observations, where the event is observed and fed into the framework.
|
||||
(ii) Reducers, where observations are collected and measured. (iii)
|
||||
Sumstats, where the main functionality is implemented.
|
||||
|
||||
So, we start by defining our observation along with a record to store
|
||||
all statistics values and an observation interval. We are conducting our
|
||||
observation on the :bro:see:`HTTP::log_http` event and we are interested
|
||||
in the MIME type, size of the file ("response_body_len") and the
|
||||
originator host ("orig_h"). We use the MIME type as our key and create
|
||||
observers for the other two values.
|
||||
|
||||
.. code:: bro
|
||||
|
||||
|
|
@ -56,15 +69,18 @@ type as our key and create observers for the other two values.
|
|||
}
|
||||
}
|
||||
|
||||
Next, we create the reducers. The first one will accumulate file sizes and the second one will make sure we only store a host ID once. Below is the partial code.
|
||||
Next, we create the reducers. The first one will accumulate file sizes
|
||||
and the second one will make sure we only store a host ID once. Below is
|
||||
the partial code.
|
||||
|
||||
.. code:: bro
|
||||
|
||||
local r1: SumStats::Reducer = [$stream="mime.bytes", $apply=set(SumStats::SUM)];
|
||||
local r2: SumStats::Reducer = [$stream="mime.hits", $apply=set(SumStats::UNIQUE)];
|
||||
|
||||
In our final step, we create the SumStats where we check for the observation interval and once it expires, we populate the record (defined above) with all the
|
||||
relevant data and write it to a log.
|
||||
In our final step, we create the SumStats where we check for the
|
||||
observation interval and once it expires, we populate the record
|
||||
(defined above) with all the relevant data and write it to a log.
|
||||
|
||||
.. code:: bro
|
||||
|
||||
|
|
@ -83,7 +99,8 @@ relevant data and write it to a log.
|
|||
Log::write(LOG, l);
|
||||
}]);
|
||||
|
||||
Putting everything together we end up with the following final code for our script.
|
||||
Putting everything together we end up with the following final code for
|
||||
our script.
|
||||
|
||||
.. code:: bro
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue