From c5ab33d88f230a48d1b76d140a83ce9a9c07dc81 Mon Sep 17 00:00:00 2001 From: Jon Siwek Date: Tue, 21 Jan 2014 11:43:35 -0600 Subject: [PATCH] Reformat line width of some docs (i.e. `fmt -72`). --- doc/broids/index.rst | 60 +++++++++++++------ doc/httpmonitor/index.rst | 120 +++++++++++++++++++++++--------------- doc/mimestats/index.rst | 49 +++++++++++----- 3 files changed, 149 insertions(+), 80 deletions(-) diff --git a/doc/broids/index.rst b/doc/broids/index.rst index 1d5f63ec7f..b3317815a7 100644 --- a/doc/broids/index.rst +++ b/doc/broids/index.rst @@ -5,17 +5,24 @@ Bro IDS ======= -An Intrusion Detection System (IDS) allows you to detect suspicious activities happening on your network as a result of a past or active -attack. Because of its programming capabilities, Bro can easily be configured to behave like traditional IDSs and detect common attacks -with well known patterns, or you can create your own scripts to detect conditions specific to your particular case. +An Intrusion Detection System (IDS) allows you to detect suspicious +activities happening on your network as a result of a past or active +attack. Because of its programming capabilities, Bro can easily be +configured to behave like traditional IDSs and detect common attacks +with well known patterns, or you can create your own scripts to detect +conditions specific to your particular case. -In the following sections, we present a few examples of common uses of Bro as an IDS. +In the following sections, we present a few examples of common uses of +Bro as an IDS. ------------------------------------------------ Detecting an FTP Bruteforce attack and notifying ------------------------------------------------ -For the purpose of this exercise, we define FTP bruteforcing as too many rejected usernames and passwords occurring from a single address. -We start by defining a threshold for the number of attempts and a monitoring interval in minutes. + +For the purpose of this exercise, we define FTP bruteforcing as too many +rejected usernames and passwords occurring from a single address. We +start by defining a threshold for the number of attempts and a +monitoring interval in minutes. .. code:: bro @@ -29,8 +36,13 @@ We start by defining a threshold for the number of attempts and a monitoring int const bruteforce_measurement_interval = 15mins &redef; } -Now, using the ftp_reply event, we check for error codes from the `500 series `_ for the "USER" and "PASS" commands, representing rejected usernames or passwords. For this, we can use the :bro:see:`FTP::parse_ftp_reply_code` function to break down the reply code and check if the first digit is a "5" or not. If true, we then use the -:ref:`Summary Statistics Framework ` to keep track of the number of failed attempts. +Now, using the ftp_reply event, we check for error codes from the `500 +series `_ +for the "USER" and "PASS" commands, representing rejected usernames or +passwords. For this, we can use the :bro:see:`FTP::parse_ftp_reply_code` +function to break down the reply code and check if the first digit is a +"5" or not. If true, we then use the :ref:`Summary Statistics Framework +` to keep track of the number of failed attempts. .. code:: bro @@ -44,7 +56,8 @@ Now, using the ftp_reply event, we check for error codes from the `500 series `. For this, we need to define a new Notice type and trigger the alarm under the right -conditions. Below is the final code for our script. +Printing a message on the console is a good start but it will be better +if we raise an alarm instead using the :ref:`Notice Framework +`. For this, we need to define a new Notice type and +trigger the alarm under the right conditions. Below is the final code +for our script. .. code:: bro @@ -135,17 +151,27 @@ conditions. Below is the final code for our script. } } -As a final note, the :doc:`detect-bruteforcing.bro ` script above is include with Bro out of the box, so you only need to load it at startup to instruct Bro to detect and notify of FTP bruteforce attacks. +As a final note, the :doc:`detect-bruteforcing.bro +` script above is +include with Bro out of the box, so you only need to load it at startup +to instruct Bro to detect and notify of FTP bruteforce attacks. ------------- Other Attacks ------------- + Detecting SQL Injection attacks ------------------------------- + Checking files against known malware hashes ------------------------------------------- -Files transmitted on your network could either be completely harmless or contain viruses and other threats. One possible action against -this threat is to compute the hashes of the files and compare them against a list of known malware hashes. Bro simplifies this task -by offering a :doc:`detect-MHR.bro ` script that creates and compares -hashes against the `Malware Hash Registry `_ maintained by Team Cymru. You only need to load this -script along with your other scripts at startup time. + +Files transmitted on your network could either be completely harmless or +contain viruses and other threats. One possible action against this +threat is to compute the hashes of the files and compare them against a +list of known malware hashes. Bro simplifies this task by offering a +:doc:`detect-MHR.bro ` +script that creates and compares hashes against the `Malware Hash +Registry `_ maintained by Team +Cymru. You only need to load this script along with your other scripts +at startup time. diff --git a/doc/httpmonitor/index.rst b/doc/httpmonitor/index.rst index 9b640c63ec..6122421e37 100644 --- a/doc/httpmonitor/index.rst +++ b/doc/httpmonitor/index.rst @@ -5,71 +5,85 @@ Monitoring HTTP Traffic with Bro ================================ -Bro can be used to log the entire HTTP traffic from your network to the http.log file. -This file can then be used for analysis and auditing purposes. +Bro can be used to log the entire HTTP traffic from your network to the +http.log file. This file can then be used for analysis and auditing +purposes. -In the sections below we briefly explain the structure of the http.log file. Then, we -show you how to perform basic HTTP traffic monitoring and analysis tasks with Bro. Some -of these ideas and techniques can later be applied to monitor different protocols in a -similar way. +In the sections below we briefly explain the structure of the http.log +file. Then, we show you how to perform basic HTTP traffic monitoring and +analysis tasks with Bro. Some of these ideas and techniques can later be +applied to monitor different protocols in a similar way. ---------------------------- Introduction to the HTTP log ---------------------------- -The http.log file contains a summary of all HTTP requests and responses sent over a Bro-monitored -network. Here are the first few columns of +The http.log file contains a summary of all HTTP requests and responses +sent over a Bro-monitored network. Here are the first few columns of ``http.log``:: # ts uid orig_h orig_p resp_h resp_p 1311627961.8 HSH4uV8KVJg 192.168.1.100 52303 192.150.187.43 80 -Every single line in this log starts with a timestamp, a unique connection identifier (UID), and a -connection 4-tuple (originator host/port and responder host/port). The UID can be used to -identify all logged activity (possibly across multiple log files) associated -with a given connection 4-tuple over its lifetime. +Every single line in this log starts with a timestamp, a unique +connection identifier (UID), and a connection 4-tuple (originator +host/port and responder host/port). The UID can be used to identify all +logged activity (possibly across multiple log files) associated with a +given connection 4-tuple over its lifetime. -The remaining columns detail the activity that's occurring. For example, the columns on the line below -(shortened for brevity) show a request to the root of Bro website:: +The remaining columns detail the activity that's occurring. For +example, the columns on the line below (shortened for brevity) show a +request to the root of Bro website:: # method host uri referrer user_agent GET bro.org / - <...>Chrome/12.0.742.122<...> -Network administrators and security engineers, for instance, can use the information in this log to understand -the HTTP activity on the network and troubleshoot network problems or search for anomalous activities. At this -point, we would like to stress out the fact that there is no just one right way to perform analysis; it will -depend on the expertise of the person doing the analysis and the specific details of the task to accomplish. +Network administrators and security engineers, for instance, can use the +information in this log to understand the HTTP activity on the network +and troubleshoot network problems or search for anomalous activities. At +this point, we would like to stress out the fact that there is no just +one right way to perform analysis; it will depend on the expertise of +the person doing the analysis and the specific details of the task to +accomplish. -For more information about how to handle the HTTP protocol in Bro, including a complete list -of the fields available in http.log, go to Bro's -:doc:`HTTP script reference `. +For more information about how to handle the HTTP protocol in Bro, +including a complete list of the fields available in http.log, go to +Bro's :doc:`HTTP script reference +`. ------------------------ Detecting a Proxy Server ------------------------ -A proxy server is a device on your network configured to request a service on behalf of a third system; one of the -most common examples is a Web proxy server. A client without Internet access connects to the proxy and requests -a Web page; the proxy then sends the request to the actual Web server, receives the response and passes it to the original +A proxy server is a device on your network configured to request a +service on behalf of a third system; one of the most common examples is +a Web proxy server. A client without Internet access connects to the +proxy and requests a Web page; the proxy then sends the request to the +actual Web server, receives the response and passes it to the original client. -Proxies were conceived to help manage a network and provide better encapsulation. By themselves, proxies are not a security -threat, but a misconfigured or unauthorized proxy can allow others, either inside or outside the network, to access any -Web site and even conduct malicious activities anonymously using the network resources. +Proxies were conceived to help manage a network and provide better +encapsulation. By themselves, proxies are not a security threat, but a +misconfigured or unauthorized proxy can allow others, either inside or +outside the network, to access any Web site and even conduct malicious +activities anonymously using the network resources. What Proxy Server traffic looks like ------------------------------------- -In general, when a client starts talking with a proxy server, the traffic consists of two parts: (i) a GET request, and -(ii) an HTTP/ reply:: +In general, when a client starts talking with a proxy server, the +traffic consists of two parts: (i) a GET request, and (ii) an HTTP/ +reply:: Request: GET http://www.bro.org/ HTTP/1.1 Reply: HTTP/1.0 200 OK -This will differ from traffic between a client and a normal Web server because GET requests should not include "http" on -the string. So we can use this to identify a proxy server. +This will differ from traffic between a client and a normal Web server +because GET requests should not include "http" on the string. So we can +use this to identify a proxy server. -We can write a basic script in Bro to handle the http_reply event and detect a reply for a ``GET http://`` request. +We can write a basic script in Bro to handle the http_reply event and +detect a reply for a ``GET http://`` request. .. code:: bro @@ -81,8 +95,10 @@ We can write a basic script in Bro to handle the http_reply event and detect a r } } -Basically, the script is checking for a "200 OK" status code on a reply for a request that includes "http:". In reality, the HTTP -protocol defines several success status codes other than 200, so we will extend our basic script to also consider the additional codes. +Basically, the script is checking for a "200 OK" status code on a reply +for a request that includes "http:". In reality, the HTTP protocol +defines several success status codes other than 200, so we will extend +our basic script to also consider the additional codes. .. code:: bro @@ -112,7 +128,8 @@ protocol defines several success status codes other than 200, so we will extend } } -Next, we will make sure that the responding proxy is part of our local network. +Next, we will make sure that the responding proxy is part of our local +network. .. code:: bro @@ -142,9 +159,12 @@ Next, we will make sure that the responding proxy is part of our local network. } } -Finally, our goal should be to generate an alert when a proxy has been detected instead of printing a message on the console output. -For that, we will tag the traffic accordingly and define a new ``Open_Proxy`` ``Notice`` type to alert of all tagged communications. Once a -notification has been fired, we will further suppress it for one day. Below is the complete script. +Finally, our goal should be to generate an alert when a proxy has been +detected instead of printing a message on the console output. For that, +we will tag the traffic accordingly and define a new ``Open_Proxy`` +``Notice`` type to alert of all tagged communications. Once a +notification has been fired, we will further suppress it for one day. +Below is the complete script. .. code:: bro @@ -216,11 +236,14 @@ notification has been fired, we will further suppress it for one day. Below is t Inspecting Files ---------------- -Files are often transmitted on regular HTTP conversations between a client and a server. Most of the time these files are harmless, -just images and some other multimedia content, but there are also types of files, specially executable files, that can damage -your system. We can instruct Bro to create a copy of all executable files that it sees for later analysis using the -:ref:`File Analysis Framework ` -(introduced with Bro 2.2) as shown in the following script. +Files are often transmitted on regular HTTP conversations between a +client and a server. Most of the time these files are harmless, just +images and some other multimedia content, but there are also types of +files, specially executable files, that can damage your system. We can +instruct Bro to create a copy of all executable files that it sees for +later analysis using the :ref:`File Analysis Framework +` (introduced with Bro 2.2) as shown in the +following script. .. code:: bro @@ -239,9 +262,11 @@ your system. We can instruct Bro to create a copy of all executable files that i Files::add_analyzer(f, Files::ANALYZER_EXTRACT, [$extract_filename=fname]); } -Bro will extract all files from the traffic and write them on a new ``extract_files/`` subdirectory and change the file name with the right -suffix (extension) based on the content of the ext_map table. So, if you want to do the same for other extracted files besides executables -you just need to add those types to the ``ext_map`` table like this. +Bro will extract all files from the traffic and write them on a new +``extract_files/`` subdirectory and change the file name with the right +suffix (extension) based on the content of the ext_map table. So, if you +want to do the same for other extracted files besides executables you +just need to add those types to the ``ext_map`` table like this. .. code:: bro @@ -253,4 +278,5 @@ you just need to add those types to the ``ext_map`` table like this. ["text/html"] = "html", } &default =""; -Bro will now write the appropriate suffix for text, JPEG, PNG, and HTML files stored in the ``extract_files/`` subdirectory. +Bro will now write the appropriate suffix for text, JPEG, PNG, and HTML +files stored in the ``extract_files/`` subdirectory. diff --git a/doc/mimestats/index.rst b/doc/mimestats/index.rst index 88499a5d6e..f81431eb8b 100644 --- a/doc/mimestats/index.rst +++ b/doc/mimestats/index.rst @@ -5,24 +5,37 @@ MIME Type Statistics ==================== -Files are constantly transmitted over HTTP on regular networks. These files belong to a specific category (i.e., executable, text, image, etc.) identified -by a `Multipurpose Internet Mail Extension (MIME) `_. Although MIME was originally developed to identify the type of -non-text attachments on email, it is also used by Web browser to identify the type of files transmitted and present them accordingly. +Files are constantly transmitted over HTTP on regular networks. These +files belong to a specific category (i.e., executable, text, image, +etc.) identified by a `Multipurpose Internet Mail Extension (MIME) +`_. Although MIME was originally +developed to identify the type of non-text attachments on email, it is +also used by Web browser to identify the type of files transmitted and +present them accordingly. -In this tutorial, we will show how to use the Sumstats Framework to collect some statistics information based on MIME types, specifically the total number of -occurrences, size in bytes, and number of unique hosts transmitting files over HTTP per each type. For instructions about extracting and creating a local copy -of these files, visit :ref:`this ` tutorial instead. +In this tutorial, we will show how to use the Sumstats Framework to +collect some statistics information based on MIME types, specifically +the total number of occurrences, size in bytes, and number of unique +hosts transmitting files over HTTP per each type. For instructions about +extracting and creating a local copy of these files, visit :ref:`this +` tutorial instead. ------------------------------------------------ MIME Statistics with Sumstats ------------------------------------------------ -When working with the :ref:`Summary Statistics Framework `, you need to define three different pieces: (i) Observations, where -the event is observed and fed into the framework. (ii) Reducers, where observations are collected and measured. (iii) Sumstats, where the main functionality -is implemented. -So, we start by defining our observation along with a record to store all statistics values and an observation interval. We are conducting our observation on -the :bro:see:`HTTP::log_http` event and we are interested in the MIME type, size of the file ("response_body_len") and the originator host ("orig_h"). We use the MIME -type as our key and create observers for the other two values. +When working with the :ref:`Summary Statistics Framework +`, you need to define three different pieces: (i) +Observations, where the event is observed and fed into the framework. +(ii) Reducers, where observations are collected and measured. (iii) +Sumstats, where the main functionality is implemented. + +So, we start by defining our observation along with a record to store +all statistics values and an observation interval. We are conducting our +observation on the :bro:see:`HTTP::log_http` event and we are interested +in the MIME type, size of the file ("response_body_len") and the +originator host ("orig_h"). We use the MIME type as our key and create +observers for the other two values. .. code:: bro @@ -56,15 +69,18 @@ type as our key and create observers for the other two values. } } -Next, we create the reducers. The first one will accumulate file sizes and the second one will make sure we only store a host ID once. Below is the partial code. +Next, we create the reducers. The first one will accumulate file sizes +and the second one will make sure we only store a host ID once. Below is +the partial code. .. code:: bro local r1: SumStats::Reducer = [$stream="mime.bytes", $apply=set(SumStats::SUM)]; local r2: SumStats::Reducer = [$stream="mime.hits", $apply=set(SumStats::UNIQUE)]; -In our final step, we create the SumStats where we check for the observation interval and once it expires, we populate the record (defined above) with all the -relevant data and write it to a log. +In our final step, we create the SumStats where we check for the +observation interval and once it expires, we populate the record +(defined above) with all the relevant data and write it to a log. .. code:: bro @@ -83,7 +99,8 @@ relevant data and write it to a log. Log::write(LOG, l); }]); -Putting everything together we end up with the following final code for our script. +Putting everything together we end up with the following final code for +our script. .. code:: bro