mirror of
https://github.com/zeek/zeek.git
synced 2025-10-16 05:28:20 +00:00
Moving the quickstart guide back in.
This commit is contained in:
Robin Sommer
parent
a4998f3780
commit
12e4dd8066
3 changed files with 424 additions and 5 deletions
|
|
@ -8,5 +8,6 @@ User Manual
|
|||
:numbered:
|
||||
|
||||
intro.rst
|
||||
starting.rst
|
||||
quickstart.rst
|
||||
scripting.rst
|
||||
|
||||
|
|
|
|||
422
doc/user-manual/quickstart.rst
Normal file
422
doc/user-manual/quickstart.rst
Normal file
|
|
@ -0,0 +1,422 @@
|
|||
|
||||
=================
|
||||
Quick Start Guide
|
||||
=================
|
||||
|
||||
.. rst-class:: opening
|
||||
|
||||
The short story for getting Bro up and running in a simple configuration
|
||||
for analysis of either live traffic from a network interface or a packet
|
||||
capture trace file.
|
||||
|
||||
.. contents::
|
||||
|
||||
Installation
|
||||
============
|
||||
|
||||
Bro works on most modern, Unix-based systems and requires no custom
|
||||
hardware. It can be downloaded in either pre-built binary package or
|
||||
source code forms. See :doc:`Installing Bro <INSTALL>` for instructions
|
||||
on how to install Bro.
|
||||
|
||||
.. note:: Below, ``$PREFIX`` is used to reference the Bro installation
|
||||
root directory.
|
||||
|
||||
Using BroControl
|
||||
================
|
||||
|
||||
BroControl is an interactive shell for easily operating/managing Bro
|
||||
installations on a single system or even across multiple systems in a
|
||||
traffic-monitoring cluster.
|
||||
|
||||
A Minimal Starting Configuration
|
||||
--------------------------------
|
||||
|
||||
These are the basic configuration changes to make for a minimal BroControl installation
|
||||
that will manage a single Bro instance on the ``localhost``:
|
||||
|
||||
1) In ``$PREFIX/etc/node.cfg``, set the right interface to monitor.
|
||||
2) In ``$PREFIX/etc/networks.cfg``, comment out the default settings and add
|
||||
the networks that Bro will consider local to the monitored environment.
|
||||
3) In ``$PREFIX/etc/broctl.cfg``, change the ``MailTo`` email address to a
|
||||
desired recipient and the ``LogRotationInterval`` to a desired log
|
||||
archival frequency.
|
||||
|
||||
Now start the BroControl shell like:
|
||||
|
||||
.. console::
|
||||
|
||||
broctl
|
||||
|
||||
Since this is the first-time use of the shell, perform an initial installation
|
||||
of the BroControl configuration:
|
||||
|
||||
.. console::
|
||||
|
||||
[BroControl] > install
|
||||
|
||||
Then start up a Bro instance:
|
||||
|
||||
.. console::
|
||||
|
||||
[BroControl] > start
|
||||
|
||||
If there are errors while trying to start the Bro instance, you can
|
||||
can view the details with the ``diag`` command. If started successfully,
|
||||
the Bro instance will begin analyzing traffic according to a default
|
||||
policy and output the results in ``$PREFIX/logs``.
|
||||
|
||||
.. note:: The user starting BroControl needs permission to capture
|
||||
network traffic. If you are not root, you may need to grant further
|
||||
privileges to the account you're using; see the :doc:`FAQ <faq>`.
|
||||
Also, if it looks like Bro is not seeing any traffic, check out
|
||||
the FAQ entry on checksum offloading.
|
||||
|
||||
You can leave it running for now, but to stop this Bro instance you would do:
|
||||
|
||||
.. console::
|
||||
|
||||
[BroControl] > stop
|
||||
|
||||
We also recommend to insert the following entry into `crontab`::
|
||||
|
||||
0-59/5 * * * * $PREFIX/bin/broctl cron
|
||||
|
||||
This will perform a number of regular housekeeping tasks, including
|
||||
verifying that the process is still running (and restarting if not in
|
||||
case of any abnormal termination).
|
||||
|
||||
Browsing Log Files
|
||||
------------------
|
||||
|
||||
By default, logs are written out in human-readable (ASCII) format and
|
||||
data is organized into columns (tab-delimited). Logs that are part of
|
||||
the current rotation interval are accumulated in
|
||||
``$PREFIX/logs/current/`` (if Bro is not running, the directory will
|
||||
be empty). For example, the ``http.log`` contains the results of Bro
|
||||
HTTP protocol analysis. Here are the first few columns of
|
||||
``http.log``::
|
||||
|
||||
# ts uid orig_h orig_p resp_h resp_p
|
||||
1311627961.8 HSH4uV8KVJg 192.168.1.100 52303 192.150.187.43 80
|
||||
|
||||
Logs that deal with analysis of a network protocol will often start like this:
|
||||
a timestamp, a unique connection identifier (UID), and a connection 4-tuple
|
||||
(originator host/port and responder host/port). The UID can be used to
|
||||
identify all logged activity (possibly across multiple log files) associated
|
||||
with a given connection 4-tuple over its lifetime.
|
||||
|
||||
The remaining columns of protocol-specific logs then detail the
|
||||
protocol-dependent activity that's occurring. E.g. ``http.log``'s next few
|
||||
columns (shortened for brevity) show a request to the root of Bro website::
|
||||
|
||||
# method host uri referrer user_agent
|
||||
GET bro.org / - <...>Chrome/12.0.742.122<...>
|
||||
|
||||
Some logs are worth explicit mention:
|
||||
|
||||
``weird.log``
|
||||
Contains unusual/exceptional activity that can indicate
|
||||
malformed connections, traffic that doesn't conform to a particular
|
||||
protocol, malfunctioning/misconfigured hardware, or even an attacker
|
||||
attempting to avoid/confuse a sensor. Without context, it's hard to
|
||||
judge whether this category of activity is interesting and so that is
|
||||
left up to the user to configure.
|
||||
|
||||
``notice.log``
|
||||
Identifies specific activity that Bro recognizes as
|
||||
potentially interesting, odd, or bad. In Bro-speak, such
|
||||
activity is called a "notice".
|
||||
|
||||
|
||||
By default, ``BroControl`` regularly takes all the logs from
|
||||
``$PREFIX/logs/current`` and archives/compresses them to a directory
|
||||
named by date, e.g. ``$PREFIX/logs/2011-10-06``. The frequency at
|
||||
which this is done can be configured via the ``LogRotationInterval``
|
||||
option in ``$PREFIX/etc/broctl.cfg``.
|
||||
|
||||
Deployment Customization
|
||||
------------------------
|
||||
|
||||
The goal of most Bro *deployments* may be to send email alarms when a network
|
||||
event requires human intervention/investigation, but sometimes that conflicts
|
||||
with Bro's goal as a *distribution* to remain policy and site neutral -- the
|
||||
events on one network may be less noteworthy than the same events on another.
|
||||
As a result, deploying Bro can be an iterative process of
|
||||
updating its policy to take different actions for events that are noticed, and
|
||||
using its scripting language to programmatically extend traffic analysis
|
||||
in a precise way.
|
||||
|
||||
One of the first steps to take in customizing Bro might be to get familiar
|
||||
with the notices it can generate by default and either tone down or escalate
|
||||
the action that's taken when specific ones occur.
|
||||
|
||||
Let's say that we've been looking at the ``notice.log`` for a bit and see two
|
||||
changes we want to make:
|
||||
|
||||
1) ``SSL::Invalid_Server_Cert`` (found in the ``note`` column) is one type of
|
||||
notice that means an SSL connection was established and the server's
|
||||
certificate couldn't be validated using Bro's default trust roots, but
|
||||
we want to ignore it.
|
||||
2) ``SSH::Login`` is a notice type that is triggered when an SSH connection
|
||||
attempt looks like it may have been successful, and we want email when
|
||||
that happens, but only for certain servers.
|
||||
|
||||
So we've defined *what* we want to do, but need to know *where* to do it.
|
||||
The answer is to use a script written in the Bro programming language, so
|
||||
let's do a quick intro to Bro scripting.
|
||||
|
||||
Bro Scripts
|
||||
~~~~~~~~~~~
|
||||
|
||||
Bro ships with many pre-written scripts that are highly customizable
|
||||
to support traffic analysis for your specific environment. By
|
||||
default, these will be installed into ``$PREFIX/share/bro`` and can be
|
||||
identified by the use of a ``.bro`` file name extension. These files
|
||||
should **never** be edited directly as changes will be lost when
|
||||
upgrading to newer versions of Bro. The exception to this rule is the
|
||||
directory ``$PREFIX/share/bro/site`` where local site-specific files
|
||||
can be put without fear of being clobbered later. The other main
|
||||
script directories under ``$PREFIX/share/bro`` are ``base`` and
|
||||
``policy``. By default, Bro automatically loads all scripts under
|
||||
``base`` (unless the ``-b`` command line option is supplied), which
|
||||
deal either with collecting basic/useful state about network
|
||||
activities or providing frameworks/utilities that extend Bro's
|
||||
functionality without any performance cost. Scripts under the
|
||||
``policy`` directory may be more situational or costly, and so users
|
||||
must explicitly choose if they want to load them.
|
||||
|
||||
The main entry point for the default analysis configuration of a standalone
|
||||
Bro instance managed by BroControl is the ``$PREFIX/share/bro/site/local.bro``
|
||||
script. So we'll be adding to that in the following sections, but first
|
||||
we have to figure out what to add.
|
||||
|
||||
Redefining Script Option Variables
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Many simple customizations just require you to redefine a variable
|
||||
from a standard Bro script with your own value, using Bro's ``redef``
|
||||
operator.
|
||||
|
||||
The typical way a standard Bro script advertises tweak-able options to users
|
||||
is by defining variables with the ``&redef`` attribute and ``const`` qualifier.
|
||||
A redefineable constant might seem strange, but what that really means is that
|
||||
the variable's value may not change at run-time, but whose initial value can be
|
||||
modified via the ``redef`` operator at parse-time.
|
||||
|
||||
So let's continue on our path to modify the behavior for the two SSL
|
||||
and SSH notices. Looking at :doc:`scripts/base/frameworks/notice/main`,
|
||||
we see that it advertises:
|
||||
|
||||
.. code:: bro
|
||||
|
||||
module Notice;
|
||||
|
||||
export {
|
||||
...
|
||||
## Ignored notice types.
|
||||
const ignored_types: set[Notice::Type] = {} &redef;
|
||||
}
|
||||
|
||||
That's exactly what we want to do for the SSL notice. So add to ``local.bro``:
|
||||
|
||||
.. code:: bro
|
||||
|
||||
redef Notice::ignored_types += { SSL::Invalid_Server_Cert };
|
||||
|
||||
.. note:: The ``Notice`` namespace scoping is necessary here because the
|
||||
variable was declared and exported inside the ``Notice`` module, but is
|
||||
being referenced from outside of it. Variables declared and exported
|
||||
inside a module do not have to be scoped if referring to them while still
|
||||
inside the module.
|
||||
|
||||
Then go into the BroControl shell to check whether the configuration change
|
||||
is valid before installing it and then restarting the Bro instance:
|
||||
|
||||
.. console::
|
||||
|
||||
[BroControl] > check
|
||||
bro is ok.
|
||||
[BroControl] > install
|
||||
removing old policies in /usr/local/bro/spool/policy/site ... done.
|
||||
removing old policies in /usr/local/bro/spool/policy/auto ... done.
|
||||
creating policy directories ... done.
|
||||
installing site policies ... done.
|
||||
generating standalone-layout.bro ... done.
|
||||
generating local-networks.bro ... done.
|
||||
generating broctl-config.bro ... done.
|
||||
updating nodes ... done.
|
||||
[BroControl] > restart
|
||||
stopping bro ...
|
||||
starting bro ...
|
||||
|
||||
Now that the SSL notice is ignored, let's look at how to send an email on
|
||||
the SSH notice. The notice framework has a similar option called
|
||||
``emailed_types``, but that can't differentiate between SSH servers and we
|
||||
only want email for logins to certain ones. Then we come to the ``PolicyItem``
|
||||
record and ``policy`` set and realize that those are actually what get used
|
||||
to implement the simple functionality of ``ignored_types`` and
|
||||
``emailed_types``, but it's extensible such that the condition and action taken
|
||||
on notices can be user-defined.
|
||||
|
||||
In ``local.bro``, let's add a new ``PolicyItem`` record to the ``policy`` set
|
||||
that only takes the email action for SSH logins to a defined set of servers:
|
||||
|
||||
.. code:: bro
|
||||
|
||||
const watched_servers: set[addr] = {
|
||||
192.168.1.100,
|
||||
192.168.1.101,
|
||||
192.168.1.102,
|
||||
} &redef;
|
||||
|
||||
redef Notice::policy += {
|
||||
[$action = Notice::ACTION_EMAIL,
|
||||
$pred(n: Notice::Info) =
|
||||
{
|
||||
return n$note == SSH::Login && n$id$resp_h in watched_servers;
|
||||
}
|
||||
]
|
||||
};
|
||||
|
||||
You'll just have to trust the syntax for now, but what we've done is
|
||||
first declare our own variable to hold a set of watched addresses,
|
||||
``watched_servers``; then added a record to the policy that will generate
|
||||
an email on the condition that the predicate function evaluates to true, which
|
||||
is whenever the notice type is an SSH login and the responding host stored
|
||||
inside the ``Info`` record's connection field is in the set of watched servers.
|
||||
|
||||
.. note:: record field member access is done with the '$' character
|
||||
instead of a '.' as might be expected from other languages, in
|
||||
order to avoid ambiguity with the builtin address type's use of '.'
|
||||
in IPv4 dotted decimal representations.
|
||||
|
||||
Remember, to finalize that configuration change perform the ``check``,
|
||||
``install``, ``restart`` commands in that order inside the BroControl shell.
|
||||
|
||||
Next Steps
|
||||
----------
|
||||
|
||||
By this point, we've learned how to set up the most basic Bro instance and
|
||||
tweak the most basic options. Here's some suggestions on what to explore next:
|
||||
|
||||
* We only looked at how to change options declared in the notice framework,
|
||||
there's many more options to look at in other script packages.
|
||||
* Look at the scripts in ``$PREFIX/share/bro/policy`` for further ones
|
||||
you may want to load.
|
||||
* Reading the code of scripts that ship with Bro is also a great way to gain
|
||||
understanding of the language and how you can start writing your own custom
|
||||
analysis.
|
||||
* Review the :doc:`FAQ <faq>`.
|
||||
* Continue reading below for another mini-tutorial on using Bro as a standalone
|
||||
command-line utility.
|
||||
|
||||
Bro, the Command-Line Utility
|
||||
=============================
|
||||
|
||||
If you prefer not to use BroControl (e.g. don't need its automation and
|
||||
management features), here's how to directly control Bro for your analysis
|
||||
activities.
|
||||
|
||||
Monitoring Live Traffic
|
||||
-----------------------
|
||||
|
||||
Analyzing live traffic from an interface is simple:
|
||||
|
||||
.. console::
|
||||
|
||||
bro -i en0 <list of scripts to load>
|
||||
|
||||
``en0`` can be replaced by the interface of your choice and for the list of
|
||||
scripts, you can just use "all" for now to perform all the default analysis
|
||||
that's available.
|
||||
|
||||
Bro will output log files into the working directory.
|
||||
|
||||
.. note:: The :doc:`FAQ <faq>` entries about
|
||||
capturing as an unprivileged user and checksum offloading are particularly
|
||||
relevant at this point.
|
||||
|
||||
To use the site-specific ``local.bro`` script, just add it to the
|
||||
command-line:
|
||||
|
||||
.. console::
|
||||
|
||||
bro -i en0 local
|
||||
|
||||
This will cause Bro to print a warning about lacking the
|
||||
``Site::local_nets`` variable being configured. You can supply this
|
||||
information at the command line like this (supply your "local" subnets
|
||||
in place of the example subnets):
|
||||
|
||||
.. console::
|
||||
|
||||
bro -r mypackets.trace local "Site::local_nets += { 1.2.3.0/24, 5.6.7.0/24 }"
|
||||
|
||||
|
||||
Reading Packet Capture (pcap) Files
|
||||
-----------------------------------
|
||||
|
||||
Capturing packets from an interface and writing them to a file can be done
|
||||
like this:
|
||||
|
||||
.. console::
|
||||
|
||||
sudo tcpdump -i en0 -s 0 -w mypackets.trace
|
||||
|
||||
Where ``en0`` can be replaced by the correct interface for your system as
|
||||
shown by e.g. ``ifconfig``. (The ``-s 0`` argument tells it to capture
|
||||
whole packets; in cases where it's not supported use ``-s 65535`` instead).
|
||||
|
||||
After a while of capturing traffic, kill the ``tcpdump`` (with ctrl-c),
|
||||
and tell Bro to perform all the default analysis on the capture which primarily includes :
|
||||
|
||||
.. console::
|
||||
|
||||
bro -r mypackets.trace
|
||||
|
||||
Bro will output log files into the working directory.
|
||||
|
||||
If you are interested in more detection, you can again load the ``local``
|
||||
script that we include as a suggested configuration:
|
||||
|
||||
.. console::
|
||||
|
||||
bro -r mypackets.trace local
|
||||
|
||||
|
||||
Telling Bro Which Scripts to Load
|
||||
---------------------------------
|
||||
|
||||
A command-line invocation of Bro typically looks like:
|
||||
|
||||
.. console::
|
||||
|
||||
bro <options> <policies...>
|
||||
|
||||
Where the last arguments are the specific policy scripts that this Bro
|
||||
instance will load. These arguments don't have to include the ``.bro``
|
||||
file extension, and if the corresponding script resides under the default
|
||||
installation path, ``$PREFIX/share/bro``, then it requires no path
|
||||
qualification. Further, a directory of scripts can be specified as
|
||||
an argument to be loaded as a "package" if it contains a ``__load__.bro``
|
||||
script that defines the scripts that are part of the package.
|
||||
|
||||
This example does all of the base analysis (primarily protocol
|
||||
logging) and adds SSL certificate validation.
|
||||
|
||||
.. console::
|
||||
|
||||
bro -r mypackets.trace protocols/ssl/validate-certs
|
||||
|
||||
You might notice that a script you load from the command line uses the
|
||||
``@load`` directive in the Bro language to declare dependence on other scripts.
|
||||
This directive is similar to the ``#include`` of C/C++, except the semantics
|
||||
are "load this script if it hasn't already been loaded".
|
||||
|
||||
.. note:: If one wants Bro to be able to load scripts that live outside the
|
||||
default directories in Bro's installation root, the ``BROPATH`` environment
|
||||
variable will need to be extended to include all the directories that need
|
||||
to be searched for scripts. See the default search path by doing
|
||||
``bro --help``.
|
||||
|
||||
|
|
@ -1,4 +0,0 @@
|
|||
|
||||
=========================
|
||||
Getting Started (Missing)
|
||||
=========================
|
||||
Loading…
Add table
Add a link
Reference in a new issue