mirror of
https://github.com/zeek/zeek.git
synced 2025-10-16 21:48:21 +00:00
Pass over upgrade guide.
This commit is contained in:
Robin Sommer
parent
59015da59f
commit
f573881c58
1 changed files with 127 additions and 55 deletions
182
doc/upgrade.rst
182
doc/upgrade.rst
|
|
@ -12,8 +12,41 @@ Upgrading From Bro 1.5 to 2.0
|
|||
.. contents::
|
||||
|
||||
|
||||
New Script Organization/Hierarchy
|
||||
=================================
|
||||
Introduction
|
||||
============
|
||||
|
||||
As the version number jump suggests, Bro 2.0 is a major upgrade and
|
||||
lots of things have changed. Most importantly, we have rewritten
|
||||
almost all of Bro's default scripts from scratch, using quite
|
||||
different structure now and focusing more on operational deployment.
|
||||
The result is a system that works much better "out of the box", even
|
||||
without much initial site-specific configuration. The down-side is
|
||||
that 1.x configurations will need to be adapted to work with the new
|
||||
version. The two rules of thumb are:
|
||||
|
||||
(1) If you have written your own Bro scripts
|
||||
that do not depend on any of the standard scripts formerly
|
||||
found in ``policy/``, they will most likely just keep working
|
||||
(although you might want to adapt them to use some of the new
|
||||
features, like the new logging framework; see below).
|
||||
|
||||
(2) If you have custom code that depends on specifics of 1.x
|
||||
default scripts (including most configuration tuning), that is
|
||||
unlikely to work with 2.x. We recommend to start by using just
|
||||
the new scripts first, and then port over any customizations
|
||||
incrementally as necessary (they may be much easier to do now,
|
||||
or even unnecessary). Send mail to the Bro user mailing list
|
||||
if you need help.
|
||||
|
||||
Below we summarize changes from 1.x to 2.x in more detail. This list
|
||||
isn't complete, see the `CHANGES <{{git("bro:CHANGES")}}>`_ file in the
|
||||
distribution for the full story.
|
||||
|
||||
Default Scripts
|
||||
===============
|
||||
|
||||
Organisation
|
||||
------------
|
||||
|
||||
In versions before 2.0, Bro scripts were all maintained in a flat
|
||||
directory called ``policy/`` in the source tree. This directory is now
|
||||
|
|
@ -45,22 +78,24 @@ subdirectories of that hierarchy are as follows:
|
|||
- ``policy/`` contains all scripts that a user will need to explicitly
|
||||
tell Bro to load. These are scripts that implement
|
||||
functionality/analysis that not all users may want to use and may have
|
||||
more significant performance costs.
|
||||
more significant performance costs. For a new installation, you
|
||||
should go through these and see what appears useful to load.
|
||||
|
||||
- ``site/`` remains a directory that can be used to store locally
|
||||
developed scripts, but now contains some extra scripts that contain
|
||||
some recommended default configurations. E.g. ``local.bro`` loads
|
||||
extra scripts from ``policy/`` and does extra tuning. These files
|
||||
can also be customized in place without being overwritten by
|
||||
upgrades/reinstalls, unlike scripts in other directories.
|
||||
developed scripts. It now comes with some preinstalled example
|
||||
scripts that contain recommended default configurations going beyond
|
||||
the ``base/`` setup. E.g. ``local.bro`` loads extra scripts from
|
||||
``policy/`` and does extra tuning. These files can be customized in
|
||||
place without being overwritten by upgrades/reinstalls, unlike
|
||||
scripts in other directories.
|
||||
|
||||
Now, with version 2.0, the default ``BROPATH`` automatically will
|
||||
With version 2.0, the default ``BROPATH`` is set to automatically
|
||||
search for scripts in ``policy/``, ``site/`` and their parent
|
||||
directory, but **not** ``base/``. Generally, everything under
|
||||
``base/`` is loaded automatically, but for users of the ``-b`` option
|
||||
(which prevents this), it's important to know that loading a
|
||||
script in that directory requires the extra ``base/`` path
|
||||
qualification. For example, the following two scripts:
|
||||
``base/`` is loaded automatically, but for users of the ``-b`` option,
|
||||
it's important to know that loading a script in that directory
|
||||
requires the extra ``base/`` path qualification. For example, the
|
||||
following two scripts:
|
||||
|
||||
* ``$PREFIX/share/bro/base/protocols/ssl/main.bro``
|
||||
* ``$PREFIX/share/bro/policy/protocols/ssl/validate-certs.bro``
|
||||
|
|
@ -73,21 +108,32 @@ are referenced from another Bro script like:
|
|||
@load protocols/ssl/validate-certs
|
||||
|
||||
Notice how ``policy/`` can be omitted as a convenience in the second
|
||||
case.
|
||||
case. ``@load`` can now also use relative path, e.g., ``@load
|
||||
../main``.
|
||||
|
||||
|
||||
Logging Framework
|
||||
-----------------
|
||||
|
||||
- The logs generated by scripts that ship with Bro are entirely redone
|
||||
to use a standardized format via the new logging framework and
|
||||
generally the content has changed towards making the logs even more
|
||||
useful.
|
||||
to use a standardized, machine-parseable format via the new logging
|
||||
framework. Generally, the log content has been restructured towards
|
||||
making it more directly useful to operations. Also, several
|
||||
analyzers have been signficantly extended and thus now log more
|
||||
information. Take a look at ``ssl.log``.
|
||||
|
||||
* A particular format change that may be useful to note is that the
|
||||
``conn.log`` ``service`` field is derived from DPD instead of
|
||||
well-known ports (while that was already possible in 1.5, it was
|
||||
not the default).
|
||||
|
||||
* Also, ``conn.log`` now reports raw number of packets/bytes per
|
||||
endpoint.
|
||||
|
||||
- The new logging framework makes it possible to extend, customize,
|
||||
and filter logs very easily. See `the logging framework
|
||||
<{{git('bro:doc/logging.rst')}}>`_ more information on usage.
|
||||
|
||||
- A common pattern found in the new scripts is to store logging stream
|
||||
records for protocols inside the ``connection`` records so that
|
||||
state can be collected until enough is seen to log a coherent unit
|
||||
|
|
@ -98,16 +144,39 @@ Logging Framework
|
|||
record by ``redef``'ing it in a
|
||||
``base/protocols/<protocol>/main.bro`` script.
|
||||
|
||||
- The new logging framework also makes it possible to extend and
|
||||
filter logs. See `the logging framework
|
||||
<{{git('bro:doc/logging.rst')}}>`_ more information on usage.
|
||||
- The logging code has been rewritten internally, with script-level
|
||||
interface and output backend now clearly separated. While ASCII
|
||||
logging is still the default, we will add further output types in
|
||||
the future (binary format, direct database logging).
|
||||
|
||||
|
||||
Scripting-Layer API Changes
|
||||
===========================
|
||||
Notice Framework
|
||||
----------------
|
||||
|
||||
The way users interact with "notices" has changed significantly in
|
||||
order to make it easier to define a site policy and more extensible
|
||||
for adding customized actions. See the `the notice framework
|
||||
<{{git('bro:doc/notice.rst')}}>`_.
|
||||
|
||||
|
||||
New Default Settings
|
||||
--------------------
|
||||
|
||||
- Dynamic Protocol Detection (DPD) is now enabled/loaded by default.
|
||||
|
||||
- The default packet filter now examines all packets instead of
|
||||
dynamically building a filter based on which protocol analysis scripts
|
||||
are loaded. See ``PacketFilter::all_packets`` for how to revert to old
|
||||
behavior.
|
||||
|
||||
- By default, Bro now sets a libpcap snaplen of 65535. Depending on
|
||||
the OS, this may have performance implications and you can use the
|
||||
``--snaplen`` option to change the value.
|
||||
|
||||
API Changes
|
||||
-----------
|
||||
|
||||
- The ``@prefixes`` directive works differently now.
|
||||
|
||||
Any added prefixes are now searched for and loaded *after* all input
|
||||
files have been parsed. After all input files are parsed, Bro
|
||||
searches ``BROPATH`` for prefixed, flattened versions of all of the
|
||||
|
|
@ -119,21 +188,6 @@ Scripting-Layer API Changes
|
|||
|
||||
- The ``make_addr`` BIF now returns a ``subnet`` versus an ``addr``
|
||||
|
||||
- The ``net`` type has been removed.
|
||||
|
||||
|
||||
New Default Settings
|
||||
====================
|
||||
|
||||
- Dynamic Protocol Detection (DPD) is now enabled/loaded by default.
|
||||
|
||||
- The default packet filter now examines all packets instead of
|
||||
dynamically building a filter based on which protocol analysis scripts
|
||||
are loaded. See ``PacketFilter::all_packets`` for how to revert to old
|
||||
behavior.
|
||||
|
||||
Script Overhaul/Modernization
|
||||
=============================
|
||||
|
||||
Variable Naming
|
||||
---------------
|
||||
|
|
@ -146,35 +200,53 @@ Variable Naming
|
|||
conventions
|
||||
<{{docroot}}/development/script-conventions.html>`_
|
||||
|
||||
Communication Framework
|
||||
-----------------------
|
||||
|
||||
- The ``remote.bro`` script has evolved into the communication framework
|
||||
BroControl
|
||||
==========
|
||||
|
||||
* ``Remote`` module renamed to ``Communication``
|
||||
BroControl looks pretty much similar to the version coming with Bro 1.x,
|
||||
but has been cleaned up and streamlined signficiantly internally.
|
||||
|
||||
* ``Remote::destinations`` renamed to ``Communication::nodes``
|
||||
(the table of peers)
|
||||
BroControl has a new ``process`` command to process a trace on disk
|
||||
offline using a similar configuration to what BroControl installs for
|
||||
live analysis.
|
||||
|
||||
* ``Remote::Destination`` renamed to ``Communication::Node``
|
||||
(the type that defines a remote peer)
|
||||
BroControl now has an extensive plugin interface for adding new
|
||||
commands and options. Note that this is still considered experimental.
|
||||
|
||||
Notice Framework
|
||||
----------------
|
||||
We have remove the ``analysis`` command, and BroControl does currently
|
||||
not not send daily alarm summaries anymore (this may be restored
|
||||
later).
|
||||
|
||||
The way users interact with "notices" has changed significantly in
|
||||
order to make it easier to define a site policy and more extensible
|
||||
for adding customized actions. See the `the notice framework
|
||||
<{{git('bro:doc/notice.rst')}}>`_.
|
||||
Removed Functionality
|
||||
=====================
|
||||
|
||||
We have remove a bunch of functionality that was rarely used and/or
|
||||
had not been maintained for a while already:
|
||||
|
||||
New Development Infrastructure
|
||||
==============================
|
||||
- The ``net`` script data type.
|
||||
- The ``alarm`` statement; use the notice framework instead.
|
||||
- Trace rewriting.
|
||||
- DFA state expiration in regexp engine.
|
||||
- Active mapping.
|
||||
- Natice DAG support (may come back eventually)
|
||||
- ClamAV support.
|
||||
- The connection compressor is now disabled by default, and will
|
||||
be removed in the future.
|
||||
|
||||
Development Infrastructure
|
||||
==========================
|
||||
|
||||
Bro development has moved from using SVN to Git for revision control.
|
||||
Users that like to use the latest Bro developments by checking it out
|
||||
from the source repositories should see the `development process
|
||||
<{{docroot}}/development/process.html>`_
|
||||
<{{docroot}}/development/process.html>`_. Note that all the various
|
||||
sub-components now reside on their own repositories. However, the
|
||||
top-level Bro repository includes them as git submodules so it's easu
|
||||
to check them all out simultaniously.
|
||||
|
||||
Bro now uses `CMake <http://www.cmake.org>`_ for its build system so
|
||||
that is a new required dependency when building from source.
|
||||
|
||||
Bro now comes with a growing suite of regresssion tests in
|
||||
``testing/``.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue