Mirror/zeek
1
0
Fork 0
mirror of https://github.com/zeek/zeek.git synced 2025-10-02 14:48:21 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 2

Merge remote-tracking branch 'origin/master' into topic/documentation

Browse source 
Conflicts:
	doc/index.rst
This commit is contained in:
Scott Runnels 2012-11-08 16:38:17 -05:00
commit 1f45d5df1e
100 changed files with 182947 additions and 709 deletions
Download patch file Download diff file Expand all files Collapse all files

55
CHANGES
View file

@ -1,4 +1,59 @@
2.1-152 | 2012-11-05 16:52:34 -0800
* Initial version of a completely reworked intelligence framework.
See doc/intel.rst for more information. (Seth Hall)
* Experimental Modbus analyzer. See policy/protocols/modbus/* for
example policies. (Dina Hadziosmanovic, Seth Hall)
2.1-112 | 2012-11-05 13:58:20 -0800
* New base script for detecting cases of checksum offloading.
Reporter messages will now tell if one has bad checksums. (Seth
Hall)
* Clarifying ownership rules for BroString constructors. (Robin
Sommer)
2.1-109 | 2012-11-05 13:39:34 -0800
* Add detection rate threshold for MHR. (Vlad Grigorescu)
* lookup_hostname_txt fixes. (Vlad Grigorescu)
2.1-104 | 2012-11-01 10:37:50 -0700
* A new built-in function lookup_hostname_txt() provides support for
DNS TXT queries. (Vlad Grigorescu)
2.1-101 | 2012-10-31 14:30:26 -0700
* Documentation reorg: The install info has been consolidated into a
single document (INSTALL), the upgrade info has been moved from
the FAQ to a section in the install doc, and the "upgrading from
1.5 to 2.0" document has been updated (and renamed) to also
include 2.0 to 2.1 upgrade info. (Daniel Thayer)
2.1-96 | 2012-10-31 14:23:50 -0700
* Renaming option defining the frequency of alarm summary mails to
'Logging::default_alarm_mail_interval'. (Daniel Thayer)
2.1-91 | 2012-10-24 16:04:47 -0700
* Adding PPPoE support to Bro. (Seth Hall)
2.1-87 | 2012-10-24 15:40:06 -0700
* Adding missing &redef for some TCP options. Addresses #905, #906,
#907. (Carsten Langer)
2.1-86 | 2012-10-24 15:37:11 -0700
* Add parsing rules for IPv4/IPv6 subnet literal constants.
Addresses #888. (Jon Siwek)
2.1-84 | 2012-10-19 15:12:56 -0700
* Added a BiF strptime() to wrap the corresponding C function. (Seth

262
INSTALL
View file

@ -1,35 +1,55 @@
.. _CMake: http://www.cmake.org
.. _SWIG: http://www.swig.org
.. _Xcode: https://developer.apple.com/xcode/
.. _MacPorts: http://www.macports.org
.. _Fink: http://www.finkproject.org
.. _Homebrew: http://mxcl.github.com/homebrew
.. _bro downloads page: http://bro-ids.org/download/index.html
==============
Installing Bro
==============
Bro can be downloaded in either pre-built binary package or
source code forms.
Prerequisites
=============
Bro requires the following libraries and tools to be installed
before you begin:
* CMake 2.6.3 or greater http://www.cmake.org
* Libpcap http://www.tcpdump.org
* Perl (used only during the Bro build process)
* OpenSSL libraries http://www.openssl.org
* Libpcap headers and libraries http://www.tcpdump.org
* OpenSSL headers and libraries http://www.openssl.org
* BIND8 headers and libraries
* BIND8 library
* Libmagic
* Libz
* Bash (for BroControl)
To build Bro from source, the following additional dependencies are required:
* CMake 2.6.3 or greater http://www.cmake.org
* SWIG http://www.swig.org
* Bison (GNU Parser Generator)
* Flex (Fast Lexical Analyzer)
* Bash (for BroControl)
* Libpcap headers http://www.tcpdump.org
* OpenSSL headers http://www.openssl.org
* libmagic headers
* zlib headers
* Perl
Bro can make use of some optional libraries and tools if they are found at
build time:
@ -45,29 +65,161 @@ build time:
* Ruby executable, library, and headers (for Broccoli Ruby bindings)
Installation
============
Installing From Pre-Built Binary Release Packages
=================================================
To build and install into ``/usr/local/bro``::
See the `bro downloads page`_ for currently supported/targeted platforms.
* RPM
.. console::
sudo yum localinstall Bro-*.rpm
* DEB
.. console::
sudo gdebi Bro-*.deb
* MacOS Disk Image with Installer
Just open the ``Bro-*.dmg`` and then run the ``.pkg`` installer.
Everything installed by the package will go into ``/opt/bro``.
The primary install prefix for binary packages is ``/opt/bro``.
Non-MacOS packages that include BroControl also put variable/runtime
data (e.g. Bro logs) in ``/var/opt/bro``.
Installing From Source
======================
Required Dependencies
~~~~~~~~~~~~~~~~~~~~~
The following dependencies are required to build Bro:
* RPM/RedHat-based Linux:
.. console::
sudo yum install cmake make gcc gcc-c++ flex bison libpcap-devel openssl-devel python-devel swig zlib-devel file-devel
* DEB/Debian-based Linux:
.. console::
sudo apt-get install cmake make gcc g++ flex bison libpcap-dev libssl-dev python-dev swig zlib1g-dev libmagic-dev
* FreeBSD
Most required dependencies should come with a minimal FreeBSD install
except for the following.
.. console::
sudo pkg_add -r bash cmake swig bison python
Note that ``bash`` needs to be in ``PATH``, which by default it is
not. The FreeBSD package installs the binary into
``/usr/local/bin``.
* Mac OS X
Compiling source code on Macs requires first downloading Xcode_,
then going through its "Preferences..." -> "Downloads" menus to
install the "Command Line Tools" component.
Lion (10.7) and Mountain Lion (10.8) come with all required
dependencies except for CMake_, SWIG_, and ``libmagic``.
Distributions of these dependencies can likely be obtained from your
preferred Mac OS X package management system (e.g. MacPorts_, Fink_,
or Homebrew_).
Specifically for MacPorts, the ``swig``, ``swig-ruby``, ``swig-python``
and ``file`` packages provide the required dependencies.
Optional Dependencies
~~~~~~~~~~~~~~~~~~~~~
Bro can use libGeoIP for geo-locating IP addresses, and sendmail for
sending emails.
* RedHat Enterprise Linux:
.. console::
sudo yum install geoip-devel sendmail
* CentOS Linux:
.. console::
sudo yum install GeoIP-devel sendmail
* DEB/Debian-based Linux:
.. console::
sudo apt-get install libgeoip-dev sendmail
* Ports-based FreeBSD
.. console::
sudo pkg_add -r GeoIP
sendmail is typically already available.
* Mac OS X
Vanilla OS X installations don't ship with libGeoIP, but
if installed from your preferred package management system (e.g. MacPorts,
Fink, or Homebrew), they should be automatically detected and Bro will
compile against them.
Additional steps may be needed to :doc:`get the right GeoIP database <geoip>`.
Compiling Bro Source Code
~~~~~~~~~~~~~~~~~~~~~~~~~
Bro releases are bundled into source packages for convenience and
available from the `bro downloads page`_.
Alternatively, the latest Bro development version can be obtained through git
repositories hosted at `git.bro-ids.org <http://git.bro-ids.org>`_. See
our `git development documentation
<http://bro-ids.org/development/process.html>`_ for comprehensive
information on Bro's use of git revision control, but the short story
for downloading the full source code experience for Bro via git is:
.. console::
git clone --recursive git://git.bro-ids.org/bro
.. note:: If you choose to clone the ``bro`` repository non-recursively for
a "minimal Bro experience", be aware that compiling it depends on
BinPAC, which has its own ``binpac`` repository. Either install it
first or initialize/update the cloned ``bro`` repository's
``aux/binpac`` submodule.
The typical way to build and install from source is (for more options,
run ``./configure --help``):
.. console::
./configure
make
make install
This will first build Bro in a directory inside the distribution
called ``build/``, using default build options. It then installs all
required files into ``/usr/local/bro``, including the Bro binary in
``/usr/local/bro/bin/bro``.
You can specify a different installation directory with::
./configure --prefix=<dir>
Note that ``/usr`` and ``/opt/bro`` are the standard prefixes for
binary Bro packages to be installed, so those are typically not good
choices unless you are creating such a package.
Run ``./configure --help`` for more options.
The default installation path is ``/usr/local/bro``, which would typically
require root privileges when doing the ``make install``. A different
installation path can be chosen by specifying the ``--prefix`` option.
Note that ``/usr`` and ``/opt/bro`` are the
standard prefixes for binary Bro packages to be installed, so those are
typically not good choices unless you are creating such a package.
Depending on the Bro package you downloaded, there may be auxiliary
tools and libraries available in the ``aux/`` directory. Some of them
@ -81,6 +233,66 @@ OpenBSD users, please see our FAQ at
http://www.bro-ids.org/documentation/faq.html if you are having
problems installing Bro.
Upgrading From a Previous Version of Bro
========================================
If you're doing an upgrade install (rather than a fresh install),
there's two suggested approaches: either install Bro using the same
installation prefix directory as before, or pick a new prefix and copy
local customizations over.
Re-Use Previous Install Prefix
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
If you choose to configure and install Bro with the same prefix
directory as before, local customization and configuration to files in
``$prefix/share/bro/site`` and ``$prefix/etc`` won't be overwritten
(``$prefix`` indicating the root of where Bro was installed). Also, logs
generated at run-time won't be touched by the upgrade. (But making
a backup of local changes before upgrading is still recommended.)
After upgrading, remember to check ``$prefix/share/bro/site`` and
``$prefix/etc`` for ``.example`` files, which indicate the
distribution's version of the file differs from the local one, which may
include local changes. Review the differences, and make adjustments
as necessary (for differences that aren't the result of a local change,
use the new version's).
Pick a New Install prefix
~~~~~~~~~~~~~~~~~~~~~~~~~
If you want to install the newer version in a different prefix
directory than before, you can just copy local customization and
configuration files from ``$prefix/share/bro/site`` and ``$prefix/etc``
to the new location (``$prefix`` indicating the root of where Bro was
originally installed). Make sure to review the files for difference
before copying and make adjustments as necessary (for differences that
aren't the result of a local change, use the new version's). Of
particular note, the copied version of ``$prefix/etc/broctl.cfg`` is
likely to need changes to the ``SpoolDir`` and ``LogDir`` settings.
Configure the Run-Time Environment
==================================
Just remember that you may need to adjust your ``PATH`` environment variable
according to the platform/shell/package you're using. For example:
Bourne-Shell Syntax:
.. console::
export PATH=/usr/local/bro/bin:$PATH
C-Shell Syntax:
.. console::
setenv PATH /usr/local/bro/bin:$PATH
Or substitute ``/opt/bro/bin`` instead if you installed from a binary package.
Running Bro
===========

5
NEWS
View file

@ -39,6 +39,11 @@ Changed Functionality
completed, but also after the last event of a whole-file-read (or
whole-db-read, etc.).
- Renamed the option defining the frequency of alarm summary mails to
'Logging::default_alarm_mail_interval'. When using BroControl, the
value can now be set with the new broctl.cfg option
"MailAlarmsInterval".
Bro 2.1
-------

2
VERSION
View file

@ -1 +1 @@
2.1-84
2.1-152

2
aux/bro-aux

@ -1 +1 @@
Subproject commit 01bb93cb23f31a98fb400584e8d2f2fbe8a589ef
Subproject commit bea556198b69d30d64c0cf1b594e6de71176df6f

2
aux/broctl

@ -1 +1 @@
Subproject commit fd0e7e0b0cf50131efaf536a5683266cfe169455
Subproject commit 834131cd0ec0f63cce9de818726fe6167dedbf34

37
doc.old/faq.rst
View file

@ -12,43 +12,6 @@ Frequently Asked Questions
Installation and Configuration
==============================
How do I upgrade to a new version of Bro?
-----------------------------------------
There's two suggested approaches, either install Bro using the same
installation prefix directory as before, or pick a new prefix and copy
local customizations over.
Re-Use Previous Install Prefix
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
If you choose to configure and install Bro with the same prefix
directory as before, local customization and configuration to files in
``$prefix/share/bro/site`` and ``$prefix/etc`` won't be overwritten
(``$prefix`` indicating the root of where Bro was installed). Also, logs
generated at run-time won't be touched by the upgrade. (But making
a backup of local changes before proceeding is still recommended.)
After upgrading, remember to check ``$prefix/share/bro/site`` and
``$prefix/etc`` for ``.example`` files, which indicate the
distribution's version of the file differs from the local one, which may
include local changes. Review the differences, and make adjustments
as necessary (for differences that aren't the result of a local change,
use the new version's).
Pick a New Install prefix
^^^^^^^^^^^^^^^^^^^^^^^^^
If you want to install the newer version in a different prefix
directory than before, you can just copy local customization and
configuration files from ``$prefix/share/bro/site`` and ``$prefix/etc``
to the new location (``$prefix`` indicating the root of where Bro was
originally installed). Make sure to review the files for difference
before copying and make adjustments as necessary (for differences that
aren't the result of a local change, use the new version's). Of
particular note, the copied version of ``$prefix/etc/broctl.cfg`` is
likely to need changes to the ``SpoolDir`` and ``LogDir`` settings.
How can I tune my operating system for best capture performance?
----------------------------------------------------------------

188
doc.old/quickstart.rst
View file

@ -1,10 +1,3 @@
.. _CMake: http://www.cmake.org
.. _SWIG: http://www.swig.org
.. _Xcode: https://developer.apple.com/xcode/
.. _MacPorts: http://www.macports.org
.. _Fink: http://www.finkproject.org
.. _Homebrew: http://mxcl.github.com/homebrew
.. _bro downloads page: http://bro-ids.org/download/index.html
=================
Quick Start Guide
@ -23,181 +16,11 @@ Installation
Bro works on most modern, Unix-based systems and requires no custom
hardware. It can be downloaded in either pre-built binary package or
source code forms.
source code forms. See :doc:`Installing Bro <INSTALL>` for instructions
on how to install Bro.
Pre-Built Binary Release Packages
---------------------------------
See the `bro downloads page`_ for currently supported/targeted platforms.
* RPM
.. console::
sudo yum localinstall Bro-*.rpm
* DEB
.. console::
sudo gdebi Bro-*.deb
* MacOS Disk Image with Installer
Just open the ``Bro-*.dmg`` and then run the ``.pkg`` installer.
Everything installed by the package will go into ``/opt/bro``.
The primary install prefix for binary packages is ``/opt/bro``.
Non-MacOS packages that include BroControl also put variable/runtime
data (e.g. Bro logs) in ``/var/opt/bro``.
Building From Source
--------------------
Required Dependencies
~~~~~~~~~~~~~~~~~~~~~
The following dependencies are required to build Bro:
* RPM/RedHat-based Linux:
.. console::
sudo yum install cmake make gcc gcc-c++ flex bison libpcap-devel openssl-devel python-devel swig zlib-devel file-devel
* DEB/Debian-based Linux:
.. console::
sudo apt-get install cmake make gcc g++ flex bison libpcap-dev libssl-dev python-dev swig zlib1g-dev libmagic-dev
* FreeBSD
Most required dependencies should come with a minimal FreeBSD install
except for the following.
.. console::
sudo pkg_add -r bash cmake swig bison python
Note that ``bash`` needs to be in ``PATH``, which by default it is
not. The FreeBSD package installs the binary into
``/usr/local/bin``.
* Mac OS X
Compiling source code on Macs requires first downloading Xcode_,
then going through its "Preferences..." -> "Downloads" menus to
install the "Command Line Tools" component.
Lion (10.7) and Mountain Lion (10.8) come with all required
dependencies except for CMake_, SWIG_, and ``libmagic``.
Distributions of these dependencies can be obtained from the project
websites linked above, but they're also likely available from your
preferred Mac OS X package management system (e.g. MacPorts_, Fink_,
or Homebrew_).
Specifically for MacPorts, the ``swig``, ``swig-ruby``, ``swig-python``
and ``file`` packages provide the required dependencies.
Optional Dependencies
~~~~~~~~~~~~~~~~~~~~~
Bro can use libGeoIP for geo-locating IP addresses, and sendmail for
sending emails.
* RedHat Enterprise Linux:
.. console::
sudo yum install geoip-devel sendmail
* CentOS Linux:
.. console::
sudo yum install GeoIP-devel sendmail
* DEB/Debian-based Linux:
.. console::
sudo apt-get install libgeoip-dev sendmail
* Ports-based FreeBSD
.. console::
sudo pkg_add -r GeoIP
sendmail is typically already available.
* Mac OS X
Vanilla OS X installations don't ship with libmagic or libGeoIP, but
if installed from your preferred package management system (e.g. MacPorts,
Fink, or Homebrew), they should be automatically detected and Bro will compile
against them.
Additional steps may be needed to :doc:`get the right GeoIP database <geoip>`
Compiling Bro Source Code
~~~~~~~~~~~~~~~~~~~~~~~~~
Bro releases are bundled into source packages for convenience and
available from the `bro downloads page`_.
The latest Bro development versions are obtainable through git
repositories hosted at `git.bro-ids.org <http://git.bro-ids.org>`_. See
our `git development documentation
<http://bro-ids.org/development/process.html>`_ for comprehensive
information on Bro's use of git revision control, but the short story
for downloading the full source code experience for Bro via git is:
.. console::
git clone --recursive git://git.bro-ids.org/bro
.. note:: If you choose to clone the ``bro`` repository non-recursively for
a "minimal Bro experience", be aware that compiling it depends on
BinPAC, which has its own ``binpac`` repository. Either install it
first or initialize/update the cloned ``bro`` repository's
``aux/binpac`` submodule.
See the ``INSTALL`` file included with the source code for more information
on compiling, but this is the typical way to build and install from source
(of course, changing the value of the ``--prefix`` option to point to the
desired root install path):
.. console::
./configure --prefix=/desired/install/path
make
make install
The default installation prefix is ``/usr/local/bro``, which would typically
require root privileges when doing the ``make install``.
Configure the Run-Time Environment
----------------------------------
Just remember that you may need to adjust your ``PATH`` environment variable
according to the platform/shell/package you're using. For example:
Bourne-Shell Syntax:
.. console::
export PATH=/usr/local/bro/bin:$PATH
C-Shell Syntax:
.. console::
setenv PATH /usr/local/bro/bin:$PATH
Or substitute ``/opt/bro/bin`` instead if you installed from a binary package.
.. note:: Below, ``$PREFIX`` is used to reference the Bro installation
root directory.
Using BroControl
================
@ -206,9 +29,6 @@ BroControl is an interactive shell for easily operating/managing Bro
installations on a single system or even across multiple systems in a
traffic-monitoring cluster.
.. note:: Below, ``$PREFIX`` is used to reference the Bro installation
root directory.
A Minimal Starting Configuration
--------------------------------

73
doc.old/upgrade.rst
View file

@ -1,19 +1,80 @@
=============================
Upgrading From Bro 1.5 to 2.0
=============================
==========================================
Upgrading From the Previous Version of Bro
==========================================
.. rst-class:: opening
This guide details differences between Bro versions 1.5 and 2.0
This guide details specific differences between Bro versions
that may be important for users to know as they work on updating
their Bro deployment/configuration to the later version.
.. contents::
Introduction
============
Upgrading From Bro 2.0 to 2.1
=============================
In Bro 2.1, IPv6 is enabled by default. Therefore, when building Bro from
source, the "--enable-brov6" configure option has been removed because it
is no longer relevant.
Other configure changes include renaming the "--enable-perftools" option
to "--enable-perftools-debug" to indicate that the option is only relevant
for debugging the heap. One other change involves what happens when
tcmalloc (part of Google perftools) is found at configure time. On Linux,
it will automatically be linked with Bro, but on other platforms you
need to use the "--enable-perftools" option to enable linking to tcmalloc.
There are a couple of changes to the Bro scripting language to better
support IPv6. First, IPv6 literals appearing in a Bro script must now be
enclosed in square brackets (for example, ``[fe80::db15]``). For subnet
literals, the slash "/" appears after the closing square bracket (for
example, ``[fe80:1234::]/32``). Second, when an IP address variable or IP
address literal is enclosed in pipes (for example, ``|[fe80::db15]|``) the
result is now the size of the address in bits (32 for IPv4 and 128 for IPv6).
In the Bro scripting language, "match" and "using" are no longer reserved
keywords.
Some built-in functions have been removed: "addr_to_count" (use
"addr_to_counts" instead), "bro_has_ipv6" (this is no longer relevant
because Bro now always supports IPv6), "active_connection" (use
"connection_exists" instead), and "connection_record" (use "lookup_connection"
instead).
The "NFS3::mode2string" built-in function has been renamed to "file_mode".
Some built-in functions have been changed: "exit" (now takes the exit code
as a parameter), "to_port" (now takes a string as parameter instead
of a count and transport protocol, but "count_to_port" is still available),
"connect" (now takes an additional string parameter specifying the zone of
a non-global IPv6 address), and "listen" (now takes three additional
parameters to enable listening on IPv6 addresses).
Some Bro script variables have been renamed: "LogAscii::header_prefix"
has been renamed to "LogAscii::meta_prefix", "LogAscii::include_header"
has been renamed to "LogAscii::include_meta".
Some Bro script variables have been removed: "tunnel_port",
"parse_udp_tunnels", "use_connection_compressor", "cc_handle_resets",
"cc_handle_only_syns", and "cc_instantiate_on_data".
A couple events have changed: the "icmp_redirect" event now includes
the target and destination addresses and any Neighbor Discovery options
in the message, and the last parameter of the "dns_AAAA_reply" event has
been removed because it was unused.
The format of the ASCII log files has changed very slightly. Two new lines
are automatically added, one to record the time when the log was opened,
and the other to record the time when the log was closed.
In BroControl, the option (in broctl.cfg) "CFlowAddr" was renamed
to "CFlowAddress".
Upgrading From Bro 1.5 to 2.0
=============================
As the version number jump suggests, Bro 2.0 is a major upgrade and
lots of things have changed. Most importantly, we have rewritten

27
doc/index.rst
View file

@ -5,6 +5,33 @@
Bro Documentation
=================
Guides
------
.. toctree::
:maxdepth: 1
INSTALL
upgrade
quickstart
faq
reporting-problems
xFrameworks
----------
.. toctree::
:maxdepth: 1
notice
logging
input
cluster
signatures
How-Tos
-------
.. toctree::
:maxdepth: 2
:numbered:

125
doc/intel.rst Normal file
View file

@ -0,0 +1,125 @@
Intel Framework
===============
Intro
-----
Intelligence data is critical to the process of monitoring for
security purposes. There is always data which will be discovered
through the incident response process and data which is shared through
private communities. The goals of Bro's Intelligence Framework are to
consume that data, make it available for matching, and provide
infrastructure around improving performance, memory utilization, and
generally making all of this easier.
Data in the Intelligence Framework is the atomic piece of intelligence
such as an IP address or an e-mail address along with a suite of
metadata about it such as a freeform source field, a freeform
descriptive field and a URL which might lead to more information about
the specific item. The metadata in the default scripts has been
deliberately kept minimal so that the community can find the
appropriate fields that need added by writing scripts which extend the
base record using the normal record extension mechanism.
Quick Start
-----------
Load the package of scripts that sends data into the Intelligence
Framework to be checked by loading this script in local.bro::
@load policy/frameworks/intel
(TODO: find some good mechanism for getting setup with good data
quickly)
Refer to the "Loading Intelligence" section below to see the format
for Intelligence Framework text files, then load those text files with
this line in local.bro::
redef Intel::read_files += { "/somewhere/yourdata.txt" };
The data itself only needs to reside on the manager if running in a
cluster.
Architecture
------------
The Intelligence Framework can be thought of as containing three
separate portions. The first part is how intelligence is loaded,
followed by the mechanism for indicating to the intelligence framework
that a piece of data which needs to be checked has been seen, and
thirdly the part where a positive match has been discovered.
Loading Intelligence
********************
Intelligence data can only be loaded through plain text files using
the Input Framework conventions. Additionally, on clusters the
manager is the only node that needs the intelligence data. The
intelligence framework has distribution mechanisms which will push
data out to all of the nodes that need it.
Here is an example of the intelligence data format. Note that all
whitespace separators are literal tabs and fields containing only a
hyphen a considered to be null values.::
#fields host net str str_type meta.source meta.desc meta.url
1.2.3.4 - - - source1 Sending phishing email http://source1.com/badhosts/1.2.3.4
- 31.131.248.0/21 - - spamhaus-drop SBL154982 - -
- - a.b.com Intel::DOMAIN source2 Name used for data exfiltration -
For more examples of built in `str_type` values, please refer to the
autogenerated documentation for the intelligence framework (TODO:
figure out how to do this link).
To load the data once files are created, use the following example
code to define files to load with your own file names of course::
redef Intel::read_files += {
"/somewhere/feed1.txt",
"/somewhere/feed2.txt",
};
Remember, the files only need to be present on the file system of the
manager node on cluster deployments.
Seen Data
*********
When some bit of data is extracted (such as an email address in the
"From" header in a message over SMTP), the Intelligence Framework
needs to be informed that this data was discovered and it's presence
should be checked within the intelligence data set. This is
accomplished through the Intel::seen (TODO: do a reference link)
function.
Typically users won't need to work with this function due to built in
hook scripts that Bro ships with that will "see" data and send it into
the intelligence framework. A user may only need to load the entire
package of hook scripts as a module or pick and choose specific
scripts to load. Keep in mind that as more data is sent into the
intelligence framework, the CPU load consumed by Bro will increase
depending on how many times the Intel::seen function is being called
which is heavily traffic dependent.
The full package of hook scripts that Bro ships with for sending this
"seen" data into the intelligence framework can be loading by adding
this line to local.bro::
@load policy/frameworks/intel
Intelligence Matches
********************
Against all hopes, most networks will eventually have a hit on
intelligence data which could indicate a possible compromise or other
unwanted activity. The Intelligence Framework provides an event that
is generated whenever a match is discovered named Intel::match (TODO:
make a link to inline docs). Due to design restrictions placed upon
the intelligence framework, there is no assurance as to where this
event will be generated. It could be generated on the worker where
the data was seen or on the manager. When the Intel::match event is
handled, only the data given as event arguments to the event can be
assured since the host where the data was seen may not be where
Intel::match is handled.

19
doc/scripts/DocSourcesList.cmake
View file

@ -36,6 +36,8 @@ rest_target(${psd} base/frameworks/input/main.bro)
rest_target(${psd} base/frameworks/input/readers/ascii.bro)
rest_target(${psd} base/frameworks/input/readers/benchmark.bro)
rest_target(${psd} base/frameworks/input/readers/raw.bro)
rest_target(${psd} base/frameworks/intel/cluster.bro)
rest_target(${psd} base/frameworks/intel/input.bro)
rest_target(${psd} base/frameworks/intel/main.bro)
rest_target(${psd} base/frameworks/logging/main.bro)
rest_target(${psd} base/frameworks/logging/postprocessors/scp.bro)
@ -62,6 +64,7 @@ rest_target(${psd} base/frameworks/reporter/main.bro)
rest_target(${psd} base/frameworks/signatures/main.bro)
rest_target(${psd} base/frameworks/software/main.bro)
rest_target(${psd} base/frameworks/tunnels/main.bro)
rest_target(${psd} base/misc/find-checksum-offloading.bro)
rest_target(${psd} base/protocols/conn/contents.bro)
rest_target(${psd} base/protocols/conn/inactivity.bro)
rest_target(${psd} base/protocols/conn/main.bro)
@ -79,6 +82,8 @@ rest_target(${psd} base/protocols/http/main.bro)
rest_target(${psd} base/protocols/http/utils.bro)
rest_target(${psd} base/protocols/irc/dcc-send.bro)
rest_target(${psd} base/protocols/irc/main.bro)
rest_target(${psd} base/protocols/modbus/consts.bro)
rest_target(${psd} base/protocols/modbus/main.bro)
rest_target(${psd} base/protocols/smtp/entities-excerpt.bro)
rest_target(${psd} base/protocols/smtp/entities.bro)
rest_target(${psd} base/protocols/smtp/main.bro)
@ -100,11 +105,21 @@ rest_target(${psd} base/utils/patterns.bro)
rest_target(${psd} base/utils/site.bro)
rest_target(${psd} base/utils/strings.bro)
rest_target(${psd} base/utils/thresholds.bro)
rest_target(${psd} base/utils/urls.bro)
rest_target(${psd} policy/frameworks/communication/listen.bro)
rest_target(${psd} policy/frameworks/control/controllee.bro)
rest_target(${psd} policy/frameworks/control/controller.bro)
rest_target(${psd} policy/frameworks/dpd/detect-protocols.bro)
rest_target(${psd} policy/frameworks/dpd/packet-segment-logging.bro)
rest_target(${psd} policy/frameworks/intel/conn-established.bro)
rest_target(${psd} policy/frameworks/intel/dns.bro)
rest_target(${psd} policy/frameworks/intel/http-host-header.bro)
rest_target(${psd} policy/frameworks/intel/http-url.bro)
rest_target(${psd} policy/frameworks/intel/http-user-agents.bro)
rest_target(${psd} policy/frameworks/intel/smtp-url-extraction.bro)
rest_target(${psd} policy/frameworks/intel/smtp.bro)
rest_target(${psd} policy/frameworks/intel/ssl.bro)
rest_target(${psd} policy/frameworks/intel/where-locations.bro)
rest_target(${psd} policy/frameworks/metrics/conn-example.bro)
rest_target(${psd} policy/frameworks/metrics/http-example.bro)
rest_target(${psd} policy/frameworks/metrics/ssl-example.bro)
@ -112,6 +127,7 @@ rest_target(${psd} policy/frameworks/software/version-changes.bro)
rest_target(${psd} policy/frameworks/software/vulnerable.bro)
rest_target(${psd} policy/integration/barnyard2/main.bro)
rest_target(${psd} policy/integration/barnyard2/types.bro)
rest_target(${psd} policy/integration/collective-intel/main.bro)
rest_target(${psd} policy/misc/analysis-groups.bro)
rest_target(${psd} policy/misc/capture-loss.bro)
rest_target(${psd} policy/misc/loaded-scripts.bro)
@ -126,7 +142,6 @@ rest_target(${psd} policy/protocols/dns/detect-external-names.bro)
rest_target(${psd} policy/protocols/ftp/detect.bro)
rest_target(${psd} policy/protocols/ftp/software.bro)
rest_target(${psd} policy/protocols/http/detect-MHR.bro)
rest_target(${psd} policy/protocols/http/detect-intel.bro)
rest_target(${psd} policy/protocols/http/detect-sqli.bro)
rest_target(${psd} policy/protocols/http/detect-webapps.bro)
rest_target(${psd} policy/protocols/http/header-names.bro)
@ -134,6 +149,8 @@ rest_target(${psd} policy/protocols/http/software-browser-plugins.bro)
rest_target(${psd} policy/protocols/http/software.bro)
rest_target(${psd} policy/protocols/http/var-extraction-cookies.bro)
rest_target(${psd} policy/protocols/http/var-extraction-uri.bro)
rest_target(${psd} policy/protocols/modbus/known-masters-slaves.bro)
rest_target(${psd} policy/protocols/modbus/track-memmap.bro)
rest_target(${psd} policy/protocols/smtp/blocklists.bro)
rest_target(${psd} policy/protocols/smtp/detect-suspicious-orig.bro)
rest_target(${psd} policy/protocols/smtp/software.bro)

4
scripts/base/frameworks/cluster/nodes/manager.bro
View file

@ -13,8 +13,12 @@
## Turn off remote logging since this is the manager and should only log here.
redef Log::enable_remote_logging = F;
## Log rotation interval.
redef Log::default_rotation_interval = 1 hrs;
## Alarm summary mail interval.
redef Log::default_mail_alarms_interval = 24 hrs;
## Use the cluster's archive logging script.
redef Log::default_rotation_postprocessor_cmd = "archive-log";

12
scripts/base/frameworks/intel/__load__.bro
View file

@ -1 +1,11 @@
@load ./main
@load ./main
# The cluster framework must be loaded first.
@load base/frameworks/cluster
@if ( Cluster::is_enabled() )
@load ./cluster
@endif
# This needs cluster support to only read on the manager.
@load ./input

61
scripts/base/frameworks/intel/cluster.bro Normal file
View file

@ -0,0 +1,61 @@
##! Cluster transparency support for the intelligence framework. This is mostly oriented
##! toward distributing intelligence information across clusters.
@load base/frameworks/cluster
@load ./input
module Intel;
redef record Item += {
## This field is used internally for cluster transparency to avoid
## re-dispatching intelligence items over and over from workers.
first_dispatch: bool &default=T;
};
# If this process is not a manager process, we don't want the full metadata
@if ( Cluster::local_node_type() != Cluster::MANAGER )
redef have_full_data = F;
@endif
global cluster_new_item: event(item: Item);
# Primary intelligence distribution comes from manager.
redef Cluster::manager2worker_events += /^Intel::(cluster_new_item)$/;
# If a worker finds intelligence and adds it, it should share it back to the manager.
redef Cluster::worker2manager_events += /^Intel::(cluster_new_item|match_no_items)$/;
@if ( Cluster::local_node_type() == Cluster::MANAGER )
event Intel::match_no_items(s: Seen) &priority=5
{
event Intel::match(s, Intel::get_items(s));
}
event remote_connection_handshake_done(p: event_peer)
{
# When a worker connects, send it the complete minimal data store.
# It will be kept up to date after this by the cluster_new_item event.
if ( Cluster::nodes[p$descr]$node_type == Cluster::WORKER )
{
send_id(p, "Intel::min_data_store");
}
}
@endif
event Intel::cluster_new_item(item: Intel::Item) &priority=5
{
# Ignore locally generated events to avoid event storms.
if ( is_remote_event() )
Intel::insert(item);
}
event Intel::new_item(item: Intel::Item) &priority=5
{
# The cluster manager always rebroadcasts intelligence.
# Workers redistribute it if it was locally generated.
if ( Cluster::local_node_type() == Cluster::MANAGER ||
item$first_dispatch )
{
item$first_dispatch=F;
event Intel::cluster_new_item(item);
}
}

33
scripts/base/frameworks/intel/input.bro Normal file
View file

@ -0,0 +1,33 @@
@load ./main
module Intel;
export {
## Intelligence files that will be read off disk. The files are
## reread everytime they are updated so updates much be atomic with
## "mv" instead of writing the file in place.
const read_files: set[string] = {} &redef;
}
event Intel::read_entry(desc: Input::EventDescription, tpe: Input::Event, item: Intel::Item)
{
Intel::insert(item);
}
event bro_init() &priority=5
{
if ( ! Cluster::is_enabled() ||
Cluster::local_node_type() == Cluster::MANAGER )
{
for ( a_file in read_files )
{
Input::add_event([$source=a_file,
$reader=Input::READER_ASCII,
$mode=Input::REREAD,
$name=cat("intel-", a_file),
$fields=Intel::Item,
$ev=Intel::read_entry]);
}
}
}

574
scripts/base/frameworks/intel/main.bro
View file

@ -1,323 +1,345 @@
##! The intelligence framework provides a way to store and query IP addresses,
##! strings (with a subtype), and numeric (with a subtype) data. Metadata
##! also be associated with the intelligence like tags which are arbitrary
##! strings, time values, and longer descriptive strings.
# Example string subtypes:
# url
# email
# domain
# software
# user_name
# file_name
# file_md5
# x509_md5
# Example tags:
# infrastructure
# malicious
# sensitive
# canary
# friend
##! and strings (with a str_type). Metadata can
##! also be associated with the intelligence like for making more informed
##! decisions about matching and handling of intelligence.
@load base/frameworks/notice
module Intel;
export {
## The intel logging stream identifier.
redef enum Log::ID += { LOG };
redef enum Notice::Type += {
## This notice should be used in all detector scripts to indicate
## an intelligence based detection.
Detection,
## String data needs to be further categoried since it could represent
## and number of types of data.
type StrType: enum {
## A complete URL without the prefix "http://".
URL,
## User-Agent string, typically HTTP or mail message body.
USER_AGENT,
## Email address.
EMAIL,
## DNS domain name.
DOMAIN,
## A user name.
USER_NAME,
## File hash which is non-hash type specific. It's up to the user to query
## for any relevant hash types.
FILE_HASH,
## Certificate SHA-1 hash.
CERT_HASH,
};
## Record type used for logging information from the intelligence framework.
## Primarily for problems or oddities with inserting and querying data.
## This is important since the content of the intelligence framework can
## change quite dramatically during runtime and problems may be introduced
## into the data.
type Info: record {
## The current network time.
ts: time &log;
## Represents the severity of the message.
## This value should be one of: "info", "warn", "error"
level: string &log;
## The message.
message: string &log;
};
## Record to represent metadata associated with a single piece of
## intelligence.
## Data about an :bro:type:`Intel::Item`
type MetaData: record {
## A description for the data.
## An arbitrary string value representing the data source. Typically,
## the convention for this field will be the source name and feed name
## separated by a hyphen. For example: "source1-c&c".
source: string;
## A freeform description for the data.
desc: string &optional;
## A URL where more information may be found about the intelligence.
## A URL for more information about the data.
url: string &optional;
## The time at which the data was first declared to be intelligence.
first_seen: time &optional;
## When this data was most recent inserted into the framework.
latest_seen: time &optional;
## Arbitrary text tags for the data.
tags: set[string];
};
## Record to represent a singular piece of intelligence.
## Represents a piece of intelligence.
type Item: record {
## If the data is an IP address, this hold the address.
ip: addr &optional;
## If the data is textual, this holds the text.
str: string &optional;
## If the data is numeric, this holds the number.
num: int &optional;
## The subtype of the data for when either the $str or $num fields are
## given. If one of those fields are given, this field must be present.
subtype: string &optional;
## The IP address if the intelligence is about an IP address.
host: addr &optional;
## The network if the intelligence is about a CIDR block.
net: subnet &optional;
## The string if the intelligence is about a string.
str: string &optional;
## The type of data that is in the string if the $str field is set.
str_type: StrType &optional;
## The next five fields are temporary until a better model for
## attaching metadata to an intelligence item is created.
desc: string &optional;
url: string &optional;
first_seen: time &optional;
latest_seen: time &optional;
tags: set[string];
## These single string tags are throw away until pybroccoli supports sets.
tag1: string &optional;
tag2: string &optional;
tag3: string &optional;
## Metadata for the item. Typically represents more deeply \
## descriptive data for a piece of intelligence.
meta: MetaData;
};
## Record model used for constructing queries against the intelligence
## framework.
type QueryItem: record {
## If an IP address is being queried for, this field should be given.
ip: addr &optional;
## If a string is being queried for, this field should be given.
str: string &optional;
## If numeric data is being queried for, this field should be given.
num: int &optional;
## If either a string or number is being queried for, this field should
## indicate the subtype of the data.
subtype: string &optional;
## A set of tags where if a single metadata record attached to an item
## has any one of the tags defined in this field, it will match.
or_tags: set[string] &optional;
## A set of tags where a single metadata record attached to an item
## must have all of the tags defined in this field.
and_tags: set[string] &optional;
## The predicate can be given when searching for a match. It will
## be tested against every :bro:type:`Intel::MetaData` item associated
## with the data being matched on. If it returns T a single time, the
## matcher will consider that the item has matched. This field can
## be used for constructing arbitrarily complex queries that may not
## be possible with the $or_tags or $and_tags fields.
pred: function(meta: Intel::MetaData): bool &optional;
## Enum to represent where data came from when it was discovered.
## The convenction is to prefix the name with "IN_".
type Where: enum {
## A catchall value to represent data of unknown provenance.
IN_ANYWHERE,
};
## Function to insert data into the intelligence framework.
##
## item: The data item.
## The $host field and combination of $str and $str_type fields are mutually
## exclusive. These records *must* represent either an IP address being
## seen or a string being seen.
type Seen: record {
## The IP address if the data seen is an IP address.
host: addr &log &optional;
## The string if the data is about a string.
str: string &log &optional;
## The type of data that is in the string if the $str field is set.
str_type: StrType &log &optional;
## Where the data was discovered.
where: Where &log;
## If the data was discovered within a connection, the
## connection record should go into get to give context to the data.
conn: connection &optional;
};
## Record used for the logging framework representing a positive
## hit within the intelligence framework.
type Info: record {
## Timestamp when the data was discovered.
ts: time &log;
## If a connection was associated with this intelligence hit,
## this is the uid for the connection
uid: string &log &optional;
## If a connection was associated with this intelligence hit,
## this is the conn_id for the connection.
id: conn_id &log &optional;
## Where the data was seen.
seen: Seen &log;
## Sources which supplied data that resulted in this match.
sources: set[string] &log;
};
## Intelligence data manipulation functions.
global insert: function(item: Item);
## Function to declare discovery of a piece of data in order to check
## it against known intelligence for matches.
global seen: function(s: Seen);
## Event to represent a match in the intelligence data from data that was seen.
## On clusters there is no assurance as to where this event will be generated
## so do not assume that arbitrary global state beyond the given data
## will be available.
##
## Returns: T if the data was successfully inserted into the framework,
## otherwise it returns F.
global insert: function(item: Item): bool;
## A wrapper for the :bro:id:`Intel::insert` function. This is primarily
## used as the external API for inserting data into the intelligence
## using Broccoli.
global insert_event: event(item: Item);
## Function for matching data within the intelligence framework.
global matcher: function(item: QueryItem): bool;
## This is the primary mechanism where a user will take actions based on data
## within the intelligence framework.
global match: event(s: Seen, items: set[Item]);
global log_intel: event(rec: Info);
}
type MetaDataStore: table[count] of MetaData;
# Internal handler for matches with no metadata available.
global match_no_items: event(s: Seen);
# Internal events for cluster data distribution
global new_item: event(item: Item);
global updated_item: event(item: Item);
# Optionally store metadata. This is used internally depending on
# if this is a cluster deployment or not.
const have_full_data = T &redef;
# The in memory data structure for holding intelligence.
type DataStore: record {
ip_data: table[addr] of MetaDataStore;
# The first string is the actual value and the second string is the subtype.
string_data: table[string, string] of MetaDataStore;
int_data: table[int, string] of MetaDataStore;
net_data: table[subnet] of set[MetaData];
string_data: table[string, StrType] of set[MetaData];
};
global data_store: DataStore;
global data_store: DataStore &redef;
event bro_init()
# The in memory data structure for holding the barest matchable intelligence.
# This is primarily for workers to do the initial quick matches and store
# a minimal amount of data for the full match to happen on the manager.
type MinDataStore: record {
net_data: set[subnet];
string_data: set[string, StrType];
};
global min_data_store: MinDataStore &redef;
event bro_init() &priority=5
{
Log::create_stream(Intel::LOG, [$columns=Info]);
Log::create_stream(LOG, [$columns=Info, $ev=log_intel]);
}
function insert(item: Item): bool
function find(s: Seen): bool
{
local err_msg = "";
if ( (item?$str || item?$num) && ! item?$subtype )
err_msg = "You must provide a subtype to insert_sync or this item doesn't make sense.";
if ( err_msg == "" )
if ( s?$host &&
((have_full_data && s$host in data_store$net_data) ||
(s$host in min_data_store$net_data)))
{
# Create and fill out the meta data item.
local meta: MetaData;
if ( item?$first_seen )
meta$first_seen = item$first_seen;
if ( item?$latest_seen )
meta$latest_seen = item$latest_seen;
if ( item?$tags )
meta$tags = item$tags;
if ( item?$desc )
meta$desc = item$desc;
if ( item?$url )
meta$url = item$url;
# This is hopefully only temporary until pybroccoli supports sets.
if ( item?$tag1 )
add item$tags[item$tag1];
if ( item?$tag2 )
add item$tags[item$tag2];
if ( item?$tag3 )
add item$tags[item$tag3];
if ( item?$ip )
{
if ( item$ip !in data_store$ip_data )
data_store$ip_data[item$ip] = table();
data_store$ip_data[item$ip][|data_store$ip_data[item$ip]|] = meta;
return T;
}
else if ( item?$str )
{
if ( [item$str, item$subtype] !in data_store$string_data )
data_store$string_data[item$str, item$subtype] = table();
data_store$string_data[item$str, item$subtype][|data_store$string_data[item$str, item$subtype]|] = meta;
return T;
}
else if ( item?$num )
{
if ( [item$num, item$subtype] !in data_store$int_data )
data_store$int_data[item$num, item$subtype] = table();
return T;
}
else if ( s?$str && s?$str_type &&
((have_full_data && [s$str, s$str_type] in data_store$string_data) ||
([s$str, s$str_type] in min_data_store$string_data)))
{
return T;
}
else
{
return F;
}
}
data_store$int_data[item$num, item$subtype][|data_store$int_data[item$num, item$subtype]|] = meta;
return T;
function get_items(s: Seen): set[Item]
{
local item: Item;
local return_data: set[Item] = set();
if ( ! have_full_data )
{
# A reporter warning should be generated here because this function
# should never be called from a host that doesn't have the full data.
# TODO: do a reporter warning.
return return_data;
}
if ( s?$host )
{
# See if the host is known about and it has meta values
if ( s$host in data_store$net_data )
{
for ( m in data_store$net_data[s$host] )
{
# TODO: the lookup should be finding all and not just most specific
# and $host/$net should have the correct value.
item = [$host=s$host, $meta=m];
add return_data[item];
}
}
}
else if ( s?$str && s?$str_type )
{
# See if the string is known about and it has meta values
if ( [s$str, s$str_type] in data_store$string_data )
{
for ( m in data_store$string_data[s$str, s$str_type] )
{
item = [$str=s$str, $str_type=s$str_type, $meta=m];
add return_data[item];
}
}
}
return return_data;
}
function Intel::seen(s: Seen)
{
if ( find(s) )
{
if ( have_full_data )
{
local items = get_items(s);
event Intel::match(s, items);
}
else
err_msg = "Failed to insert intelligence item for some unknown reason.";
}
if ( err_msg != "" )
Log::write(Intel::LOG, [$ts=network_time(), $level="warn", $message=fmt(err_msg)]);
return F;
}
event insert_event(item: Item)
{
insert(item);
}
function match_item_with_metadata(item: QueryItem, meta: MetaData): bool
{
if ( item?$and_tags )
{
local matched = T;
# Every tag given has to match in a single MetaData entry.
for ( tag in item$and_tags )
{
if ( tag !in meta$tags )
matched = F;
event Intel::match_no_items(s);
}
if ( matched )
}
}
function has_meta(check: MetaData, metas: set[MetaData]): bool
{
local check_hash = md5_hash(check);
for ( m in metas )
{
if ( check_hash == md5_hash(m) )
return T;
}
else if ( item?$or_tags )
{
# For OR tags, only a single tag has to match.
for ( tag in item$or_tags )
{
if ( tag in meta$tags )
return T;
}
}
else if ( item?$pred )
return item$pred(meta);
# This indicates some sort of failure in the query
# The records must not be equivalent if we made it this far.
return F;
}
function matcher(item: QueryItem): bool
event Intel::match(s: Seen, items: set[Item]) &priority=5
{
local err_msg = "";
if ( ! (item?$ip || item?$str || item?$num) )
err_msg = "You must supply one of the $ip, $str, or $num fields to search on";
else if ( (item?$or_tags || item?$and_tags) && item?$pred )
err_msg = "You can't match with both tags and a predicate.";
else if ( item?$or_tags && item?$and_tags )
err_msg = "You can't match with both OR'd together tags and AND'd together tags";
else if ( (item?$str || item?$num) && ! item?$subtype )
err_msg = "You must provide a subtype to matcher or this item doesn't make sense.";
else if ( item?$str && item?$num )
err_msg = "You must only provide $str or $num, not both.";
local meta: MetaData;
local empty_set: set[string] = set();
local info: Info = [$ts=network_time(), $seen=s, $sources=empty_set];
if ( err_msg == "" )
if ( s?$conn )
{
if ( item?$ip )
{
if ( item$ip in data_store$ip_data )
{
if ( ! item?$and_tags && ! item?$or_tags && ! item?$pred )
return T;
for ( i in data_store$ip_data[item$ip] )
{
meta = data_store$ip_data[item$ip][i];
if ( match_item_with_metadata(item, meta) )
return T;
}
}
}
else if ( item?$str )
{
if ( [item$str, item$subtype] in data_store$string_data )
{
if ( ! item?$and_tags && ! item?$or_tags && ! item?$pred )
return T;
for ( i in data_store$string_data[item$str, item$subtype] )
{
meta = data_store$string_data[item$str, item$subtype][i];
if ( match_item_with_metadata(item, meta) )
return T;
}
}
}
else if ( item?$num )
{
if ( [item$num, item$subtype] in data_store$int_data )
{
if ( ! item?$and_tags && ! item?$or_tags && ! item?$pred )
return T;
for ( i in data_store$int_data[item$num, item$subtype] )
{
meta = data_store$int_data[item$num, item$subtype][i];
if ( match_item_with_metadata(item, meta) )
return T;
}
}
}
else
err_msg = "Failed to query intelligence data for some unknown reason.";
info$uid = s$conn$uid;
info$id = s$conn$id;
}
if ( err_msg != "" )
Log::write(Intel::LOG, [$ts=network_time(), $level="error", $message=fmt(err_msg)]);
return F;
for ( item in items )
add info$sources[item$meta$source];
Log::write(Intel::LOG, info);
}
function insert(item: Item)
{
if ( item?$str && !item?$str_type )
{
event reporter_warning(network_time(), fmt("You must provide a str_type for strings or this item doesn't make sense. Item: %s", item), "");
return;
}
# Create and fill out the meta data item.
local meta = item$meta;
local metas: set[MetaData];
if ( item?$host )
{
local host = mask_addr(item$host, is_v4_addr(item$host) ? 32 : 128);
if ( have_full_data )
{
if ( host !in data_store$net_data )
data_store$net_data[host] = set();
metas = data_store$net_data[host];
}
add min_data_store$net_data[host];
}
else if ( item?$net )
{
if ( have_full_data )
{
if ( item$net !in data_store$net_data )
data_store$net_data[item$net] = set();
metas = data_store$net_data[item$net];
}
add min_data_store$net_data[item$net];
}
else if ( item?$str )
{
if ( have_full_data )
{
if ( [item$str, item$str_type] !in data_store$string_data )
data_store$string_data[item$str, item$str_type] = set();
metas = data_store$string_data[item$str, item$str_type];
}
add min_data_store$string_data[item$str, item$str_type];
}
local updated = F;
if ( have_full_data )
{
for ( m in metas )
{
if ( meta$source == m$source )
{
if ( has_meta(meta, metas) )
{
# It's the same item being inserted again.
return;
}
else
{
# Same source, different metadata means updated item.
updated = T;
}
}
}
add metas[item$meta];
}
if ( updated )
event Intel::updated_item(item);
else
event Intel::new_item(item);
}

3
scripts/base/frameworks/logging/main.bro
View file

@ -60,6 +60,9 @@ export {
## Default rotation interval. Zero disables rotation.
const default_rotation_interval = 0secs &redef;
## Default alarm summary mail interval. Zero disables alarm summary mails.
const default_mail_alarms_interval = 0secs &redef;
## Default naming format for timestamps embedded into filenames.
## Uses a ``strftime()`` style.
const default_rotation_date_format = "%Y-%m-%d-%H-%M-%S" &redef;

2
scripts/base/frameworks/notice/actions/pp-alarms.bro
View file

@ -101,7 +101,7 @@ event bro_init()
# This replaces the standard non-pretty-printing filter.
Log::add_filter(Notice::ALARM_LOG,
[$name="alarm-mail", $writer=Log::WRITER_NONE,
$interv=Log::default_rotation_interval,
$interv=Log::default_mail_alarms_interval,
$postprocessor=pp_postprocessor]);
}

16
scripts/base/init-bare.bro
View file

@ -826,7 +826,7 @@ const tcp_storm_interarrival_thresh = 1 sec &redef;
## peer's ACKs. Set to zero to turn off this determination.
##
## .. bro:see:: tcp_max_above_hole_without_any_acks tcp_excessive_data_without_further_acks
const tcp_max_initial_window = 4096;
const tcp_max_initial_window = 4096 &redef;
## If we're not seeing our peer's ACKs, the maximum volume of data above a sequence
## hole that we'll tolerate before assuming that there's been a packet drop and we
@ -834,7 +834,7 @@ const tcp_max_initial_window = 4096;
## up.
##
## .. bro:see:: tcp_max_initial_window tcp_excessive_data_without_further_acks
const tcp_max_above_hole_without_any_acks = 4096;
const tcp_max_above_hole_without_any_acks = 4096 &redef;
## If we've seen this much data without any of it being acked, we give up
## on that connection to avoid memory exhaustion due to buffering all that
@ -843,7 +843,7 @@ const tcp_max_above_hole_without_any_acks = 4096;
## has in fact gone too far, but for now we just make this quite beefy.
##
## .. bro:see:: tcp_max_initial_window tcp_max_above_hole_without_any_acks
const tcp_excessive_data_without_further_acks = 10 * 1024 * 1024;
const tcp_excessive_data_without_further_acks = 10 * 1024 * 1024 &redef;
## For services without an a handler, these sets define originator-side ports that
## still trigger reassembly.
@ -2457,6 +2457,16 @@ type bittorrent_benc_dir: table[string] of bittorrent_benc_value;
## bt_tracker_response_not_ok
type bt_tracker_headers: table[string] of string;
type ModbusCoils: vector of bool;
type ModbusRegisters: vector of count;
type ModbusHeaders: record {
tid: count;
pid: count;
uid: count;
function_code: count;
};
module SOCKS;
export {
## This record is for a SOCKS client or server to provide either a

4
scripts/base/init-default.bro
View file

@ -14,6 +14,7 @@
@load base/utils/patterns
@load base/utils/strings
@load base/utils/thresholds
@load base/utils/urls
# This has some deep interplay between types and BiFs so it's
# loaded in base/init-bare.bro
@ -36,8 +37,11 @@
@load base/protocols/ftp
@load base/protocols/http
@load base/protocols/irc
@load base/protocols/modbus
@load base/protocols/smtp
@load base/protocols/socks
@load base/protocols/ssh
@load base/protocols/ssl
@load base/protocols/syslog
@load base/misc/find-checksum-offloading

57
scripts/base/misc/find-checksum-offloading.bro Normal file
View file

@ -0,0 +1,57 @@
##! Discover cases where the local interface is sniffed and outbound packets
##! have checksum offloading. Load this script to receive a notice if it's
##! likely that checksum offload effects are being seen on a live interface or
##! in a packet trace file.
@load base/frameworks/notice
module ChecksumOffloading;
export {
## The interval which is used for checking packet statistics
## to see if checksum offloading is affecting analysis.
const check_interval = 10secs &redef;
}
# Keep track of how many bad checksums have been seen.
global bad_checksums = 0;
# Track to see if this script is done so that messages aren't created multiple times.
global done = F;
event ChecksumOffloading::check()
{
if ( done )
return;
local pkts_recvd = net_stats()$pkts_recvd;
if ( (bad_checksums*1.0 / net_stats()$pkts_recvd*1.0) > 0.05 )
{
local packet_src = reading_traces() ? "trace file likely has" : "interface is likely receiving";
local message = fmt("Your %s invalid IP checksums, most likely from NIC checksum offloading.", packet_src);
Reporter::warning(message);
done = T;
}
else if ( pkts_recvd < 20 )
{
# Keep scheduling this event until we've seen some lower threshold of
# total packets.
schedule check_interval { ChecksumOffloading::check() };
}
}
event bro_init()
{
schedule check_interval { ChecksumOffloading::check() };
}
event net_weird(name: string)
{
if ( name == "bad_IP_checksum" )
++bad_checksums;
}
event bro_done()
{
event ChecksumOffloading::check();
}

2
scripts/base/protocols/modbus/__load__.bro Normal file
View file

@ -0,0 +1,2 @@
@load ./consts
@load ./main

67
scripts/base/protocols/modbus/consts.bro Normal file
View file

@ -0,0 +1,67 @@
module Modbus;
export {
## Standard defined Modbus function codes.
const function_codes = {
[0x01] = "READ_COILS",
[0x02] = "READ_DISCRETE_INPUTS",
[0x03] = "READ_HOLDING_REGISTERS",
[0x04] = "READ_INPUT_REGISTERS",
[0x05] = "WRITE_SINGLE_COIL",
[0x06] = "WRITE_SINGLE_REGISTER",
[0x07] = "READ_EXCEPTION_STATUS",
[0x08] = "DIAGNOSTICS",
[0x0B] = "GET_COMM_EVENT_COUNTER",
[0x0C] = "GET_COMM_EVENT_LOG",
[0x0F] = "WRITE_MULTIPLE_COILS",
[0x10] = "WRITE_MULTIPLE_REGISTERS",
[0x11] = "REPORT_SLAVE_ID",
[0x14] = "READ_FILE_RECORD",
[0x15] = "WRITE_FILE_RECORD",
[0x16] = "MASK_WRITE_REGISTER",
[0x17] = "READ_WRITE_MULTIPLE_REGISTERS",
[0x18] = "READ_FIFO_QUEUE",
[0x2B] = "ENCAP_INTERFACE_TRANSPORT",
# Machine/vendor/network specific functions
[0x09] = "PROGRAM_484",
[0x0A] = "POLL_484",
[0x0D] = "PROGRAM_584_984",
[0x0E] = "POLL_584_984",
[0x12] = "PROGRAM_884_U84",
[0x13] = "RESET_COMM_LINK_884_U84",
[0x28] = "PROGRAM_CONCEPT",
[0x7D] = "FIRMWARE_REPLACEMENT",
[0x7E] = "PROGRAM_584_984_2",
[0x7F] = "REPORT_LOCAL_ADDRESS",
# Exceptions
[0x81] = "READ_COILS_EXCEPTION",
[0x82] = "READ_DISCRETE_INPUTS_EXCEPTION",
[0x83] = "READ_HOLDING_REGISTERS_EXCEPTION",
[0x84] = "READ_INPUT_REGISTERS_EXCEPTION",
[0x85] = "WRITE_SINGLE_COIL_EXCEPTION",
[0x86] = "WRITE_SINGLE_REGISTER_EXCEPTION",
[0x87] = "READ_EXCEPTION_STATUS_EXCEPTION",
[0x8F] = "WRITE_MULTIPLE_COILS_EXCEPTION",
[0x90] = "WRITE_MULTIPLE_REGISTERS_EXCEPTION",
[0x94] = "READ_FILE_RECORD_EXCEPTION",
[0x95] = "WRITE_FILE_RECORD_EXCEPTION",
[0x96] = "MASK_WRITE_REGISTER_EXCEPTION",
[0x97] = "READ_WRITE_MULTIPLE_REGISTERS_EXCEPTION",
[0x98] = "READ_FIFO_QUEUE_EXCEPTION",
} &default=function(i: count):string { return fmt("unknown-%d", i); } &redef;
const exception_codes = {
[0x01] = "ILLEGAL_FUNCTION",
[0x02] = "ILLEGAL_DATA_ADDRESS",
[0x03] = "ILLEGAL_DATA_VALUE",
[0x04] = "SLAVE_DEVICE_FAILURE",
[0x05] = "ACKNOWLEDGE",
[0x06] = "SLAVE_DEVICE_BUSY",
[0x08] = "MEMORY_PARITY_ERROR",
[0x0A] = "GATEWAY_PATH_UNAVAILABLE",
[0x0B] = "GATEWAY_TARGET_DEVICE_FAILED_TO_RESPOND",
} &default=function(i: count):string { return fmt("unknown-%d", i); } &redef;
}

71
scripts/base/protocols/modbus/main.bro Normal file
View file

@ -0,0 +1,71 @@
##! Base Modbus analysis script.
module Modbus;
@load ./consts
export {
redef enum Log::ID += { LOG };
type Info: record {
## Time of the request.
ts: time &log;
## Unique identifier for the connnection.
uid: string &log;
## Identifier for the connection.
id: conn_id &log;
## The name of the function message that was sent.
func: string &log &optional;
## The exception if the response was a failure.
exception: string &log &optional;
};
## Event that can be handled to access the Modbus record as it is sent on
## to the logging framework.
global log_modbus: event(rec: Info);
}
redef record connection += {
modbus: Info &optional;
};
# Configure DPD and the packet filter.
redef capture_filters += { ["modbus"] = "tcp port 502" };
redef dpd_config += { [ANALYZER_MODBUS] = [$ports = set(502/tcp)] };
redef likely_server_ports += { 502/tcp };
event bro_init() &priority=5
{
Log::create_stream(Modbus::LOG, [$columns=Info, $ev=log_modbus]);
}
event modbus_message(c: connection, headers: ModbusHeaders, is_orig: bool) &priority=5
{
if ( ! c?$modbus )
{
c$modbus = [$ts=network_time(), $uid=c$uid, $id=c$id];
}
c$modbus$ts = network_time();
c$modbus$func = function_codes[headers$function_code];
}
event modbus_message(c: connection, headers: ModbusHeaders, is_orig: bool) &priority=-5
{
# Only log upon replies.
# Also, don't log now if this is an exception (log in the exception event handler)
if ( ! is_orig && ( headers$function_code <= 0x81 || headers$function_code >= 0x98 ) )
Log::write(LOG, c$modbus);
}
event modbus_exception(c: connection, headers: ModbusHeaders, code: count) &priority=5
{
c$modbus$exception = exception_codes[code];
}
event modbus_exception(c: connection, headers: ModbusHeaders, code: count) &priority=-5
{
Log::write(LOG, c$modbus);
delete c$modbus$exception;
}

2
scripts/base/utils/patterns.bro
View file

@ -1,5 +1,7 @@
##! Functions for creating and working with patterns.
module GLOBAL;
## Given a pattern as a string with two tildes (~~) contained in it, it will
## return a pattern with string set's elements OR'd together where the
## double-tilde was given (this function only works at or before init time).

25
scripts/base/utils/urls.bro Normal file
View file

@ -0,0 +1,25 @@
## Functions for URL handling.
## A regular expression for matching and extracting URLs.
const url_regex = /^([a-zA-Z\-]{3,5})(:\/\/[^\/?#"'\r\n><]*)([^?#"'\r\n><]*)([^[:blank:]\r\n"'><]*|\??[^"'\r\n><]*)/ &redef;
## Extracts URLs discovered in arbitrary text.
function find_all_urls(s: string): string_set
{
return find_all(s, url_regex);
}
## Extracts URLs discovered in arbitrary text without
## the URL scheme included.
function find_all_urls_without_scheme(s: string): string_set
{
local urls = find_all_urls(s);
local return_urls: set[string] = set();
for ( url in urls )
{
local no_scheme = sub(url, /^([a-zA-Z\-]{3,5})(:\/\/)/, "");
add return_urls[no_scheme];
}
return return_urls;
}

8
scripts/policy/frameworks/intel/__load__.bro Normal file
View file

@ -0,0 +1,8 @@
@load ./conn-established
@load ./dns
@load ./http-host-header
@load ./http-url
@load ./http-user-agents
@load ./ssl
@load ./smtp
@load ./smtp-url-extraction

8
scripts/policy/frameworks/intel/conn-established.bro Normal file
View file

@ -0,0 +1,8 @@
@load base/frameworks/intel
@load ./where-locations
event connection_established(c: connection)
{
Intel::seen([$host=c$id$orig_h, $conn=c, $where=Conn::IN_ORIG]);
Intel::seen([$host=c$id$resp_h, $conn=c, $where=Conn::IN_RESP]);
}

10
scripts/policy/frameworks/intel/dns.bro Normal file
View file

@ -0,0 +1,10 @@
@load base/frameworks/intel
@load ./where-locations
event dns_request(c: connection, msg: dns_msg, query: string, qtype: count, qclass: count)
{
Intel::seen([$str=query,
$str_type=Intel::DOMAIN,
$conn=c,
$where=DNS::IN_REQUEST]);
}

11
scripts/policy/frameworks/intel/http-host-header.bro Normal file
View file

@ -0,0 +1,11 @@
@load base/frameworks/intel
@load ./where-locations
event http_header(c: connection, is_orig: bool, name: string, value: string)
{
if ( is_orig && name == "HOST" )
Intel::seen([$str=value,
$str_type=Intel::DOMAIN,
$conn=c,
$where=HTTP::IN_HOST_HEADER]);
}

12
scripts/policy/frameworks/intel/http-url.bro Normal file
View file

@ -0,0 +1,12 @@
@load base/frameworks/intel
@load base/protocols/http/utils
@load ./where-locations
event http_message_done(c: connection, is_orig: bool, stat: http_message_stat)
{
if ( is_orig && c?$http )
Intel::seen([$str=HTTP::build_url(c$http),
$str_type=Intel::URL,
$conn=c,
$where=HTTP::IN_URL]);
}

12
scripts/policy/frameworks/intel/http-user-agents.bro Normal file
View file

@ -0,0 +1,12 @@
@load base/frameworks/intel
@load ./where-locations
event http_header(c: connection, is_orig: bool, name: string, value: string)
{
if ( is_orig && name == "USER-AGENT" )
Intel::seen([$str=value,
$str_type=Intel::USER_AGENT,
$conn=c,
$where=HTTP::IN_USER_AGENT_HEADER]);
}

15
scripts/policy/frameworks/intel/smtp-url-extraction.bro Normal file
View file

@ -0,0 +1,15 @@
@load base/frameworks/intel
@load base/utils/urls
@load ./where-locations
event mime_segment_data(c: connection, length: count, data: string) &priority=3
{
local urls = find_all_urls_without_scheme(data);
for ( url in urls )
{
Intel::seen([$str=url,
$str_type=Intel::URL,
$conn=c,
$where=SMTP::IN_MESSAGE]);
}
}

71
scripts/policy/frameworks/intel/smtp.bro Normal file
View file

@ -0,0 +1,71 @@
@load base/frameworks/intel
@load base/protocols/smtp
@load ./where-locations
event mime_end_entity(c: connection)
{
if ( c?$smtp )
{
if ( c$smtp?$path )
{
local path = c$smtp$path;
for ( i in path )
{
Intel::seen([$host=path[i],
$conn=c,
$where=SMTP::IN_RECEIVED_HEADER]);
}
}
if ( c$smtp?$user_agent )
Intel::seen([$str=c$smtp$user_agent,
$str_type=Intel::USER_AGENT,
$conn=c,
$where=SMTP::IN_HEADER]);
if ( c$smtp?$x_originating_ip )
Intel::seen([$host=c$smtp$x_originating_ip,
$conn=c,
$where=SMTP::IN_X_ORIGINATING_IP_HEADER]);
if ( c$smtp?$mailfrom )
Intel::seen([$str=c$smtp$mailfrom,
$str_type=Intel::EMAIL,
$conn=c,
$where=SMTP::IN_MAIL_FROM]);
if ( c$smtp?$rcptto )
{
for ( rcptto in c$smtp$rcptto )
{
Intel::seen([$str=rcptto,
$str_type=Intel::EMAIL,
$conn=c,
$where=SMTP::IN_RCPT_TO]);
}
}
if ( c$smtp?$from )
Intel::seen([$str=c$smtp$from,
$str_type=Intel::EMAIL,
$conn=c,
$where=SMTP::IN_FROM]);
if ( c$smtp?$to )
{
for ( email_to in c$smtp$to )
{
Intel::seen([$str=email_to,
$str_type=Intel::EMAIL,
$conn=c,
$where=SMTP::IN_TO]);
}
}
if ( c$smtp?$reply_to )
Intel::seen([$str=c$smtp$reply_to,
$str_type=Intel::EMAIL,
$conn=c,
$where=SMTP::IN_REPLY_TO]);
}
}

34
scripts/policy/frameworks/intel/ssl.bro Normal file
View file

@ -0,0 +1,34 @@
@load base/frameworks/intel
@load base/protocols/ssl
@load ./where-locations
event x509_certificate(c: connection, is_orig: bool, cert: X509, chain_idx: count, chain_len: count, der_cert: string)
{
if ( chain_idx == 0 )
{
if ( /emailAddress=/ in cert$subject )
{
local email = sub(cert$subject, /^.*emailAddress=/, "");
email = sub(email, /,.*$/, "");
Intel::seen([$str=email,
$str_type=Intel::EMAIL,
$conn=c,
$where=(is_orig ? SSL::IN_CLIENT_CERT : SSL::IN_SERVER_CERT)]);
}
Intel::seen([$str=sha1_hash(der_cert),
$str_type=Intel::CERT_HASH,
$conn=c,
$where=(is_orig ? SSL::IN_CLIENT_CERT : SSL::IN_SERVER_CERT)]);
}
}
event ssl_extension(c: connection, is_orig: bool, code: count, val: string)
{
if ( is_orig && SSL::extensions[code] == "server_name" &&
c?$ssl && c$ssl?$server_name )
Intel::seen([$str=c$ssl$server_name,
$str_type=Intel::DOMAIN,
$conn=c,
$where=SSL::IN_SERVER_NAME]);
}

25
scripts/policy/frameworks/intel/where-locations.bro Normal file
View file

@ -0,0 +1,25 @@
@load base/frameworks/intel
export {
redef enum Intel::Where += {
Conn::IN_ORIG,
Conn::IN_RESP,
DNS::IN_REQUEST,
DNS::IN_RESPONSE,
HTTP::IN_HOST_HEADER,
HTTP::IN_USER_AGENT_HEADER,
HTTP::IN_URL,
SMTP::IN_MAIL_FROM,
SMTP::IN_RCPT_TO,
SMTP::IN_FROM,
SMTP::IN_TO,
SMTP::IN_RECEIVED_HEADER,
SMTP::IN_REPLY_TO,
SMTP::IN_X_ORIGINATING_IP_HEADER,
SMTP::IN_MESSAGE,
SSL::IN_SERVER_CERT,
SSL::IN_CLIENT_CERT,
SSL::IN_SERVER_NAME,
SMTP::IN_HEADER,
};
}

6
scripts/policy/integration/collective-intel/README Normal file
View file

@ -0,0 +1,6 @@
Collective Intelligence Framework Integration
=============================================
The scripts in this module are for deeper integration with the Collective Intelligence
Framework (CIF) since Bro's Intel framework doesn't natively behave the same as CIF nor
does it store and maintain the same data in all cases.

1
scripts/policy/integration/collective-intel/__load__.bro Normal file
View file

@ -0,0 +1 @@
@load ./main

15
scripts/policy/integration/collective-intel/main.bro Normal file
View file

@ -0,0 +1,15 @@
@load base/frameworks/intel
module Intel;
## These are some fields to add extended compatibility between Bro and the Collective
## Intelligence Framework
redef record Intel::MetaData += {
## Maps to the Impact field in the Collective Intelligence Framework.
cif_impact: string &optional;
## Maps to the Severity field in the Collective Intelligence Framework.
cif_severity: string &optional;
## Maps to the Confidence field in the Collective Intelligence Framework.
cif_confidence: double &optional;
};

19
scripts/policy/protocols/http/detect-MHR.bro
View file

@ -1,4 +1,4 @@
##! Detect file downloads over HTTP that have MD5 sums matching files in Team
##! Detect file downloads over HTTP that have MD5 sums matching files in Team
##! Cymru's Malware Hash Registry (http://www.team-cymru.org/Services/MHR/).
##! By default, not all file transfers will have MD5 sums calculated. Read the
##! documentation for the :doc:base/protocols/http/file-hash.bro script to see
@ -10,11 +10,17 @@
module HTTP;
export {
redef enum Notice::Type += {
redef enum Notice::Type += {
## The MD5 sum of a file transferred over HTTP matched in the
## malware hash registry.
Malware_Hash_Registry_Match
};
## The malware hash registry runs each malware sample through several A/V engines.
## Team Cymru returns a percentage to indicate how many A/V engines flagged the
## sample as malicious. This threshold allows you to require a minimum detection
## rate (default: 50%).
const MHR_threshold = 50 &redef;
}
event log_http(rec: HTTP::Info)
@ -22,14 +28,15 @@ event log_http(rec: HTTP::Info)
if ( rec?$md5 )
{
local hash_domain = fmt("%s.malware.hash.cymru.com", rec$md5);
when ( local addrs = lookup_hostname(hash_domain) )
when ( local MHR_result = lookup_hostname_txt(hash_domain) )
{
# 127.0.0.2 indicates that the md5 sum was found in the MHR.
if ( 127.0.0.2 in addrs )
# Data is returned as "<dateFirstDetected> <detectionRate>"
local MHR_answer = split1(MHR_result, / /);
if ( length(MHR_answer) == 2 && to_count(MHR_answer[2]) >= MHR_threshold )
{
local url = HTTP::build_url_http(rec);
local message = fmt("%s %s %s", rec$id$orig_h, rec$md5, url);
NOTICE([$note=Malware_Hash_Registry_Match,
NOTICE([$note=Malware_Hash_Registry_Match,
$msg=message, $id=rec$id, $URL=url]);
}
}

21
scripts/policy/protocols/http/detect-intel.bro
View file

@ -1,21 +0,0 @@
##! Intelligence based HTTP detections. Not yet working!
@load base/protocols/http/main
@load base/protocols/http/utils
@load base/frameworks/intel/main
module HTTP;
event log_http(rec: Info)
{
local url = HTTP::build_url(rec);
local query = [$str=url, $subtype="url", $or_tags=set("malicious", "malware")];
if ( Intel::matcher(query) )
{
local msg = fmt("%s accessed a malicious URL from the intelligence framework", rec$id$orig_h);
NOTICE([$note=Intel::Detection,
$msg=msg,
$sub=HTTP::build_url_http(rec),
$id=rec$id]);
}
}

58
scripts/policy/protocols/modbus/known-masters-slaves.bro Normal file
View file

@ -0,0 +1,58 @@
##! Script for tracking known Modbus masters and slaves.
##!
##! .. todo: This script needs a lot of work. What might be more interesting is to track
##! master/slave relationships based on commands sent and successful (non-exception)
##! responses.
@load base/protocols/modbus
module Known;
export {
redef enum Log::ID += { MODBUS_LOG };
type ModbusDeviceType: enum {
MODBUS_MASTER,
MODBUS_SLAVE,
};
type ModbusInfo: record {
## The time the device was discovered.
ts: time &log;
## The IP address of the host.
host: addr &log;
## The type of device being tracked.
device_type: ModbusDeviceType &log;
};
## The Modbus nodes being tracked.
global modbus_nodes: set[addr, ModbusDeviceType] &create_expire=1day &redef;
## Event that can be handled to access the loggable record as it is sent
## on to the logging framework.
global log_known_modbus: event(rec: ModbusInfo);
}
event bro_init() &priority=5
{
Log::create_stream(Known::MODBUS_LOG, [$columns=ModbusInfo, $ev=log_known_modbus]);
}
event modbus_message(c: connection, headers: ModbusHeaders, is_orig: bool)
{
local master = c$id$orig_h;
local slave = c$id$resp_h;
if ( [master, MODBUS_MASTER] !in modbus_nodes )
{
add modbus_nodes[master, MODBUS_MASTER];
Log::write(MODBUS_LOG, [$ts=network_time(), $host=master, $device_type=MODBUS_MASTER]);
}
if ( [slave, MODBUS_SLAVE] !in modbus_nodes )
{
add modbus_nodes[slave, MODBUS_SLAVE];
Log::write(MODBUS_LOG, [$ts=network_time(), $host=slave, $device_type=MODBUS_SLAVE]);
}
}

104
scripts/policy/protocols/modbus/track-memmap.bro Normal file
View file

@ -0,0 +1,104 @@
##! This script tracks the memory map of holding (read/write) registers and logs
##! changes as they are discovered.
##!
##! .. todo: Not all register reads and write functions are being supported yet.
module Modbus;
@load base/protocols/modbus
@load base/utils/directions-and-hosts
export {
redef enum Log::ID += { Modbus::REGISTER_CHANGE_LOG };
## The hosts that should have memory mapping enabled.
const track_memmap: Host = ALL_HOSTS &redef;
type MemmapInfo: record {
## Timestamp for the detected register change
ts: time &log;
## Unique ID for the connection
uid: string &log;
## Connection ID.
id: conn_id &log;
## The device memory offset.
register: count &log;
## The old value stored in the register.
old_val: count &log;
## The new value stored in the register.
new_val: count &log;
## The time delta between when the 'old_val' and 'new_val' were seen.
delta: interval &log;
};
type RegisterValue: record {
last_set: time;
value: count;
};
## Indexed on the device register value and yielding the register value.
type Registers: table[count] of RegisterValue;
## The memory map of slaves is tracked with this variable.
global device_registers: table[addr] of Registers;
## This event is generated every time a register is seen to be different than
## it was previously seen to be.
global changed_register: event(c: connection, register: count, old_val: count, new_val: count, delta: interval);
}
redef record Modbus::Info += {
track_address: count &default=0;
};
event bro_init() &priority=5
{
Log::create_stream(Modbus::REGISTER_CHANGE_LOG, [$columns=MemmapInfo]);
}
event modbus_read_holding_registers_request(c: connection, headers: ModbusHeaders, start_address: count, quantity: count)
{
c$modbus$track_address = start_address+1;
}
event modbus_read_holding_registers_response(c: connection, headers: ModbusHeaders, registers: ModbusRegisters)
{
local slave = c$id$resp_h;
if ( ! addr_matches_host(slave, track_memmap ) )
return;
if ( slave !in device_registers )
device_registers[slave] = table();
local slave_regs = device_registers[slave];
for ( i in registers )
{
if ( c$modbus$track_address in slave_regs )
{
if ( slave_regs[c$modbus$track_address]$value != registers[i] )
{
local delta = network_time() - slave_regs[c$modbus$track_address]$last_set;
event Modbus::changed_register(c, c$modbus$track_address,
slave_regs[c$modbus$track_address]$value, registers[i],
delta);
slave_regs[c$modbus$track_address]$last_set = network_time();
slave_regs[c$modbus$track_address]$value = registers[i];
}
}
else
{
local tmp_reg: RegisterValue = [$last_set=network_time(), $value=registers[i]];
slave_regs[c$modbus$track_address] = tmp_reg;
}
++c$modbus$track_address;
}
}
event Modbus::changed_register(c: connection, register: count, old_val: count, new_val: count, delta: interval)
{
local rec: MemmapInfo = [$ts=network_time(), $uid=c$uid, $id=c$id,
$register=register, $old_val=old_val, $new_val=new_val, $delta=delta];
Log::write(REGISTER_CHANGE_LOG, rec);
}

15
scripts/test-all-policy.bro
View file

@ -14,6 +14,16 @@
# @load frameworks/control/controller.bro
@load frameworks/dpd/detect-protocols.bro
@load frameworks/dpd/packet-segment-logging.bro
@load frameworks/intel/__load__.bro
@load frameworks/intel/conn-established.bro
@load frameworks/intel/dns.bro
@load frameworks/intel/http-host-header.bro
@load frameworks/intel/http-url.bro
@load frameworks/intel/http-user-agents.bro
@load frameworks/intel/smtp-url-extraction.bro
@load frameworks/intel/smtp.bro
@load frameworks/intel/ssl.bro
@load frameworks/intel/where-locations.bro
@load frameworks/metrics/conn-example.bro
@load frameworks/metrics/http-example.bro
@load frameworks/metrics/ssl-example.bro
@ -22,6 +32,8 @@
@load integration/barnyard2/__load__.bro
@load integration/barnyard2/main.bro
@load integration/barnyard2/types.bro
@load integration/collective-intel/__load__.bro
@load integration/collective-intel/main.bro
@load misc/analysis-groups.bro
@load misc/capture-loss.bro
@load misc/loaded-scripts.bro
@ -35,7 +47,6 @@
@load protocols/dns/detect-external-names.bro
@load protocols/ftp/detect.bro
@load protocols/ftp/software.bro
@load protocols/http/detect-intel.bro
@load protocols/http/detect-MHR.bro
@load protocols/http/detect-sqli.bro
@load protocols/http/detect-webapps.bro
@ -44,6 +55,8 @@
@load protocols/http/software.bro
@load protocols/http/var-extraction-cookies.bro
@load protocols/http/var-extraction-uri.bro
@load protocols/modbus/known-masters-slaves.bro
@load protocols/modbus/track-memmap.bro
@load protocols/smtp/blocklists.bro
@load protocols/smtp/detect-suspicious-orig.bro
@load protocols/smtp/software.bro

4
src/Analyzer.cc
View file

@ -28,6 +28,7 @@
#include "DCE_RPC.h"
#include "Gnutella.h"
#include "Ident.h"
#include "Modbus.h"
#include "NCP.h"
#include "NetbiosSSN.h"
#include "SMB.h"
@ -129,6 +130,9 @@ const Analyzer::Config Analyzer::analyzer_configs[] = {
{ AnalyzerTag::SYSLOG_BINPAC, "SYSLOG_BINPAC",
Syslog_Analyzer_binpac::InstantiateAnalyzer,
Syslog_Analyzer_binpac::Available, 0, false },
{ AnalyzerTag::Modbus, "MODBUS",
ModbusTCP_Analyzer::InstantiateAnalyzer,
ModbusTCP_Analyzer::Available, 0, false },
{ AnalyzerTag::AYIYA, "AYIYA",
AYIYA_Analyzer::InstantiateAnalyzer,

1
src/AnalyzerTags.h
View file

@ -32,6 +32,7 @@ namespace AnalyzerTag {
// Application-layer analyzers, binpac-generated.
DHCP_BINPAC, DNS_TCP_BINPAC, DNS_UDP_BINPAC,
HTTP_BINPAC, SSL, SYSLOG_BINPAC,
Modbus,
// Decapsulation analyzers.
AYIYA,

6
src/BroString.h
View file

@ -34,11 +34,15 @@ public:
typedef IdxVec::iterator IdxVecIt;
typedef IdxVec::const_iterator IdxVecCIt;
BroString(int arg_final_NUL, byte_vec str, int arg_n);
// Constructors creating internal copies of the data passed in.
BroString(const u_char* str, int arg_n, int add_NUL);
BroString(const char* str);
BroString(const string& str);
BroString(const BroString& bs);
// Constructor that takes owernship of the vector passed in.
BroString(int arg_final_NUL, byte_vec str, int arg_n);
BroString();
~BroString() { Reset(); }

3
src/CMakeLists.txt
View file

@ -216,6 +216,8 @@ binpac_target(ssl.pac
ssl-defs.pac ssl-protocol.pac ssl-analyzer.pac)
binpac_target(syslog.pac
syslog-protocol.pac syslog-analyzer.pac)
binpac_target(modbus.pac
modbus-protocol.pac modbus-analyzer.pac)
########################################################################
## bro target
@ -346,6 +348,7 @@ set(bro_SRCS
Reporter.cc
Login.cc
MIME.cc
Modbus.cc
NCP.cc
NFA.cc
NFS.cc

172
src/DNS_Mgr.cc
View file

@ -46,13 +46,15 @@ extern int select(int, fd_set *, fd_set *, fd_set *, struct timeval *);
class DNS_Mgr_Request {
public:
DNS_Mgr_Request(const char* h, int af) { host = copy_string(h); fam = af; }
DNS_Mgr_Request(const char* h, int af, bool is_txt)
{ host = copy_string(h); fam = af; qtype = is_txt ? 16 : 0; }
DNS_Mgr_Request(const IPAddr& a) { addr = a; host = 0; fam = 0; }
~DNS_Mgr_Request() { delete [] host; }
// Returns nil if this was an address request.
const char* ReqHost() const { return host; }
const IPAddr& ReqAddr() const { return addr; }
const bool ReqIsTxt() const { return qtype == 16; }
int MakeRequest(nb_dns_info* nb_dns);
int RequestPending() const { return request_pending; }
@ -61,7 +63,8 @@ public:
protected:
char* host; // if non-nil, this is a host request
int fam; // address family query type for host requests
int fam; // address family query type for host requests
int qtype; // Query type
IPAddr addr;
int request_pending;
};
@ -75,7 +78,7 @@ int DNS_Mgr_Request::MakeRequest(nb_dns_info* nb_dns)
char err[NB_DNS_ERRSIZE];
if ( host )
return nb_dns_host_request2(nb_dns, host, fam, (void*) this, err) >= 0;
return nb_dns_host_request2(nb_dns, host, fam, qtype, (void*) this, err) >= 0;
else
{
const uint32* bytes;
@ -475,8 +478,8 @@ TableVal* DNS_Mgr::LookupHost(const char* name)
// Not found, or priming.
switch ( mode ) {
case DNS_PRIME:
requests.append(new DNS_Mgr_Request(name, AF_INET));
requests.append(new DNS_Mgr_Request(name, AF_INET6));
requests.append(new DNS_Mgr_Request(name, AF_INET, false));
requests.append(new DNS_Mgr_Request(name, AF_INET6, false));
return empty_addr_set();
case DNS_FORCE:
@ -484,8 +487,8 @@ TableVal* DNS_Mgr::LookupHost(const char* name)
return 0;
case DNS_DEFAULT:
requests.append(new DNS_Mgr_Request(name, AF_INET));
requests.append(new DNS_Mgr_Request(name, AF_INET6));
requests.append(new DNS_Mgr_Request(name, AF_INET, false));
requests.append(new DNS_Mgr_Request(name, AF_INET6, false));
Resolve();
return LookupHost(name);
@ -636,6 +639,7 @@ int DNS_Mgr::Save()
Save(f, host_mappings);
Save(f, addr_mappings);
// Save(f, text_mappings); // We don't save the TXT mappings (yet?).
fclose(f);
@ -704,39 +708,48 @@ void DNS_Mgr::AddResult(DNS_Mgr_Request* dr, struct nb_dns_result* r)
new_dm = new DNS_Mapping(dr->ReqHost(), h, ttl);
prev_dm = 0;
HostMap::iterator it = host_mappings.find(dr->ReqHost());
if ( it == host_mappings.end() )
if ( dr->ReqIsTxt() )
{
host_mappings[dr->ReqHost()].first =
new_dm->Type() == AF_INET ? new_dm : 0;
host_mappings[dr->ReqHost()].second =
new_dm->Type() == AF_INET ? 0 : new_dm;
TextMap::iterator it = text_mappings.find(dr->ReqHost());
if ( it == text_mappings.end() )
text_mappings[dr->ReqHost()] = new_dm;
}
else
{
if ( new_dm->Type() == AF_INET )
HostMap::iterator it = host_mappings.find(dr->ReqHost());
if ( it == host_mappings.end() )
{
prev_dm = it->second.first;
it->second.first = new_dm;
host_mappings[dr->ReqHost()].first =
new_dm->Type() == AF_INET ? new_dm : 0;
host_mappings[dr->ReqHost()].second =
new_dm->Type() == AF_INET ? 0 : new_dm;
}
else
{
prev_dm = it->second.second;
it->second.second = new_dm;
if ( new_dm->Type() == AF_INET )
{
prev_dm = it->second.first;
it->second.first = new_dm;
}
else
{
prev_dm = it->second.second;
it->second.second = new_dm;
}
}
}
if ( new_dm->Failed() && prev_dm && prev_dm->Valid() )
{
// Put previous, valid entry back - CompareMappings
// will generate a corresponding warning.
if ( prev_dm->Type() == AF_INET )
host_mappings[dr->ReqHost()].first = prev_dm;
else
host_mappings[dr->ReqHost()].second = prev_dm;
if ( new_dm->Failed() && prev_dm && prev_dm->Valid() )
{
// Put previous, valid entry back - CompareMappings
// will generate a corresponding warning.
if ( prev_dm->Type() == AF_INET )
host_mappings[dr->ReqHost()].first = prev_dm;
else
host_mappings[dr->ReqHost()].second = prev_dm;
++keep_prev;
++keep_prev;
}
}
}
else
@ -928,7 +941,10 @@ TableVal* DNS_Mgr::LookupNameInCache(string name)
{
HostMap::iterator it = dns_mgr->host_mappings.find(name);
if ( it == dns_mgr->host_mappings.end() )
{
it = dns_mgr->host_mappings.begin();
return 0;
}
DNS_Mapping* d4 = it->second.first;
DNS_Mapping* d6 = it->second.second;
@ -951,6 +967,26 @@ TableVal* DNS_Mgr::LookupNameInCache(string name)
return tv6;
}
const char* DNS_Mgr::LookupTextInCache(string name)
{
TextMap::iterator it = dns_mgr->text_mappings.find(name);
if ( it == dns_mgr->text_mappings.end() )
return 0;
DNS_Mapping* d = it->second;
if ( d->Expired() )
{
dns_mgr->text_mappings.erase(it);
delete d;
return 0;
}
// The escapes in the following strings are to avoid having it
// interpreted as a trigraph sequence.
return d->names ? d->names[0] : "<\?\?\?>";
}
void DNS_Mgr::AsyncLookupAddr(const IPAddr& host, LookupCallback* callback)
{
if ( ! did_init )
@ -976,6 +1012,7 @@ void DNS_Mgr::AsyncLookupAddr(const IPAddr& host, LookupCallback* callback)
// A new one.
req = new AsyncRequest;
req->host = host;
req->is_txt = false;
asyncs_queued.push_back(req);
asyncs_addrs.insert(AsyncRequestAddrMap::value_type(host, req));
}
@ -1020,6 +1057,35 @@ void DNS_Mgr::AsyncLookupName(string name, LookupCallback* callback)
IssueAsyncRequests();
}
void DNS_Mgr::AsyncLookupNameText(string name, LookupCallback* callback)
{
if ( ! did_init )
Init();
// Do we already know the answer?
TableVal* addrs;
AsyncRequest* req = 0;
// Have we already a request waiting for this host?
AsyncRequestTextMap::iterator i = asyncs_texts.find(name);
if ( i != asyncs_texts.end() )
req = i->second;
else
{
// A new one.
req = new AsyncRequest;
req->name = name;
req->is_txt = true;
asyncs_queued.push_back(req);
asyncs_texts.insert(AsyncRequestTextMap::value_type(name, req));
}
req->callbacks.push_back(callback);
IssueAsyncRequests();
}
void DNS_Mgr::IssueAsyncRequests()
{
while ( asyncs_queued.size() && asyncs_pending < MAX_PENDING_REQUESTS )
@ -1036,8 +1102,9 @@ void DNS_Mgr::IssueAsyncRequests()
dr = new DNS_Mgr_Request(req->host);
else
{
dr = new DNS_Mgr_Request(req->name.c_str(), AF_INET);
dr6 = new DNS_Mgr_Request(req->name.c_str(), AF_INET6);
dr = new DNS_Mgr_Request(req->name.c_str(), AF_INET, req->is_txt);
if ( ! req->is_txt )
dr6 = new DNS_Mgr_Request(req->name.c_str(), AF_INET6, req->is_txt);
}
if ( ! dr->MakeRequest(nb_dns) )
@ -1109,6 +1176,38 @@ void DNS_Mgr::CheckAsyncAddrRequest(const IPAddr& addr, bool timeout)
}
void DNS_Mgr::CheckAsyncTextRequest(const char* host, bool timeout)
{
// Note that this code is a mirror of that for CheckAsyncAddrRequest.
AsyncRequestTextMap::iterator i = asyncs_texts.find(host);
if ( i != asyncs_texts.end() )
{
const char* name = LookupTextInCache(host);
if ( name )
{
++successful;
i->second->Resolved(name);
}
else if ( timeout )
{
AsyncRequestTextMap::iterator it = asyncs_texts.begin();
++failed;
i->second->Timeout();
}
else
return;
asyncs_texts.erase(i);
--asyncs_pending;
// Don't delete the request. That will be done once it
// eventually times out.
}
}
void DNS_Mgr::CheckAsyncHostRequest(const char* host, bool timeout)
{
// Note that this code is a mirror of that for CheckAsyncAddrRequest.
@ -1157,8 +1256,12 @@ void DNS_Mgr::Flush()
for ( AddrMap::iterator it2 = addr_mappings.begin(); it2 != addr_mappings.end(); ++it2 )
delete it2->second;
for ( TextMap::iterator it3 = text_mappings.begin(); it3 != text_mappings.end(); ++it3 )
delete it3->second;
host_mappings.clear();
addr_mappings.clear();
text_mappings.clear();
}
void DNS_Mgr::Process()
@ -1177,6 +1280,8 @@ void DNS_Mgr::DoProcess(bool flush)
if ( req->IsAddrReq() )
CheckAsyncAddrRequest(req->host, true);
else if ( req->is_txt )
CheckAsyncTextRequest(req->name.c_str(), true);
else
CheckAsyncHostRequest(req->name.c_str(), true);
@ -1184,7 +1289,7 @@ void DNS_Mgr::DoProcess(bool flush)
delete req;
}
if ( asyncs_addrs.size() == 0 && asyncs_names.size() == 0 )
if ( asyncs_addrs.size() == 0 && asyncs_names.size() == 0 && asyncs_texts.size() == 0 )
return;
if ( AnswerAvailable(0) <= 0 )
@ -1217,6 +1322,8 @@ void DNS_Mgr::DoProcess(bool flush)
if ( ! dr->ReqHost() )
CheckAsyncAddrRequest(dr->ReqAddr(), true);
else if ( dr->ReqIsTxt() )
CheckAsyncTextRequest(dr->ReqHost(), do_host_timeout);
else
CheckAsyncHostRequest(dr->ReqHost(), do_host_timeout);
@ -1271,5 +1378,6 @@ void DNS_Mgr::GetStats(Stats* stats)
stats->pending = asyncs_pending;
stats->cached_hosts = host_mappings.size();
stats->cached_addresses = addr_mappings.size();
stats->cached_texts = text_mappings.size();
}

10
src/DNS_Mgr.h
View file

@ -63,6 +63,7 @@ public:
const char* LookupAddrInCache(const IPAddr& addr);
TableVal* LookupNameInCache(string name);
const char* LookupTextInCache(string name);
// Support for async lookups.
class LookupCallback {
@ -77,6 +78,7 @@ public:
void AsyncLookupAddr(const IPAddr& host, LookupCallback* callback);
void AsyncLookupName(string name, LookupCallback* callback);
void AsyncLookupNameText(string name, LookupCallback* callback);
struct Stats {
unsigned long requests; // These count only async requests.
@ -85,6 +87,7 @@ public:
unsigned long pending;
unsigned long cached_hosts;
unsigned long cached_addresses;
unsigned long cached_texts;
};
void GetStats(Stats* stats);
@ -106,6 +109,7 @@ protected:
typedef map<string, pair<DNS_Mapping*, DNS_Mapping*> > HostMap;
typedef map<IPAddr, DNS_Mapping*> AddrMap;
typedef map<string, DNS_Mapping*> TextMap;
void LoadCache(FILE* f);
void Save(FILE* f, const AddrMap& m);
void Save(FILE* f, const HostMap& m);
@ -122,6 +126,7 @@ protected:
// requested.
void CheckAsyncAddrRequest(const IPAddr& addr, bool timeout);
void CheckAsyncHostRequest(const char* host, bool timeout);
void CheckAsyncTextRequest(const char* host, bool timeout);
// Process outstanding requests.
void DoProcess(bool flush);
@ -138,6 +143,7 @@ protected:
HostMap host_mappings;
AddrMap addr_mappings;
TextMap text_mappings;
DNS_mgr_request_list requests;
@ -165,6 +171,7 @@ protected:
double time;
IPAddr host;
string name;
bool is_txt;
CallbackList callbacks;
bool IsAddrReq() const { return name.length() == 0; }
@ -210,6 +217,9 @@ protected:
typedef map<string, AsyncRequest*> AsyncRequestNameMap;
AsyncRequestNameMap asyncs_names;
typedef map<string, AsyncRequest*> AsyncRequestTextMap;
AsyncRequestTextMap asyncs_texts;
typedef list<AsyncRequest*> QueuedList;
QueuedList asyncs_queued;

4
src/IPAddr.cc
View file

@ -248,10 +248,10 @@ IPPrefix::IPPrefix(const in6_addr& in6, uint8_t length)
prefix.Mask(this->length);
}
IPPrefix::IPPrefix(const IPAddr& addr, uint8_t length)
IPPrefix::IPPrefix(const IPAddr& addr, uint8_t length, bool len_is_v6_relative)
: prefix(addr)
{
if ( prefix.GetFamily() == IPv4 )
if ( prefix.GetFamily() == IPv4 && ! len_is_v6_relative )
{
if ( length > 32 )
reporter->InternalError("Bad IPAddr(v4) IPPrefix length : %d",

9
src/IPAddr.h
View file

@ -496,8 +496,15 @@ public:
* @param addr The IP address.
*
* @param length The prefix length in the range from 0 to 128
*
* @param len_is_v6_relative Whether \a length is relative to the full
* 128 bits of an IPv6 address. If false and \a addr is an IPv4
* address, then \a length is expected to range from 0 to 32. If true
* \a length is expected to range from 0 to 128 even if \a addr is IPv4,
* meaning that the mask is to apply to the IPv4-mapped-IPv6 representation.
*/
IPPrefix(const IPAddr& addr, uint8_t length);
IPPrefix(const IPAddr& addr, uint8_t length,
bool len_is_v6_relative = false);
/**
* Copy constructor.

41
src/Modbus.cc Normal file
View file

@ -0,0 +1,41 @@
#include "Modbus.h"
#include "TCP_Reassembler.h"
ModbusTCP_Analyzer::ModbusTCP_Analyzer(Connection* c)
: TCP_ApplicationAnalyzer(AnalyzerTag::Modbus, c)
{
interp = new binpac::ModbusTCP::ModbusTCP_Conn(this);
}
ModbusTCP_Analyzer::~ModbusTCP_Analyzer()
{
delete interp;
}
void ModbusTCP_Analyzer::Done()
{
TCP_ApplicationAnalyzer::Done();
interp->FlowEOF(true);
interp->FlowEOF(false);
}
void ModbusTCP_Analyzer::DeliverStream(int len, const u_char* data, bool orig)
{
TCP_ApplicationAnalyzer::DeliverStream(len, data, orig);
interp->NewData(orig, data, data + len);
}
void ModbusTCP_Analyzer::Undelivered(int seq, int len, bool orig)
{
TCP_ApplicationAnalyzer::Undelivered(seq, len, orig);
interp->NewGap(orig, len);
}
void ModbusTCP_Analyzer::EndpointEOF(bool is_orig)
{
TCP_ApplicationAnalyzer::EndpointEOF(is_orig);
interp->FlowEOF(is_orig);
}

58
src/Modbus.h Normal file
View file

@ -0,0 +1,58 @@
#ifndef MODBUS_H
#define MODBUS_H
#include "TCP.h"
#include "modbus_pac.h"
class ModbusTCP_Analyzer : public TCP_ApplicationAnalyzer {
public:
ModbusTCP_Analyzer(Connection* conn);
virtual ~ModbusTCP_Analyzer();
virtual void Done();
virtual void DeliverStream(int len, const u_char* data, bool orig);
virtual void Undelivered(int seq, int len, bool orig);
virtual void EndpointEOF(bool is_orig);
static Analyzer* InstantiateAnalyzer(Connection* conn)
{ return new ModbusTCP_Analyzer(conn); }
// Put event names in this function
static bool Available()
{
return modbus_message
| modbus_exception
| modbus_read_coils_request
| modbus_read_coils_response
| modbus_read_discrete_inputs_request
| modbus_read_discrete_inputs_response
| modbus_read_holding_registers_request
| modbus_read_holding_registers_response
| modbus_read_input_registers_request
| modbus_read_input_registers_response
| modbus_write_single_coil_request
| modbus_write_single_coil_response
| modbus_write_single_register_request
| modbus_write_single_register_response
| modbus_write_multiple_coils_request
| modbus_write_multiple_coils_response
| modbus_write_multiple_registers_request
| modbus_write_multiple_registers_response
| modbus_read_file_record_request
| modbus_read_file_record_response
| modbus_write_file_record_request
| modbus_write_file_record_response
| modbus_mask_write_register_request
| modbus_mask_write_register_response
| modbus_read_write_multiple_registers_request
| modbus_read_write_multiple_registers_response
| modbus_read_fifo_queue_request
| modbus_read_fifo_queue_response;
}
protected:
binpac::ModbusTCP::ModbusTCP_Conn* interp;
};
#endif

37
src/PktSrc.cc
View file

@ -219,16 +219,35 @@ void PktSrc::Process()
// Get protocol being carried from the ethernet frame.
protocol = (data[12] << 8) + data[13];
// MPLS carried over the ethernet frame.
if ( protocol == 0x8847 )
have_mpls = true;
// VLAN carried over ethernet frame.
else if ( protocol == 0x8100 )
switch ( protocol )
{
data += get_link_header_size(datalink);
data += 4; // Skip the vlan header
pkt_hdr_size = 0;
// MPLS carried over the ethernet frame.
case 0x8847:
have_mpls = true;
break;
// VLAN carried over the ethernet frame.
case 0x8100:
data += get_link_header_size(datalink);
data += 4; // Skip the vlan header
pkt_hdr_size = 0;
break;
// PPPoE carried over the ethernet frame.
case 0x8864:
data += get_link_header_size(datalink);
protocol = (data[6] << 8) + data[7];
data += 8; // Skip the PPPoE session and PPP header
pkt_hdr_size = 0;
if ( protocol != 0x0021 && protocol != 0x0057 )
{
// Neither IPv4 nor IPv6.
sessions->Weird("non_ip_packet_in_pppoe_encapsulation", &hdr, data);
data = 0;
return;
}
break;
}
break;

29
src/bro.bif
View file

@ -3770,6 +3770,35 @@ function lookup_addr%(host: addr%) : string
return 0;
%}
## Issues an asynchronous TEXT DNS lookup and delays the function result.
## This function can therefore only be called inside a ``when`` condition,
## e.g., ``when ( local h = lookup_hostname_txt("www.bro-ids.org") ) { f(h); }``.
##
## host: The hostname to lookup.
##
## Returns: The DNS TXT record associated with *host*.
##
## .. bro:see:: lookup_hostname
function lookup_hostname_txt%(host: string%) : string
%{
// FIXME: Is should be easy to adapt the function to synchronous
// lookups if we're reading a trace.
Trigger* trigger = frame->GetTrigger();
if ( ! trigger)
{
builtin_error("lookup_hostname_txt() can only be called inside a when-condition");
return new StringVal("<error>");
}
frame->SetDelayed();
trigger->Hold();
dns_mgr->AsyncLookupNameText(host->CheckString(),
new LookupHostCallback(trigger, frame->GetCall(), true));
return 0;
%}
## Issues an asynchronous DNS lookup and delays the function result.
## This function can therefore only be called inside a ``when`` condition,
## e.g., ``when ( local h = lookup_hostname("www.bro-ids.org") ) { f(h); }``.

297
src/event.bif
View file

@ -150,7 +150,7 @@ event new_connection%(c: connection%);
## encapsulation value of *e* after this event is raised. If the desired
## behavior is to track the latest tunnel encapsulation per-connection,
## then a handler of this event should assign *e* to ``c$tunnel`` (which Bro's
## default scripts are doing).
## default scripts are doing).
##
## c: The connection whose tunnel/encapsulation changed.
##
@ -6547,6 +6547,301 @@ event netflow_v5_header%(h: nf_v5_header%);
## .. bro:see:: netflow_v5_record
event netflow_v5_record%(r: nf_v5_record%);
## Generated for any modbus message regardless if the particular function
## is further supported or not.
##
## c: The connection.
##
## headers: The headers for the modbus function.
##
## is_orig: True if the event is raised for the originator side.
event modbus_message%(c: connection, headers: ModbusHeaders, is_orig: bool%);
## Generated for any modbus exception message.
##
## c: The connection.
##
## headers: The headers for the modbus function.
##
## code: The exception code.
event modbus_exception%(c: connection, headers: ModbusHeaders, code: count%);
## Generated for a Modbus read coils request.
##
## c: The connection.
##
## headers: The headers for the modbus function.
##
## start_address: The memory address where of the first coil to be read.
##
## quantity: The number of coils to be read.
event modbus_read_coils_request%(c: connection, headers: ModbusHeaders, start_address: count, quantity: count%);
## Generated for a Modbus read coils response.
##
## c: The connection.
##
## headers: The headers for the modbus function.
##
## coils: The coil values returned from the device.
event modbus_read_coils_response%(c: connection, headers: ModbusHeaders, coils: ModbusCoils%);
## Generated for a Modbus read discrete inputs request.
##
## c: The connection.
##
## headers: The headers for the modbus function.
##
## start_address: The memory address of the first coil to be read.
##
## quantity: The number of coils to be read.
event modbus_read_discrete_inputs_request%(c: connection, headers: ModbusHeaders, start_address: count, quantity: count%);
## Generated for a Modbus read discrete inputs response.
##
## c: The connection.
##
## headers: The headers for the modbus function.
##
## coils: The coil values returned from the device.
event modbus_read_discrete_inputs_response%(c: connection, headers: ModbusHeaders, coils: ModbusCoils%);
## Generated for a Modbus read holding registers request.
##
## c: The connection.
##
## headers: The headers for the modbus function.
##
## start_address: The memory address of the first register to be read.
##
## quantity: The number of registers to be read.
event modbus_read_holding_registers_request%(c: connection, headers: ModbusHeaders, start_address: count, quantity: count%);
## Generated for a Modbus read holding registers response.
##
## c: The connection.
##
## headers: The headers for the modbus function.
##
## registers: The register values returned from the device.
event modbus_read_holding_registers_response%(c: connection, headers: ModbusHeaders, registers: ModbusRegisters%);
## Generated for a Modbus read input registers request.
##
## c: The connection.
##
## headers: The headers for the modbus function.
##
## start_address: The memory address of the first register to be read.
##
## quantity: The number of registers to be read.
event modbus_read_input_registers_request%(c: connection, headers: ModbusHeaders, start_address: count, quantity: count%);
## Generated for a Modbus read input registers response.
##
## c: The connection.
##
## headers: The headers for the modbus function.
##
## registers: The register values returned from the device.
event modbus_read_input_registers_response%(c: connection, headers: ModbusHeaders, registers: ModbusRegisters%);
## Generated for a Modbus write single coil request.
##
## c: The connection.
##
## headers: The headers for the modbus function.
##
## address: The memory address of the coil to be written.
##
## value: The value to be written to the coil.
event modbus_write_single_coil_request%(c: connection, headers: ModbusHeaders, address: count, value: bool%);
## Generated for a Modbus write single coil response.
##
## c: The connection.
##
## headers: The headers for the modbus function.
##
## address: The memory address of the coil that was written.
##
## value: The value that was written to the coil.
event modbus_write_single_coil_response%(c: connection, headers: ModbusHeaders, address: count, value: bool%);
## Generated for a Modbus write single register request.
##
## c: The connection.
##
## headers: The headers for the modbus function.
##
## address: The memory address of the register to be written.
##
## value: The value to be written to the register.
event modbus_write_single_register_request%(c: connection, headers: ModbusHeaders, address: count, value: count%);
## Generated for a Modbus write single register response.
##
## c: The connection.
##
## headers: The headers for the modbus function.
##
## address: The memory address of the register that was written.
##
## value: The value that was written to the register.
event modbus_write_single_register_response%(c: connection, headers: ModbusHeaders, address: count, value: count%);
## Generated for a Modbus write multiple coils request.
##
## c: The connection.
##
## headers: The headers for the modbus function.
##
## start_address: The memory address of the first coil to be written.
##
## value: The values to be written to the coils.
event modbus_write_multiple_coils_request%(c: connection, headers: ModbusHeaders, start_address: count, coils: ModbusCoils%);
## Generated for a Modbus write multiple coils response.
##
## c: The connection.
##
## headers: The headers for the modbus function.
##
## start_address: The memory address of the first coil that was written.
##
## quantity: The quantity of coils that were written.
event modbus_write_multiple_coils_response%(c: connection, headers: ModbusHeaders, start_address: count, quantity: count%);
## Generated for a Modbus write multiple registers request.
##
## c: The connection.
##
## headers: The headers for the modbus function.
##
## start_address: The memory address of the first register to be written.
##
## registers: The values to be written to the registers.
event modbus_write_multiple_registers_request%(c: connection, headers: ModbusHeaders, start_address: count, registers: ModbusRegisters%);
## Generated for a Modbus write multiple registers response.
##
## c: The connection.
##
## headers: The headers for the modbus function.
##
## start_address: The memory address of the first register that was written.
##
## quantity: The quantity of registers that were written.
event modbus_write_multiple_registers_response%(c: connection, headers: ModbusHeaders, start_address: count, quantity: count%);
## Generated for a Modbus read file record request.
##
## c: The connection.
##
## headers: The headers for the modbus function.
##
## .. note: This event is incomplete. The information from the data structure is not
## yet passed through to the event.
event modbus_read_file_record_request%(c: connection, headers: ModbusHeaders%);
## Generated for a Modbus read file record response.
##
## c: The connection.
##
## headers: The headers for the modbus function.
##
## .. note: This event is incomplete. The information from the data structure is not
## yet passed through to the event.
event modbus_read_file_record_response%(c: connection, headers: ModbusHeaders%);
## Generated for a Modbus write file record request.
##
## c: The connection.
##
## headers: The headers for the modbus function.
##
## .. note: This event is incomplete. The information from the data structure is not
## yet passed through to the event.
event modbus_write_file_record_request%(c: connection, headers: ModbusHeaders%);
## Generated for a Modbus write file record response.
##
## c: The connection.
##
## headers: The headers for the modbus function.
##
## .. note: This event is incomplete. The information from the data structure is not
## yet passed through to the event.
event modbus_write_file_record_response%(c: connection, headers: ModbusHeaders%);
## Generated for a Modbus mask write register request.
##
## c: The connection.
##
## headers: The headers for the modbus function.
##
## address: The memory address of the register where the masks should be applied.
##
## and_mask: The value of the logical AND mask to apply to the register.
##
## or_mask: The value of the logical OR mask to apply to the register.
event modbus_mask_write_register_request%(c: connection, headers: ModbusHeaders, address: count, and_mask: count, or_mask: count%);
## Generated for a Modbus mask write register request.
##
## c: The connection.
##
## headers: The headers for the modbus function.
##
## address: The memory address of the register where the masks were applied.
##
## and_mask: The value of the logical AND mask applied register.
##
## or_mask: The value of the logical OR mask applied to the register.
event modbus_mask_write_register_response%(c: connection, headers: ModbusHeaders, address: count, and_mask: count, or_mask: count%);
## Generated for a Modbus read/write multiple registers request.
##
## c: The connection.
##
## headers: The headers for the modbus function.
##
## read_start_address: The memory address of the first register to be read.
##
## read_quantity: The number of registers to read.
##
## write_start_address: The memory address of the first register to be written.
##
## write_registers: The values to be written to the registers.
event modbus_read_write_multiple_registers_request%(c: connection, headers: ModbusHeaders, read_start_address: count, read_quantity: count, write_start_address: count, write_registers: ModbusRegisters%);
## Generated for a Modbus read/write multiple registers response.
##
## c: The connection.
##
## headers: The headers for the modbus function.
##
## written_registers: The register values read from the registers specified in the request.
event modbus_read_write_multiple_registers_response%(c: connection, headers: ModbusHeaders, written_registers: ModbusRegisters%);
## Generated for a Modbus read FIFO queue request.
##
## c: The connection.
##
## headers: The headers for the modbus function.
##
## start_address: The address of the FIFO queue to read.
event modbus_read_fifo_queue_request%(c: connection, headers: ModbusHeaders, start_address: count%);
## Generated for a Modbus read FIFO queue response.
##
## c: The connection.
##
## headers: The headers for the modbus function.
##
## fifos: The register values read from the FIFO queue on the device.
event modbus_read_fifo_queue_response%(c: connection, headers: ModbusHeaders, fifos: ModbusRegisters%);
## Raised for informational messages reported via Bro's reporter framework. Such
## messages may be generated internally by the event engine and also by other
## scripts calling :bro:id:`Reporter::info`.

564
src/modbus-analyzer.pac Normal file
View file

@ -0,0 +1,564 @@
#
# The development of Bro's Modbus analyzer has been made possible thanks to
# the support of the Ministry of Security and Justice of the Kingdom of the
# Netherlands within the projects of Hermes, Castor and Midas.
#
# Useful references: http://www.modbus.org/docs/Modbus_Application_Protocol_V1_1b.pdf
# http://www.simplymodbus.ca/faq.htm
#
%header{
VectorVal* bytestring_to_coils(bytestring coils, uint quantity);
RecordVal* HeaderToBro(ModbusTCP_TransportHeader *header);
%}
%code{
VectorVal* bytestring_to_coils(bytestring coils, uint quantity)
{
VectorVal* modbus_coils = new VectorVal(BifType::Vector::ModbusCoils);
return modbus_coils;
}
RecordVal* HeaderToBro(ModbusTCP_TransportHeader *header)
{
RecordVal* modbus_header = new RecordVal(BifType::Record::ModbusHeaders);
modbus_header->Assign(0, new Val(header->tid(), TYPE_COUNT));
modbus_header->Assign(1, new Val(header->pid(), TYPE_COUNT));
modbus_header->Assign(2, new Val(header->uid(), TYPE_COUNT));
modbus_header->Assign(3, new Val(header->fc(), TYPE_COUNT));
return modbus_header;
}
%}
refine flow ModbusTCP_Flow += {
function deliver_message(header: ModbusTCP_TransportHeader): bool
%{
if ( ::modbus_message )
{
BifEvent::generate_modbus_message(connection()->bro_analyzer(),
connection()->bro_analyzer()->Conn(),
HeaderToBro(header),
is_orig());
}
return true;
%}
# EXCEPTION
function deliver_Exception(header: ModbusTCP_TransportHeader, message: Exception): bool
%{
if ( ::modbus_exception )
{
BifEvent::generate_modbus_exception(connection()->bro_analyzer(),
connection()->bro_analyzer()->Conn(),
HeaderToBro(header),
${message.code});
}
return true;
%}
# REQUEST FC=1
function deliver_ReadCoilsRequest(header: ModbusTCP_TransportHeader, message: ReadCoilsRequest): bool
%{
if ( ::modbus_read_coils_request )
{
BifEvent::generate_modbus_read_coils_request(connection()->bro_analyzer(),
connection()->bro_analyzer()->Conn(),
HeaderToBro(header),
${message.start_address},
${message.quantity});
}
return true;
%}
# RESPONSE FC=1
function deliver_ReadCoilsResponse(header: ModbusTCP_TransportHeader, message: ReadCoilsResponse): bool
%{
if ( ::modbus_read_coils_response )
{
BifEvent::generate_modbus_read_coils_response(connection()->bro_analyzer(),
connection()->bro_analyzer()->Conn(),
HeaderToBro(header),
bytestring_to_coils(${message.bits}, ${message.bits}.length()*8));
}
return true;
%}
# REQUEST FC=2
function deliver_ReadDiscreteInputsRequest(header: ModbusTCP_TransportHeader, message: ReadDiscreteInputsRequest): bool
%{
if ( ::modbus_read_discrete_inputs_request )
{
BifEvent::generate_modbus_read_discrete_inputs_request(connection()->bro_analyzer(),
connection()->bro_analyzer()->Conn(),
HeaderToBro(header),
${message.start_address}, ${message.quantity});
}
return true;
%}
# RESPONSE FC=2
function deliver_ReadDiscreteInputsResponse(header: ModbusTCP_TransportHeader, message: ReadDiscreteInputsResponse): bool
%{
if ( ::modbus_read_discrete_inputs_response )
{
BifEvent::generate_modbus_read_discrete_inputs_response(connection()->bro_analyzer(),
connection()->bro_analyzer()->Conn(),
HeaderToBro(header),
bytestring_to_coils(${message.bits}, ${message.bits}.length()*8));
}
return true;
%}
# REQUEST FC=3
function deliver_ReadHoldingRegistersRequest(header: ModbusTCP_TransportHeader, message: ReadHoldingRegistersRequest): bool
%{
if ( ::modbus_read_holding_registers_request )
{
BifEvent::generate_modbus_read_holding_registers_request(connection()->bro_analyzer(),
connection()->bro_analyzer()->Conn(),
HeaderToBro(header),
${message.start_address}, ${message.quantity});
}
return true;
%}
# RESPONSE FC=3
function deliver_ReadHoldingRegistersResponse(header: ModbusTCP_TransportHeader, message: ReadHoldingRegistersResponse): bool
%{
if ( ::modbus_read_holding_registers_response )
{
VectorVal* t = new VectorVal(BifType::Vector::ModbusRegisters);
for ( unsigned int i=0; i < ${message.registers}->size(); ++i )
{
Val* r = new Val(${message.registers[i]}, TYPE_COUNT);
t->Assign(i, r, 0, OP_ASSIGN);
}
BifEvent::generate_modbus_read_holding_registers_response(connection()->bro_analyzer(),
connection()->bro_analyzer()->Conn(),
HeaderToBro(header),
t);
}
return true;
%}
# REQUEST FC=4
function deliver_ReadInputRegistersRequest(header: ModbusTCP_TransportHeader, message: ReadInputRegistersRequest): bool
%{
if ( ::modbus_read_input_registers_request )
{
BifEvent::generate_modbus_read_input_registers_request(connection()->bro_analyzer(),
connection()->bro_analyzer()->Conn(),
HeaderToBro(header),
${message.start_address}, ${message.quantity});
}
return true;
%}
# RESPONSE FC=4
function deliver_ReadInputRegistersResponse(header: ModbusTCP_TransportHeader, message: ReadInputRegistersResponse): bool
%{
if ( ::modbus_read_input_registers_response )
{
VectorVal* t = new VectorVal(BifType::Vector::ModbusRegisters);
for ( unsigned int i=0; i < (${message.registers})->size(); ++i )
{
Val* r = new Val(${message.registers[i]}, TYPE_COUNT);
t->Assign(i, r, 0, OP_ASSIGN);
}
BifEvent::generate_modbus_read_input_registers_response(connection()->bro_analyzer(),
connection()->bro_analyzer()->Conn(),
HeaderToBro(header), t);
}
return true;
%}
# REQUEST FC=5
function deliver_WriteSingleCoilRequest(header: ModbusTCP_TransportHeader, message: WriteSingleCoilRequest): bool
%{
if ( ::modbus_write_single_coil_request )
{
int val;
if ( ${message.value} == 0x0000 )
val = 0;
else if ( ${message.value} == 0xFF00 )
val = 1;
else
{
connection()->bro_analyzer()->ProtocolViolation(fmt("invalid value for modbus write single coil request %d",
${message.value}));
return false;
}
BifEvent::generate_modbus_write_single_coil_request(connection()->bro_analyzer(),
connection()->bro_analyzer()->Conn(),
HeaderToBro(header),
${message.address},
val);
}
return true;
%}
# RESPONSE FC=5
function deliver_WriteSingleCoilResponse(header: ModbusTCP_TransportHeader, message: WriteSingleCoilResponse): bool
%{
if ( ::modbus_write_single_coil_response )
{
int val;
if ( ${message.value} == 0x0000 )
val = 0;
else if ( ${message.value} == 0xFF00 )
val = 1;
else
{
connection()->bro_analyzer()->ProtocolViolation(fmt("invalid value for modbus write single coil response %d",
${message.value}));
return false;
}
BifEvent::generate_modbus_write_single_coil_response(connection()->bro_analyzer(),
connection()->bro_analyzer()->Conn(),
HeaderToBro(header),
${message.address},
val);
}
return true;
%}
# REQUEST FC=6
function deliver_WriteSingleRegisterRequest(header: ModbusTCP_TransportHeader, message: WriteSingleRegisterRequest): bool
%{
if ( ::modbus_write_single_register_request )
{
BifEvent::generate_modbus_write_single_register_request(connection()->bro_analyzer(),
connection()->bro_analyzer()->Conn(),
HeaderToBro(header),
${message.address}, ${message.value});
}
return true;
%}
# RESPONSE FC=6
function deliver_WriteSingleRegisterResponse(header: ModbusTCP_TransportHeader, message: WriteSingleRegisterResponse): bool
%{
if ( ::modbus_write_single_register_response )
{
BifEvent::generate_modbus_write_single_register_response(connection()->bro_analyzer(),
connection()->bro_analyzer()->Conn(),
HeaderToBro(header),
${message.address}, ${message.value});
}
return true;
%}
# REQUEST FC=15
function deliver_WriteMultipleCoilsRequest(header: ModbusTCP_TransportHeader, message: WriteMultipleCoilsRequest): bool
%{
if ( ::modbus_write_multiple_coils_request )
{
BifEvent::generate_modbus_write_multiple_coils_request(connection()->bro_analyzer(),
connection()->bro_analyzer()->Conn(),
HeaderToBro(header),
${message.start_address},
bytestring_to_coils(${message.coils}, ${message.quantity}));
}
return true;
%}
# RESPONSE FC=15
function deliver_WriteMultipleCoilsResponse(header: ModbusTCP_TransportHeader, message: WriteMultipleCoilsResponse): bool
%{
if ( ::modbus_write_multiple_coils_response )
{
BifEvent::generate_modbus_write_multiple_coils_response(connection()->bro_analyzer(),
connection()->bro_analyzer()->Conn(),
HeaderToBro(header),
${message.start_address}, ${message.quantity});
}
return true;
%}
# REQUEST FC=16
function deliver_WriteMultipleRegistersRequest(header: ModbusTCP_TransportHeader, message: WriteMultipleRegistersRequest): bool
%{
if ( ::modbus_write_multiple_registers_request )
{
VectorVal * t = new VectorVal(BifType::Vector::ModbusRegisters);
for ( unsigned int i = 0; i < (${message.registers}->size()); ++i )
{
Val* r = new Val(${message.registers[i]}, TYPE_COUNT);
t->Assign(i, r, 0, OP_ASSIGN);
}
BifEvent::generate_modbus_write_multiple_registers_request(connection()->bro_analyzer(),
connection()->bro_analyzer()->Conn(),
HeaderToBro(header),
${message.start_address}, t);
}
return true;
%}
# RESPONSE FC=16
function deliver_WriteMultipleRegistersResponse(header: ModbusTCP_TransportHeader, message: WriteMultipleRegistersResponse): bool
%{
if ( ::modbus_write_multiple_registers_response )
{
BifEvent::generate_modbus_write_multiple_registers_response(connection()->bro_analyzer(),
connection()->bro_analyzer()->Conn(),
HeaderToBro(header),
${message.start_address}, ${message.quantity});
}
return true;
%}
# REQUEST FC=20
function deliver_ReadFileRecordRequest(header: ModbusTCP_TransportHeader, message: ReadFileRecordRequest): bool
%{
if ( ::modbus_read_file_record_request )
{
//TODO: this need to be a vector of some Reference Request record type
//VectorVal *t = new VectorVal(new VectorType(base_type(TYPE_COUNT)));
//for ( unsigned int i = 0; i < (${message.references}->size()); ++i )
// {
// Val* r = new Val((${message.references[i].ref_type}), TYPE_COUNT);
// t->Assign(i, r, 0, OP_ASSIGN);
//
// Val* k = new Val((${message.references[i].file_num}), TYPE_COUNT);
// t->Assign(i, k, 0, OP_ASSIGN);
//
// Val* l = new Val((${message.references[i].record_num}), TYPE_COUNT);
// t->Assign(i, l, 0, OP_ASSIGN);
// }
BifEvent::generate_modbus_read_file_record_request(connection()->bro_analyzer(),
connection()->bro_analyzer()->Conn(),
HeaderToBro(header));
}
return true;
%}
# RESPONSE FC=20
function deliver_ReadFileRecordResponse(header: ModbusTCP_TransportHeader, message: ReadFileRecordResponse): bool
%{
if ( ::modbus_read_file_record_response )
{
//VectorVal *t = new VectorVal(new VectorType(base_type(TYPE_COUNT)));
//for ( unsigned int i = 0; i < ${message.references}->size(); ++i )
// {
// //TODO: work the reference type in here somewhere
// Val* r = new Val(${message.references[i].record_data}), TYPE_COUNT);
// t->Assign(i, r, 0, OP_ASSIGN);
// }
BifEvent::generate_modbus_read_file_record_response(connection()->bro_analyzer(),
connection()->bro_analyzer()->Conn(),
HeaderToBro(header));
}
return true;
%}
# REQUEST FC=21
function deliver_WriteFileRecordRequest(header: ModbusTCP_TransportHeader, message: WriteFileRecordRequest): bool
%{
if ( ::modbus_write_file_record_request )
{
//VectorVal* t = new VectorVal(new VectorType(base_type(TYPE_COUNT)));
//for ( unsigned int i = 0; i < (${message.references}->size()); ++i )
// {
// Val* r = new Val((${message.references[i].ref_type}), TYPE_COUNT);
// t->Assign(i, r, 0, OP_ASSIGN);
//
// Val* k = new Val((${message.references[i].file_num}), TYPE_COUNT);
// t->Assign(i, k, 0, OP_ASSIGN);
//
// Val* n = new Val((${message.references[i].record_num}), TYPE_COUNT);
// t->Assign(i, n, 0, OP_ASSIGN);
//
// for ( unsigned int j = 0; j < (${message.references[i].register_value}->size()); ++j )
// {
// k = new Val((${message.references[i].register_value[j]}), TYPE_COUNT);
// t->Assign(i, k, 0, OP_ASSIGN);
// }
// }
BifEvent::generate_modbus_write_file_record_request(connection()->bro_analyzer(),
connection()->bro_analyzer()->Conn(),
HeaderToBro(header));
}
return true;
%}
# RESPONSE FC=21
function deliver_WriteFileRecordResponse(header: ModbusTCP_TransportHeader, message: WriteFileRecordResponse): bool
%{
if ( ::modbus_write_file_record_response )
{
//VectorVal* t = new VectorVal(new VectorType(base_type(TYPE_COUNT)));
//for ( unsigned int i = 0; i < (${messages.references}->size()); ++i )
// {
// Val* r = new Val((${message.references[i].ref_type}), TYPE_COUNT);
// t->Assign(i, r, 0, OP_ASSIGN);
//
// Val* f = new Val((${message.references[i].file_num}), TYPE_COUNT);
// t->Assign(i, f, 0, OP_ASSIGN);
//
// Val* rn = new Val((${message.references[i].record_num}), TYPE_COUNT);
// t->Assign(i, rn, 0, OP_ASSIGN);
//
// for ( unsigned int j = 0; j<(${message.references[i].register_value}->size()); ++j )
// {
// Val* k = new Val((${message.references[i].register_value[j]}), TYPE_COUNT);
// t->Assign(i, k, 0, OP_ASSIGN);
// }
BifEvent::generate_modbus_write_file_record_response(connection()->bro_analyzer(),
connection()->bro_analyzer()->Conn(),
HeaderToBro(header));
}
return true;
%}
# REQUEST FC=22
function deliver_MaskWriteRegisterRequest(header: ModbusTCP_TransportHeader, message: MaskWriteRegisterRequest): bool
%{
if ( ::modbus_mask_write_register_request )
{
BifEvent::generate_modbus_mask_write_register_request(connection()->bro_analyzer(),
connection()->bro_analyzer()->Conn(),
HeaderToBro(header),
${message.address},
${message.and_mask}, ${message.or_mask});
}
return true;
%}
# RESPONSE FC=22
function deliver_MaskWriteRegisterResponse(header: ModbusTCP_TransportHeader, message: MaskWriteRegisterResponse): bool
%{
if ( ::modbus_mask_write_register_response )
{
BifEvent::generate_modbus_mask_write_register_response(connection()->bro_analyzer(),
connection()->bro_analyzer()->Conn(),
HeaderToBro(header),
${message.address},
${message.and_mask}, ${message.or_mask});
}
return true;
%}
# REQUEST FC=23
function deliver_ReadWriteMultipleRegistersRequest(header: ModbusTCP_TransportHeader, message: ReadWriteMultipleRegistersRequest): bool
%{
if ( ::modbus_read_write_multiple_registers_request )
{
VectorVal* t = new VectorVal(BifType::Vector::ModbusRegisters);
for ( unsigned int i = 0; i < ${message.write_register_values}->size(); ++i )
{
Val* r = new Val(${message.write_register_values[i]}, TYPE_COUNT);
t->Assign(i, r, 0, OP_ASSIGN);
}
BifEvent::generate_modbus_read_write_multiple_registers_request(connection()->bro_analyzer(),
connection()->bro_analyzer()->Conn(),
HeaderToBro(header),
${message.read_start_address},
${message.read_quantity},
${message.write_start_address},
t);
}
return true;
%}
# RESPONSE FC=23
function deliver_ReadWriteMultipleRegistersResponse(header: ModbusTCP_TransportHeader, message: ReadWriteMultipleRegistersResponse): bool
%{
if ( ::modbus_read_write_multiple_registers_response )
{
VectorVal* t = new VectorVal(BifType::Vector::ModbusRegisters);
for ( unsigned int i = 0; i < ${message.registers}->size(); ++i )
{
Val* r = new Val(${message.registers[i]}, TYPE_COUNT);
t->Assign(i, r, 0, OP_ASSIGN);
}
BifEvent::generate_modbus_read_write_multiple_registers_response(connection()->bro_analyzer(),
connection()->bro_analyzer()->Conn(),
HeaderToBro(header), t);
}
return true;
%}
# REQUEST FC=24
function deliver_ReadFIFOQueueRequest(header: ModbusTCP_TransportHeader, message: ReadFIFOQueueRequest): bool
%{
if ( ::modbus_read_fifo_queue_request )
{
BifEvent::generate_modbus_read_fifo_queue_request(connection()->bro_analyzer(),
connection()->bro_analyzer()->Conn(),
HeaderToBro(header),
${message.start_address});
}
return true;
%}
# RESPONSE FC=24
function deliver_ReadFIFOQueueResponse(header: ModbusTCP_TransportHeader, message: ReadFIFOQueueResponse): bool
%{
if ( ::modbus_read_fifo_queue_response )
{
VectorVal* t = new VectorVal(new VectorType(base_type(TYPE_COUNT)));
for ( unsigned int i = 0; i < (${message.register_data})->size(); ++i )
{
Val* r = new Val(${message.register_data[i]}, TYPE_COUNT);
t->Assign(i, r, 0, OP_ASSIGN);
}
BifEvent::generate_modbus_read_fifo_queue_response(connection()->bro_analyzer(),
connection()->bro_analyzer()->Conn(),
HeaderToBro(header), t);
}
return true;
%}
};

394
src/modbus-protocol.pac Normal file
View file

@ -0,0 +1,394 @@
#
# The development of Bro's Modbus analyzer has been made possible thanks to
# the support of the Ministry of Security and Justice of the Kingdom of the
# Netherlands within the projects of Hermes, Castor and Midas.
#
# Useful references: http://www.modbus.org/docs/Modbus_Application_Protocol_V1_1b.pdf
# http://www.simplymodbus.ca/faq.htm
enum function_codes {
# Standard functions
READ_COILS = 0x01,
READ_DISCRETE_INPUTS = 0x02,
READ_HOLDING_REGISTERS = 0x03,
READ_INPUT_REGISTERS = 0x04,
WRITE_SINGLE_COIL = 0x05,
WRITE_SINGLE_REGISTER = 0x06,
# READ_EXCEPTION_STATUS = 0x07,
# DIAGNOSTICS = 0x08,
# GET_COMM_EVENT_COUNTER = 0x0B,
# GET_COMM_EVENT_LOG = 0x0C,
WRITE_MULTIPLE_COILS = 0x0F,
WRITE_MULTIPLE_REGISTERS = 0x10,
# REPORT_SLAVE_ID = 0x11,
READ_FILE_RECORD = 0x14,
WRITE_FILE_RECORD = 0x15,
MASK_WRITE_REGISTER = 0x16,
READ_WRITE_MULTIPLE_REGISTERS = 0x17,
READ_FIFO_QUEUE = 0x18,
ENCAP_INTERFACE_TRANSPORT = 0x2B,
# Machine/vendor/network specific functions
PROGRAM_484 = 0x09,
POLL_484 = 0x0A,
PROGRAM_584_984 = 0x0D,
POLL_584_984 = 0x0E,
PROGRAM_884_U84 = 0x12,
RESET_COMM_LINK_884_U84 = 0x13,
PROGRAM_CONCEPT = 0x28,
FIRMWARE_REPLACEMENT = 0x7D,
PROGRAM_584_984_2 = 0x7E,
REPORT_LOCAL_ADDRESS = 0x7F,
# Exceptions (not really function codes but they are used similarly)
READ_COILS_EXCEPTION = 0x81,
READ_DISCRETE_INPUTS_EXCEPTION = 0x82,
READ_HOLDING_REGISTERS_EXCEPTION = 0x83,
READ_INPUT_REGISTERS_EXCEPTION = 0x84,
WRITE_SINGLE_COIL_EXCEPTION = 0x85,
WRITE_SINGLE_REGISTER_EXCEPTION = 0x86,
READ_EXCEPTION_STATUS_EXCEPTION = 0x87,
WRITE_MULTIPLE_COILS_EXCEPTION = 0x8F,
WRITE_MULTIPLE_REGISTERS_EXCEPTION = 0x90,
READ_FILE_RECORD_EXCEPTION = 0x94,
WRITE_FILE_RECORD_EXCEPTION = 0x95,
MASK_WRITE_REGISTER_EXCEPTION = 0x96,
READ_WRITE_MULTIPLE_REGISTERS_EXCEPTION = 0x97,
READ_FIFO_QUEUE_EXCEPTION = 0x98,
};
# Main Modbus/TCP PDU
type ModbusTCP_PDU(is_orig: bool) = record {
header: ModbusTCP_TransportHeader;
body: case is_orig of {
true -> request: ModbusTCP_Request(header);
false -> response: ModbusTCP_Response(header);
};
} &length=header.len+6, &byteorder=bigendian;
type ModbusTCP_TransportHeader = record {
tid: uint16; # Transaction identifier
pid: uint16; # Protocol identifier
len: uint16; # Length of everyting after this field
uid: uint8; # Unit identifier (previously 'slave address')
fc: uint8; # MODBUS function code (see function_codes enum)
} &byteorder=bigendian, &let {
deliver: bool = $context.flow.deliver_message(this);
};
type ModbusTCP_Request(header: ModbusTCP_TransportHeader) = case header.fc of {
READ_COILS -> readCoils: ReadCoilsRequest(header);
READ_DISCRETE_INPUTS -> readDiscreteInputs: ReadDiscreteInputsRequest(header);
READ_HOLDING_REGISTERS -> readHoldingRegisters: ReadHoldingRegistersRequest(header);
READ_INPUT_REGISTERS -> readInputRegisters: ReadInputRegistersRequest(header);
WRITE_SINGLE_COIL -> writeSingleCoil: WriteSingleCoilRequest(header);
WRITE_SINGLE_REGISTER -> writeSingleRegister: WriteSingleRegisterRequest(header);
#READ_EXCEPTION_STATUS -> readExceptionStatus: ReadExceptionStatusRequest(header);
#DIAGNOSTICS -> diagnostics: DiagnosticsRequest(header);
#GET_COMM_EVENT_COUNTER -> getCommEventCounter: GetCommEventCounterRequest(header);
#GET_COMM_EVENT_LOG -> getCommEventLog: GetCommEventLogRequest(header);
WRITE_MULTIPLE_COILS -> writeMultipleCoils: WriteMultipleCoilsRequest(header);
WRITE_MULTIPLE_REGISTERS -> writeMultRegisters: WriteMultipleRegistersRequest(header);
#REPORT_SLAVE_ID
READ_FILE_RECORD -> readFileRecord: ReadFileRecordRequest(header);
WRITE_FILE_RECORD -> writeFileRecord: WriteFileRecordRequest(header);
MASK_WRITE_REGISTER -> maskWriteRegister: MaskWriteRegisterRequest(header);
READ_WRITE_MULTIPLE_REGISTERS -> readWriteMultipleRegisters: ReadWriteMultipleRegistersRequest(header);
READ_FIFO_QUEUE -> readFIFOQueue: ReadFIFOQueueRequest(header);
#ENCAP_INTERFACE_TRANSPORT
# All the rest
default -> unknown: bytestring &restofdata;
};
# Responses
#
type ModbusTCP_Response(header: ModbusTCP_TransportHeader) = case header.fc of {
READ_COILS -> readCoils: ReadCoilsResponse(header);
READ_DISCRETE_INPUTS -> readDiscreteInputs: ReadDiscreteInputsResponse(header);
READ_HOLDING_REGISTERS -> readHoldingRegisters: ReadHoldingRegistersResponse(header);
READ_INPUT_REGISTERS -> readInputRegisters: ReadInputRegistersResponse(header);
WRITE_SINGLE_COIL -> writeSingleCoil: WriteSingleCoilResponse(header);
WRITE_SINGLE_REGISTER -> writeSingleRegister: WriteSingleRegisterResponse(header);
#READ_EXCEPTION_STATUS -> readExceptionStatus: ReadExceptionStatusResponse(header);
#DIAGNOSTICS -> diagnostics: DiagnosticsResponse(header);
#GET_COMM_EVENT_COUNTER -> getCommEventCounter: GetCommEventCounterResponse(header);
#GET_COMM_EVENT_LOG -> getCommEventLog: GetCommEventLogResponse(header);
WRITE_MULTIPLE_COILS -> writeMultipleCoils: WriteMultipleCoilsResponse(header);
WRITE_MULTIPLE_REGISTERS -> writeMultRegisters: WriteMultipleRegistersResponse(header);
#REPORT_SLAVE_ID
READ_FILE_RECORD -> readFileRecord: ReadFileRecordResponse(header);
WRITE_FILE_RECORD -> writeFileRecord: WriteFileRecordResponse(header);
MASK_WRITE_REGISTER -> maskWriteRegister: MaskWriteRegisterResponse(header);
READ_WRITE_MULTIPLE_REGISTERS -> readWriteMultipleRegisters: ReadWriteMultipleRegistersResponse(header);
READ_FIFO_QUEUE -> readFIFOQueue: ReadFIFOQueueResponse(header);
# Exceptions
READ_HOLDING_REGISTERS_EXCEPTION -> readHoldingRegistersException: Exception(header);
WRITE_MULTIPLE_REGISTERS_EXCEPTION -> writeMultRegistersException: Exception(header);
READ_COILS_EXCEPTION -> readCoilsException: Exception(header);
READ_DISCRETE_INPUTS_EXCEPTION -> readDiscreteInputsException: Exception(header);
READ_INPUT_REGISTERS_EXCEPTION -> readInputRegistersException: Exception(header);
WRITE_SINGLE_COIL_EXCEPTION -> writeCoilException: Exception(header);
WRITE_SINGLE_REGISTER_EXCEPTION -> writeSingleRegisterException: Exception(header);
READ_EXCEPTION_STATUS_EXCEPTION -> readExceptionStatusException: Exception(header);
WRITE_MULTIPLE_COILS_EXCEPTION -> forceMultipleCoilsException: Exception(header);
READ_FILE_RECORD_EXCEPTION -> readGeneralReferenceException: Exception(header);
WRITE_FILE_RECORD_EXCEPTION -> writeGeneralReferenceException: Exception(header);
MASK_WRITE_REGISTER_EXCEPTION -> maskWriteRegisterException: Exception(header);
READ_WRITE_MULTIPLE_REGISTERS_EXCEPTION -> readWriteRegistersException: Exception(header);
READ_FIFO_QUEUE_EXCEPTION -> readFIFOQueueException: Exception(header);
# All the rest
default -> unknown: bytestring &restofdata;
};
type Exception(header: ModbusTCP_TransportHeader) = record {
code: uint8;
} &let {
deliver: bool = $context.flow.deliver_Exception(header, this);
};
# REQUEST FC=1
type ReadCoilsRequest(header: ModbusTCP_TransportHeader) = record {
start_address: uint16;
quantity: uint16 &check(quantity <= 2000);
} &let {
deliver: bool = $context.flow.deliver_ReadCoilsRequest(header, this);
} &byteorder=bigendian;
# RESPONSE FC=1
type ReadCoilsResponse(header: ModbusTCP_TransportHeader) = record {
byte_count: uint8;
bits: bytestring &length=byte_count;
} &let {
deliver: bool = $context.flow.deliver_ReadCoilsResponse(header, this);
} &byteorder=bigendian;
# REQUEST FC=2
type ReadDiscreteInputsRequest(header: ModbusTCP_TransportHeader) = record {
start_address: uint16;
quantity: uint16 &check(quantity <= 2000);
} &let {
deliver: bool = $context.flow.deliver_ReadDiscreteInputsRequest(header, this);
} &byteorder=bigendian;
# RESPONSE FC=2
type ReadDiscreteInputsResponse(header: ModbusTCP_TransportHeader) = record {
byte_count: uint8;
bits: bytestring &length=byte_count;
} &let {
deliver: bool = $context.flow.deliver_ReadDiscreteInputsResponse(header, this);
} &byteorder=bigendian;
# REQUEST FC=3
type ReadHoldingRegistersRequest(header: ModbusTCP_TransportHeader) = record {
start_address: uint16;
quantity: uint16 &check(1 <= quantity && quantity <= 125);
} &let {
deliver: bool = $context.flow.deliver_ReadHoldingRegistersRequest(header, this);
} &byteorder=bigendian;
# RESPONSE FC=3
type ReadHoldingRegistersResponse(header: ModbusTCP_TransportHeader) = record {
byte_count: uint8;
registers: uint16[] &length=byte_count;
} &let {
deliver: bool = $context.flow.deliver_ReadHoldingRegistersResponse(header, this);
} &byteorder=bigendian;
# REQUEST FC=4
type ReadInputRegistersRequest(header: ModbusTCP_TransportHeader) = record {
start_address: uint16;
quantity: uint16 &check(1 <= quantity && quantity <= 125);
} &let {
deliver: bool = $context.flow.deliver_ReadInputRegistersRequest(header, this);
} &byteorder=bigendian;
# RESPONSE FC=4
type ReadInputRegistersResponse(header: ModbusTCP_TransportHeader) = record {
byte_count: uint8;
registers: uint16[] &length=byte_count;
} &let {
deliver: bool = $context.flow.deliver_ReadInputRegistersResponse(header, this);
} &byteorder=bigendian;
# REQUEST FC=5
type WriteSingleCoilRequest(header: ModbusTCP_TransportHeader) = record {
address: uint16;
value: uint16 &check(value == 0x0000 || value == 0xFF00);
} &let {
deliver: bool = $context.flow.deliver_WriteSingleCoilRequest(header, this);
} &byteorder=bigendian;
# RESPONSE FC=5
type WriteSingleCoilResponse(header: ModbusTCP_TransportHeader) = record {
address: uint16;
value: uint16 &check(value == 0x0000 || value == 0xFF00);
} &let {
deliver: bool = $context.flow.deliver_WriteSingleCoilResponse(header, this);
} &byteorder=bigendian;
# REQUEST FC=6
type WriteSingleRegisterRequest(header: ModbusTCP_TransportHeader) = record {
address: uint16;
value: uint16;
} &let {
deliver: bool = $context.flow.deliver_WriteSingleRegisterRequest(header, this);
} &byteorder=bigendian;
# RESPONSE FC=6
type WriteSingleRegisterResponse(header: ModbusTCP_TransportHeader) = record {
address: uint16;
value: uint16;
} &let {
deliver: bool = $context.flow.deliver_WriteSingleRegisterResponse(header, this);
} &byteorder=bigendian;
# REQUEST FC=15
type WriteMultipleCoilsRequest(header: ModbusTCP_TransportHeader) = record {
start_address: uint16;
quantity: uint16 &check(quantity <= 0x07B0);
byte_count: uint8 &check(byte_count == (quantity + 7)/8);
coils: bytestring &length=byte_count;
} &let {
deliver: bool = $context.flow.deliver_WriteMultipleCoilsRequest(header, this);
} &byteorder=bigendian;
# RESPONSE FC=15
type WriteMultipleCoilsResponse(header: ModbusTCP_TransportHeader) = record {
start_address: uint16;
quantity: uint16 &check(quantity <= 0x07B0);
} &let {
deliver: bool = $context.flow.deliver_WriteMultipleCoilsResponse(header, this);
} &byteorder=bigendian;
# REQUEST FC=16
type WriteMultipleRegistersRequest(header: ModbusTCP_TransportHeader) = record {
start_address: uint16;
quantity: uint16;
byte_count: uint8;
# We specify registers buffer with quantity and byte_count so that the analyzer
# will choke if something doesn't match right (correct devices should make it right).
registers: uint16[quantity] &length=byte_count;
} &let {
deliver: bool = $context.flow.deliver_WriteMultipleRegistersRequest(header, this);
} &byteorder=bigendian;
# RESPONSE FC=16
type WriteMultipleRegistersResponse(header: ModbusTCP_TransportHeader) = record {
start_address: uint16;
quantity: uint16;
} &let {
deliver: bool = $context.flow.deliver_WriteMultipleRegistersResponse(header, this);
} &byteorder=bigendian;
# Support data structure for following message type.
type FileRecordRequest = record {
ref_type: uint8 &check(ref_type == 6);
file_num: uint16 &check(file_num > 0);
record_num: uint16 &check(record_num <= 0x270F);
record_len: uint16;
} &byteorder=bigendian;
# REQUEST FC=20
type ReadFileRecordRequest(header: ModbusTCP_TransportHeader) = record {
byte_count: uint8 &check(byte_count >= 0x07 && byte_count <= 0xF5);
references: FileRecordRequest[] &length=byte_count;
} &let {
deliver: bool = $context.flow.deliver_ReadFileRecordRequest(header, this);
} &byteorder=bigendian;
# Support data structure for the following message type.
type FileRecordResponse = record {
file_len: uint8 &check(file_len >= 0x07 && file_len <= 0xF5);
ref_type: uint8 &check(ref_type == 6);
record_data: uint16[] &length=file_len;
} &byteorder=bigendian;
# RESPONSE FC=20
type ReadFileRecordResponse(header: ModbusTCP_TransportHeader) = record {
byte_count: uint8 &check(byte_count >= 0x07 && byte_count <= 0xF5);
references: FileRecordResponse[] &length=byte_count;
} &let {
deliver: bool = $context.flow.deliver_ReadFileRecordResponse(header, this);
} &byteorder=bigendian;
# Support data structure for the two following message types.
type ReferenceWithData = record {
ref_type: uint8;
file_num: uint16;
record_num: uint16;
word_count: uint16;
register_value: uint16[word_count];
} &byteorder=bigendian;
# REQUEST FC=21
type WriteFileRecordRequest(header: ModbusTCP_TransportHeader) = record {
byte_count: uint8;
references: ReferenceWithData[] &length=byte_count;
} &let {
deliver: bool = $context.flow.deliver_WriteFileRecordRequest(header, this);
} &byteorder=bigendian;
# RESPONSE FC=21
type WriteFileRecordResponse(header: ModbusTCP_TransportHeader) = record {
byte_count: uint8;
references: ReferenceWithData[] &length=byte_count;
} &let {
deliver: bool = $context.flow.deliver_WriteFileRecordResponse(header, this);
} &byteorder=bigendian;
# REQUEST FC=22
type MaskWriteRegisterRequest(header: ModbusTCP_TransportHeader) = record {
address: uint16;
and_mask: uint16;
or_mask: uint16;
} &let {
deliver: bool = $context.flow.deliver_MaskWriteRegisterRequest(header, this);
} &byteorder=bigendian;
# RESPONSE FC=22
type MaskWriteRegisterResponse(header: ModbusTCP_TransportHeader) = record {
address: uint16;
and_mask: uint16;
or_mask: uint16;
} &let {
deliver: bool = $context.flow.deliver_MaskWriteRegisterResponse(header, this);
} &byteorder=bigendian;
# REQUEST FC=23
type ReadWriteMultipleRegistersRequest(header: ModbusTCP_TransportHeader) = record {
read_start_address: uint16;
read_quantity: uint16 &check(read_quantity <= 125);
write_start_address: uint16;
write_quantity: uint16 &check(write_quantity <= 100);
write_byte_count: uint8;
write_register_values: uint16[write_quantity] &length=write_byte_count;
} &let {
deliver: bool = $context.flow.deliver_ReadWriteMultipleRegistersRequest(header, this);
} &byteorder=bigendian;
# RESPONSE FC=23
type ReadWriteMultipleRegistersResponse(header: ModbusTCP_TransportHeader) = record {
byte_count: uint8;
registers: uint16[] &length=byte_count;
} &let {
deliver: bool = $context.flow.deliver_ReadWriteMultipleRegistersResponse(header, this);
} &byteorder=bigendian;
# REQUEST FC=24
type ReadFIFOQueueRequest(header: ModbusTCP_TransportHeader) = record {
start_address: uint16;
} &let{
deliver: bool = $context.flow.deliver_ReadFIFOQueueRequest(header, this);
} &byteorder=bigendian;
# RESPONSE FC=24
type ReadFIFOQueueResponse(header: ModbusTCP_TransportHeader) = record {
byte_count: uint16 &check(byte_count <= 62);
fifo_count: uint16 &check(fifo_count <= 31);
register_data: uint16[fifo_count] &length=byte_count/2;
} &let {
deliver: bool = $context.flow.deliver_ReadFIFOQueueResponse(header, this);
} &byteorder=bigendian;

28
src/modbus.pac Normal file
View file

@ -0,0 +1,28 @@
#
# The development of Bro's Modbus analyzer has been made possible thanks to
# the support of the Ministry of Security and Justice of the Kingdom of the
# Netherlands within the projects of Hermes, Castor and Midas.
#
# Useful references: http://www.modbus.org/docs/Modbus_Application_Protocol_V1_1b.pdf
# http://www.simplymodbus.ca/faq.htm
%include binpac.pac
%include bro.pac
analyzer ModbusTCP withcontext {
connection: ModbusTCP_Conn;
flow: ModbusTCP_Flow;
};
connection ModbusTCP_Conn(bro_analyzer: BroAnalyzer) {
upflow = ModbusTCP_Flow(true);
downflow = ModbusTCP_Flow(false);
};
%include modbus-protocol.pac
flow ModbusTCP_Flow(is_orig: bool) {
flowunit = ModbusTCP_PDU(is_orig) withcontext (connection, this);
}
%include modbus-analyzer.pac

44
src/nb_dns.c
View file

@ -307,31 +307,32 @@ nb_dns_host_request(register struct nb_dns_info *nd, register const char *name,
register void *cookie, register char *errstr)
{
return (nb_dns_host_request2(nd, name, AF_INET, cookie, errstr));
return (nb_dns_host_request2(nd, name, AF_INET, 0, cookie, errstr));
}
int
nb_dns_host_request2(register struct nb_dns_info *nd, register const char *name,
register int af, register void *cookie, register char *errstr)
register int af, register int qtype, register void *cookie, register char *errstr)
{
register int qtype;
if (qtype != 16) {
switch (af) {
switch (af) {
case AF_INET:
qtype = T_A;
break;
case AF_INET:
qtype = T_A;
break;
#ifdef AF_INET6
case AF_INET6:
qtype = T_AAAA;
break;
case AF_INET6:
qtype = T_AAAA;
break;
#endif
default:
snprintf(errstr, NB_DNS_ERRSIZE,
"nb_dns_host_request2(): uknown address family %d", af);
return (-1);
default:
snprintf(errstr, NB_DNS_ERRSIZE,
"nb_dns_host_request2(): unknown address family %d", af);
return (-1);
}
}
return (_nb_dns_mkquery(nd, name, af, qtype, cookie, errstr));
}
@ -579,6 +580,7 @@ nb_dns_activity(struct nb_dns_info *nd, struct nb_dns_result *nr, char *errstr)
nr->host_errno = NO_RECOVERY;
return (-1);
}
memcpy(bp, rdata, rdlen);
*hap++ = bp;
bp += rdlen;
@ -587,6 +589,20 @@ nb_dns_activity(struct nb_dns_info *nd, struct nb_dns_result *nr, char *errstr)
/* Keep looking for more A records */
break;
case T_TXT:
if (bp + rdlen >= ep) {
snprintf(errstr, NB_DNS_ERRSIZE,
"nb_dns_activity(): overflow 1 for txt");
nr->host_errno = NO_RECOVERY;
return (-1);
}
memcpy(bp, rdata, rdlen);
he->h_name = bp+1; /* First char is a control character. */
nr->hostent = he;
nr->ttl = rttl;
return (1);
case T_PTR:
n = dn_expand((const u_char *)msg,
(const u_char *)msg + msglen, rdata, bp, ep - bp);

2
src/nb_dns.h
View file

@ -22,7 +22,7 @@ void nb_dns_finish(struct nb_dns_info *);
int nb_dns_fd(struct nb_dns_info *);
int nb_dns_host_request(struct nb_dns_info *, const char *, void *, char *);
int nb_dns_host_request2(struct nb_dns_info *, const char *, int,
int nb_dns_host_request2(struct nb_dns_info *, const char *, int, int,
void *, char *);
int nb_dns_addr_request(struct nb_dns_info *, nb_uint32_t, void *, char *);

19
src/rule-scan.l
View file

@ -11,16 +11,6 @@
#include "rule-parse.h"
int rules_line_number = 0;
static string extract_ipv6(string s)
{
if ( s.substr(0, 3) == "[0x" )
s = s.substr(3, s.find("]") - 3);
else
s = s.substr(1, s.find("]") - 1);
return s;
}
%}
%x PS
@ -49,15 +39,14 @@ PID ([0-9a-zA-Z_-]|"::")+
}
{IP6} {
rules_lval.prefixval = new IPPrefix(IPAddr(extract_ipv6(yytext)), 128);
rules_lval.prefixval = new IPPrefix(IPAddr(extract_ip(yytext)), 128, true);
return TOK_IP6;
}
{IP6}{OWS}"/"{OWS}{D} {
char* l = strchr(yytext, '/');
*l++ = '\0';
int len = atoi(l);
rules_lval.prefixval = new IPPrefix(IPAddr(extract_ipv6(yytext)), len);
int len = 0;
string ip = extract_ip_and_len(yytext, &len);
rules_lval.prefixval = new IPPrefix(IPAddr(ip), len, true);
return TOK_IP6;
}

31
src/scan.l
View file

@ -148,6 +148,7 @@ D [0-9]+
HEX [0-9a-fA-F]+
IDCOMPONENT [A-Za-z_][A-Za-z_0-9]*
ID {IDCOMPONENT}(::{IDCOMPONENT})*
IP6 ("["({HEX}:){7}{HEX}"]")|("["0x{HEX}({HEX}|:)*"::"({HEX}|:)*"]")|("["({HEX}|:)*"::"({HEX}|:)*"]")|("["({HEX}|:)*"::"({HEX}|:)*({D}"."){3}{D}"]")
FILE [^ \t\n]+
PREFIX [^ \t\n]+
FLOAT (({D}*"."?{D})|({D}"."?{D}*))([eE][-+]?{D})?
@ -229,21 +230,23 @@ ESCSEQ (\\([^\n]|[0-7]+|x[[:xdigit:]]+))
}
/* IPv6 literal constant patterns */
"["({HEX}:){7}{HEX}"]" {
string s(yytext+1);
RET_CONST(new AddrVal(s.erase(s.size()-1)))
{IP6} {
RET_CONST(new AddrVal(extract_ip(yytext)))
}
"["0x{HEX}({HEX}|:)*"::"({HEX}|:)*"]" {
string s(yytext+3);
RET_CONST(new AddrVal(s.erase(s.size()-1)))
{IP6}{OWS}"/"{OWS}{D} {
int len = 0;
string ip = extract_ip_and_len(yytext, &len);
RET_CONST(new SubNetVal(IPPrefix(IPAddr(ip), len, true)))
}
"["({HEX}|:)*"::"({HEX}|:)*"]" {
string s(yytext+1);
RET_CONST(new AddrVal(s.erase(s.size()-1)))
}
"["({HEX}|:)*"::"({HEX}|:)*({D}"."){3}{D}"]" {
string s(yytext+1);
RET_CONST(new AddrVal(s.erase(s.size()-1)))
/* IPv4 literal constant patterns */
({D}"."){3}{D} RET_CONST(new AddrVal(yytext))
({D}"."){3}{D}{OWS}"/"{OWS}{D} {
int len = 0;
string ip = extract_ip_and_len(yytext, &len);
RET_CONST(new SubNetVal(IPPrefix(IPAddr(ip), len)))
}
[!%*/+\-,:;<=>?()\[\]{}~$|] return yytext[0];
@ -484,8 +487,6 @@ F RET_CONST(new Val(false, TYPE_BOOL))
{FLOAT}{OWS}msec(s?) RET_CONST(new IntervalVal(atof(yytext),Milliseconds))
{FLOAT}{OWS}usec(s?) RET_CONST(new IntervalVal(atof(yytext),Microseconds))
({D}"."){3}{D} RET_CONST(new AddrVal(yytext))
"0x"{HEX}+ RET_CONST(new Val(static_cast<bro_uint_t>(strtoull(yytext, 0, 16)), TYPE_COUNT))
{H}("."{H})+ RET_CONST(dns_mgr->LookupHost(yytext))

7
src/types.bif
View file

@ -156,6 +156,13 @@ type readdir_reply_t: record;
type fsstat_t: record;
module GLOBAL;
type ModbusHeaders: record;
type ModbusCoils: vector;
type ModbusRegisters: vector;
module Log;
enum Writer %{

34
src/util.cc
View file

@ -43,6 +43,40 @@
#include "Net.h"
#include "Reporter.h"
/**
* Return IP address without enclosing brackets and any leading 0x.
*/
std::string extract_ip(const std::string& i)
{
std::string s(skip_whitespace(i.c_str()));
if ( s.size() > 0 && s[0] == '[' )
s.erase(0, 1);
if ( s.size() > 1 && s.substr(0, 2) == "0x" )
s.erase(0, 2);
size_t pos = 0;
if ( (pos = s.find(']')) != std::string::npos )
s = s.substr(0, pos);
return s;
}
/**
* Given a subnet string, return IP address and subnet length separately.
*/
std::string extract_ip_and_len(const std::string& i, int* len)
{
size_t pos = i.find('/');
if ( pos == std::string::npos )
return i;
if ( len )
*len = atoi(i.substr(pos + 1).c_str());
return extract_ip(i.substr(0, pos));
}
/**
* Takes a string, unescapes all characters that are escaped as hex codes
* (\x##) and turns them into the equivalent ascii-codes. Returns a string

3
src/util.h
View file

@ -91,6 +91,9 @@ void delete_each(T* t)
delete *it;
}
std::string extract_ip(const std::string& i);
std::string extract_ip_and_len(const std::string& i, int* len);
std::string get_unescaped_string(const std::string& str);
std::string get_escaped_string(const std::string& str, bool escape_all);

16
testing/btest/Baseline/core.pppoe/conn.log Normal file
View file

@ -0,0 +1,16 @@
#separator \x09
#set_separator ,
#empty_field (empty)
#unset_field -
#path conn
#open 2012-10-24-05-04-16
#fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p proto service duration orig_bytes resp_bytes conn_state local_orig missed_bytes history orig_pkts orig_ip_bytes resp_pkts resp_ip_bytes tunnel_parents
#types time string addr port addr port enum string interval count count string bool count string count count count count table[string]
1284385418.014560 TEfuqmmG4bh fe80::c801:eff:fe88:8 547 fe80::ce05:eff:fe88:0 546 udp - 0.096000 192 0 S0 - 0 D 2 288 0 0 (empty)
1284385417.962560 j4u32Pc5bif fe80::ce05:eff:fe88:0 546 ff02::1:2 547 udp - 0.078000 114 0 S0 - 0 D 2 210 0 0 (empty)
1284385411.091560 arKYeMETxOg fe80::c801:eff:fe88:8 136 ff02::1 135 icmp - - - - OTH - 0 - 1 64 0 0 (empty)
1284385411.035560 UWkUyAuUGXf fe80::c801:eff:fe88:8 143 ff02::16 0 icmp - 0.835000 160 0 OTH - 0 - 8 608 0 0 (empty)
1284385451.658560 FrJExwHcSal fc00:0:2:100::1:1 128 fc00::1 129 icmp - 0.156000 260 260 OTH - 0 - 5 500 5 500 (empty)
1284385413.027560 nQcgTWjvg4c fe80::c801:eff:fe88:8 134 fe80::ce05:eff:fe88:0 133 icmp - - - - OTH - 0 - 1 64 0 0 (empty)
1284385412.963560 k6kgXLOoSKl fe80::ce05:eff:fe88:0 133 ff02::2 134 icmp - - - - OTH - 0 - 1 48 0 0 (empty)
#close 2012-10-24-05-04-16

24
testing/btest/Baseline/core.print-bpf-filters/output
View file

@ -3,38 +3,38 @@
#empty_field (empty)
#unset_field -
#path packet_filter
#open 2012-10-08-16-16-08
#open 2012-11-06-00-53-09
#fields ts node filter init success
#types time string string bool bool
1349712968.812610 - ip or not ip T T
#close 2012-10-08-16-16-08
1352163189.729807 - ip or not ip T T
#close 2012-11-06-00-53-09
#separator \x09
#set_separator ,
#empty_field (empty)
#unset_field -
#path packet_filter
#open 2012-10-08-16-16-09
#open 2012-11-06-00-53-10
#fields ts node filter init success
#types time string string bool bool
1349712969.042094 - (((((((((((((((((((((((((port 53) or (tcp port 989)) or (tcp port 443)) or (port 6669)) or (udp and port 5353)) or (port 6668)) or (tcp port 1080)) or (udp and port 5355)) or (tcp port 995)) or (tcp port 22)) or (port 21 and port 2811)) or (tcp port 25 or tcp port 587)) or (tcp port 614)) or (tcp port 990)) or (port 6667)) or (udp port 137)) or (tcp port 993)) or (tcp port 5223)) or (port 514)) or (tcp port 585)) or (tcp port 992)) or (tcp port 563)) or (tcp port 994)) or (tcp port 636)) or (tcp and port (80 or 81 or 631 or 1080 or 3138 or 8000 or 8080 or 8888))) or (port 6666) T T
#close 2012-10-08-16-16-09
1352163190.114261 - ((((((((((((((((((((((((((port 53) or (tcp port 989)) or (tcp port 443)) or (port 6669)) or (udp and port 5353)) or (port 6668)) or (tcp port 1080)) or (udp and port 5355)) or (tcp port 502)) or (tcp port 995)) or (tcp port 22)) or (port 21 and port 2811)) or (tcp port 25 or tcp port 587)) or (tcp port 614)) or (tcp port 990)) or (port 6667)) or (udp port 137)) or (tcp port 993)) or (tcp port 5223)) or (port 514)) or (tcp port 585)) or (tcp port 992)) or (tcp port 563)) or (tcp port 994)) or (tcp port 636)) or (tcp and port (80 or 81 or 631 or 1080 or 3138 or 8000 or 8080 or 8888))) or (port 6666) T T
#close 2012-11-06-00-53-10
#separator \x09
#set_separator ,
#empty_field (empty)
#unset_field -
#path packet_filter
#open 2012-10-08-16-16-09
#open 2012-11-06-00-53-10
#fields ts node filter init success
#types time string string bool bool
1349712969.270826 - port 42 T T
#close 2012-10-08-16-16-09
1352163190.484506 - port 42 T T
#close 2012-11-06-00-53-10
#separator \x09
#set_separator ,
#empty_field (empty)
#unset_field -
#path packet_filter
#open 2012-10-08-16-16-09
#open 2012-11-06-00-53-10
#fields ts node filter init success
#types time string string bool bool
1349712969.499878 - port 56730 T T
#close 2012-10-08-16-16-09
1352163190.855090 - port 56730 T T
#close 2012-11-06-00-53-10

10
testing/btest/Baseline/coverage.default-load-baseline/canonified_loaded_scripts.log
View file

@ -3,7 +3,7 @@
#empty_field (empty)
#unset_field -
#path loaded_scripts
#open 2012-07-20-14-34-40
#open 2012-11-05-23-29-45
#fields name
#types string
scripts/base/init-bare.bro
@ -40,6 +40,7 @@ scripts/base/init-default.bro
scripts/base/utils/paths.bro
scripts/base/utils/strings.bro
scripts/base/utils/thresholds.bro
scripts/base/utils/urls.bro
scripts/base/frameworks/notice/__load__.bro
scripts/base/frameworks/notice/./main.bro
scripts/base/frameworks/notice/./weird.bro
@ -69,6 +70,7 @@ scripts/base/init-default.bro
scripts/base/frameworks/metrics/./non-cluster.bro
scripts/base/frameworks/intel/__load__.bro
scripts/base/frameworks/intel/./main.bro
scripts/base/frameworks/intel/./input.bro
scripts/base/frameworks/reporter/__load__.bro
scripts/base/frameworks/reporter/./main.bro
scripts/base/frameworks/tunnels/__load__.bro
@ -99,6 +101,9 @@ scripts/base/init-default.bro
scripts/base/protocols/irc/__load__.bro
scripts/base/protocols/irc/./main.bro
scripts/base/protocols/irc/./dcc-send.bro
scripts/base/protocols/modbus/__load__.bro
scripts/base/protocols/modbus/./consts.bro
scripts/base/protocols/modbus/./main.bro
scripts/base/protocols/smtp/__load__.bro
scripts/base/protocols/smtp/./main.bro
scripts/base/protocols/smtp/./entities.bro
@ -111,5 +116,6 @@ scripts/base/init-default.bro
scripts/base/protocols/syslog/__load__.bro
scripts/base/protocols/syslog/./consts.bro
scripts/base/protocols/syslog/./main.bro
scripts/base/misc/find-checksum-offloading.bro
scripts/policy/misc/loaded-scripts.bro
#close 2012-07-20-14-34-40
#close 2012-11-05-23-29-45

1
testing/btest/Baseline/coverage.init-default/missing_loads
View file

@ -2,5 +2,6 @@
-./frameworks/cluster/nodes/proxy.bro
-./frameworks/cluster/nodes/worker.bro
-./frameworks/cluster/setup-connections.bro
-./frameworks/intel/cluster.bro
-./frameworks/metrics/cluster.bro
-./frameworks/notice/cluster.bro

2
testing/btest/Baseline/language.addr/out
View file

@ -13,3 +13,5 @@ IPv6 address not case-sensitive (PASS)
size of IPv6 address (PASS)
IPv6 address type inference (PASS)
IPv4 and IPv6 address inequality (PASS)
IPv4-mapped-IPv6 equality to IPv4 (PASS)
IPv4-mapped-IPv6 is IPv4 (PASS)

8
testing/btest/Baseline/language.subnet/out
View file

@ -10,3 +10,11 @@ IPv6 subnet !in operator (PASS)
IPv6 subnet type inference (PASS)
IPv4 and IPv6 subnet inequality (PASS)
IPv4 address and IPv6 subnet (PASS)
IPv4 in IPv4-mapped-IPv6 subnet (PASS)
IPv6 !in IPv4-mapped-IPv6 subnet (PASS)
IPv4-mapped-IPv6 in IPv4-mapped-IPv6 subnet (PASS)
IPv4-mapped-IPv6 subnet equality (PASS)
subnet literal const whitespace (PASS)
subnet literal const whitespace (PASS)
subnet literal const whitespace (PASS)
subnet literal const whitespace (PASS)

2
testing/btest/Baseline/scripts.base.frameworks.intel.cluster-transparency/manager-1..stdout Normal file
View file

@ -0,0 +1,2 @@
cluster_new_item: 123.123.123.123 inserted by worker-1 (from peer: worker-1)
cluster_new_item: 4.3.2.1 inserted by worker-2 (from peer: worker-2)

10
testing/btest/Baseline/scripts.base.frameworks.intel.cluster-transparency/manager-1.intel.log Normal file
View file

@ -0,0 +1,10 @@
#separator \x09
#set_separator ,
#empty_field (empty)
#unset_field -
#path intel
#open 2012-10-03-20-20-39
#fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p seen.host seen.str seen.str_type seen.where sources
#types time string addr port addr port addr string enum enum table[string]
1349295639.424940 - - - - - 123.123.123.123 - - Intel::IN_ANYWHERE worker-1
#close 2012-10-03-20-20-49

3
testing/btest/Baseline/scripts.base.frameworks.intel.cluster-transparency/worker-1..stdout Normal file
View file

@ -0,0 +1,3 @@
cluster_new_item: 1.2.3.4 inserted by manager (from peer: manager-1)
cluster_new_item: 123.123.123.123 inserted by worker-1 (from peer: manager-1)
cluster_new_item: 4.3.2.1 inserted by worker-2 (from peer: manager-1)

4
testing/btest/Baseline/scripts.base.frameworks.intel.cluster-transparency/worker-2..stdout Normal file
View file

@ -0,0 +1,4 @@
cluster_new_item: 1.2.3.4 inserted by manager (from peer: manager-1)
cluster_new_item: 123.123.123.123 inserted by worker-1 (from peer: manager-1)
cluster_new_item: 4.3.2.1 inserted by worker-2 (from peer: manager-1)
Doing a lookup

11
testing/btest/Baseline/scripts.base.frameworks.intel.input-and-match/broproc.intel.log Normal file
View file

@ -0,0 +1,11 @@
#separator \x09
#set_separator ,
#empty_field (empty)
#unset_field -
#path intel
#open 2012-10-03-20-18-05
#fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p seen.host seen.str seen.str_type seen.where sources
#types time string addr port addr port addr string enum enum table[string]
1349295485.114156 - - - - - - e@mail.com Intel::EMAIL SOMEWHERE source1
1349295485.114156 - - - - - 1.2.3.4 - - SOMEWHERE source1
#close 2012-10-03-20-18-05

3
testing/btest/Baseline/scripts.base.frameworks.intel.insert-and-matcher/out
View file

@ -1,3 +0,0 @@
VALID
VALID
VALID

0
testing/btest/Baseline/scripts.base.frameworks.intel.read-file-dist-cluster/manager-1..stdout Normal file
View file

13
testing/btest/Baseline/scripts.base.frameworks.intel.read-file-dist-cluster/manager-1.intel.log Normal file
View file

@ -0,0 +1,13 @@
#separator \x09
#set_separator ,
#empty_field (empty)
#unset_field -
#path intel
#open 2012-10-10-15-05-23
#fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p seen.host seen.str seen.str_type seen.where sources
#types time string addr port addr port addr string enum enum table[string]
1349881523.548946 - - - - - 1.2.3.4 - - Intel::IN_A_TEST source1
1349881523.548946 - - - - - - e@mail.com Intel::EMAIL Intel::IN_A_TEST source1
1349881524.567896 - - - - - 1.2.3.4 - - Intel::IN_A_TEST source1
1349881524.567896 - - - - - - e@mail.com Intel::EMAIL Intel::IN_A_TEST source1
#close 2012-10-10-15-05-24

0
testing/btest/Baseline/scripts.base.frameworks.intel.read-file-dist-cluster/worker-1..stdout Normal file
View file

0
testing/btest/Baseline/scripts.base.frameworks.intel.read-file-dist-cluster/worker-2..stdout Normal file
View file

1
testing/btest/Baseline/scripts.base.protocols.modbus.events/coverage Normal file
View file

@ -0,0 +1 @@
2 of 28 events triggered by trace

159115
testing/btest/Baseline/scripts.base.protocols.modbus.events/output Normal file
View file

File diff suppressed because it is too large Load diff

17
testing/btest/Baseline/scripts.base.protocols.modbus.policy/known_modbus.log Normal file
View file

@ -0,0 +1,17 @@
#separator \x09
#set_separator ,
#empty_field (empty)
#unset_field -
#path known_modbus
#open 2012-11-06-00-51-15
#fields ts host device_type
#types time addr enum
1093521694.211940 10.0.0.57 Known::MODBUS_MASTER
1093521694.211940 10.0.0.3 Known::MODBUS_SLAVE
1093521958.375300 10.0.0.8 Known::MODBUS_SLAVE
1093522338.985618 10.0.0.9 Known::MODBUS_MASTER
1153491892.212845 192.168.66.235 Known::MODBUS_MASTER
1153491892.212845 166.161.16.230 Known::MODBUS_SLAVE
1342774499.589057 10.1.1.234 Known::MODBUS_MASTER
1342774499.589057 10.10.5.85 Known::MODBUS_SLAVE
#close 2012-11-06-00-51-23

19990
testing/btest/Baseline/scripts.base.protocols.modbus.policy/modbus.log Normal file
View file

File diff suppressed because it is too large Load diff

49
testing/btest/Baseline/scripts.base.protocols.modbus.policy/modbus_register_change.log Normal file
View file

@ -0,0 +1,49 @@
#separator \x09
#set_separator ,
#empty_field (empty)
#unset_field -
#path modbus_register_change
#open 2012-11-06-00-51-15
#fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p register old_val new_val delta
#types time string addr port addr port count count count interval
1342774501.024564 3PKsZ2Uye21 10.1.1.234 51411 10.10.5.85 502 101 0 1 1.250066
1342774540.946501 3PKsZ2Uye21 10.1.1.234 51411 10.10.5.85 502 101 1 0 39.921937
1342774540.946501 3PKsZ2Uye21 10.1.1.234 51411 10.10.5.85 502 103 0 1 41.172003
1342774811.727563 3PKsZ2Uye21 10.1.1.234 51411 10.10.5.85 502 102 0 1 311.953065
1342774811.727563 3PKsZ2Uye21 10.1.1.234 51411 10.10.5.85 502 103 1 0 270.781062
1342774831.727542 3PKsZ2Uye21 10.1.1.234 51411 10.10.5.85 502 101 0 1 290.781041
1342774831.727542 3PKsZ2Uye21 10.1.1.234 51411 10.10.5.85 502 102 1 0 19.999979
1342774872.821282 3PKsZ2Uye21 10.1.1.234 51411 10.10.5.85 502 101 1 0 41.093740
1342774872.821282 3PKsZ2Uye21 10.1.1.234 51411 10.10.5.85 502 103 0 1 61.093719
1342775143.602482 3PKsZ2Uye21 10.1.1.234 51411 10.10.5.85 502 102 0 1 311.874940
1342775143.602482 3PKsZ2Uye21 10.1.1.234 51411 10.10.5.85 502 103 1 0 270.781200
1342775164.774350 3PKsZ2Uye21 10.1.1.234 51411 10.10.5.85 502 101 0 1 291.953068
1342775164.774350 3PKsZ2Uye21 10.1.1.234 51411 10.10.5.85 502 102 1 0 21.171868
1342775204.696194 3PKsZ2Uye21 10.1.1.234 51411 10.10.5.85 502 101 1 0 39.921844
1342775204.696194 3PKsZ2Uye21 10.1.1.234 51411 10.10.5.85 502 103 0 1 61.093712
1342775475.477365 3PKsZ2Uye21 10.1.1.234 51411 10.10.5.85 502 102 0 1 310.703015
1342775475.477365 3PKsZ2Uye21 10.1.1.234 51411 10.10.5.85 502 103 1 0 270.781171
1342775495.477389 3PKsZ2Uye21 10.1.1.234 51411 10.10.5.85 502 101 0 1 290.781195
1342775495.477389 3PKsZ2Uye21 10.1.1.234 51411 10.10.5.85 502 102 1 0 20.000024
1342775535.399236 3PKsZ2Uye21 10.1.1.234 51411 10.10.5.85 502 101 1 0 39.921847
1342775535.399236 3PKsZ2Uye21 10.1.1.234 51411 10.10.5.85 502 103 0 1 59.921871
1342775806.180404 3PKsZ2Uye21 10.1.1.234 51411 10.10.5.85 502 102 0 1 310.703015
1342775806.180404 3PKsZ2Uye21 10.1.1.234 51411 10.10.5.85 502 103 1 0 270.781168
1342775826.180415 3PKsZ2Uye21 10.1.1.234 51411 10.10.5.85 502 101 0 1 290.781179
1342775826.180415 3PKsZ2Uye21 10.1.1.234 51411 10.10.5.85 502 102 1 0 20.000011
1342775848.508596 3PKsZ2Uye21 10.1.1.234 51411 10.10.5.85 502 501 80 90 1348.671590
1342775871.961652 3PKsZ2Uye21 10.1.1.234 51411 10.10.5.85 502 101 1 0 45.781237
1342775871.961652 3PKsZ2Uye21 10.1.1.234 51411 10.10.5.85 502 103 0 1 65.781248
1342776142.758456 3PKsZ2Uye21 10.1.1.234 51411 10.10.5.85 502 102 0 1 316.578041
1342776142.758456 3PKsZ2Uye21 10.1.1.234 51411 10.10.5.85 502 103 1 0 270.796804
1342776167.445943 3PKsZ2Uye21 10.1.1.234 51411 10.10.5.85 502 101 0 1 295.484291
1342776167.445943 3PKsZ2Uye21 10.1.1.234 51411 10.10.5.85 502 102 1 0 24.687487
1342776213.274085 3PKsZ2Uye21 10.1.1.234 51411 10.10.5.85 502 101 1 0 45.828142
1342776213.274085 3PKsZ2Uye21 10.1.1.234 51411 10.10.5.85 502 103 0 1 70.515629
1342776484.055366 3PKsZ2Uye21 10.1.1.234 51411 10.10.5.85 502 102 0 1 316.609423
1342776484.055366 3PKsZ2Uye21 10.1.1.234 51411 10.10.5.85 502 103 1 0 270.781281
1342776507.570851 3PKsZ2Uye21 10.1.1.234 51411 10.10.5.85 502 101 0 1 294.296766
1342776507.570851 3PKsZ2Uye21 10.1.1.234 51411 10.10.5.85 502 102 1 0 23.515485
1342776553.352098 3PKsZ2Uye21 10.1.1.234 51411 10.10.5.85 502 101 1 0 45.781247
1342776553.352098 3PKsZ2Uye21 10.1.1.234 51411 10.10.5.85 502 103 0 1 69.296732
#close 2012-11-06-00-51-23

BIN
testing/btest/Traces/modbus.trace Normal file
View file

Binary file not shown.

BIN
testing/btest/Traces/pppoe.trace Normal file
View file

Binary file not shown.

2
testing/btest/core/pppoe.test Normal file
View file

@ -0,0 +1,2 @@
# @TEST-EXEC: bro -r $TRACES/pppoe.trace %INPUT
# @TEST-EXEC: btest-diff conn.log

7
testing/btest/language/addr.bro
View file

@ -1,4 +1,4 @@
# @TEST-EXEC: bro %INPUT >out
# @TEST-EXEC: bro -b %INPUT >out
# @TEST-EXEC: btest-diff out
function test_case(msg: string, expect: bool)
@ -43,5 +43,10 @@ event bro_init()
test_case( "IPv4 and IPv6 address inequality", a1 != b1 );
# IPv4-mapped-IPv6 (internally treated as IPv4)
local c1: addr = [::ffff:1.2.3.4];
test_case( "IPv4-mapped-IPv6 equality to IPv4", c1 == 1.2.3.4 );
test_case( "IPv4-mapped-IPv6 is IPv4", is_v4_addr(c1) == T );
}

19
testing/btest/language/subnet.bro
View file

@ -1,4 +1,4 @@
# @TEST-EXEC: bro %INPUT >out
# @TEST-EXEC: bro -b %INPUT >out
# @TEST-EXEC: btest-diff out
function test_case(msg: string, expect: bool)
@ -43,5 +43,22 @@ event bro_init()
test_case( "IPv4 and IPv6 subnet inequality", s1 != t1 );
test_case( "IPv4 address and IPv6 subnet", a1 !in t2 );
# IPv4-mapped-IPv6 subnets
local u1: subnet = [::ffff:0:0]/96;
test_case( "IPv4 in IPv4-mapped-IPv6 subnet", 1.2.3.4 in u1 );
test_case( "IPv6 !in IPv4-mapped-IPv6 subnet", [fe80::1] !in u1 );
test_case( "IPv4-mapped-IPv6 in IPv4-mapped-IPv6 subnet",
[::ffff:1.2.3.4] in u1 );
test_case( "IPv4-mapped-IPv6 subnet equality",
[::ffff:1.2.3.4]/112 == 1.2.0.0/16 );
test_case( "subnet literal const whitespace",
[::ffff:1.2.3.4] / 112 == 1.2.0.0 / 16 );
test_case( "subnet literal const whitespace",
[::ffff:1.2.3.4]/ 128 == 1.2.3.4/ 32 );
test_case( "subnet literal const whitespace",
[::ffff:1.2.3.4] /96 == 1.2.3.4 /0 );
test_case( "subnet literal const whitespace",
[::ffff:1.2.3.4] / 92 == [::fffe:1.2.3.4] / 92 );
}

80
testing/btest/scripts/base/frameworks/intel/cluster-transparency.bro Normal file
View file

@ -0,0 +1,80 @@
# @TEST-SERIALIZE: comm
#
# @TEST-EXEC: btest-bg-run manager-1 BROPATH=$BROPATH:.. CLUSTER_NODE=manager-1 bro %INPUT
# @TEST-EXEC: btest-bg-run worker-1 BROPATH=$BROPATH:.. CLUSTER_NODE=worker-1 bro %INPUT
# @TEST-EXEC: btest-bg-run worker-2 BROPATH=$BROPATH:.. CLUSTER_NODE=worker-2 bro %INPUT
# @TEST-EXEC: btest-bg-wait -k 10
# @TEST-EXEC: btest-diff manager-1/.stdout
# @TEST-EXEC: btest-diff manager-1/intel.log
# @TEST-EXEC: btest-diff worker-1/.stdout
# @TEST-EXEC: btest-diff worker-2/.stdout
@TEST-START-FILE cluster-layout.bro
redef Cluster::nodes = {
["manager-1"] = [$node_type=Cluster::MANAGER, $ip=127.0.0.1, $p=37757/tcp, $workers=set("worker-1", "worker-2")],
["worker-1"] = [$node_type=Cluster::WORKER, $ip=127.0.0.1, $p=37760/tcp, $manager="manager-1"],
["worker-2"] = [$node_type=Cluster::WORKER, $ip=127.0.0.1, $p=37761/tcp, $manager="manager-1"],
};
@TEST-END-FILE
@load base/frameworks/control
module Intel;
redef Log::default_rotation_interval=0sec;
event remote_connection_handshake_done(p: event_peer)
{
# Insert the data once both workers are connected.
if ( Cluster::local_node_type() == Cluster::MANAGER && Cluster::worker_count == 2 )
{
Intel::insert([$host=1.2.3.4,$meta=[$source="manager"]]);
}
}
global worker2_data = 0;
global sent_data = F;
event Intel::cluster_new_item(item: Intel::Item)
{
if ( ! is_remote_event() )
return;
print fmt("cluster_new_item: %s inserted by %s (from peer: %s)", item$host, item$meta$source, get_event_peer()$descr);
if ( ! sent_data )
{
# We wait to insert data here because we can now be sure the
# full cluster is constructed.
sent_data = T;
if ( Cluster::node == "worker-1" )
Intel::insert([$host=123.123.123.123,$meta=[$source="worker-1"]]);
if ( Cluster::node == "worker-2" )
Intel::insert([$host=4.3.2.1,$meta=[$source="worker-2"]]);
}
# We're forcing worker-2 to do a lookup when it has three intelligence items
# which were distributed over the cluster (data inserted locally is resent).
if ( Cluster::node == "worker-2" )
{
++worker2_data;
if ( worker2_data == 3 )
{
# Now that everything is inserted, see if we can match on the data inserted
# by worker-1.
print "Doing a lookup";
Intel::seen([$host=123.123.123.123, $where=Intel::IN_ANYWHERE]);
}
}
}
event Intel::log_intel(rec: Intel::Info)
{
event Control::shutdown_request();
}
event remote_connection_closed(p: event_peer)
{
# Cascading termination
#print fmt("disconnected from: %s", p);
terminate_communication();
}

40
testing/btest/scripts/base/frameworks/intel/input-and-match.bro Normal file
View file

@ -0,0 +1,40 @@
# @TEST-SERIALIZE: comm
# @TEST-EXEC: btest-bg-run broproc bro %INPUT
# @TEST-EXEC: btest-bg-wait -k 5
# @TEST-EXEC: btest-diff broproc/intel.log
@TEST-START-FILE intel.dat
#fields host net str str_type meta.source meta.desc meta.url
1.2.3.4 - - - source1 this host is just plain baaad http://some-data-distributor.com/1234
1.2.3.4 - - - source1 this host is just plain baaad http://some-data-distributor.com/1234
- - e@mail.com Intel::EMAIL source1 Phishing email source http://some-data-distributor.com/100000
@TEST-END-FILE
@load frameworks/communication/listen
redef Intel::read_files += { "../intel.dat" };
redef enum Intel::Where += { SOMEWHERE };
event do_it()
{
Intel::seen([$str="e@mail.com",
$str_type=Intel::EMAIL,
$where=SOMEWHERE]);
Intel::seen([$host=1.2.3.4,
$where=SOMEWHERE]);
}
global log_lines = 0;
event Intel::log_intel(rec: Intel::Info)
{
++log_lines;
if ( log_lines == 2 )
terminate();
}
event bro_init() &priority=-10
{
schedule 1sec { do_it() };
}

34
testing/btest/scripts/base/frameworks/intel/insert-and-matcher.bro
View file

@ -1,34 +0,0 @@
#
# @TEST-EXEC: bro %INPUT >out
# @TEST-EXEC: btest-diff out
event bro_init()
{
Intel::insert([$ip=1.2.3.4, $tags=set("zeustracker.abuse.ch", "malicious")]);
Intel::insert([$str="http://www.google.com/", $subtype="url", $tags=set("infrastructure", "google")]);
Intel::insert([$str="Ab439G32F...", $subtype="x509_cert", $tags=set("bad")]);
Intel::insert([$str="Ab439G32F...", $tags=set("bad")]);
}
event bro_done()
{
local orig_h = 1.2.3.4;
if ( Intel::matcher([$ip=orig_h, $and_tags=set("malicious")]) )
print "VALID";
if ( Intel::matcher([$ip=orig_h, $and_tags=set("don't match")]) )
print "INVALID";
if ( Intel::matcher([$ip=orig_h, $pred=function(meta: Intel::MetaData): bool { return T; } ]) )
print "VALID";
if ( Intel::matcher([$ip=orig_h, $pred=function(meta: Intel::MetaData): bool { return F; } ]) )
print "INVALID";
if ( Intel::matcher([$str="http://www.google.com/", $subtype="url", $tags=set("google")]) )
print "VALID";
if ( Intel::matcher([$str="http://www.example.com", $subtype="url"]) )
print "INVALID";
}

66
testing/btest/scripts/base/frameworks/intel/read-file-dist-cluster.bro Normal file
View file

@ -0,0 +1,66 @@
# @TEST-SERIALIZE: comm
#
# @TEST-EXEC: btest-bg-run manager-1 BROPATH=$BROPATH:.. CLUSTER_NODE=manager-1 bro %INPUT
# @TEST-EXEC: sleep 2
# @TEST-EXEC: btest-bg-run worker-1 BROPATH=$BROPATH:.. CLUSTER_NODE=worker-1 bro %INPUT
# @TEST-EXEC: btest-bg-run worker-2 BROPATH=$BROPATH:.. CLUSTER_NODE=worker-2 bro %INPUT
# @TEST-EXEC: btest-bg-wait -k 10
# @TEST-EXEC: btest-diff manager-1/.stdout
# @TEST-EXEC: btest-diff manager-1/intel.log
# @TEST-EXEC: btest-diff worker-1/.stdout
# @TEST-EXEC: btest-diff worker-2/.stdout
@TEST-START-FILE cluster-layout.bro
redef Cluster::nodes = {
["manager-1"] = [$node_type=Cluster::MANAGER, $ip=127.0.0.1, $p=37757/tcp, $workers=set("worker-1", "worker-2")],
["worker-1"] = [$node_type=Cluster::WORKER, $ip=127.0.0.1, $p=37760/tcp, $manager="manager-1"],
["worker-2"] = [$node_type=Cluster::WORKER, $ip=127.0.0.1, $p=37761/tcp, $manager="manager-1"],
};
@TEST-END-FILE
@TEST-START-FILE intel.dat
#fields host net str str_type meta.source meta.desc meta.url
1.2.3.4 - - - source1 this host is just plain baaad http://some-data-distributor.com/1234
1.2.3.4 - - - source1 this host is just plain baaad http://some-data-distributor.com/1234
- - e@mail.com Intel::EMAIL source1 Phishing email source http://some-data-distributor.com/100000
@TEST-END-FILE
@load base/frameworks/control
redef Log::default_rotation_interval=0sec;
module Intel;
@if ( Cluster::local_node_type() == Cluster::MANAGER )
redef Intel::read_files += { "../intel.dat" };
@endif
redef enum Intel::Where += {
Intel::IN_A_TEST,
};
event do_it()
{
Intel::seen([$host=1.2.3.4, $where=Intel::IN_A_TEST]);
Intel::seen([$str="e@mail.com", $str_type=Intel::EMAIL, $where=Intel::IN_A_TEST]);
}
event bro_init()
{
# Delay the workers searching for hits briefly to allow for the data distribution
# mechanism to distribute the data to the workers.
if ( Cluster::local_node_type() == Cluster::WORKER )
schedule 2sec { do_it() };
}
global intel_hits=0;
event Intel::log_intel(rec: Intel::Info)
{
++intel_hits;
# There should be 4 hits since each worker is "seeing" 2 things.
if ( intel_hits == 4 )
{
# We're delaying shutdown for a second here to make sure that no other
# matches happen (which would be wrong!).
schedule 1sec { Control::shutdown_request() };
}
}

148
testing/btest/scripts/base/protocols/modbus/events.bro Normal file
View file

@ -0,0 +1,148 @@
#
# @TEST-EXEC: bro -r $TRACES/modbus.trace %INPUT | sort | uniq -c | sed 's/^ *//g' >output
# @TEST-EXEC: btest-diff output
# @TEST-EXEC: cat output | awk '{print $1}' | sort | uniq | wc -l >covered
# @TEST-EXEC: cat ${DIST}/src/event.bif | grep "^event modbus_" | wc -l >total
# @TEST-EXEC: echo `cat covered` of `cat total` events triggered by trace >coverage
# @TEST-EXEC: btest-diff coverage
event modbus_message(c: connection, headers: ModbusHeaders, is_orig: bool)
{
print "modbus_message", c, headers, is_orig;
}
event modbus_exception(c: connection, headers: ModbusHeaders, code: count)
{
print "modbus_exception", c, headers, code;
}
event modbus_read_coils_request(c: connection, headers: ModbusHeaders, start_address: count, quantity: count)
{
print "modbus_read_coils_request", c, headers, start_address, quantity;
}
event modbus_read_coils_response(c: connection, headers: ModbusHeaders, coils: ModbusCoils)
{
print "modbus_read_coils_response", c, headers, coils;
}
event modbus_read_discrete_inputs_request(c: connection, headers: ModbusHeaders, start_address: count, quantity: count)
{
print "modbus_read_discrete_inputs_request", c, headers, start_address, quantity;
}
event modbus_read_discrete_inputs_response(c: connection, headers: ModbusHeaders, coils: ModbusCoils)
{
print "modbus_read_discrete_inputs_response", c, headers, coils;
}
event modbus_read_holding_registers_request(c: connection, headers: ModbusHeaders, start_address: count, quantity: count)
{
print "modbus_read_holding_registers_request", c, headers, start_address, quantity;
}
event modbus_read_holding_registers_response(c: connection, headers: ModbusHeaders, registers: ModbusRegisters)
{
print "modbus_read_holding_registers_response", c, headers, registers;
}
event modbus_read_input_registers_request(c: connection, headers: ModbusHeaders, start_address: count, quantity: count)
{
print "modbus_read_input_registers_request", c, headers, start_address, quantity;
}
event modbus_read_input_registers_response(c: connection, headers: ModbusHeaders, registers: ModbusRegisters)
{
print "modbus_read_input_registers_response", c, headers, registers;
}
event modbus_write_single_coil_request(c: connection, headers: ModbusHeaders, address: count, value: bool)
{
print "modbus_write_single_coil_request", c, headers, address, value;
}
event modbus_write_single_coil_response(c: connection, headers: ModbusHeaders, address: count, value: bool)
{
print "modbus_write_single_coil_response", c, headers, address, value;
}
event modbus_write_single_register_request(c: connection, headers: ModbusHeaders, address: count, value: count)
{
print "modbus_write_single_register_request", c, headers, address, value;
}
event modbus_write_single_register_response(c: connection, headers: ModbusHeaders, address: count, value: count)
{
print "modbus_write_single_register_response", c, headers, address, value;
}
event modbus_write_multiple_coils_request(c: connection, headers: ModbusHeaders, start_address: count, coils: ModbusCoils)
{
print "modbus_write_multiple_coils_request", c, headers, start_address, coils;
}
event modbus_write_multiple_coils_response(c: connection, headers: ModbusHeaders, start_address: count, quantity: count)
{
print "modbus_write_multiple_coils_response", c, headers, start_address, quantity;
}
event modbus_write_multiple_registers_request(c: connection, headers: ModbusHeaders, start_address: count, registers: ModbusRegisters)
{
print "modbus_write_multiple_registers_request", c, headers, start_address, registers;
}
event modbus_write_multiple_registers_response(c: connection, headers: ModbusHeaders, start_address: count, quantity: count)
{
print "modbus_write_multiple_registers_response", c, headers, start_address, quantity;
}
event modbus_read_file_record_request(c: connection, headers: ModbusHeaders)
{
print "modbus_read_file_record_request", c, headers;
}
event modbus_read_file_record_response(c: connection, headers: ModbusHeaders)
{
print "modbus_read_file_record_response", c, headers;
}
event modbus_write_file_record_request(c: connection, headers: ModbusHeaders)
{
print "modbus_write_file_record_request", c, headers;
}
event modbus_write_file_record_response(c: connection, headers: ModbusHeaders)
{
print "modbus_write_file_record_response", c, headers;
}
event modbus_mask_write_register_request(c: connection, headers: ModbusHeaders, address: count, and_mask: count, or_mask: count)
{
print "modbus_mask_write_register_request", c, headers, address, and_mask, or_mask;
}
event modbus_mask_write_register_response(c: connection, headers: ModbusHeaders, address: count, and_mask: count, or_mask: count)
{
print "modbus_mask_write_register_response", c, headers, address, and_mask, or_mask;
}
event modbus_read_write_multiple_registers_request(c: connection, headers: ModbusHeaders, read_start_address: count, read_quantity: count, write_start_address: count, write_registers: ModbusRegisters)
{
print "modbus_read_write_multiple_registers_request", c, headers, read_start_address, read_quantity, write_start_address, write_registers;
}
event modbus_read_write_multiple_registers_response(c: connection, headers: ModbusHeaders, written_registers: ModbusRegisters)
{
print "modbus_read_write_multiple_registers_response", c, headers, written_registers;
}
event modbus_read_fifo_queue_request(c: connection, headers: ModbusHeaders, start_address: count)
{
print "modbus_read_fifo_queue_request", c, headers, start_address;
}
event modbus_read_fifo_queue_response(c: connection, headers: ModbusHeaders, fifos: ModbusRegisters)
{
print "modbus_read_fifo_queue_response", c, headers, fifos;
}

9
testing/btest/scripts/base/protocols/modbus/policy.bro Normal file
View file

@ -0,0 +1,9 @@
#
# @TEST-EXEC: bro -r $TRACES/modbus.trace %INPUT
# @TEST-EXEC: btest-diff modbus.log
# @TEST-EXEC: btest-diff modbus_register_change.log
# @TEST-EXEC: btest-diff known_modbus.log
#
@load protocols/modbus/known-masters-slaves.bro
@load protocols/modbus/track-memmap.bro