Mirror/zeek
1
0
Fork 0
mirror of https://github.com/zeek/zeek.git synced 2025-10-02 06:38:20 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 1

Creating a branch release/1.5 with the current 1.5.3 release code.

Browse source 
This is so that people working from the current stable version can
still start using git.
This commit is contained in:
Robin Sommer 2011-03-09 15:26:01 -08:00
parent 61757ac78b
commit 2b6ad76bd5
74 changed files with 1551 additions and 856 deletions
Download patch file Download diff file Expand all files Collapse all files

169
CHANGES
View file

@ -2,30 +2,65 @@
-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1.5.2.7 Sun Sep 12 19:39:49 PDT 2010
1.5.3 Thu Mar 3 08:55:11 PST 2011
- Addressed a number of lint nits (Vern Paxson).
- Removing aux/broctl/policy/cluster-addrs.hot.bro from the
distribution. The script is no longer needed and could in fact break
an installation because it redefines an old variable that has went
away. (Robin Sommer)
- Smarter way to increase the communication module's pipe's socket
buffer size, resulting in a value closer to the allowed maximum.
(Craig Leres)
1.5.2.6 Sun Sep 12 17:00:13 PDT 2010
- BroControl now also maintains links from the log archive to the
current set of logs when running in standalone mode. (Robin Sommer)
- Bug fix for a file descriptor leak in the remote communication
module. (Scott Campbell)
- Bug fix for BroControl to now activate trace-summary's sampling in
cluster mode, but not anymore in standalone mode. (Robin Sommer)
- Broccoli updates:
* Accept empty strings ("") as values in the configuration file.
(Craig Leres)
* Support for specifying a separate host key for SSL-enabled
operation, with documentation update. (Craig Leres)
1.5.2 Wed Jan 12 17:34:55 PST 2011
- Portability fixes for --enable-int64 (Vern Paxson).
- Bug fix for Active Mapping support (Kevin Lo).
- Broccoli compiler warning fixes (Kevin Lo).
- Bug fixes for --enable-int64 and for avoiding bogus statistics /
bad memory references when generating profiling information upon
exit (Vern Paxson).
- Bug fixes for terminating connections (Tyler Schoenke and Vern Paxson).
- Removed now-quite-stale SSHv1 overflow detection, as it's more prone
to false positives than useful detection (Vern Paxson).
- The SWIG file now explicitly lists those pieces from broccoli.h which it
wants to wrap, rather than just including all of broccoli.h (Robin Sommer).
This fixes the problem that the SWIG bindings depend on what configure
finds out about the availability of libpcap even though the corresponding
functions don't need to be wrapped anyway.
- http-header.bro now includes a global include_header: set[string]
(Robin Sommer). If it contains any strings, then only those headers
will be processed. If left empty, then you continue to get the current
behavior of processing all headers.
- http-header.bro now includes a global "include_header: set[string]" If it
contains any strings, then only those headers will be processed. If left
empty, then you continue to get the current behavior of processing all
headers. (Robin Sommer).
- Several changes to drop.bro (Robin Sommer):
* If true, the new flag Drop::dont_drop_locals indicates that
* If True, the new flag Drop::dont_drop_locals indicates that
local hosts should never be dropped. On by default.
* If true, the new flag Drop::debugging activates extensive debugging
* If True, the new flag Drop::debugging activates extensive debugging
output for the catch-and-release logic. Off by default.
* The timeout for tracking dropping information is now 1 day
@ -39,14 +74,14 @@
Sommer).
- The HTTP analyzer no longer attempts to track Server/User-Agent
versions, as these are hugely voluminous (Seth Hall). Ideally this
would still be available as an option for someone who truly wants
the full set.
versions, as these are hugely voluminous (Seth Hall).
- HTTP and SMTP no longer have extra-short inactivity timeouts, as
these were too often leading to premature expiration of a connection
(Robin Sommer).
- Tracking of HTTP refer[r]er's by setting log_referrer. (Vern Paxson).
- The "rst" tool (aux/rst/) now takes an optional "-I <text>" argument
that instructs it to inject <text> as payload rather than sending a RST
packet (Vern Paxson). <text> must be NUL-terminated, and the NUL is not
@ -55,39 +90,113 @@
- Bug fix for crashes in the DNS analyzer when processing replies for
which no request was seen (Robin Sommer).
- Addressed a number of lint nits (Vern Paxson).
1.5.2.5 Mon Jul 19 16:20:58 PDT 2010
- Rotation post-processors are now passed an additional argument
indicating whether Bro is terminating (Robin Sommer).
- Removed now-quite-stale SSHv1 overflow detection, as it's more prone
to false positives than useful detection (Vern Paxson).
- Bro now consistently generates a file_opened event for all fopen() calls.
(Robin Sommer).
- The "cf" utility now ignores a leading "t=" prefix, for compatibility
with Bro's "tagged" logging format (Robin Sommer).
1.5.2.4 Fri Jun 4 16:02:11 PDT 2010
- You can now redefine the email_notice_to function (Robin Sommer).
- Bug fixes for terminating connections (Tyler Schoenke and Vern Paxson).
- Fix for packet processing resumption when a remote Bro dies during
state synchronization (Robin Sommer).
- OpenSSL/X509 portability fix, at long last (Gregor Maier & Christian
Kreibich).
1.5.2.3 Wed Mar 24 18:23:57 PDT 2010
- Fix for compatibility with newer versions of autoconf (Gregor Maier).
- Bug fixes for --enable-int64 and for avoiding bogus statistics /
bad memory references when generating profiling information upon
exit (Vern Paxson).
- A larger BroControl update (Robin Sommer, if not marked otherwise):
o Increasing default timeouts for scan detector significantly.
1.5.2.2 Tue Jan 12 12:33:42 PST 2010
o Increasing the manager's max_remote_events_processed to
something large, as it would slow down the process too much
otherwise and there's no other work to be interleaved with it
anyway.
- Broccoli compiler warning fixes (Kevin Lo).
o Adding debug output to cluster's part of catch-and-release
(extends the debugging already present in policy/debug.bro)
o Fixing typo in util.py. Closes #223.
1.5.2.1 Sun Jan 10 16:59:01 PST 2010
o Added note to README pointing to HTML version.
- Bug fix for Active Mapping support (Kevin Lo).
o Disabling print_hook for proxies' remote.log.
o broctl's capstats now reports a total as well, and stats.log
tracks these totals. Closes #160.
1.5.2 Sat Dec 26 18:38:37 PST 2009
o Avoiding spurious "waiting for lock" messages in cron mode.
Closes #206.
- Portability fixes for --enable-int64 (Vern Paxson).
o Bug fixes for installation on NFS.
o Bug fix for top command on FreeBSD 8.
o crash-diag now checks whether gdb is available.
o trace-summary reports the sample factor in use in its output,
and now also applies it to the top-local-networks output (not
doing the latter was a bug).
o Removed the default twice-a-day rotation for conn.log. The
default rotation for conn.log now is now once every 24h, just
like for all other logs with the exception of mail.log (which is
still rotated twice a day, and thus the alarms are still mailed
out twice a day).
o Fixed the problem of logs sometimes being filed into the wrong
directory (see the (now gone) FAQ entry in the README).
o One can now customize the archive naming scheme. See the
corresponding FAQ entry in the README.
o Cleaned up, and extended, collection of cluster statistics.
${logdir}/stats now looks like this:
drwxr-xr-x 4 bro wheel 59392 Apr 5 17:55 .
drwxr-xr-x 96 bro wheel 2560 Apr 6 12:00 ..
-rw-r--r-- 1 bro wheel 576 Apr 6 16:40 meta.dat
drwxr-xr-x 2 bro wheel 2048 Apr 6 16:40 profiling
-rw-r--r-- 1 bro wheel 771834825 Apr 6 16:40 stats.log
drwxr-xr-x 2 bro wheel 2048 Apr 6 16:25 www
stats.log accumulates cluster statistics collected every time
"cron" is called.
- profiling/ keeps the nodes' prof.logs.
- www/ keeps a subset of stats.log in CSV format for easy plotting.
- meta.dat contains meta information about the current cluster
state (in particular which nodes we have, and when the last
stats update was done).
Note that there is no Web setup yet to actually visualize the data in
www/.
o BroControl now automatically maintains links inside today's log
archive directory pointing to the current live version of the
corresponding log file (if Bro is running). For example:
smtp.log.11:52:18-current -> /usr/local/cluster/spool/manager/smtp.log
o Alarms mailed out by BroControl now (1) have the notice msg in the
subject; and (2) come with the full mail.log entry in the body.
o Fixing broctl's top output. (Seth Hall).
o Fixing broctl's df output in certain situations.
o BroControl fix for dealing with large vsize values reported by
"top" (Craig Leres).
1.5.1 Fri Dec 18 15:17:12 PST 2009

4
Checklist-for-Release
View file

@ -30,6 +30,10 @@
bro-1.X-current.tar.gz
bro-1.<n>-release.tar.gz
- Create symlink for HTTP:
/ftp/BROIDS/bro-XXX.tar.gz -> /www/BROIDS/download/bro-XXX.tar.gz
- Update crd:/www/BROIDS/download.html to reflect new version. This page
is generated from trunk/bro-web/download.xml. Edit this file, and also
update the (web page) version in build.xml, the copyright year in

2
README
View file

@ -1,4 +1,4 @@
This is release 1.5 of Bro, a system for detecting network intruders in
This is release 1.5.3 of Bro, a system for detecting network intruders in
real-time using passive network monitoring.
Please see the file INSTALL for installation instructions and some examples

2
VERSION
View file

@ -1 +1 @@
1.5.2.7
1.5.3

56
acinclude.m4
View file

@ -60,26 +60,24 @@ AC_DEFUN([AC_LBL_TYPE_SIGNAL],
esac]])
dnl
dnl Determine which compiler we're using (cc or gcc)
dnl If using gcc, determine the version number
dnl If using cc, require that it support ansi prototypes
dnl If using gcc, use -O2 (otherwise use -O)
dnl If using cc, explicitly specify /usr/local/include
dnl Do whatever AC_LBL_C_INIT work is necessary before using AC_PROG_CC.
dnl
dnl usage:
dnl It appears that newer versions of autoconf (2.64 and later) will,
dnl if you use AC_TRY_COMPILE in a macro, stick AC_PROG_CC at the
dnl beginning of the macro, even if the macro itself calls AC_PROG_CC.
dnl See the "Prerequisite Macros" and "Expanded Before Required" sections
dnl in the Autoconf documentation.
dnl
dnl AC_LBL_C_INIT(copt, incls)
dnl This causes a steaming heap of fail in our case, as we were, in
dnl AC_LBL_C_INIT, doing the tests we now do in AC_LBL_C_INIT_BEFORE_CC,
dnl calling AC_PROG_CC, and then doing the tests we now do in
dnl AC_LBL_C_INIT. Now, we run AC_LBL_C_INIT_BEFORE_CC, AC_PROG_CC,
dnl and AC_LBL_C_INIT at the top level.
dnl
dnl results:
dnl
dnl $1 (copt set)
dnl $2 (incls set)
dnl CC
dnl LDFLAGS
dnl LBL_CFLAGS
dnl
AC_DEFUN([AC_LBL_C_INIT],
dnl Borrowed from libpcap-1.1.1 by Gregor
AC_DEFUN([AC_LBL_C_INIT_BEFORE_CC],
[AC_PREREQ(2.12)
AC_BEFORE([$0], [AC_LBL_C_INIT])
AC_BEFORE([$0], [AC_PROG_CC])
AC_BEFORE([$0], [AC_LBL_FIXINCLUDES])
AC_BEFORE([$0], [AC_LBL_DEVEL])
@ -108,7 +106,31 @@ AC_DEFUN([AC_LBL_C_INIT],
CC=cc
export CC
fi
AC_PROG_CC
])
dnl
dnl Determine which compiler we're using (cc or gcc)
dnl If using gcc, determine the version number
dnl If using cc, require that it support ansi prototypes
dnl If using gcc, use -O2 (otherwise use -O)
dnl If using cc, explicitly specify /usr/local/include
dnl
dnl usage:
dnl
dnl AC_LBL_C_INIT(copt, incls)
dnl
dnl results:
dnl
dnl $1 (copt set)
dnl $2 (incls set)
dnl CC
dnl LDFLAGS
dnl LBL_CFLAGS
dnl
AC_DEFUN([AC_LBL_C_INIT],
[AC_PREREQ(2.12)
AC_BEFORE([$0], [AC_LBL_FIXINCLUDES])
AC_BEFORE([$0], [AC_LBL_DEVEL])
if test "$GCC" != yes ; then
AC_MSG_CHECKING(that $CC handles ansi prototypes)
AC_CACHE_VAL(ac_cv_lbl_cc_ansi_prototypes,

8
aux/broccoli/ChangeLog
View file

@ -1,9 +1,13 @@
Broccoli Changelog
========================================================================
Tue Jan 12 12:32:12 PST 2010 Christian <christian@whoop.org>
Wed Mar 2 15:38:02 PST 2011 Christian <christian@whoop.org>
- Build warning fixes (Kevin Lo).
- Accept empty strings ("") as values in the configuration file
(Craig Leres).
- Support for specifying a separate host key for SSL-enabled operation,
with documentation update (Craig Leres).
- Version bump to 1.5.3.
------------------------------------------------------------------------

2
aux/broccoli/configure.in
View file

@ -8,7 +8,7 @@ AC_CANONICAL_HOST
AC_CONFIG_AUX_DIR(.)
AM_CONFIG_HEADER(config.h)
AM_INIT_AUTOMAKE(broccoli, 1.5.0)
AM_INIT_AUTOMAKE(broccoli, 1.5.3)
dnl Commands for funkier shell output:
BLD_ON=`./shtool echo -n -e %B`

7
aux/broccoli/docs/broccoli-manual.sgml
View file

@ -1,7 +1,7 @@
<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook V4.2//EN" [
<!ENTITY bc "<function>broccoli</function>">
<!ENTITY bcc "<filename>broccoli-config</filename>">
<!ENTITY bc-latest-rel "1.5">
<!ENTITY bc-latest-rel "1.5.3">
<!ENTITY bc-header SYSTEM "sgml/broccoli.sgml">
<!ENTITY bp "<function>broping</function>">
]>
@ -19,7 +19,7 @@
<abstract>
<para>
This is documentation for release <emphasis>&bc-latest-rel;</emphasis>
of Broccoli, compatible with Bro IDS releases of <emphasis>1.4</emphasis>
of Broccoli, compatible with Bro IDS releases of <emphasis>1.5</emphasis>
or newer. Broccoli is free software under terms of the BSD license as given
in the <link linkend="license" endterm="license.title">License</link>
section. This documentation is always available on the web for download
@ -1531,6 +1531,8 @@ Bar/SomeLongStr "Hello World"
need to put the CA certificate and the peer certificate in the
<varname>/broccoli/ca_cert</varname> and
<varname>/broccoli/host_cert</varname> keys, respectively, in the configuration file.
Optionally, you can store the private key in a separate file specified by
<varname>/broccoli/host_key</varname>.
To quickly enable/disable a certificate configuration, the
<varname>/broccoli/use_ssl</varname> key can be used.
<caution>
@ -1555,6 +1557,7 @@ Bar/SomeLongStr "Hello World"
/broccoli/use_ssl yes
/broccoli/ca_cert <path>/ca_cert.pem
/broccoli/host_cert <path>/bro_cert.pem
/broccoli/host_key <path>/bro_cert.key
]]>
</programlisting>
<para>

210
aux/broccoli/docs/html/a3621.html
View file

@ -1,210 +0,0 @@
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<HTML
><HEAD
><TITLE
>Appendix</TITLE
><META
NAME="GENERATOR"
CONTENT="Modular DocBook HTML Stylesheet Version 1.7"><LINK
REL="HOME"
TITLE="Broccoli: The Bro Client Communications Library"
HREF="index.html"><LINK
REL="PREVIOUS"
TITLE="broccoli"
HREF="broccoli-broccoli.html"><LINK
REL="STYLESHEET"
TYPE="text/css"
HREF="stylesheet.css"></HEAD
><BODY
CLASS="APPENDIX"
BGCOLOR="#FFFFFF"
TEXT="#000000"
LINK="#0000FF"
VLINK="#840084"
ALINK="#0000FF"
><DIV
CLASS="NAVHEADER"
><TABLE
SUMMARY="Header navigation table"
WIDTH="100%"
BORDER="0"
CELLPADDING="0"
CELLSPACING="0"
><TR
><TH
COLSPAN="3"
ALIGN="center"
>Broccoli: The Bro Client Communications Library</TH
></TR
><TR
><TD
WIDTH="10%"
ALIGN="left"
VALIGN="bottom"
><A
HREF="broccoli-broccoli.html"
ACCESSKEY="P"
>Prev</A
></TD
><TD
WIDTH="80%"
ALIGN="center"
VALIGN="bottom"
></TD
><TD
WIDTH="10%"
ALIGN="right"
VALIGN="bottom"
>&nbsp;</TD
></TR
></TABLE
><HR
ALIGN="LEFT"
WIDTH="100%"></DIV
><DIV
CLASS="APPENDIX"
><H1
><A
NAME="AEN3621"
></A
>Appendix A. Appendix</H1
><DIV
CLASS="TOC"
><DL
><DT
><B
>Table of Contents</B
></DT
><DT
>A.1. <A
HREF="a3621.html#LICENSE"
>License</A
></DT
><DT
>A.2. <A
HREF="a3621.html#ABOUT"
>About this document</A
></DT
></DL
></DIV
><BR
CLEAR="all"><DIV
CLASS="SECT1"
><H1
CLASS="SECT1"
><A
NAME="LICENSE"
>A.1. License</A
></H1
><P
> Copyright (C) 2004-2008 Christian Kreibich and various contributors.
</P
><P
>
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to
deal in the Software without restriction, including without limitation the
rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
sell copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
</P
><P
> The above copyright notice and this permission notice shall be included in
all copies of the Software and its documentation and acknowledgment shall be
given in the documentation and software packages that this Software was
used.
</P
><P
>
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
</P
></DIV
><BR
CLEAR="all"><DIV
CLASS="SECT1"
><H1
CLASS="SECT1"
><A
NAME="ABOUT"
>A.2. About this document</A
></H1
><P
> This documentation is maintained in SGML <A
HREF="http://www.docbook.org"
TARGET="_top"
>DocBook</A
>,
API documentation is extracted from the code using the
<A
HREF="http://www.gtk.org/gtk-doc/"
TARGET="_top"
><B
CLASS="COMMAND"
>gtk-doc</B
></A
> tools.
</P
></DIV
></DIV
><DIV
CLASS="NAVFOOTER"
><HR
ALIGN="LEFT"
WIDTH="100%"><TABLE
SUMMARY="Footer navigation table"
WIDTH="100%"
BORDER="0"
CELLPADDING="0"
CELLSPACING="0"
><TR
><TD
WIDTH="33%"
ALIGN="left"
VALIGN="top"
><A
HREF="broccoli-broccoli.html"
ACCESSKEY="P"
>Prev</A
></TD
><TD
WIDTH="34%"
ALIGN="center"
VALIGN="top"
><A
HREF="index.html"
ACCESSKEY="H"
>Home</A
></TD
><TD
WIDTH="33%"
ALIGN="right"
VALIGN="top"
>&nbsp;</TD
></TR
><TR
><TD
WIDTH="33%"
ALIGN="left"
VALIGN="top"
>broccoli</TD
><TD
WIDTH="34%"
ALIGN="center"
VALIGN="top"
>&nbsp;</TD
><TD
WIDTH="33%"
ALIGN="right"
VALIGN="top"
>&nbsp;</TD
></TR
></TABLE
></DIV
></BODY
></HTML
>

210
aux/broccoli/docs/html/a3638.html Normal file
View file

@ -0,0 +1,210 @@
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<HTML
><HEAD
><TITLE
>Appendix</TITLE
><META
NAME="GENERATOR"
CONTENT="Modular DocBook HTML Stylesheet Version 1.79"><LINK
REL="HOME"
TITLE="Broccoli: The Bro Client Communications Library"
HREF="index.html"><LINK
REL="PREVIOUS"
TITLE="broccoli"
HREF="broccoli-broccoli.html"><LINK
REL="STYLESHEET"
TYPE="text/css"
HREF="stylesheet.css"></HEAD
><BODY
CLASS="APPENDIX"
BGCOLOR="#FFFFFF"
TEXT="#000000"
LINK="#0000FF"
VLINK="#840084"
ALINK="#0000FF"
><DIV
CLASS="NAVHEADER"
><TABLE
SUMMARY="Header navigation table"
WIDTH="100%"
BORDER="0"
CELLPADDING="0"
CELLSPACING="0"
><TR
><TH
COLSPAN="3"
ALIGN="center"
>Broccoli: The Bro Client Communications Library</TH
></TR
><TR
><TD
WIDTH="10%"
ALIGN="left"
VALIGN="bottom"
><A
HREF="broccoli-broccoli.html"
ACCESSKEY="P"
>Prev</A
></TD
><TD
WIDTH="80%"
ALIGN="center"
VALIGN="bottom"
></TD
><TD
WIDTH="10%"
ALIGN="right"
VALIGN="bottom"
>&nbsp;</TD
></TR
></TABLE
><HR
ALIGN="LEFT"
WIDTH="100%"></DIV
><DIV
CLASS="APPENDIX"
><H1
><A
NAME="AEN3638"
></A
>Appendix A. Appendix</H1
><DIV
CLASS="TOC"
><DL
><DT
><B
>Table of Contents</B
></DT
><DT
>A.1. <A
HREF="a3638.html#LICENSE"
>License</A
></DT
><DT
>A.2. <A
HREF="a3638.html#ABOUT"
>About this document</A
></DT
></DL
></DIV
><BR
CLEAR="all"><DIV
CLASS="SECT1"
><H1
CLASS="SECT1"
><A
NAME="LICENSE"
>A.1. License</A
></H1
><P
> Copyright (C) 2004-2008 Christian Kreibich and various contributors.
</P
><P
>
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to
deal in the Software without restriction, including without limitation the
rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
sell copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
</P
><P
> The above copyright notice and this permission notice shall be included in
all copies of the Software and its documentation and acknowledgment shall be
given in the documentation and software packages that this Software was
used.
</P
><P
>
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
</P
></DIV
><BR
CLEAR="all"><DIV
CLASS="SECT1"
><H1
CLASS="SECT1"
><A
NAME="ABOUT"
>A.2. About this document</A
></H1
><P
> This documentation is maintained in SGML <A
HREF="http://www.docbook.org"
TARGET="_top"
>DocBook</A
>,
API documentation is extracted from the code using the
<A
HREF="http://www.gtk.org/gtk-doc/"
TARGET="_top"
><B
CLASS="COMMAND"
>gtk-doc</B
></A
> tools.
</P
></DIV
></DIV
><DIV
CLASS="NAVFOOTER"
><HR
ALIGN="LEFT"
WIDTH="100%"><TABLE
SUMMARY="Footer navigation table"
WIDTH="100%"
BORDER="0"
CELLPADDING="0"
CELLSPACING="0"
><TR
><TD
WIDTH="33%"
ALIGN="left"
VALIGN="top"
><A
HREF="broccoli-broccoli.html"
ACCESSKEY="P"
>Prev</A
></TD
><TD
WIDTH="34%"
ALIGN="center"
VALIGN="top"
><A
HREF="index.html"
ACCESSKEY="H"
>Home</A
></TD
><TD
WIDTH="33%"
ALIGN="right"
VALIGN="top"
>&nbsp;</TD
></TR
><TR
><TD
WIDTH="33%"
ALIGN="left"
VALIGN="top"
>broccoli</TD
><TD
WIDTH="34%"
ALIGN="center"
VALIGN="top"
>&nbsp;</TD
><TD
WIDTH="33%"
ALIGN="right"
VALIGN="top"
>&nbsp;</TD
></TR
></TABLE
></DIV
></BODY
></HTML
>

566
aux/broccoli/docs/html/broccoli-broccoli.html
View file

File diff suppressed because it is too large Load diff

2
aux/broccoli/docs/html/c54.html
View file

@ -107,7 +107,7 @@ CLASS="COMMAND"
>--enable-debug</B
>: enables debugging output.
Please refer to the <A
HREF="c84.html#AEN818"
HREF="c84.html#AEN819"
>Broccoli debugging</A
>
section for details on configuring and using debugging output.

32
aux/broccoli/docs/html/c84.html
View file

@ -107,17 +107,17 @@ HREF="c84.html#AEN738"
></DT
><DT
>3.5. <A
HREF="c84.html#AEN784"
HREF="c84.html#AEN785"
>Configuring event reception in Bro policies</A
></DT
><DT
>3.6. <A
HREF="c84.html#AEN818"
HREF="c84.html#AEN819"
>Configuring debugging output</A
></DT
><DT
>3.7. <A
HREF="c84.html#AEN842"
HREF="c84.html#AEN843"
>Test programs</A
></DT
></DL
@ -1169,7 +1169,7 @@ CLASS="FUNCTION"
>Remote::destinations</CODE
> configuration.
See <A
HREF="c84.html#AEN784"
HREF="c84.html#AEN785"
>below</A
> for how to do this.
Finally, in order to obtain the class of a connection as indicated by the remote side, use
@ -1230,7 +1230,7 @@ CLASS="EMPHASIS"
> You need to make sure that the remote Bro agent is interested in receiving
the events you send. This interest is expressed in policy configuration.
We'll explain this in more detail <A
HREF="c84.html#AEN784"
HREF="c84.html#AEN785"
>below</A
>
and for now assume that our remote peer is configured to receive the
@ -3351,6 +3351,11 @@ CLASS="VARNAME"
CLASS="VARNAME"
>/broccoli/host_cert</CODE
> keys, respectively, in the configuration file.
Optionally, you can store the private key in a separate file specified by
<CODE
CLASS="VARNAME"
>/broccoli/host_key</CODE
>.
To quickly enable/disable a certificate configuration, the
<CODE
CLASS="VARNAME"
@ -3429,6 +3434,7 @@ CLASS="PROGRAMLISTING"
>/broccoli/use_ssl yes
/broccoli/ca_cert &#60;path&#62;/ca_cert.pem
/broccoli/host_cert &#60;path&#62;/bro_cert.pem
/broccoli/host_key &#60;path&#62;/bro_cert.key
</PRE
></TD
></TR
@ -3533,7 +3539,7 @@ CLASS="SECT1"
><H1
CLASS="SECT1"
><A
NAME="AEN784"
NAME="AEN785"
>3.5. Configuring event reception in Bro policies</A
></H1
><P
@ -3690,7 +3696,7 @@ CLASS="COMMAND"
>broping</B
> tool
explained in the <A
HREF="c84.html#AEN842"
HREF="c84.html#AEN843"
>section on testing</A
> below.
It will allow an agent on the local host to connect and send "ping" events.
@ -3708,7 +3714,7 @@ CLASS="SECT1"
><H1
CLASS="SECT1"
><A
NAME="AEN818"
NAME="AEN819"
>3.6. Configuring debugging output</A
></H1
><P
@ -3804,7 +3810,7 @@ CLASS="SECT1"
><H1
CLASS="SECT1"
><A
NAME="AEN842"
NAME="AEN843"
>3.7. Test programs</A
></H1
><P
@ -3818,8 +3824,8 @@ CLASS="FUNCTION"
>broping</CODE
>
<A
NAME="AEN847"
HREF="#FTN.AEN847"
NAME="AEN848"
HREF="#FTN.AEN848"
><SPAN
CLASS="footnote"
>[2]</SPAN
@ -3958,8 +3964,8 @@ ALIGN="LEFT"
VALIGN="TOP"
WIDTH="5%"
><A
NAME="FTN.AEN847"
HREF="c84.html#AEN847"
NAME="FTN.AEN848"
HREF="c84.html#AEN848"
><SPAN
CLASS="footnote"
>[2]</SPAN

16
aux/broccoli/docs/html/index.html
View file

@ -49,7 +49,7 @@ NAME="AEN9"
CLASS="emphasis"
><B
CLASS="EMPHASIS"
>1.5</B
>1.5.3</B
></SPAN
>
of Broccoli, compatible with Bro IDS releases of <SPAN
@ -61,7 +61,7 @@ CLASS="EMPHASIS"
>
or newer. Broccoli is free software under terms of the BSD license as given
in the <A
HREF="a3637.html#LICENSE"
HREF="a3638.html#LICENSE"
>License</A
>
section. This documentation is always available on the web for download
@ -225,17 +225,17 @@ HREF="c84.html#AEN738"
></DT
><DT
>3.5. <A
HREF="c84.html#AEN784"
HREF="c84.html#AEN785"
>Configuring event reception in Bro policies</A
></DT
><DT
>3.6. <A
HREF="c84.html#AEN818"
HREF="c84.html#AEN819"
>Configuring debugging output</A
></DT
><DT
>3.7. <A
HREF="c84.html#AEN842"
HREF="c84.html#AEN843"
>Test programs</A
></DT
></DL
@ -256,19 +256,19 @@ HREF="broccoli-broccoli.html"
></DD
><DT
>A. <A
HREF="a3637.html"
HREF="a3638.html"
>Appendix</A
></DT
><DD
><DL
><DT
>A.1. <A
HREF="a3637.html#LICENSE"
HREF="a3638.html#LICENSE"
>License</A
></DT
><DT
>A.2. <A
HREF="a3637.html#ABOUT"
HREF="a3638.html#ABOUT"
>About this document</A
></DT
></DL

2
aux/broccoli/src/bro_lexer.l
View file

@ -27,11 +27,11 @@ no|false|off { yylval.i = 0; return BROINT; }
[ \t]+ ;
[0-9]+ { yylval.i = strtol(yytext, NULL, 10); return BROINT; }
[0-9]+\.[0-9]+ { yylval.d = strtod(yytext, NULL); return BRODOUBLE; }
[[:alnum:][:punct:]]+ { yylval.s = strdup(yytext); return BROWORD; }
\".*\" { yylval.s = strdup(yytext+1);
yylval.s[strlen(yylval.s) - 1] = '\0';
return BROSTRING;
}
[[:alnum:][:punct:]]+ { yylval.s = strdup(yytext); return BROWORD; }
"#".*\n { bro_parse_lineno++; }
"//".*\n { bro_parse_lineno++; }

38
aux/broccoli/src/bro_openssl.c
View file

@ -244,7 +244,7 @@ __bro_openssl_init(void)
{
static int deja_vu = FALSE;
int use_ssl = FALSE;
const char *our_cert, *our_pass, *ca_cert;
const char *our_cert, *our_key, *our_pass, *ca_cert;
D_ENTER;
@ -284,7 +284,15 @@ __bro_openssl_init(void)
D_RETURN_(TRUE);
}
if (! (our_cert = __bro_conf_get_str("/broccoli/host_cert")))
our_cert = __bro_conf_get_str("/broccoli/host_cert");
our_key = __bro_conf_get_str("/broccoli/host_key");
if (our_key == NULL)
{
/* No private key configured; get it from the certificate file */
our_key = our_cert;
}
if (our_cert == NULL)
{
if (use_ssl)
{
@ -298,6 +306,21 @@ __bro_openssl_init(void)
}
}
if (our_key == NULL)
{
if (use_ssl)
{
D(("SSL requested but host key not given -- aborting.\n"));
D_RETURN_(FALSE);
}
else
{
D(("use_ssl not used and host key not given -- not using SSL.\n"));
D_RETURN_(TRUE);
}
}
/* At this point we either haven't seen use_ssl but a host_cert, or
* we have seen use_ssl and it is set to true. Either way, we attempt
* to set up an SSL connection now and abort if this fails in any way.
@ -326,9 +349,9 @@ __bro_openssl_init(void)
SSL_CTX_set_default_passwd_cb_userdata(ctx, (void *) our_pass);
}
if (SSL_CTX_use_PrivateKey_file(ctx, our_cert, SSL_FILETYPE_PEM) != 1)
if (SSL_CTX_use_PrivateKey_file(ctx, our_key, SSL_FILETYPE_PEM) != 1)
{
D(("SSL used but error loading private key from '%s' -- aborting.\n", our_cert));
D(("SSL used but error loading private key from '%s' -- aborting.\n", our_key));
goto error_return;
}
@ -356,6 +379,13 @@ __bro_openssl_init(void)
goto error_return;
}
/* Check the consistency of the certificate vs. the private key */
if (SSL_CTX_check_private_key(ctx) != 1)
{
D(("SSL used but private key does not match the certificate -- aborting\n"));
goto error_return;
}
/* Only use real ciphers.
*/
if (! SSL_CTX_set_cipher_list(ctx, "HIGH"))

55
aux/broctl/BroControl/control.py
View file

@ -585,7 +585,7 @@ def getTopOutput(nodes):
d = {}
d["pid"] = int(p[0])
d["proc"] = (p[0] == parents[node.tag] and "parent" or "child")
d["vsize"] = int(p[1])
d["vsize"] = int(float(p[1]))
d["rss"] = int(p[2])
d["cpu"] = p[3]
d["cmd"] = " ".join(p[4:])
@ -761,6 +761,9 @@ def attachGdb(nodes):
#
# Tags are those as returned by capstats on the command-line
#
# There is one "pseudo-node" of the name "$total" with the sum of all
# individual values.
#
# We do all the stuff in parallel across all nodes which is why this looks
# a bit confusing ...
@ -798,6 +801,8 @@ def getCapstatsOutput(nodes, interval):
outputs = execute.runHelperParallel(cmds)
totals = {}
for (node, success, output) in outputs:
if not success:
@ -810,13 +815,22 @@ def getCapstatsOutput(nodes, interval):
try:
for field in fields[1:]:
(key, val) = field.split("=")
vals[key] = float(val)
val = float(val)
vals[key] = val
try:
totals[key] += val
except KeyError:
totals[key] = val
results += [(node, None, vals)]
except ValueError:
results += [(node, "%s: unexpected capstats output: %s" % (node.tag, output[0]), {})]
# Add pseudo-node for totals
results += [(config.Node("$total"), None, totals)]
return results
# Get current statistics from cFlow.
@ -861,16 +875,9 @@ def calculateCFlowRate(start, stop, interval):
def capstats(nodes, interval):
def output(tag, data):
util.output("\n%-12s %-10s %-10s (%ds average)" % (tag, "kpps", "mbps", interval))
util.output("-" * 30)
for (port, error, vals) in data:
if error:
util.output(error)
continue
util.output("%-12s " % port, nl=False)
def outputOne(tag, vals):
util.output("%-12s " % tag, nl=False)
if not error:
util.output("%-10s " % vals["kpps"], nl=False)
@ -880,6 +887,27 @@ def capstats(nodes, interval):
else:
util.output("<%s> " % error)
util.output("\n%-12s %-10s %-10s (%ds average)" % (tag, "kpps", "mbps", interval))
util.output("-" * 30)
totals = None
for (port, error, vals) in data:
if error:
util.output(error)
continue
if str(port) != "$total":
outputOne(port, vals)
else:
totals = vals
if totals:
util.output("")
outputOne("Total", totals)
util.output("")
have_cflow = config.Config.cflowaddress and config.Config.cflowuser and config.Config.cflowpassword
have_capstats = config.Config.capstats
@ -960,6 +988,11 @@ def getDf(nodes):
cmds = []
for node in nodes:
if dir == "logdir" and node.type != "manager":
# Don't need this on the workers/proxies.
continue
cmds += [(node, "df", [path])]
results = execute.runHelperParallel(cmds)

29
aux/broctl/BroControl/cron.py
View file

@ -20,11 +20,12 @@ def doCron():
if config.Config.cronenabled == "0":
return
config.Config.config["cron"] = "1" # Flag to indicate that we're running from cron.
if not util.lock():
return
util.bufferOutput()
config.Config.config["cron"] = "1" # Flag to indicate that we're running from cron.
# Check whether nodes are still running an restart if neccessary.
for (node, isrunning) in control.isRunning(config.Config.nodes()):
@ -55,10 +56,10 @@ def doCron():
if output:
util.sendMail("cron: " + output.split("\n")[0], output)
config.Config.config["cron"] = "0"
util.unlock()
config.Config.config["cron"] = "0"
def logAction(node, action):
t = time.time()
out = open(config.Config.statslog, "a")
@ -191,15 +192,6 @@ def _checkHosts():
config.Config._setState(tag, alive)
def _getProfLogs():
dir = config.Config.statsdir
if not os.path.exists(dir):
os.mkdir(dir)
if not os.path.exists(dir) or not os.path.isdir(dir):
util.output("cannot create directory %s" % dir)
return
cmds = []
for node in config.Config.hosts():
@ -211,14 +203,10 @@ def _getProfLogs():
util.output("cannot get prof.log from %s" % node.tag)
def _updateHTTPStats():
# Get the prof.logs.
_getProfLogs()
# Copy stats.dat.
shutil.copy(config.Config.statslog, config.Config.statsdir)
# Creat meta file.
# Create meta file.
meta = open(os.path.join(config.Config.statsdir, "meta.dat"), "w")
for node in config.Config.hosts():
print >>meta, "node", node.tag, node.type, node.host
@ -238,5 +226,12 @@ def _updateHTTPStats():
meta.close()
# Run the update-stats script.
(success, output) = execute.runLocalCmd(os.path.join(config.Config.scriptsdir, "update-stats"))
if not success:
util.output("error running update-stats\n\n")
util.output(output)

9
aux/broctl/BroControl/execute.py
View file

@ -79,7 +79,7 @@ def mkdirs(dirs):
else:
cmds += [(node, [], [])]
# Need to be careful here as our helper scripts may not be installed yet.
fullcmds += [("test -d %s || mkdir %s 2>/dev/null; echo $?; echo ~~~" % (dir, dir))]
fullcmds += [("test -d %s || mkdir -p %s 2>/dev/null; echo $?; echo ~~~" % (dir, dir))]
for (node, success, output) in runHelperParallel(cmds, fullcmds=fullcmds):
results += [(node, success)]
@ -147,7 +147,14 @@ def install(host, src, dst):
os.remove(dst)
util.debug(1, "cp %s %s" % (src, dst))
try:
shutil.copy2(src, dst)
except OSError:
# Python 2.6 has a bug where this may fail on NFS. So we just
# ignore errors.
pass
return True
else:
util.error("install() not yet supported for remote hosts")

19
aux/broctl/BroControl/install.py
View file

@ -72,6 +72,7 @@ Targets = [
("${distdir}/aux/broctl/bin/delete-log", "${scriptsdir}", True),
("${distdir}/aux/broctl/bin/expire-logs.in", "${scriptsdir}", True),
("${distdir}/aux/broctl/bin/post-terminate.in", "${scriptsdir}", True),
("${distdir}/aux/broctl/bin/stat-ctime", "${scriptsdir}", True),
("${distdir}/aux/broctl/bin/crash-diag.in", "${scriptsdir}", True),
("${distdir}/aux/broctl/bin/send-mail.in", "${scriptsdir}", True),
("${distdir}/aux/broctl/bin/mail-alarm.in", "${scriptsdir}", True),
@ -82,6 +83,12 @@ Targets = [
("${distdir}/aux/broctl/bin/cflow-stats.in", "${scriptsdir}", True),
("${distdir}/aux/broctl/bin/get-prof-log.in", "${scriptsdir}", True),
("${distdir}/aux/broctl/bin/mail-contents.in", "${scriptsdir}", True),
("${distdir}/aux/broctl/bin/make-archive-name", "${scriptsdir}", True),
("${distdir}/aux/broctl/bin/create-link-for-log.in", "${scriptsdir}", True),
("${distdir}/aux/broctl/bin/remove-link-for-log.in", "${scriptsdir}", True),
("${distdir}/aux/broctl/bin/update-stats.in", "${scriptsdir}", True),
("${distdir}/aux/broctl/bin/stats-to-csv", "${scriptsdir}", True),
("${distdir}/aux/broctl/bin/fmt-time", "${scriptsdir}", True),
("${distdir}/aux/broctl/bin/helpers/start.in", "${helperdir}", True),
("${distdir}/aux/broctl/bin/helpers/stop", "${helperdir}", True),
("${distdir}/aux/broctl/bin/helpers/check-pid", "${helperdir}", True),
@ -318,7 +325,7 @@ def install(local_only, make_install):
try:
os.symlink(manager.cwd(), current)
except (IOError, OSError), e:
util.warn("cannot link %s to %s: %s" % (manager.cwd(), current, e))
pass
if local_only:
return
@ -362,17 +369,21 @@ def install(local_only, make_install):
# already take care of that.
else:
# NFS. We only need to take care of the spool/log directoryies.
# NFS. We only need to take care of the spool/log directories.
paths = [config.Config.spooldir]
paths += [config.Config.logdir]
paths += [config.Config.tmpdir]
dirs = []
for dir in paths:
dirs += [(n, dir) for n in nodes]
# We need this only on the manager.
dirs += [(manager, config.Config.logdir)]
for (node, success) in execute.mkdirs(dirs):
if not success:
util.warn("cannot create directory on %s" % (dir, node.tag))
util.warn("cannot create (some of the) directories %s on %s" % (",".join(paths), node.tag))
util.output("done.")
# Create Bro-side broctl configuration broctl-layout.bro.

2
aux/broctl/BroControl/options.py
View file

@ -33,6 +33,8 @@ options = [
Option("LogDir", "${BroBase}/logs", "string", Option.USER, False,
"Directory for archived log files."),
Option("MakeArchiveName", "${BroBase}/share/broctl/scripts/make-archive-name", "string", Option.USER, False,
"Script to generate filenames for archived log files."),
Option("SendMail", "1", "bool", Option.USER, False,
"True if shell may send mails."),

2
aux/broctl/BroControl/util.py
View file

@ -177,7 +177,7 @@ def lock():
else:
do_output = 2
if do_ouput:
if do_output:
output("waiting for lock ...", nl=False)
count = 0

27
aux/broctl/README
View file

@ -2,7 +2,11 @@
//
// $Id: README 6948 2009-12-03 20:59:41Z robin $
//
// FIXME: This needs asciidoc 8.2.x plus some custom config files.
// NOTE: This README contains only parts of the BroControl documentation.
// Please see README.html for the complete document.
// (To generate the HTML version, one needs asciidoc 8.2.x plus some custom
// config files.)
BroControl
===========
@ -223,7 +227,7 @@ expects commands on its command-line (alternatively, +broctl+ can
also be started with a single command directly on the shell's
command line):
> cluster
> broctl
Welcome to BroControl 0.2
Type "help" for help.
@ -425,17 +429,6 @@ Note for folks who have used the old "cluster shell": the
development mode corresponds to the old default behaviour, which
worked with any +make install-broctl+.
After a Bro crash, the timestamps of the archived log files sometimes seem to be wrong???
When Bro crashes, broctl archives the log files produced so far
at the normal location. However, for some files it can't (easily)
determine the right timestamps to put into the filename. This
affects in particular those log files that are not rotated on
regular basis (e.g., +stdout.log+, +prof.log+); their filenames
will indicate as their start time the point when all the other
files were _rotated_ most recently. In addition, for all log
files, after a crash the start/end times indicated by the file
names might be off a few seconds.
[[devversion]]Anything special to consider when using development versions???
If you are using a _development version_, _BroControl_ might
require patching Bro itself to work correctly. A "development
@ -448,3 +441,11 @@ After a Bro crash, the timestamps of the archived log files sometimes seem to be
> cd /path/to/bro/source/distribution
> patch -p0 <aux/broctl/patch-bro.diff
> ./autogen.sh
Can I change the naming scheme that BroControl uses for archived log files?
Yes, set xref:opt_MakeArchiveName[+MakeArchiveName+]+ to a
script that outputs the desired destination file name for an
archived log file. The default script for that task is
+<BroBase>/share/broctl/scripts/make-archive-name+, which you
can use that as a template for creating your own version. See
the beginning of that script for instructions.

35
aux/broctl/README.html
View file

@ -764,7 +764,7 @@ also be started with a single command directly on the shell's
command line):</p>
<div class="literalblock">
<div class="content">
<pre><tt>&gt; cluster
<pre><tt>&gt; broctl
Welcome to BroControl 0.2</tt></pre>
</div></div>
<div class="literalblock">
@ -1454,6 +1454,14 @@ Destination address for broctl-generated non-alarm mails. Default is to use the
</p>
</dd>
<dt>
<a id="opt_MakeArchiveName"></a> <strong>MakeArchiveName</strong> (string, default "${BroBase}/share/broctl/scripts/make-archive-name")
</dt>
<dd>
<p>
Script to generate filenames for archived log files.
</p>
</dd>
<dt>
<a id="opt_MemLimit"></a> <strong>MemLimit</strong> (string, default "unlimited")
</dt>
<dd>
@ -1941,22 +1949,6 @@ worked with any <tt>make install-broctl</tt>.</p>
</li>
<li>
<p><em>
After a Bro crash, the timestamps of the archived log files sometimes seem to be wrong?
</em></p>
<p>
When Bro crashes, broctl archives the log files produced so far
at the normal location. However, for some files it can't (easily)
determine the right timestamps to put into the filename. This
affects in particular those log files that are not rotated on
regular basis (e.g., <tt>stdout.log</tt>, <tt>prof.log</tt>); their filenames
will indicate as their start time the point when all the other
files were <em>rotated</em> most recently. In addition, for all log
files, after a crash the start/end times indicated by the file
names might be off a few seconds.
</p>
</li>
<li>
<p><em>
<a id="devversion"></a>Anything special to consider when using development versions?
</em></p>
<p>
@ -1976,10 +1968,17 @@ After a Bro crash, the timestamps of the archived log files sometimes seem to be
</div></div>
</li>
</ol>
<p>Can I change the naming scheme that BroControl uses for archived log files?
Yes, set <a href="#opt_MakeArchiveName"><tt>MakeArchiveName</tt></a><tt> to a
script that outputs the desired destination file name for an
archived log file. The default script for that task is
</tt>&lt;BroBase&gt;/share/broctl/scripts/make-archive-name+, which you
can use that as a template for creating your own version. See
the beginning of that script for instructions.</p>
</div>
<div id="footer">
<div id="footer-text">
Last modified at 2009-12-03 12:58:36 PDT - Robin Sommer
Last modified at 2010-10-18 16:49:08 PDT - Robin Sommer
</div>
</div>
</body>

2
aux/broctl/README.options
View file

@ -46,6 +46,8 @@ Reply-to address for broctl-generated mails.
General Subject prefix for broctl-generated mails.
[[opt_MailTo]] *MailTo* (string, default "<user>")::
Destination address for broctl-generated non-alarm mails. Default is to use the same address as +MailTo+.
[[opt_MakeArchiveName]] *MakeArchiveName* (string, default "$\{BroBase}/share/broctl/scripts/make-archive-name")::
Script to generate filenames for archived log files.
[[opt_MemLimit]] *MemLimit* (string, default "unlimited")::
Maximum amount of memory for Bro processes to use (in KB, or the string 'unlimited').
[[opt_MinDiskSpace]] *MinDiskSpace* (int, default 5)::

13
aux/broctl/aux/trace-summary/trace-summary
View file

@ -184,6 +184,9 @@ class Interval:
s += fmt("Connections", self.pkts) + \
fmt("Payload", self.payload)
if Options.factor != 1:
s += "Sampling %.2f%% -" % ( 100.0 / Options.factor )
if Options.verbose:
ports = topx(self.ports)
srcs = topx(self.srcs)
@ -848,13 +851,18 @@ print Total.format(conns=Options.conns, title="Total")
locals = LocalNets.keys()
for net in locals:
(txt, i) = LocalNets[net]
if i.updates:
i.applySampleFactor()
if locals:
type = "packets"
if Options.conns:
type = "connections"
locals.sort(lambda x,y: LocalNets[y][1].pkts - LocalNets[x][1].pkts)
locals.sort(lambda x,y: int(LocalNets[y][1].pkts - LocalNets[x][1].pkts))
print "\n>== Top %d local networks by number of %s\n" % (Options.topx, type)
@ -876,9 +884,6 @@ for net in locals:
(txt, i) = LocalNets[net]
if i.updates:
# i.start += TotalIntervals.start
# i.end += TotalIntervals.start
i.applySampleFactor()
print i.format(conns=Options.conns, title=net + " " + txt)
print "First: %16s (%.6f) Last: %s %.6f" % (isoTime(Total.start), Total.start, isoTime(Total.end), Total.end)

50
aux/broctl/bin/archive-log.in
View file

@ -4,9 +4,7 @@
#
# Bro postprocessor script to archive log files.
#
# archive-log <rotated-file-name> <base-name> <timestamp-when-opened> <timestamp-when-closed> [<tag>]
base=${logdir}
# archive-log <rotated-file-name> <base-name> <timestamp-when-opened> <timestamp-when-closed> <terminating> [<tag>]
delete=1
if [ "$1" == "-c" ]; then
@ -14,31 +12,36 @@ if [ "$1" == "-c" ]; then
shift
fi
# Record time of last rotation.
date +%y-%m-%d_%H.%M.%S >.rotate # Bro default format when rotating files.
# We do not keep the logs for workers/proxies.
if [ -e .worker -o -e .proxy ]; then
test $delete = 0 || rm -rf $1
exit 0
fi
# Build archive name
day=`echo $3 | sed 's/_.*$//'`
from=`echo $3 | sed 's/^.*_//' | sed 's/\./:/g'`
to=`echo $4 | sed 's/^.*._//' | sed 's/\./:/g'`
century=`date +%Y | sed 's/..$//g'`
day="$century$day"
terminating=$5
if [ ! -d "$base/$day" ]; then
mkdir "$base/$day" 2>/dev/null
century=`date +%Y | sed 's/..$//g'`
from=`echo $3 | sed 's/[_.]/-/g'`
from="$century$from"
to=`echo $4 | sed 's/[_.]/-/g'`
to="$century$to"
dest=`${makearchivename} $2 $from $to`
echo $dest | grep -q '^/'
if [ $? != 0 ]; then
dest="${logdir}/$dest"
fi
#if [ $# == 5 ]; then
# dest="$base/$day/$5.$2.$from-$to.gz"
#else
dest="$base/$day/$2.$from-$to.gz"
#fi
dest_dir=`dirname $dest`
mkdir -p $dest_dir # Makes sure all parent directories exist.
# Record time of last rotation.
date +%y-%m-%d_%H.%M.%S >.rotated.$2 # Bro default format when rotating files.
# Run other postprocessors.
for pp in ${postprocdir}/*; do
@ -46,9 +49,11 @@ for pp in ${postprocdir}/*; do
done
if [ -e $1 ]; then
nice gzip -9 <$1 >$dest 2>/dev/null
nice gzip -9 <$1 >$dest.gz 2>/dev/null &
fi
wait
if [ "$?" == "0" ]; then
if [ "$delete" == "1" ]; then
rm -rf $1
@ -57,3 +62,8 @@ if [ "$?" == "0" ]; then
find $1 -size +104857600c -delete
fi
fi
if [ "$terminating" == "1" ]; then
${scriptsdir}/remove-link-for-log $2
fi

3
aux/broctl/bin/broctl.in
View file

@ -323,9 +323,9 @@ class BroCtlCmdLoop(cmd.Cmd):
manually: all the maintainance tasks will then just be performed one
more time."""
if len(args) > 0:
self.lock()
if len(args) > 0:
if args == "enable":
config.Config._setState("cronenabled", "1")
util.output("cron enabled")
@ -336,6 +336,7 @@ class BroCtlCmdLoop(cmd.Cmd):
util.output("cron " + (config.Config.cronenabled == "1" and "enabled" or "disabled"))
else:
util.output("wrong cron argument")
return
cron.doCron()

1
aux/broctl/bin/check-config.in
View file

@ -25,6 +25,7 @@ shift
export PATH=${bindir}:${scriptsdir}:$PATH
echo $@ >.cmdline
touch .checking
if [ "${devmode}" == "0" ]; then
${bro} $@

8
aux/broctl/bin/crash-diag.in
View file

@ -44,12 +44,16 @@ echo
core=`ls -t *core* 2>&1`
for c in $core; do
if which gdb >/dev/null 2>&1; then
for c in $core; do
if [ -e $c ]; then
echo $c
echo "bt" | gdb --batch -x /dev/stdin ${bro} $c
fi
done
done
else
echo "No gdb installed."
fi
) >.crash-diag.log

65
aux/broctl/bin/create-link-for-log.in Normal file
View file

@ -0,0 +1,65 @@
#! /usr/bin/env bash
#
# create-link-for <file-name>
#
# Creates a link from `pwd`/$1 into the current archive directory.
if [ ! -e .manager -a ! -e .standalone ]; then
# We only create links on the manager/standalone.
exit 0
fi
if [ -e .checking ]; then
# Just checking configuration, don't create links.
exit 0
fi
if [ ! -f $1 ]; then
# Doesn't exist.
exit 0
fi
echo $1 | grep -q '^\.'
if [ $? == 0 ]; then
# Don't link internal files.
exit 0
fi
date=`date +%Y-%m-%d-%H-%M-%S`
link=`${makearchivename} $1 $date`
echo $link | grep -q '^/'
if [ $? != 0 ]; then
link="${logdir}/$link"
fi
dest_dir=`dirname $link`
mkdir -p $dest_dir # Makes sure all parent directories exist.
if [ -e $link ]; then
if [ ! -L $link ]; then
# Exists, but isn't a link. Don't touch.
exit 0
fi
# Link exists already for some reason, remove it.
rm -f $link
fi
# Remove last link we did for this file.
if [ -e .link.$1 ]; then
rm -f `cat .link.$1 | tail -1`
fi
# Do the link.
ln -s `pwd`/$1 $link
# Record the link.
echo $link >.link.$1

2
aux/broctl/bin/delete-log
View file

@ -4,7 +4,7 @@
#
# Bro postprocessor script to archive log files.
#
# archive-log <rotated-file-name> <base-name> <timestamp-when-opened> <timestamp-when-closed> [<tag>]
# archive-log <rotated-file-name> <base-name> <timestamp-when-opened> <timestamp-when-closed> <terminating> [<tag>]
rm -rf $1

19
aux/broctl/bin/fmt-time Normal file
View file

@ -0,0 +1,19 @@
#! /usr/bin/env python
#
# Prints a Unix timestamp $1 in the format Bro uses for its rotation timestamps.
fmt="%y-%m-%d_%H.%M.%S" # From rotate-logs.bro
import sys
import time
if len(sys.argv) != 2:
print >>sys.stderr, "usage: fmt-time unix-timestamp"
sys.exit(1)
t = float(sys.argv[1])
print time.strftime(fmt, time.localtime(int(t)))

4
aux/broctl/bin/get-prof-log.in
View file

@ -8,7 +8,9 @@ tag=$1
host=$2
path=$3
dstbase=${statsdir}/prof.$tag
mkdir -p ${statsdir}/profiling
dstbase=${statsdir}/profiling/prof.$tag
tmp=$dstbase.$$.log.tmp
# Ignore errors.

2
aux/broctl/bin/helpers/df.in
View file

@ -7,5 +7,5 @@
# Returns: <fs> <fs-size> <fs-used> <fs-avail>
echo 0
df -h $1 | awk '{print $1, $2, $3, $4}' | tail -1 | awk -f ${helperdir}/to-bytes.awk
df -kP $1 | awk '{print $1, $2, $3, $4}' | tail -1 | awk -v def_factor=1024 -f ${helperdir}/to-bytes.awk
echo ~~~

7
aux/broctl/bin/helpers/to-bytes.awk
View file

@ -1,6 +1,12 @@
# $Id: to-bytes.awk 6811 2009-07-06 20:41:10Z robin $
# Converts strings such as 12K, 42M, etc. into bytes.
# If def_factor is set, it's applied to values without any unit.
BEGIN {
if ( def_factor == 0 )
def_factor = 1;
}
{
for ( i = 1; i <= NF; i++) {
@ -9,6 +15,7 @@
else if ( match($i, "^(-?[0-9.]+)Mi?$") ){ $i = substr($i, RSTART, RLENGTH-1) * 1024 * 1024; }
else if ( match($i, "^(-?[0-9.]+)Gi?$") ){ $i = substr($i, RSTART, RLENGTH-1) * 1024 * 1024 * 1024; }
else if ( match($i, "^(-?[0-9.]+)Te?$") ){ $i = substr($i, RSTART, RLENGTH-1) * 1024 * 1024 * 1024 * 1024; }
else if ( match($i, "^(-?[0-9.]+)$") ) { $i = substr($i, RSTART, RLENGTH) * def_factor; }
printf("%s ", $i);
}

4
aux/broctl/bin/helpers/top.in
View file

@ -14,11 +14,11 @@ cmd_freebsd_nonsmp='top -u -b all | awk "/^ *[0-9]+ /{printf(\"%d %s %s %d %s\\n
cmd_darwin='top -l 1 | awk "/^ *[0-9]+ /{printf(\"%d %dK %dK %d %s\\n\", \$1, \$11, \$10, \$3, \$2)}"'
cmd_netbsd='top -b -u | awk "/^ *[0-9]+ /{printf(\"%d %s %s %d %s\\n\", \$1, \$5, \$6, \$10, \$11)}"'
cmd="$cmd_${os}"
eval cmd="\$cmd_${os}"
if [ "${os}" == "freebsd" ]; then
# Top's output looks different on non-SMP FreeBSD machines.
top -u -b all | grep -q "STATE C TIME" || cmd="$cmd_freebsd_nonsmp"
top -u -b all | grep -q "STATE *C *TIME" || cmd="$cmd_freebsd_nonsmp"
fi
unset LINES

41
aux/broctl/bin/make-archive-name Normal file
View file

@ -0,0 +1,41 @@
#! /usr/bin/env bash
#
# $Id: archive-log.in 6847 2009-07-30 16:54:58Z robin $
#
# Returns a path for archived log files. This script is called
# once for each log file being archived. Usage is:
#
# make-archive-name <basename> <timestamp-when-opened> [<timestamp-when-closed>]
#
# basename: The base file name of the log file being archived (e.g., conn.log).
# timestamp-when-opened: The timestamp when the log file being archived was created.
# timestamp-when-closed: The timestamp when the log file being archived was finished.
# Optional. If not given, the name is used to create a link to
# the current live version of the file.
#
# Times are given in the form "year-month-day-hour-minute-second",
# e.g., "2010-03-30-13-12-04"
#
# The script must return the path under which the file should be
# archived. If it's a relative path, it will be interpreted as
# relative to BroControl's standard log directory.
#
# Note that even though the logs will later be compressed, this
# script should return the filename without any .gz extension; that
# extension will be appended later.
name=$1
opened=$2
closed=$3
day=`echo $opened | awk -F - '{printf "%s-%s-%s", $1, $2, $3}'`
from=`echo $opened | awk -F - '{printf "%s:%s:%s", $4, $5, $6}'`
to=`echo $closed | awk -F - '{printf "%s:%s:%s", $4, $5, $6}'`
if [ "$closed" != "" ]; then
echo $day/$name.$from-$to
else
echo $day/$name.$from-current
fi

23
aux/broctl/bin/post-terminate.in
View file

@ -57,20 +57,33 @@ if [ ! -f .startup ]; then
exit
fi
brostart=`cat .startup | tail -1`
end=`date +%y-%m-%d_%H.%M.%S`
start=`cat .startup | tail -1`
if [ "$crash" = "1" -a -e .rotate ]; then
start=`cat .rotate | tail -1`
fi
# Old. Remove later.
#
#if [ "$crash" = "1" -a -e .rotate ]; then
# start=`cat .rotate | tail -1`
#fi
# Likewise old.
#if [ -e .peer_description ]; then
# tag=`cat .peer_description | tail -1`
#fi
( for i in *.log; do
if [ -s $i ]; then
${scriptsdir}/archive-log $archive_flags $i $i $start $end $tag >/dev/null &
if [ -e .rotated.$i ]; then
start=`cat .rotated.$i`
else
start=$brostart
fi
${scriptsdir}/archive-log $archive_flags "$i" "$i" "$start" "$end" 1 "$tag" >/dev/null &
fi
${scriptsdir}/remove-link-for-log $i
done && wait && if [ "$crash" = "0" ]; then rm -rf $tmp; fi ) &

5
aux/broctl/bin/postprocessors/mail-log.in
View file

@ -5,7 +5,7 @@
# Formats Bro's mail.log, archives, encrypts and mails it (if requested).
#
# It's called as a Bro postprocessor so its arguments are:
# mail-log <logfile> <basename> <timestamp-when-opened> <timestamp-when-closed> [<tag>]
# mail-log <logfile> <basename> <timestamp-when-opened> <timestamp-when-closed> <terminating> [<tag>]
if [ "$2" != "mail.log" ]; then
exit 0
@ -15,7 +15,8 @@ log=$1
base=$2
open=$3
close=$4
tag=$5
terminating=$5
tag=$6
# Do nothing if log is empty
if [ ! -s $log ]; then

4
aux/broctl/bin/postprocessors/summarize-connections.in
View file

@ -6,7 +6,7 @@
#
# Needs trace-summary script.
#
# summarize-conns <rotated-file-name> <base-name> <timestamp-when-opened> <timestamp-when-closed> [<tag>]
# summarize-conns <rotated-file-name> <base-name> <timestamp-when-opened> <timestamp-when-closed> <terminating> [<tag>]
if [ "$2" != "conn.log" ]; then
exit 0
@ -15,7 +15,7 @@ fi
summary_options="-c -r"
# If we're a cluster installation, we assume we have lots of traffic and activate sampling.
if [ "${standalone}" != "0" ]; then
if [ "${standalone}" = "0" ]; then
summary_options="$summary_options -S 0.01"
fi

8
aux/broctl/bin/remove-link-for-log.in Normal file
View file

@ -0,0 +1,8 @@
#! /usr/bin/env bash
#
# remove-link-for-log <filename>
if [ -e .link.$1 ]; then
rm -f `cat .link.$1 | tail -1`
rm -f .link.$1
fi

25
aux/broctl/bin/stat-ctime Normal file
View file

@ -0,0 +1,25 @@
#! /usr/bin/env python
#
# Prints a given paths ctime in the format Bro uses for its rotation
# timestamps.
fmt="%y-%m-%d_%H.%M.%S" # From rotate-logs.bro
import sys
import os
import stat
import time
if len(sys.argv) != 2:
print >>sys.stderr, "usage: stat-ctime <path>"
try:
ctime = os.stat(sys.argv[1])[stat.ST_CTIME]
except OSError, e:
print e
sys.exit(1)
print time.strftime(fmt, time.localtime(ctime))

137
aux/broctl/bin/stats-to-csv Normal file
View file

@ -0,0 +1,137 @@
#! /usr/bin/env python
#
# stats-to-csv <stats.log> <meta.dat> <wwwdir>
#
# Reads information from stats directory and outputs csv files <wwwdir>/<node>.<type>.csv.
# If any of these files already exists, we append (without writing the header line again).
import os.path
import os
import sys
Workers = set()
Proxies = set()
def readNodes(meta):
for line in open(meta):
m = line.split()
if m[0] == "node":
if m[2] == "worker":
Workers.add(m[1])
if m[2] == "proxy":
Workers.add(m[1])
def processNode(stats, wwwdir, node, iface):
print node, "..."
def openFile(tag, columns):
name = os.path.join(wwwdir, "%s.%s.csv" % (node, tag))
if os.path.exists(name):
return open(name, "a")
else:
f = open(name, "w")
print >>f, "time," + ",".join(columns)
return f
iface_mbps = openFile("mbps", ["MBits/sec"])
iface_pkts = openFile("pkts", ["TCP", "UDP", "ICMP", "Other"])
cpu = openFile("cpu", ["CPU"])
mem = openFile("mem", ["Memory"])
cflow = openFile("in", ["MBits/sec"])
def printEntry(t, entry):
try:
val = int(entry["parent-cpu"]) + int(entry["child-cpu"])
print >>cpu, "%s,%s" % (t, val)
except KeyError:
pass
try:
val = int(entry["parent-vsize"]) + int(entry["child-vsize"])
print >>mem, "%s,%s" % (t, val)
except KeyError:
pass
if iface:
try:
print >>iface_mbps, "%s,%s" % (t, entry["interface-mbps"])
except KeyError:
pass
try:
tc = float(entry["interface-t"])
ud = float(entry["interface-u"])
ic = float(entry["interface-i"])
ot = float(entry["interface-o"])
print >>iface_pkts, "%s,%s,%s,%s,%s" % (t, tc, ud, ic, ot)
except KeyError:
pass
if "in-mbps" in entry:
print >>cflow, "%s,%s" % (t, entry["in-mbps"])
entry = {}
first = -1
for line in open(stats):
m = line.split()
if m[1] != node:
continue
t = m[0]
if t != first and first >= 0:
printEntry(t, entry)
entry = {}
first = t
try:
entry["%s-%s" % (m[2], m[3])] = m[4]
except IndexError:
pass
if first >= 0:
printEntry(t, entry)
iface_mbps.close()
iface_pkts.close()
cpu.close()
mem.close()
cflow.close()
if len(sys.argv) != 4:
print "usage: %s <stats.log> <meta.dat> <www-dir>" % sys.argv[0]
sys.exit(1)
stats = sys.argv[1]
meta = sys.argv[2]
wwwdir = sys.argv[3]
try:
os.mkdir(wwwdir)
except OSError:
pass
readNodes(meta)
for w in Workers:
processNode(stats, wwwdir, w, True)
for p in Proxies:
processNode(stats, wwwdir, p, False)
processNode(stats, wwwdir, "manager", False)
processNode(stats, wwwdir, "cflow", False)

15
aux/broctl/bin/update-stats.in Normal file
View file

@ -0,0 +1,15 @@
#! /usr/bin/env bash
#
# $Id: archive-log.in 6847 2009-07-30 16:54:58Z robin $
#
# Saves the current stats.log from spool to ${statsdir}, and
# updates the WWW data.
dst=${statsdir}/`basename ${statslog}`
cat ${statslog} >>$dst
cp ${statsdir}/meta.dat ${statsdir}/www
${scriptsdir}/stats-to-csv ${statslog} ${statsdir}/meta.dat ${statsdir}/www
rm -f ${statslog}

6
aux/broctl/policy/broctl-check.bro
View file

@ -2,9 +2,5 @@
#
# Only loaded when checking configuration, not when running live.
@load rotate-logs
redef RotateLogs::rotate_on_shutdown=F;
redef RotateLogs::rotate_on_shutdown = F;

1
aux/broctl/policy/broctl-live.bro
View file

@ -3,4 +3,3 @@
# Only loaded when running live, not when just checking configuration.

1
aux/broctl/policy/broctl.bro
View file

@ -20,6 +20,7 @@ redef MANAGER = MANAGER > 0 ? MANAGER : 1;
@load cluster-by-addrs
@load remote-update
@load checkpoint
@load rotate-logs
# FIXME: Load them here to work around a namespace bug.
@load conn

4
aux/broctl/policy/cluster-manager.bro
View file

@ -8,7 +8,6 @@
@load filter-duplicates
@load notice
@load remote
@load rotate-logs
@load mail-alarms
# Since we don't capture, don't bother with this.
@ -32,6 +31,9 @@ redef interfaces = "";
# Give us a name.
redef peer_description = BroCtl::manager$tag;
# We're processing essentially *only* remote events.
redef max_remote_events_processed = 10000;
# Reraise remote notices locally.
event notice_action(n: notice_info, action: NoticeAction)
{

23
aux/broctl/policy/cluster-manager.drop.bro
View file

@ -3,27 +3,8 @@
# These will be generated by the workers.
event Drop::address_seen_again(a: addr)
{
if ( ! use_catch_release )
return;
if ( a !in drop_info )
# Never dropped.
return;
local di = drop_info[a];
if ( is_dropped(a) )
# Still dropped.
return;
NOTICE([$note=AddressSeenAgain, $src=a,
$msg=fmt("%s seen again after release", a)]);
}
# $Id$
# These will be generated by the workers.
event Drop::address_seen_again(a: addr)
{
debug_log(fmt("received seen_again for %s", a));
if ( ! use_catch_release )
return;

6
aux/broctl/policy/cluster-manager.icmp.bro
View file

@ -4,9 +4,3 @@ redef FilterDuplicates::filters += {
[ICMPAddressScan] = FilterDuplicates::match_src_num
};
# $Id: cluster-manager.scan.bro 6740 2009-06-12 17:59:44Z robin $
redef FilterDuplicates::filters += {
[ICMPAddressScan] = FilterDuplicates::match_src_num
};

8
aux/broctl/policy/cluster-manager.rotate-logs.bro
View file

@ -4,4 +4,10 @@ redef log_rotate_interval = 24hrs;
redef log_rotate_base_time = "0:00";
redef RotateLogs::default_postprocessor = "archive-log";
redef conn_file &rotate_interval = 12hrs;
event file_opened(f: file)
{
# Create a link from the archive directory to the newly created file.
if ( MANAGER == 1 && ! bro_is_terminating() )
system(fmt("create-link-for-log %s", get_file_name(f)));
}

1
aux/broctl/policy/cluster-proxy.bro
View file

@ -6,7 +6,6 @@
@load broctl
@load remote
@load rotate-logs
# Since we don't capture, don't bother with this.
@unload print-filter

3
aux/broctl/policy/cluster-proxy.remote.bro
View file

@ -1,5 +1,8 @@
# $Id: cluster-proxy.remote.bro 6811 2009-07-06 20:41:10Z robin $
# Do not copy the proxies's remote.log to the manager
redef Remote::rm_log &disable_print_hook;
event bro_init()
{
# Set up worker connections.

1
aux/broctl/policy/cluster-worker.bro
View file

@ -7,7 +7,6 @@
@load broctl
@load remote
@load rotate-logs
@load trim-trace-file

10
aux/broctl/policy/cluster-worker.drop.bro
View file

@ -7,18 +7,21 @@ global watch_addr_table: set[addr] &read_expire=7days &persistent;
global address_seen_again: event(a: addr);
event address_restored(a: addr)
event Drop::address_restored(a: addr)
{
debug_log(fmt("received restored for %s", a));
add watch_addr_table[a];
}
event address_dropped(a: addr)
event Drop::address_dropped(a: addr)
{
debug_log(fmt("received dropped for %s", a));
delete watch_addr_table[a];
}
event address_cleared(a: addr)
event Drop::address_cleared(a: addr)
{
debug_log(fmt("received cleared for %s", a));
delete watch_addr_table[a];
}
@ -28,6 +31,7 @@ event new_connection(c: connection)
local a = c$id$orig_h;
if ( a in watch_addr_table )
{
debug_log(fmt("sending seen_again for %s", a));
event Drop::address_seen_again(a);
delete watch_addr_table[a];
}

10
aux/broctl/policy/cluster.scan.bro
View file

@ -5,9 +5,9 @@ redef ignore_scanners_threshold = 500;
redef pre_distinct_peers &read_expire = 12hrs;
redef distinct_backscatter_peers &read_expire = 30mins;
redef distinct_peers &read_expire = 30mins;
redef distinct_ports &read_expire = 30mins;
redef distinct_low_ports &read_expire = 30mins;
redef possible_scan_sources &read_expire = 30mins;
redef distinct_backscatter_peers &create_expire = 5hrs;
redef distinct_peers &create_expire = 5hrs;
redef distinct_ports &create_expire = 5hrs;
redef distinct_low_ports &create_expire = 5hrs;
redef possible_scan_sources &create_expire = 5hrs;

71
aux/broctl/policy/mail-alarms.bro
View file

@ -22,18 +22,38 @@ export {
global output = open_log_file( "mail" );
}
function do_msg(line1: string, line2: string, line3: string, host: addr, name: string)
function do_msg(n: notice_info, line1: string, line2: string, line3: string, host: addr, name: string, dest: string)
{
if ( host != 0.0.0.0 )
name = fmt("%s = %s", host, name);
print output, cat(line1, name);
line1 = cat(line1, name);
if ( dest == "" )
{
# Append to mail.log.
print output, line1;
print output, line2;
if ( line3 != "" )
print output, line3;
}
function message(msg: string, flag: bool, host: addr, n: notice_info)
else
{
line1 = str_shell_escape(line1);
line2 = str_shell_escape(line2);
line3 = str_shell_escape(line3);
# Mail out an individual alarm.
local mail_cmd =
fmt("( echo \"%s\"; echo \"%s\"; echo \"%s\" ) | %s -s \"[Bro Alarm] %s: %s\" %s",
line1, line2, line3, mail_script, n$note, str_shell_escape(n$msg), dest);
system(mail_cmd);
}
}
function message(msg: string, flag: bool, host: addr, n: notice_info, dest: string)
{
if ( length(include_only) > 0 && n$note !in include_only )
return;
@ -52,30 +72,22 @@ function message(msg: string, flag: bool, host: addr, n: notice_info)
if ( host == 0.0.0.0 )
{
do_msg(line1, line2, line3, 0.0.0.0, "");
do_msg(n, line1, line2, line3, 0.0.0.0, "", dest);
return;
}
when ( local name = lookup_addr(host) )
{
do_msg(line1, line2, line3, host, name);
do_msg(n, line1, line2, line3, host, name, dest);
}
timeout 5secs
{
do_msg(line1, line2, line3, host, "(dns timeout)");
do_msg(n, line1, line2, line3, host, "(dns timeout)", dest);
}
}
event bro_init()
function make_alarm(n: notice_info, dest: string)
{
set_buf( output, F );
}
event notice_alarm(n: notice_info, action: NoticeAction) &priority = -10
{
if ( is_remote_event() )
return;
if ( n$note in ignore )
return;
@ -112,6 +124,33 @@ event notice_alarm(n: notice_info, action: NoticeAction) &priority = -10
if ( orig in flag_nets || resp in flag_nets )
flag = T;
message(msg, flag, host, n);
message(msg, flag, host, n, dest);
}
event bro_init()
{
set_buf( output, F );
}
event notice_alarm(n: notice_info, action: NoticeAction) &priority = -10
{
if ( is_remote_event() )
return;
make_alarm(n, "");
}
function broctl_email_notice_to(n: notice_info, dest: string)
{
if ( reading_traces() || dest == "" )
return;
if ( dest == "" )
return;
make_alarm(n, dest);
}
# Make the alarm mails nicer.
redef email_notice_to = broctl_email_notice_to;

1
aux/broctl/policy/standalone.bro
View file

@ -10,7 +10,6 @@
@load broctl
@load notice
@load remote
@load rotate-logs
@load mail-alarms
@load trim-trace-file

10
aux/broctl/policy/standalone.rotate-logs.bro
View file

@ -1,9 +1,13 @@
# $Id: standalone.rotate-logs.bro 6811 2009-07-06 20:41:10Z robin $
@load mail-alarms
redef log_rotate_interval = 24hrs;
redef log_rotate_base_time = "0:00";
redef RotateLogs::default_postprocessor = "archive-log";
redef conn_file &rotate_interval = 12hrs;
event file_opened(f: file)
{
# Create a link from the archive directory to the newly created file.
if ( ! bro_is_terminating() )
system(fmt("create-link-for-log %s", get_file_name(f)));
}

8
aux/cf/cf.1
View file

@ -40,7 +40,7 @@ with a formated time and date time and date. For example:
.RS
.na
.nh
\% echo '1074558944 default format' | cf
% echo '1074558944 default format' | cf
.br
Jan 19 16:35:44 default format
.ad
@ -66,6 +66,12 @@ and
flags override the
.B CFTIMEFMT
environment variable.
Note that filter skips over an instance of "t=" at the beginning of
a line, to provide compatibility with Bro's
.I
tagged
logging format.
.SH OPTIONS
.LP
.TP

6
aux/cf/cf.c
View file

@ -143,6 +143,12 @@ doone(fin, fout)
while (fgets(buf, sizeof(buf), fin)) {
bp = buf;
dotbp = NULL;
if (*bp == 't' && *(bp+1) == '=') {
fputs("t=", fout);
bp += 2;
}
if (isdigit(*bp)) {
ts = atol(bp);
++bp;

48
configure.in
View file

@ -30,6 +30,8 @@ AC_CANONICAL_SYSTEM
#AM_INIT_AUTOMAKE(bro, 0.1.0)
AM_INIT_AUTOMAKE(bro, esyscmd([tr -d '\n' < VERSION]))
AM_CONFIG_HEADER(config.h)
AC_LBL_C_INIT_BEFORE_CC(V_CCOPT, V_INCLS)
AC_PROG_CC
AC_LBL_C_INIT(V_CCOPT, V_INCLS)
AM_PROG_LEX
@ -136,6 +138,20 @@ AC_LBL_ENABLE_CHECK([activemapping binpac broccoli brov6 debug \
expire-dfa-states gtk-doc int64 openssl perftools perl \
select-loop shippedpcap broctl cluster nbdns])
dnl ################################################
dnl # Writing around broken autoconf
dnl ################################################
dnl It seems that AC_CHECK_HEADER defines a bash function called
dnl ac_fn_c_check_header_compile in the output when it is first
dnl encountered. While in general a neat idea, this fails, if the
dnl first use of AC_CHECK_HEADER is in an if/else clause. In this
dnl case the function's scope is limited to the enclosing if/els
dnl block and later calls to the function fail (more or less silently)
dnl Solution: we just place a phony AC_CHECK_HEADER call here.
AC_CHECK_HEADER([stdio.h])
AC_CHECK_HEADERS([stdio.h stdio.h])
dnl ################################################
dnl # OpenSSL
dnl ################################################
@ -168,9 +184,9 @@ if test "$use_openssl" = "yes"; then
# (CHECK_HEADER doesn't work here)
saved_cflags="${CFLAGS}"
CFLAGS="${CFLAGS} -I${OPENSSL}/include"
AC_COMPILE_IFELSE([#include <openssl/ssl.h>],,
AC_COMPILE_IFELSE([AC_LANG_SOURCE([[#include <openssl/ssl.h>]])],,
CFLAGS="${CFLAGS} -I/usr/kerberos/include"
AC_CHECK_HEADER(krb5.h,
AC_CHECK_HEADER([krb5.h],
V_INCLS="${V_INCLS} -I/usr/kerberos/include"
AC_DEFINE(NEED_KRB5_H,,[Include krb5.h]),
use_openssl=no
@ -188,7 +204,7 @@ if test "$use_openssl" = "yes"; then
saved_libs="${LIBS}"
LIBS="${LIBS} -lssl -lcrypto"
AC_MSG_CHECKING([for OpenSSL >= 0.9.7])
AC_LINK_IFELSE(AC_LANG_PROGRAM([[#include <openssl/evp.h>]], [[OPENSSL_add_all_algorithms_conf();]]),
AC_LINK_IFELSE([AC_LANG_PROGRAM([[#include <openssl/evp.h>]], [[OPENSSL_add_all_algorithms_conf();]])],
AC_MSG_RESULT(yes)
use_openssl=yes,
AC_MSG_RESULT(no)
@ -212,9 +228,9 @@ if test "$use_openssl" = "yes"; then
AC_MSG_CHECKING([whether d2i_X509() uses a const unsigned char**])
AC_LANG_PUSH([C++])
AC_COMPILE_IFELSE(
AC_LANG_PROGRAM([[#include <openssl/x509.h>]],
[AC_LANG_PROGRAM([[#include <openssl/x509.h>]],
[[const unsigned char** cpp = 0;
X509** x = 0; d2i_X509(x, cpp, 0);]]),
X509** x = 0; d2i_X509(x, cpp, 0);]])],
AC_DEFINE(OPENSSL_D2I_X509_USES_CONST_CHAR,,[d2i_x509 uses const char**])
AC_MSG_RESULT(yes),
AC_MSG_RESULT(no))
@ -288,7 +304,7 @@ freebsd*)
darwin*)
AC_MSG_CHECKING([if we need to include arpa/nameser_compat.h])
AC_COMPILE_IFELSE(AC_LANG_PROGRAM([[#include <arpa/nameser.h>]], [[HEADER *hdr; int d = NS_IN6ADDRSZ;]]), bro_ns_header_defined=yes, bro_ns_header_defined=no)
AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[#include <arpa/nameser.h>]], [[HEADER *hdr; int d = NS_IN6ADDRSZ;]])], bro_ns_header_defined=yes, bro_ns_header_defined=no)
# if the header is found, we don't need compatibility
if test "x$bro_ns_header_defined" = xyes; then
AC_MSG_RESULT(no)
@ -353,14 +369,14 @@ AC_LBL_CHECK_TYPE(u_int16_t, u_short)
AC_LBL_CHECK_TYPE(u_int8_t, u_char)
AC_HEADER_TIME
AC_CHECK_HEADERS(memory.h netinet/in.h socket.h getopt.h)
AC_CHECK_HEADERS(net/ethernet.h netinet/ether.h netinet/if_ether.h sys/ethernet.h,,,
AC_CHECK_HEADERS([memory.h netinet/in.h socket.h getopt.h])
AC_CHECK_HEADERS([net/ethernet.h netinet/ether.h netinet/if_ether.h sys/ethernet.h],,,
[#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <net/if.h>])
AC_CHECK_HEADERS(netinet/ip6.h,,,
AC_CHECK_HEADERS([netinet/ip6.h],,,
[#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>
@ -416,7 +432,7 @@ if test "$pcap_local" = "NO"; then
dnl ################################################
AC_MSG_CHECKING([for pcap_version in libpcap])
AC_LINK_IFELSE(
AC_LANG_PROGRAM([extern char pcap_version[];], [puts(pcap_version);]),
[AC_LANG_PROGRAM([extern char pcap_version[];], [puts(pcap_version);])],
AC_MSG_RESULT(yes)
AC_DEFINE(PCAP_VERSION_STRING,,[Have a version string in libpcap]),
AC_MSG_RESULT(no))
@ -448,7 +464,7 @@ dnl #
AC_MSG_CHECKING([if char_traits defines all methods])
AC_LANG_PUSH([C++])
AC_LINK_IFELSE(
AC_LANG_PROGRAM([[
[AC_LANG_PROGRAM([[
#include <string>
using namespace std;
class Foo { };
@ -456,7 +472,7 @@ class Foo { };
char_traits<Foo*> foo;
Foo f;
Foo *fp;
foo.assign(&fp, 10, &f);]]),
foo.assign(&fp, 10, &f);]])],
AC_MSG_RESULT([yes])
basic_string_works=yes,
AC_MSG_RESULT([no])
@ -575,17 +591,17 @@ else
bro_ns_initparse_works=no
bro_res_mkquery_works=no
AC_LINK_IFELSE(AC_LANG_PROGRAM([[#include <arpa/nameser.h>]],
[[ns_initparse(0,0,0);]]),
AC_LINK_IFELSE([AC_LANG_PROGRAM([[#include <arpa/nameser.h>]],
[[ns_initparse(0,0,0);]])],
bro_ns_initparse_works=yes)
AC_LINK_IFELSE(AC_LANG_PROGRAM([[
AC_LINK_IFELSE([AC_LANG_PROGRAM([[
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <arpa/nameser.h>
#include <resolv.h>]],
[[int (*p)() = res_mkquery]]), bro_res_mkquery_works=yes)
[[int (*p)() = res_mkquery]])], bro_res_mkquery_works=yes)
if test $bro_ns_initparse_works = yes && test $bro_res_mkquery_works = yes && test $nbdns = yes; then
AC_MSG_RESULT(yes)

8
policy/http-reply.bro
View file

@ -54,7 +54,13 @@ function http_reply_done(c: connection, stat: http_message_stat)
--s$num_pending_requests;
++s$first_pending_request;
if ( log_referrer )
req = fmt("%s %s [ref %s]", r$method, r$URI,
req_msg$referrer == "" ?
"<NONE>" : req_msg$referrer);
else
req = fmt("%s %s", r$method, r$URI);
log_it = r$log_it;
}
@ -113,5 +119,7 @@ event http_header(c: connection, is_orig: bool, name: string, value: string)
else
msg$host = value;
}
else if ( is_orig && name == "REFERER" )
msg$referrer = value;
}
}

3
policy/http-request.bro
View file

@ -51,6 +51,9 @@ export {
&redef;
const sensitive_post_URIs = /wwwroot|WWWROOT/ &redef;
# Include the referrer header in the log.
const log_referrer = F &redef;
}
redef capture_filters += {

3
policy/http.bro
View file

@ -50,6 +50,7 @@ type http_message: record {
abstract: string; # data abstract
skip_abstract: bool; # to skip abstract for certain content types
host: string; # host indicated in Host header
referrer: string; # "Referer" [sic] field
};
type http_pending_request_stream: record {
@ -105,7 +106,7 @@ function init_http_message(msg: http_message)
msg$header_slot = 0;
msg$abstract = "";
msg$skip_abstract = F;
msg$host = "";
msg$referrer = msg$host = "";
}
function new_http_message(): http_message

2
policy/notice.bro
View file

@ -272,6 +272,8 @@ function build_notice_info_string_tagged(n: notice_info) : string
return cur_info;
}
global email_notice_to: function(n: notice_info, dest: string) &redef;
function email_notice_to(n: notice_info, dest: string)
{
if ( reading_traces() || dest == "" )

3
policy/rotate-logs.bro
View file

@ -56,10 +56,11 @@ function run_pp(info: rotate_info)
if ( pp != "" )
# The date format is hard-coded here to provide a standardized
# script interface.
system(fmt("%s %s %s %s %s %s",
system(fmt("%s %s %s %s %s %s %s",
pp, info$new_name, info$old_name,
strftime("%y-%m-%d_%H.%M.%S", info$open),
strftime("%y-%m-%d_%H.%M.%S", info$close),
bro_is_terminating() ? "1" : "0",
tag));
else
system(fmt("/bin/mv %s %s %s",

20
src/File.cc
View file

@ -217,11 +217,8 @@ bool BroFile::Open(FILE* file)
return false;
}
val_list* vl = new val_list;
Ref(this);
vl->append(new Val(this));
Event* event = new ::Event(::file_opened, vl);
mgr.Dispatch(event, true);
RaiseOpenEvent();
return true;
}
@ -305,6 +302,7 @@ FILE* BroFile::BringIntoCache()
return f;
}
RaiseOpenEvent();
UpdateFileSize();
if ( fseek(f, position, SEEK_SET) < 0 )
@ -809,6 +807,18 @@ int BroFile::Write(const char* data, int len)
return true;
}
void BroFile::RaiseOpenEvent()
{
if ( ! ::file_opened )
return;
val_list* vl = new val_list;
Ref(this);
vl->append(new Val(this));
Event* event = new ::Event(::file_opened, vl);
mgr.Dispatch(event, true);
}
void BroFile::UpdateFileSize()
{
struct stat s;

3
src/File.h
View file

@ -114,6 +114,9 @@ protected:
// Stats the file to get its current size.
void UpdateFileSize();
// Raises a file_opened event.
void RaiseOpenEvent();
// Initialize encryption with the given public key.
void InitEncrypt(const char* keyfile);
// Finalize encryption.

71
src/RemoteSerializer.cc
View file

@ -544,6 +544,36 @@ void RemoteSerializer::Init()
initialized = 1;
}
void RemoteSerializer::SetSocketBufferSize(int fd, int opt, const char *what, int size, int verbose)
{
int defsize = 0;
socklen_t len = sizeof(defsize);
if ( getsockopt(fd, SOL_SOCKET, opt, (void *)&defsize, &len) < 0 )
{
if ( verbose )
Log(LogInfo, fmt("warning: cannot get socket buffer size (%s): %s", what, strerror(errno)));
return;
}
for ( int trysize = size; trysize > defsize; trysize -= 1024 )
{
if ( setsockopt(fd, SOL_SOCKET, opt, &trysize, sizeof(trysize)) >= 0 )
{
if ( verbose )
{
if ( trysize == size )
Log(LogInfo, fmt("raised pipe's socket buffer size from %dK to %dK", defsize / 1024, trysize / 1024));
else
Log(LogInfo, fmt("raised pipe's socket buffer size from %dK to %dK (%dK was requested)", defsize / 1024, trysize / 1024, size / 1024));
}
return;
}
}
Log(LogInfo, fmt("warning: cannot increase %s socket buffer size from %dK (%dK was requested)", what, defsize / 1024, size / 1024));
}
void RemoteSerializer::Fork()
{
if ( child_pid )
@ -562,25 +592,11 @@ void RemoteSerializer::Fork()
return;
}
int bufsize;
socklen_t len = sizeof(bufsize);
if ( getsockopt(pipe[0], SOL_SOCKET, SO_SNDBUF, &bufsize, &len ) < 0 )
Log(LogInfo, fmt("warning: cannot get socket buffer size: %s", strerror(errno)));
else
Log(LogInfo, fmt("pipe's socket buffer size is %d, setting to %d", bufsize, SOCKBUF_SIZE));
bufsize = SOCKBUF_SIZE;
if ( setsockopt(pipe[0], SOL_SOCKET, SO_SNDBUF,
&bufsize, sizeof(bufsize) ) < 0 ||
setsockopt(pipe[0], SOL_SOCKET, SO_RCVBUF,
&bufsize, sizeof(bufsize) ) < 0 ||
setsockopt(pipe[1], SOL_SOCKET, SO_SNDBUF,
&bufsize, sizeof(bufsize) ) < 0 ||
setsockopt(pipe[1], SOL_SOCKET, SO_RCVBUF,
&bufsize, sizeof(bufsize) ) < 0 )
Log(LogInfo, fmt("warning: cannot set socket buffer size to %dK: %s", bufsize / 1024, strerror(errno)));
// Try to increase the size of the socket send and receive buffers.
SetSocketBufferSize(pipe[0], SO_SNDBUF, "SO_SNDBUF", SOCKBUF_SIZE, 1);
SetSocketBufferSize(pipe[0], SO_RCVBUF, "SO_RCVBUF", SOCKBUF_SIZE, 0);
SetSocketBufferSize(pipe[1], SO_SNDBUF, "SO_SNDBUF", SOCKBUF_SIZE, 0);
SetSocketBufferSize(pipe[1], SO_RCVBUF, "SO_RCVBUF", SOCKBUF_SIZE, 0);
child_pid = 0;
@ -681,7 +697,7 @@ bool RemoteSerializer::CloseConnection(Peer* peer)
if ( peer->suspended_processing )
{
net_continue_processing();
current_peer->suspended_processing = false;
peer->suspended_processing = false;
}
if ( peer->state == Peer::CLOSING )
@ -1614,6 +1630,12 @@ void RemoteSerializer::PeerDisconnected(Peer* peer)
{
assert(peer);
if ( peer->suspended_processing )
{
net_continue_processing();
peer->suspended_processing = false;
}
if ( peer->state == Peer::CLOSED || peer->state == Peer::INIT )
return;
@ -1744,6 +1766,12 @@ void RemoteSerializer::UnregisterHandlers(Peer* peer)
void RemoteSerializer::RemovePeer(Peer* peer)
{
if ( peer->suspended_processing )
{
net_continue_processing();
peer->suspended_processing = false;
}
peers.remove(peer);
UnregisterHandlers(peer);
@ -2941,7 +2969,7 @@ void SocketComm::Run()
struct timeval small_timeout;
small_timeout.tv_sec = 0;
small_timeout.tv_usec =
io->CanWrite() || io->CanRead() ? 10 : 10000;
io->CanWrite() || io->CanRead() ? 1 : 10;
int a = select(max_fd + 1, &fd_read, &fd_write, &fd_except,
&small_timeout);
@ -3575,6 +3603,7 @@ bool SocketComm::Listen(uint32 ip, uint16 port, bool expect_ssl)
if ( bind(*listen_fd, (sockaddr*) &server, sizeof(server)) < 0 )
{
Error(fmt("can't bind to port %d, %s", port, strerror(errno)));
close(*listen_fd);
*listen_fd = -1;
if ( errno == EADDRINUSE )

2
src/RemoteSerializer.h
View file

@ -297,6 +297,8 @@ protected:
bool SendToChild(char type, Peer* peer, int nargs, ...); // can send uints32 only
bool SendToChild(ChunkedIO::Chunk* c);
void SetSocketBufferSize(int fd, int opt, const char *what, int size, int verbose);
private:
enum { TYPE, ARGS } msgstate; // current state of reading comm.
Peer* current_peer;

6
src/X509.cc
View file

@ -192,7 +192,7 @@ int X509_Cert::verifyChain(Contents_SSL* e, const u_char* data, uint32 len)
// but in chain format).
// Init the stack.
STACK_OF(X509)* untrustedCerts = sk_new_null();
STACK_OF(X509)* untrustedCerts = sk_X509_new_null();
if ( ! untrustedCerts )
{
// Internal error allocating stack of untrusted certs.
@ -233,7 +233,7 @@ int X509_Cert::verifyChain(Contents_SSL* e, const u_char* data, uint32 len)
else
// The remaining certificates (if any) are put into
// the list of untrusted certificates
sk_push(untrustedCerts, (char*) pTemp);
sk_X509_push(untrustedCerts, pTemp);
tempLength += certLength + 3;
}
@ -259,7 +259,7 @@ int X509_Cert::verifyChain(Contents_SSL* e, const u_char* data, uint32 len)
// Free the stack, incuding. contents.
// FIXME: could this break Bro's memory tracking?
sk_pop_free(untrustedCerts, free);
sk_X509_pop_free(untrustedCerts, X509_free);
return ret;
}

6
src/ssl-analyzer.pac
View file

@ -382,7 +382,7 @@ refine analyzer SSLAnalyzer += {
STACK_OF(X509)* untrusted_certs = 0;
if ( certificates->size() > 1 )
{
untrusted_certs = sk_new_null();
untrusted_certs = sk_X509_new_null();
if ( ! untrusted_certs )
{
// X509_V_ERR_OUT_OF_MEM;
@ -405,7 +405,7 @@ refine analyzer SSLAnalyzer += {
return false;
}
sk_push(untrusted_certs, (char*) pTemp);
sk_X509_push(untrusted_certs, pTemp);
}
}
@ -417,7 +417,7 @@ refine analyzer SSLAnalyzer += {
certificate_error(csc.error);
X509_STORE_CTX_cleanup(&csc);
sk_pop_free(untrusted_certs, free_X509);
sk_X509_pop_free(untrusted_certs, X509_free);
}
X509_free(pCert);