mirror of
https://github.com/zeek/zeek.git
synced 2025-10-02 14:48:21 +00:00
Merge remote-tracking branch 'origin/master' into topic/bernhard/input-threads
Conflicts: src/CMakeLists.txt testing/btest/Baseline/coverage.bare-load-baseline/canonified_loaded_scripts.log testing/btest/Baseline/coverage.default-load-baseline/canonified_loaded_scripts.log
This commit is contained in:
Bernhard Amann
commit
3b82d69eb3
167 changed files with 3528 additions and 1066 deletions
143
CHANGES
143
CHANGES
|
|
@ -1,4 +1,147 @@
|
|||
|
||||
2.0-372 | 2012-05-17 13:59:45 -0700
|
||||
|
||||
* Fix compile errors. (Jon Siwek)
|
||||
|
||||
* Linking in the DS docs. (Robin Sommer)
|
||||
|
||||
* Fix mobility checksums unit test. (Jon Siwek)
|
||||
|
||||
2.0-367 | 2012-05-17 12:42:30 -0700
|
||||
|
||||
* Adding support for binary output via DataSeries. See
|
||||
logging-dataseries.rst for more information. (Gilbert Clark and
|
||||
Robin Sommer)
|
||||
|
||||
* Adding target update-doc-sources to top-level Makefile that runs
|
||||
genDocSourcesList.sh. (Robin Sommer)
|
||||
|
||||
* Moving trace for rotation test into traces directory. (Robin Sommer)
|
||||
|
||||
* Fixing a rotation race condition at termination. (Robin Sommer)
|
||||
|
||||
* Extending log post-processor call to include the name of the
|
||||
writer. (Robin Sommer)
|
||||
|
||||
* In threads, an internal error now immediately aborts. Otherwise,
|
||||
the error won't make it back to the main thread for a while and
|
||||
subsequent code in the thread would still execute. (Robin Sommer)
|
||||
|
||||
* DataSeries cleanup. (Robin Sommer)
|
||||
|
||||
* Fixing threads' DoFinish() method. It wasn't called reliably. Now,
|
||||
it's always called before the thread is destroyed (assuming
|
||||
processing has went normally so far). (Robin Sommer)
|
||||
|
||||
2.0-341 | 2012-05-17 09:54:30 -0700
|
||||
|
||||
* Add a comment to explain the ICMPv6 error message types. (Daniel Thayer)
|
||||
|
||||
* Quieting external test output somehwat. (Robin Sommer)
|
||||
|
||||
2.0-336 | 2012-05-14 17:15:44 -0700
|
||||
|
||||
* Don't print the various "weird" events to stderr. Address #805.
|
||||
(Daniel Thayer)
|
||||
|
||||
* Generate icmp_error_message event for ICMPv6 error msgs.
|
||||
Previously, icmp_sent was being generated, but icmp_error_message
|
||||
contains more info.
|
||||
|
||||
* Improved documentation comments for icmp-related events. (Daniel
|
||||
Thayer)
|
||||
|
||||
2.0-330 | 2012-05-14 17:05:56 -0700
|
||||
|
||||
* Add `addr_to_uri` script-level function that adds brackets to an
|
||||
address if it's IPv6 and will be included in a URI or when a
|
||||
":<port>" needs to be appended to it. (Jon Siwek)
|
||||
|
||||
* Also add a test case for content extraction. (Jon Siwek)
|
||||
|
||||
* Fix typos and improve INSTALL document. (Daniel Thayer)
|
||||
|
||||
* Switching to new btest command TEST-SERIALIZE for communication
|
||||
tests. (Robin Sommer)
|
||||
|
||||
2.0-323 | 2012-05-04 21:04:34 -0700
|
||||
|
||||
* Add SHA1 and SHA256 hashing BIFs. Addresses #542.
|
||||
|
||||
* Refactor all internal MD5 stuff to use OpenSSL's. (Jon Siwek)
|
||||
|
||||
* Changes to open-file caching limits and uncached file unserialization. (Jon Siwek)
|
||||
|
||||
- Unserializing files that were previously kicked out of the open-file
|
||||
cache would cause them to be fopen'd with the original access
|
||||
permissions which is usually 'w' and causes truncation. They
|
||||
are now opened in 'a' mode. (addresses #780)
|
||||
|
||||
- Add 'max_files_in_cache' script option to manually set the maximum
|
||||
amount of opened files to keep cached. Mainly this just helped
|
||||
to create a simple test case for the above change.
|
||||
|
||||
- Remove unused NO_HAVE_SETRLIMIT preprocessor switch.
|
||||
|
||||
- On systems that don't enforce a limit on number of files opened for
|
||||
the process, raise default max size of open-file cache from
|
||||
32 to 512.
|
||||
|
||||
2.0-319 | 2012-05-03 13:24:44 -0700
|
||||
|
||||
* SSL bugfixes and cleanup. (Seth Hall)
|
||||
|
||||
- SSL related files and classes renamed to remove the "binpac" term.
|
||||
|
||||
- A small fix for DPD scripts to make the DPD log more helpful if
|
||||
there are multiple continued failures.
|
||||
|
||||
- Fixed the SSL analyzer to make it stop doing repeated violation
|
||||
messages for some handshake failures.
|
||||
|
||||
- Added a $issuer_subject to the SSL log.
|
||||
|
||||
- Created a basic test for SSL.
|
||||
|
||||
- Fixed parsing of TLS server extensions. (Seth Hall)
|
||||
|
||||
2.0-315 | 2012-05-03 11:44:17 -0700
|
||||
|
||||
* Add two more TLS extension values that we see in live traffic.
|
||||
(Bernhard Amann)
|
||||
|
||||
* Fixed IPv6 link local unicast CIDR and added IPv6 loopback to
|
||||
private address space. (Seth Hall)
|
||||
|
||||
* Fixed a problem where cluster workers were still processing
|
||||
notices in some cases. (Seth Hall)
|
||||
|
||||
* Added a configure option to specify the 'etc' directory. Addresses
|
||||
#801. (Daniel Thayer)
|
||||
|
||||
|
||||
2.0-306 | 2012-04-24 14:37:00 -0700
|
||||
|
||||
* Add further TLS extension values "extended_random" and
|
||||
"heartbeat". (Seth Hall)
|
||||
|
||||
* Fix problem with extracting FTP passwords and add "ftpuser" as
|
||||
another anonymous username. (Seth Hall, discovered by Patrik
|
||||
Lundin).
|
||||
|
||||
2.0-303 | 2012-04-19 10:01:06 -0700
|
||||
|
||||
* Changes related to ICMPv6 Neighbor Discovery messages. (Jon Siwek)
|
||||
|
||||
- The 'icmp_conn' record now contains an 'hlim' field since hop limit
|
||||
in the IP header is an interesting field for at least these ND
|
||||
messages.
|
||||
|
||||
- Fixed and extended 'icmp_router_advertisement' event parameters.
|
||||
|
||||
- Changed 'icmp_neighbor_advertisement' event parameters to add
|
||||
more of the known boolean flags.
|
||||
|
||||
2.0-301 | 2012-04-17 17:58:55 -0700
|
||||
|
||||
* Bro now support ICMPv6. (Matti Mantere, Jon Siwek, Robin Sommer,
|
||||
|
|
|
|||
|
|
@ -107,6 +107,21 @@ if (GOOGLEPERFTOOLS_FOUND)
|
|||
endif ()
|
||||
endif ()
|
||||
|
||||
set(USE_DATASERIES false)
|
||||
find_package(Lintel)
|
||||
find_package(DataSeries)
|
||||
find_package(LibXML2)
|
||||
|
||||
if (LINTEL_FOUND AND DATASERIES_FOUND AND LIBXML2_FOUND)
|
||||
set(USE_DATASERIES true)
|
||||
include_directories(BEFORE ${Lintel_INCLUDE_DIR})
|
||||
include_directories(BEFORE ${DataSeries_INCLUDE_DIR})
|
||||
include_directories(BEFORE ${LibXML2_INCLUDE_DIR})
|
||||
list(APPEND OPTLIBS ${Lintel_LIBRARIES})
|
||||
list(APPEND OPTLIBS ${DataSeries_LIBRARIES})
|
||||
list(APPEND OPTLIBS ${LibXML2_LIBRARIES})
|
||||
endif()
|
||||
|
||||
if (ENABLE_PERFTOOLS_DEBUG)
|
||||
# Just a no op to prevent CMake from complaining about manually-specified
|
||||
# ENABLE_PERFTOOLS_DEBUG not being used if google perftools weren't found
|
||||
|
|
@ -198,6 +213,7 @@ message(
|
|||
"\nGeoIP: ${USE_GEOIP}"
|
||||
"\nGoogle perftools: ${USE_PERFTOOLS}"
|
||||
"\n debugging: ${USE_PERFTOOLS_DEBUG}"
|
||||
"\nDataSeries: ${USE_DATASERIES}"
|
||||
"\n"
|
||||
"\n================================================================\n"
|
||||
)
|
||||
|
|
|
|||
144
DocSourcesList.cmake
Normal file
144
DocSourcesList.cmake
Normal file
|
|
@ -0,0 +1,144 @@
|
|||
# DO NOT EDIT
|
||||
# This file is auto-generated from the genDocSourcesList.sh script.
|
||||
#
|
||||
# This is a list of Bro script sources for which to generate reST documentation.
|
||||
# It will be included inline in the CMakeLists.txt found in the same directory
|
||||
# in order to create Makefile targets that define how to generate reST from
|
||||
# a given Bro script.
|
||||
#
|
||||
# Note: any path prefix of the script (2nd argument of rest_target macro)
|
||||
# will be used to derive what path under scripts/ the generated documentation
|
||||
# will be placed.
|
||||
|
||||
set(psd ${PROJECT_SOURCE_DIR}/scripts)
|
||||
|
||||
rest_target(${CMAKE_CURRENT_SOURCE_DIR} example.bro internal)
|
||||
rest_target(${psd} base/init-default.bro internal)
|
||||
rest_target(${psd} base/init-bare.bro internal)
|
||||
|
||||
rest_target(${CMAKE_BINARY_DIR}/src base/bro.bif.bro)
|
||||
rest_target(${CMAKE_BINARY_DIR}/src base/const.bif.bro)
|
||||
rest_target(${CMAKE_BINARY_DIR}/src base/event.bif.bro)
|
||||
rest_target(${CMAKE_BINARY_DIR}/src base/logging.bif.bro)
|
||||
rest_target(${CMAKE_BINARY_DIR}/src base/reporter.bif.bro)
|
||||
rest_target(${CMAKE_BINARY_DIR}/src base/strings.bif.bro)
|
||||
rest_target(${CMAKE_BINARY_DIR}/src base/types.bif.bro)
|
||||
rest_target(${psd} base/frameworks/cluster/main.bro)
|
||||
rest_target(${psd} base/frameworks/cluster/nodes/manager.bro)
|
||||
rest_target(${psd} base/frameworks/cluster/nodes/proxy.bro)
|
||||
rest_target(${psd} base/frameworks/cluster/nodes/worker.bro)
|
||||
rest_target(${psd} base/frameworks/cluster/setup-connections.bro)
|
||||
rest_target(${psd} base/frameworks/communication/main.bro)
|
||||
rest_target(${psd} base/frameworks/control/main.bro)
|
||||
rest_target(${psd} base/frameworks/dpd/main.bro)
|
||||
rest_target(${psd} base/frameworks/intel/main.bro)
|
||||
rest_target(${psd} base/frameworks/logging/main.bro)
|
||||
rest_target(${psd} base/frameworks/logging/postprocessors/scp.bro)
|
||||
rest_target(${psd} base/frameworks/logging/postprocessors/sftp.bro)
|
||||
rest_target(${psd} base/frameworks/logging/writers/ascii.bro)
|
||||
rest_target(${psd} base/frameworks/logging/writers/dataseries.bro)
|
||||
rest_target(${psd} base/frameworks/metrics/cluster.bro)
|
||||
rest_target(${psd} base/frameworks/metrics/main.bro)
|
||||
rest_target(${psd} base/frameworks/metrics/non-cluster.bro)
|
||||
rest_target(${psd} base/frameworks/notice/actions/add-geodata.bro)
|
||||
rest_target(${psd} base/frameworks/notice/actions/drop.bro)
|
||||
rest_target(${psd} base/frameworks/notice/actions/email_admin.bro)
|
||||
rest_target(${psd} base/frameworks/notice/actions/page.bro)
|
||||
rest_target(${psd} base/frameworks/notice/actions/pp-alarms.bro)
|
||||
rest_target(${psd} base/frameworks/notice/cluster.bro)
|
||||
rest_target(${psd} base/frameworks/notice/extend-email/hostnames.bro)
|
||||
rest_target(${psd} base/frameworks/notice/main.bro)
|
||||
rest_target(${psd} base/frameworks/notice/weird.bro)
|
||||
rest_target(${psd} base/frameworks/packet-filter/main.bro)
|
||||
rest_target(${psd} base/frameworks/packet-filter/netstats.bro)
|
||||
rest_target(${psd} base/frameworks/reporter/main.bro)
|
||||
rest_target(${psd} base/frameworks/signatures/main.bro)
|
||||
rest_target(${psd} base/frameworks/software/main.bro)
|
||||
rest_target(${psd} base/protocols/conn/contents.bro)
|
||||
rest_target(${psd} base/protocols/conn/inactivity.bro)
|
||||
rest_target(${psd} base/protocols/conn/main.bro)
|
||||
rest_target(${psd} base/protocols/dns/consts.bro)
|
||||
rest_target(${psd} base/protocols/dns/main.bro)
|
||||
rest_target(${psd} base/protocols/ftp/file-extract.bro)
|
||||
rest_target(${psd} base/protocols/ftp/main.bro)
|
||||
rest_target(${psd} base/protocols/ftp/utils-commands.bro)
|
||||
rest_target(${psd} base/protocols/http/file-extract.bro)
|
||||
rest_target(${psd} base/protocols/http/file-hash.bro)
|
||||
rest_target(${psd} base/protocols/http/file-ident.bro)
|
||||
rest_target(${psd} base/protocols/http/main.bro)
|
||||
rest_target(${psd} base/protocols/http/utils.bro)
|
||||
rest_target(${psd} base/protocols/irc/dcc-send.bro)
|
||||
rest_target(${psd} base/protocols/irc/main.bro)
|
||||
rest_target(${psd} base/protocols/smtp/entities-excerpt.bro)
|
||||
rest_target(${psd} base/protocols/smtp/entities.bro)
|
||||
rest_target(${psd} base/protocols/smtp/main.bro)
|
||||
rest_target(${psd} base/protocols/ssh/main.bro)
|
||||
rest_target(${psd} base/protocols/ssl/consts.bro)
|
||||
rest_target(${psd} base/protocols/ssl/main.bro)
|
||||
rest_target(${psd} base/protocols/ssl/mozilla-ca-list.bro)
|
||||
rest_target(${psd} base/protocols/syslog/consts.bro)
|
||||
rest_target(${psd} base/protocols/syslog/main.bro)
|
||||
rest_target(${psd} base/utils/addrs.bro)
|
||||
rest_target(${psd} base/utils/conn-ids.bro)
|
||||
rest_target(${psd} base/utils/directions-and-hosts.bro)
|
||||
rest_target(${psd} base/utils/files.bro)
|
||||
rest_target(${psd} base/utils/numbers.bro)
|
||||
rest_target(${psd} base/utils/paths.bro)
|
||||
rest_target(${psd} base/utils/patterns.bro)
|
||||
rest_target(${psd} base/utils/site.bro)
|
||||
rest_target(${psd} base/utils/strings.bro)
|
||||
rest_target(${psd} base/utils/thresholds.bro)
|
||||
rest_target(${psd} policy/frameworks/communication/listen.bro)
|
||||
rest_target(${psd} policy/frameworks/control/controllee.bro)
|
||||
rest_target(${psd} policy/frameworks/control/controller.bro)
|
||||
rest_target(${psd} policy/frameworks/dpd/detect-protocols.bro)
|
||||
rest_target(${psd} policy/frameworks/dpd/packet-segment-logging.bro)
|
||||
rest_target(${psd} policy/frameworks/metrics/conn-example.bro)
|
||||
rest_target(${psd} policy/frameworks/metrics/http-example.bro)
|
||||
rest_target(${psd} policy/frameworks/metrics/ssl-example.bro)
|
||||
rest_target(${psd} policy/frameworks/software/version-changes.bro)
|
||||
rest_target(${psd} policy/frameworks/software/vulnerable.bro)
|
||||
rest_target(${psd} policy/integration/barnyard2/main.bro)
|
||||
rest_target(${psd} policy/integration/barnyard2/types.bro)
|
||||
rest_target(${psd} policy/misc/analysis-groups.bro)
|
||||
rest_target(${psd} policy/misc/capture-loss.bro)
|
||||
rest_target(${psd} policy/misc/loaded-scripts.bro)
|
||||
rest_target(${psd} policy/misc/profiling.bro)
|
||||
rest_target(${psd} policy/misc/stats.bro)
|
||||
rest_target(${psd} policy/misc/trim-trace-file.bro)
|
||||
rest_target(${psd} policy/protocols/conn/known-hosts.bro)
|
||||
rest_target(${psd} policy/protocols/conn/known-services.bro)
|
||||
rest_target(${psd} policy/protocols/conn/weirds.bro)
|
||||
rest_target(${psd} policy/protocols/dns/auth-addl.bro)
|
||||
rest_target(${psd} policy/protocols/dns/detect-external-names.bro)
|
||||
rest_target(${psd} policy/protocols/ftp/detect.bro)
|
||||
rest_target(${psd} policy/protocols/ftp/software.bro)
|
||||
rest_target(${psd} policy/protocols/http/detect-MHR.bro)
|
||||
rest_target(${psd} policy/protocols/http/detect-intel.bro)
|
||||
rest_target(${psd} policy/protocols/http/detect-sqli.bro)
|
||||
rest_target(${psd} policy/protocols/http/detect-webapps.bro)
|
||||
rest_target(${psd} policy/protocols/http/header-names.bro)
|
||||
rest_target(${psd} policy/protocols/http/software-browser-plugins.bro)
|
||||
rest_target(${psd} policy/protocols/http/software.bro)
|
||||
rest_target(${psd} policy/protocols/http/var-extraction-cookies.bro)
|
||||
rest_target(${psd} policy/protocols/http/var-extraction-uri.bro)
|
||||
rest_target(${psd} policy/protocols/smtp/blocklists.bro)
|
||||
rest_target(${psd} policy/protocols/smtp/detect-suspicious-orig.bro)
|
||||
rest_target(${psd} policy/protocols/smtp/software.bro)
|
||||
rest_target(${psd} policy/protocols/ssh/detect-bruteforcing.bro)
|
||||
rest_target(${psd} policy/protocols/ssh/geo-data.bro)
|
||||
rest_target(${psd} policy/protocols/ssh/interesting-hostnames.bro)
|
||||
rest_target(${psd} policy/protocols/ssh/software.bro)
|
||||
rest_target(${psd} policy/protocols/ssl/cert-hash.bro)
|
||||
rest_target(${psd} policy/protocols/ssl/expiring-certs.bro)
|
||||
rest_target(${psd} policy/protocols/ssl/extract-certs-pem.bro)
|
||||
rest_target(${psd} policy/protocols/ssl/known-certs.bro)
|
||||
rest_target(${psd} policy/protocols/ssl/validate-certs.bro)
|
||||
rest_target(${psd} policy/tuning/defaults/packet-fragments.bro)
|
||||
rest_target(${psd} policy/tuning/defaults/warnings.bro)
|
||||
rest_target(${psd} policy/tuning/track-all-assets.bro)
|
||||
rest_target(${psd} site/local-manager.bro)
|
||||
rest_target(${psd} site/local-proxy.bro)
|
||||
rest_target(${psd} site/local-worker.bro)
|
||||
rest_target(${psd} site/local.bro)
|
||||
rest_target(${psd} test-all-policy.bro)
|
||||
56
INSTALL
56
INSTALL
|
|
@ -5,34 +5,44 @@ Installing Bro
|
|||
Prerequisites
|
||||
=============
|
||||
|
||||
Bro relies on the following libraries and tools, which need to be installed
|
||||
Bro requires the following libraries and tools to be installed
|
||||
before you begin:
|
||||
|
||||
* CMake 2.6.3 or greater http://www.cmake.org
|
||||
|
||||
* Libpcap (headers and libraries) http://www.tcpdump.org
|
||||
* Perl (used only during the Bro build process)
|
||||
|
||||
* OpenSSL (headers and libraries) http://www.openssl.org
|
||||
* Libpcap headers and libraries http://www.tcpdump.org
|
||||
|
||||
* SWIG http://www.swig.org
|
||||
* OpenSSL headers and libraries http://www.openssl.org
|
||||
|
||||
* BIND8 headers and libraries
|
||||
|
||||
* Libmagic
|
||||
|
||||
* Libz
|
||||
|
||||
Bro can make uses of some optional libraries if they are found at
|
||||
installation time:
|
||||
* SWIG http://www.swig.org
|
||||
|
||||
* LibGeoIP For geo-locating IP addresses.
|
||||
|
||||
Bro also needs the following tools, but on most systems they will
|
||||
already come preinstalled:
|
||||
|
||||
* Bash (For Bro Control).
|
||||
* BIND8 (headers and libraries)
|
||||
* Bison (GNU Parser Generator)
|
||||
|
||||
* Flex (Fast Lexical Analyzer)
|
||||
* Perl (Used only during the Bro build process)
|
||||
|
||||
* Bash (for BroControl)
|
||||
|
||||
|
||||
Bro can make use of some optional libraries and tools if they are found at
|
||||
build time:
|
||||
|
||||
* LibGeoIP (for geo-locating IP addresses)
|
||||
|
||||
* gperftools (tcmalloc is used to improve memory and CPU usage)
|
||||
|
||||
* sendmail (for BroControl)
|
||||
|
||||
* ipsumdump (for trace-summary) http://www.cs.ucla.edu/~kohler/ipsumdump
|
||||
|
||||
* Ruby executable, library, and headers (for Broccoli Ruby bindings)
|
||||
|
||||
|
||||
Installation
|
||||
|
|
@ -44,7 +54,7 @@ To build and install into ``/usr/local/bro``::
|
|||
make
|
||||
make install
|
||||
|
||||
This will first build Bro into a directory inside the distribution
|
||||
This will first build Bro in a directory inside the distribution
|
||||
called ``build/``, using default build options. It then installs all
|
||||
required files into ``/usr/local/bro``, including the Bro binary in
|
||||
``/usr/local/bro/bin/bro``.
|
||||
|
|
@ -60,22 +70,22 @@ choices unless you are creating such a package.
|
|||
Run ``./configure --help`` for more options.
|
||||
|
||||
Depending on the Bro package you downloaded, there may be auxiliary
|
||||
tools and libraries available in the ``aux/`` directory. All of them
|
||||
except for ``aux/bro-aux`` will also be built and installed by doing
|
||||
``make install``. To install the programs that come in the
|
||||
``aux/bro-aux`` directory, use ``make install-aux``. There are
|
||||
tools and libraries available in the ``aux/`` directory. Some of them
|
||||
will be automatically built and installed along with Bro. There are
|
||||
``--disable-*`` options that can be given to the configure script to
|
||||
turn off unwanted auxiliary projects.
|
||||
turn off unwanted auxiliary projects that would otherwise be installed
|
||||
automatically. Finally, use ``make install-aux`` to install some of
|
||||
the other programs that are in the ``aux/bro-aux`` directory.
|
||||
|
||||
OpenBSD users, please see our `FAQ
|
||||
<http://www.bro-ids.org/documentation/faq.html>` if you are having
|
||||
OpenBSD users, please see our FAQ at
|
||||
http://www.bro-ids.org/documentation/faq.html if you are having
|
||||
problems installing Bro.
|
||||
|
||||
Running Bro
|
||||
===========
|
||||
|
||||
Bro is a complex program and it takes a bit of time to get familiar
|
||||
with it. A good place for newcomers to start is the Quickstart Guide
|
||||
with it. A good place for newcomers to start is the Quick Start Guide
|
||||
at http://www.bro-ids.org/documentation/quickstart.html.
|
||||
|
||||
For developers that wish to run Bro directly from the ``build/``
|
||||
|
|
|
|||
3
Makefile
3
Makefile
|
|
@ -41,6 +41,9 @@ broxygen: configured
|
|||
broxygenclean: configured
|
||||
$(MAKE) -C $(BUILD) $@
|
||||
|
||||
update-doc-sources:
|
||||
./doc/scripts/genDocSourcesList.sh ./doc/scripts/DocSourcesList.cmake
|
||||
|
||||
dist:
|
||||
@rm -rf $(VERSION_FULL) $(VERSION_FULL).tgz
|
||||
@rm -rf $(VERSION_MIN) $(VERSION_MIN).tgz
|
||||
|
|
|
|||
7
NEWS
7
NEWS
|
|
@ -47,6 +47,13 @@ Bro 2.1
|
|||
joint set of events. The `icmp_conn` record got a new boolean field
|
||||
'v6' that indicates whether the ICMP message is v4 or v6.
|
||||
|
||||
- Log postprocessor scripts get an additional argument indicating the
|
||||
type of the log writer in use (e.g., "ascii").
|
||||
|
||||
- BroControl's make-archive-name scripts also receives the writer
|
||||
type, but as it's 2nd(!) argument. If you're using a custom version
|
||||
of that script, you need to adapt it. See the shipped version for
|
||||
details.
|
||||
|
||||
TODO: Extend.
|
||||
|
||||
|
|
|
|||
2
VERSION
2
VERSION
|
|
@ -1 +1 @@
|
|||
2.0-301
|
||||
2.0-372
|
||||
|
|
|
|||
|
|
@ -1 +1 @@
|
|||
Subproject commit 71c37019bc371eb7863fb6aa47a7daa4540f4f1f
|
||||
Subproject commit b4094cb75e0a7769123f7db1f5d73f3f9f1c3977
|
||||
|
|
@ -1 +1 @@
|
|||
Subproject commit d885987e7968669e34504b0403ac89bd13928e9a
|
||||
Subproject commit 2038e3de042115c3caa706426e16c830c1fd1e9e
|
||||
|
|
@ -1 +1 @@
|
|||
Subproject commit bead1168ae9c2d2ae216dd58522fbc05498ff2c8
|
||||
Subproject commit 95c93494d7192f69d30f208c4caa3bd38adda6fd
|
||||
|
|
@ -1 +1 @@
|
|||
Subproject commit 44cc3de5f6f98a86b2516bdc48dd168e6a6a28fd
|
||||
Subproject commit ba9e1aa2f2159deac0cf96863f54405643764df0
|
||||
|
|
@ -1 +1 @@
|
|||
Subproject commit 1897d224ce295e91d20e458851759c99734a0a74
|
||||
Subproject commit e0da8d0e284bbebbaef711c91c1b961580f225d2
|
||||
2
cmake
2
cmake
|
|
@ -1 +1 @@
|
|||
Subproject commit 49278736c1404cb8c077272b80312c947e68bf52
|
||||
Subproject commit 96f3d92acadbe1ae64f410e974c5ff503903394b
|
||||
|
|
@ -114,6 +114,9 @@
|
|||
/* Analyze Mobile IPv6 traffic */
|
||||
#cmakedefine ENABLE_MOBILE_IPV6
|
||||
|
||||
/* Use the DataSeries writer. */
|
||||
#cmakedefine USE_DATASERIES
|
||||
|
||||
/* Version number of package */
|
||||
#define VERSION "@VERSION@"
|
||||
|
||||
|
|
|
|||
40
configure
vendored
40
configure
vendored
|
|
@ -24,6 +24,7 @@ Usage: $0 [OPTION]... [VAR=VALUE]...
|
|||
--prefix=PREFIX installation directory [/usr/local/bro]
|
||||
--scriptdir=PATH root installation directory for Bro scripts
|
||||
[PREFIX/share/bro]
|
||||
--conf-files-dir=PATH config files installation directory [PREFIX/etc]
|
||||
|
||||
Optional Features:
|
||||
--enable-debug compile in debugging mode
|
||||
|
|
@ -31,7 +32,7 @@ Usage: $0 [OPTION]... [VAR=VALUE]...
|
|||
--enable-perftools-debug use Google's perftools for debugging
|
||||
--disable-broccoli don't build or install the Broccoli library
|
||||
--disable-broctl don't install Broctl
|
||||
--disable-auxtools don't build or install auxilliary tools
|
||||
--disable-auxtools don't build or install auxiliary tools
|
||||
--disable-python don't try to build python bindings for broccoli
|
||||
--disable-ruby don't try to build ruby bindings for broccoli
|
||||
|
||||
|
|
@ -55,6 +56,8 @@ Usage: $0 [OPTION]... [VAR=VALUE]...
|
|||
--with-ruby-lib=PATH path to ruby library
|
||||
--with-ruby-inc=PATH path to ruby headers
|
||||
--with-swig=PATH path to SWIG executable
|
||||
--with-dataseries=PATH path to DataSeries and Lintel libraries
|
||||
--with-xml2=PATH path to libxml2 installation (for DataSeries)
|
||||
|
||||
Packaging Options (for developers):
|
||||
--binary-package toggle special logic for binary packaging
|
||||
|
|
@ -86,13 +89,15 @@ append_cache_entry () {
|
|||
|
||||
# set defaults
|
||||
builddir=build
|
||||
prefix=/usr/local/bro
|
||||
CMakeCacheEntries=""
|
||||
append_cache_entry CMAKE_INSTALL_PREFIX PATH /usr/local/bro
|
||||
append_cache_entry BRO_ROOT_DIR PATH /usr/local/bro
|
||||
append_cache_entry PY_MOD_INSTALL_DIR PATH /usr/local/bro/lib/broctl
|
||||
append_cache_entry BRO_SCRIPT_INSTALL_PATH STRING /usr/local/bro/share/bro
|
||||
append_cache_entry CMAKE_INSTALL_PREFIX PATH $prefix
|
||||
append_cache_entry BRO_ROOT_DIR PATH $prefix
|
||||
append_cache_entry PY_MOD_INSTALL_DIR PATH $prefix/lib/broctl
|
||||
append_cache_entry BRO_SCRIPT_INSTALL_PATH STRING $prefix/share/bro
|
||||
append_cache_entry BRO_ETC_INSTALL_DIR PATH $prefix/etc
|
||||
append_cache_entry ENABLE_DEBUG BOOL false
|
||||
append_cache_entry ENABLE_PERFTOOLS_DEBUG BOOL false
|
||||
append_cache_entry ENABLE_PERFTOOLS_DEBUG BOOL false
|
||||
append_cache_entry BinPAC_SKIP_INSTALL BOOL true
|
||||
append_cache_entry BUILD_SHARED_LIBS BOOL true
|
||||
append_cache_entry INSTALL_AUX_TOOLS BOOL true
|
||||
|
|
@ -120,17 +125,19 @@ while [ $# -ne 0 ]; do
|
|||
CMakeGenerator="$optarg"
|
||||
;;
|
||||
--prefix=*)
|
||||
prefix=$optarg
|
||||
append_cache_entry CMAKE_INSTALL_PREFIX PATH $optarg
|
||||
append_cache_entry BRO_ROOT_DIR PATH $optarg
|
||||
append_cache_entry PY_MOD_INSTALL_DIR PATH $optarg/lib/broctl
|
||||
if [ "$user_set_scriptdir" != "true" ]; then
|
||||
append_cache_entry BRO_SCRIPT_INSTALL_PATH STRING $optarg/share/bro
|
||||
fi
|
||||
;;
|
||||
--scriptdir=*)
|
||||
append_cache_entry BRO_SCRIPT_INSTALL_PATH STRING $optarg
|
||||
user_set_scriptdir="true"
|
||||
;;
|
||||
--conf-files-dir=*)
|
||||
append_cache_entry BRO_ETC_INSTALL_DIR PATH $optarg
|
||||
user_set_conffilesdir="true"
|
||||
;;
|
||||
--enable-debug)
|
||||
append_cache_entry ENABLE_DEBUG BOOL true
|
||||
;;
|
||||
|
|
@ -208,6 +215,13 @@ while [ $# -ne 0 ]; do
|
|||
--with-swig=*)
|
||||
append_cache_entry SWIG_EXECUTABLE PATH $optarg
|
||||
;;
|
||||
--with-dataseries=*)
|
||||
append_cache_entry DataSeries_ROOT_DIR PATH $optarg
|
||||
append_cache_entry Lintel_ROOT_DIR PATH $optarg
|
||||
;;
|
||||
--with-xml2=*)
|
||||
append_cache_entry LibXML2_ROOT_DIR PATH $optarg
|
||||
;;
|
||||
--binary-package)
|
||||
append_cache_entry BINARY_PACKAGING_MODE BOOL true
|
||||
;;
|
||||
|
|
@ -231,6 +245,14 @@ while [ $# -ne 0 ]; do
|
|||
shift
|
||||
done
|
||||
|
||||
if [ "$user_set_scriptdir" != "true" ]; then
|
||||
append_cache_entry BRO_SCRIPT_INSTALL_PATH STRING $prefix/share/bro
|
||||
fi
|
||||
|
||||
if [ "$user_set_conffilesdir" != "true" ]; then
|
||||
append_cache_entry BRO_ETC_INSTALL_DIR PATH $prefix/etc
|
||||
fi
|
||||
|
||||
if [ -d $builddir ]; then
|
||||
# If build directory exists, check if it has a CMake cache
|
||||
if [ -f $builddir/CMakeCache.txt ]; then
|
||||
|
|
|
|||
168
doc/logging-dataseries.rst
Normal file
168
doc/logging-dataseries.rst
Normal file
|
|
@ -0,0 +1,168 @@
|
|||
|
||||
=============================
|
||||
Binary Output with DataSeries
|
||||
=============================
|
||||
|
||||
.. rst-class:: opening
|
||||
|
||||
Bro's default ASCII log format is not exactly the most efficient
|
||||
way for storing large volumes of data. An an alternative, Bro comes
|
||||
with experimental support for `DataSeries
|
||||
<http://www.hpl.hp.com/techreports/2009/HPL-2009-323.html>`_
|
||||
output, an efficient binary format for recording structured bulk
|
||||
data. DataSeries is developed and maintained at HP Labs.
|
||||
|
||||
.. contents::
|
||||
|
||||
Installing DataSeries
|
||||
---------------------
|
||||
|
||||
To use DataSeries, its libraries must be available at compile-time,
|
||||
along with the supporting *Lintel* package. Generally, both are
|
||||
distributed on `HP Labs' web site
|
||||
<http://tesla.hpl.hp.com/opensource/>`_. Currently, however, you need
|
||||
to use recent developments versions for both packages, which you can
|
||||
download from github like this::
|
||||
|
||||
git clone http://github.com/dataseries/Lintel
|
||||
git clone http://github.com/dataseries/DataSeries
|
||||
|
||||
To build and install the two into ``<prefix>``, do::
|
||||
|
||||
( cd Lintel && mkdir build && cd build && cmake -DCMAKE_INSTALL_PREFIX=<prefix> .. && make && make install )
|
||||
( cd DataSeries && mkdir build && cd build && cmake -DCMAKE_INSTALL_PREFIX=<prefix> .. && make && make install )
|
||||
|
||||
Please refer to the packages' documentation for more information about
|
||||
the installation process. In particular, there's more information on
|
||||
required and optional `dependencies for Lintel
|
||||
<https://raw.github.com/eric-anderson/Lintel/master/doc/dependencies.txt>`_
|
||||
and `dependencies for DataSeries
|
||||
<https://raw.github.com/eric-anderson/DataSeries/master/doc/dependencies.txt>`_
|
||||
|
||||
Compiling Bro with DataSeries Support
|
||||
-------------------------------------
|
||||
|
||||
Once you have installed DataSeries, Bro's ``configure`` should pick it
|
||||
up automatically as long as it finds it in a standard system location.
|
||||
Alternatively, you can specify the DataSeries installation prefix
|
||||
manually with ``--with-dataseries=<prefix>``. Keep an eye on
|
||||
``configure``'s summary output, if it looks like the following, Bro
|
||||
found DataSeries and will compile in the support::
|
||||
|
||||
# ./configure --with-dataseries=/usr/local
|
||||
[...]
|
||||
====================| Bro Build Summary |=====================
|
||||
[...]
|
||||
DataSeries: true
|
||||
[...]
|
||||
================================================================
|
||||
|
||||
Activating DataSeries
|
||||
---------------------
|
||||
|
||||
The direct way to use DataSeries is to switch *all* log files over to
|
||||
the binary format. To do that, just add ``redef
|
||||
Log::default_writer=Log::WRITER_DATASERIES;`` to your ``local.bro``.
|
||||
For testing, you can also just pass that on the command line::
|
||||
|
||||
bro -r trace.pcap Log::default_writer=Log::WRITER_DATASERIES
|
||||
|
||||
With that, Bro will now write all its output into DataSeries files
|
||||
``*.ds``. You can inspect these using DataSeries's set of command line
|
||||
tools, which its installation process installs into ``<prefix>/bin``.
|
||||
For example, to convert a file back into an ASCII representation::
|
||||
|
||||
$ ds2txt conn.log
|
||||
[... We skip a bunch of meta data here ...]
|
||||
ts uid id.orig_h id.orig_p id.resp_h id.resp_p proto service duration orig_bytes resp_bytes conn_state local_orig missed_bytes history orig_pkts orig_ip_bytes resp_pkts resp_ip_bytes
|
||||
1300475167.096535 CRCC5OdDlXe 141.142.220.202 5353 224.0.0.251 5353 udp dns 0.000000 0 0 S0 F 0 D 1 73 0 0
|
||||
1300475167.097012 o7XBsfvo3U1 fe80::217:f2ff:fed7:cf65 5353 ff02::fb 5353 udp 0.000000 0 0 S0 F 0 D 1 199 0 0
|
||||
1300475167.099816 pXPi1kPMgxb 141.142.220.50 5353 224.0.0.251 5353 udp 0.000000 0 0 S0 F 0 D 1 179 0 0
|
||||
1300475168.853899 R7sOc16woCj 141.142.220.118 43927 141.142.2.2 53 udp dns 0.000435 38 89 SF F 0 Dd 1 66 1 117
|
||||
1300475168.854378 Z6dfHVmt0X7 141.142.220.118 37676 141.142.2.2 53 udp dns 0.000420 52 99 SF F 0 Dd 1 80 1 127
|
||||
1300475168.854837 k6T92WxgNAh 141.142.220.118 40526 141.142.2.2 53 udp dns 0.000392 38 183 SF F 0 Dd 1 66 1 211
|
||||
[...]
|
||||
|
||||
(``--skip-all`` suppresses the meta data.)
|
||||
|
||||
Note that the ASCII conversion is *not* equivalent to Bro's default
|
||||
output format.
|
||||
|
||||
You can also switch only individual files over to DataSeries by adding
|
||||
code like this to your ``local.bro``::
|
||||
|
||||
.. code:: bro
|
||||
|
||||
event bro_init()
|
||||
{
|
||||
local f = Log::get_filter(Conn::LOG, "default"); # Get default filter for connection log.
|
||||
f$writer = Log::WRITER_DATASERIES; # Change writer type.
|
||||
Log::add_filter(Conn::LOG, f); # Replace filter with adapted version.
|
||||
}
|
||||
|
||||
Bro's DataSeries writer comes with a few tuning options, see
|
||||
:doc:`scripts/base/frameworks/logging/writers/dataseries`.
|
||||
|
||||
Working with DataSeries
|
||||
=======================
|
||||
|
||||
Here are few examples of using DataSeries command line tools to work
|
||||
with the output files.
|
||||
|
||||
* Printing CSV::
|
||||
|
||||
$ ds2txt --csv conn.log
|
||||
ts,uid,id.orig_h,id.orig_p,id.resp_h,id.resp_p,proto,service,duration,orig_bytes,resp_bytes,conn_state,local_orig,missed_bytes,history,orig_pkts,orig_ip_bytes,resp_pkts,resp_ip_bytes
|
||||
1258790493.773208,ZTtgbHvf4s3,192.168.1.104,137,192.168.1.255,137,udp,dns,3.748891,350,0,S0,F,0,D,7,546,0,0
|
||||
1258790451.402091,pOY6Rw7lhUd,192.168.1.106,138,192.168.1.255,138,udp,,0.000000,0,0,S0,F,0,D,1,229,0,0
|
||||
1258790493.787448,pn5IiEslca9,192.168.1.104,138,192.168.1.255,138,udp,,2.243339,348,0,S0,F,0,D,2,404,0,0
|
||||
1258790615.268111,D9slyIu3hFj,192.168.1.106,137,192.168.1.255,137,udp,dns,3.764626,350,0,S0,F,0,D,7,546,0,0
|
||||
[...]
|
||||
|
||||
Add ``--separator=X`` to set a different separator.
|
||||
|
||||
* Extracting a subset of columns::
|
||||
|
||||
$ ds2txt --select '*' ts,id.resp_h,id.resp_p --skip-all conn.log
|
||||
1258790493.773208 192.168.1.255 137
|
||||
1258790451.402091 192.168.1.255 138
|
||||
1258790493.787448 192.168.1.255 138
|
||||
1258790615.268111 192.168.1.255 137
|
||||
1258790615.289842 192.168.1.255 138
|
||||
[...]
|
||||
|
||||
* Filtering rows::
|
||||
|
||||
$ ds2txt --where '*' 'duration > 5 && id.resp_p > 1024' --skip-all conn.ds
|
||||
1258790631.532888 V8mV5WLITu5 192.168.1.105 55890 239.255.255.250 1900 udp 15.004568 798 0 S0 F 0 D 6 966 0 0
|
||||
1258792413.439596 tMcWVWQptvd 192.168.1.105 55890 239.255.255.250 1900 udp 15.004581 798 0 S0 F 0 D 6 966 0 0
|
||||
1258794195.346127 cQwQMRdBrKa 192.168.1.105 55890 239.255.255.250 1900 udp 15.005071 798 0 S0 F 0 D 6 966 0 0
|
||||
1258795977.253200 i8TEjhWd2W8 192.168.1.105 55890 239.255.255.250 1900 udp 15.004824 798 0 S0 F 0 D 6 966 0 0
|
||||
1258797759.160217 MsLsBA8Ia49 192.168.1.105 55890 239.255.255.250 1900 udp 15.005078 798 0 S0 F 0 D 6 966 0 0
|
||||
1258799541.068452 TsOxRWJRGwf 192.168.1.105 55890 239.255.255.250 1900 udp 15.004082 798 0 S0 F 0 D 6 966 0 0
|
||||
[...]
|
||||
|
||||
* Calculate some statistics:
|
||||
|
||||
Mean/stdev/min/max over a column::
|
||||
|
||||
$ dsstatgroupby '*' basic duration from conn.ds
|
||||
# Begin DSStatGroupByModule
|
||||
# processed 2159 rows, where clause eliminated 0 rows
|
||||
# count(*), mean(duration), stddev, min, max
|
||||
2159, 42.7938, 1858.34, 0, 86370
|
||||
[...]
|
||||
|
||||
Quantiles of total connection volume::
|
||||
|
||||
> dsstatgroupby '*' quantile 'orig_bytes + resp_bytes' from conn.ds
|
||||
[...]
|
||||
2159 data points, mean 24616 +- 343295 [0,1.26615e+07]
|
||||
quantiles about every 216 data points:
|
||||
10%: 0, 124, 317, 348, 350, 350, 601, 798, 1469
|
||||
tails: 90%: 1469, 95%: 7302, 99%: 242629, 99.5%: 1226262
|
||||
[...]
|
||||
|
||||
The ``man`` pages for these tool show further options, and their
|
||||
``-h`` option gives some more information (either can be a bit cryptic
|
||||
unfortunately though).
|
||||
|
|
@ -373,3 +373,13 @@ record, care must be given to when and how long data is stored.
|
|||
Normally data saved to the connection record will remain there for the
|
||||
duration of the connection and from a practical perspective it's not
|
||||
uncommon to need to delete that data before the end of the connection.
|
||||
|
||||
Other Writers
|
||||
-------------
|
||||
|
||||
Bro support the following output formats other than ASCII:
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
|
||||
logging-dataseries
|
||||
|
|
|
|||
|
|
@ -39,6 +39,7 @@ rest_target(${psd} base/frameworks/logging/main.bro)
|
|||
rest_target(${psd} base/frameworks/logging/postprocessors/scp.bro)
|
||||
rest_target(${psd} base/frameworks/logging/postprocessors/sftp.bro)
|
||||
rest_target(${psd} base/frameworks/logging/writers/ascii.bro)
|
||||
rest_target(${psd} base/frameworks/logging/writers/dataseries.bro)
|
||||
rest_target(${psd} base/frameworks/metrics/cluster.bro)
|
||||
rest_target(${psd} base/frameworks/metrics/main.bro)
|
||||
rest_target(${psd} base/frameworks/metrics/non-cluster.bro)
|
||||
|
|
|
|||
|
|
@ -105,5 +105,8 @@ event protocol_violation(c: connection, atype: count, aid: count,
|
|||
reason: string) &priority=-5
|
||||
{
|
||||
if ( c?$dpd )
|
||||
{
|
||||
Log::write(DPD::LOG, c$dpd);
|
||||
delete c$dpd;
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -1,3 +1,4 @@
|
|||
@load ./main
|
||||
@load ./postprocessors
|
||||
@load ./writers/ascii
|
||||
@load ./writers/dataseries
|
||||
|
|
|
|||
|
|
@ -376,13 +376,16 @@ function run_rotation_postprocessor_cmd(info: RotationInfo, npath: string) : boo
|
|||
if ( pp_cmd == "" )
|
||||
return T;
|
||||
|
||||
# Turn, e.g., Log::WRITER_ASCII into "ascii".
|
||||
local writer = subst_string(to_lower(fmt("%s", info$writer)), "log::writer_", "");
|
||||
|
||||
# The date format is hard-coded here to provide a standardized
|
||||
# script interface.
|
||||
system(fmt("%s %s %s %s %s %d",
|
||||
system(fmt("%s %s %s %s %s %d %s",
|
||||
pp_cmd, npath, info$path,
|
||||
strftime("%y-%m-%d_%H.%M.%S", info$open),
|
||||
strftime("%y-%m-%d_%H.%M.%S", info$close),
|
||||
info$terminating));
|
||||
info$terminating, writer));
|
||||
|
||||
return T;
|
||||
}
|
||||
|
|
|
|||
60
scripts/base/frameworks/logging/writers/dataseries.bro
Normal file
60
scripts/base/frameworks/logging/writers/dataseries.bro
Normal file
|
|
@ -0,0 +1,60 @@
|
|||
##! Interface for the DataSeries log writer.
|
||||
|
||||
module LogDataSeries;
|
||||
|
||||
export {
|
||||
## Compression to use with the DS output file. Options are:
|
||||
##
|
||||
## 'none' -- No compression.
|
||||
## 'lzf' -- LZF compression. Very quick, but leads to larger output files.
|
||||
## 'lzo' -- LZO compression. Very fast decompression times.
|
||||
## 'gz' -- GZIP compression. Slower than LZF, but also produces smaller output.
|
||||
## 'bz2' -- BZIP2 compression. Slower than GZIP, but also produces smaller output.
|
||||
const compression = "lzo" &redef;
|
||||
|
||||
## The extent buffer size.
|
||||
## Larger values here lead to better compression and more efficient writes, but
|
||||
## also increase the lag between the time events are received and the time they
|
||||
## are actually written to disk.
|
||||
const extent_size = 65536 &redef;
|
||||
|
||||
## Should we dump the XML schema we use for this DS file to disk?
|
||||
## If yes, the XML schema shares the name of the logfile, but has
|
||||
## an XML ending.
|
||||
const dump_schema = F &redef;
|
||||
|
||||
## How many threads should DataSeries spawn to perform compression?
|
||||
## Note that this dictates the number of threads per log stream. If
|
||||
## you're using a lot of streams, you may want to keep this number
|
||||
## relatively small.
|
||||
##
|
||||
## Default value is 1, which will spawn one thread / stream.
|
||||
##
|
||||
## Maximum is 128, minimum is 1.
|
||||
const num_threads = 1 &redef;
|
||||
|
||||
## Should time be stored as an integer or a double?
|
||||
## Storing time as a double leads to possible precision issues and
|
||||
## can (significantly) increase the size of the resulting DS log.
|
||||
## That said, timestamps stored in double form are consistent
|
||||
## with the rest of Bro, including the standard ASCII log. Hence, we
|
||||
## use them by default.
|
||||
const use_integer_for_time = F &redef;
|
||||
}
|
||||
|
||||
# Default function to postprocess a rotated DataSeries log file. It moves the
|
||||
# rotated file to a new name that includes a timestamp with the opening time, and
|
||||
# then runs the writer's default postprocessor command on it.
|
||||
function default_rotation_postprocessor_func(info: Log::RotationInfo) : bool
|
||||
{
|
||||
# Move file to name including both opening and closing time.
|
||||
local dst = fmt("%s.%s.ds", info$path,
|
||||
strftime(Log::default_rotation_date_format, info$open));
|
||||
|
||||
system(fmt("/bin/mv %s %s", info$fname, dst));
|
||||
|
||||
# Run default postprocessor.
|
||||
return Log::run_rotation_postprocessor_cmd(info, dst);
|
||||
}
|
||||
|
||||
redef Log::default_rotation_postprocessors += { [Log::WRITER_DATASERIES] = default_rotation_postprocessor_func };
|
||||
|
|
@ -23,7 +23,10 @@ redef Cluster::worker2manager_events += /Notice::cluster_notice/;
|
|||
@if ( Cluster::local_node_type() != Cluster::MANAGER )
|
||||
# The notice policy is completely handled by the manager and shouldn't be
|
||||
# done by workers or proxies to save time for packet processing.
|
||||
redef policy = {};
|
||||
event bro_init() &priority=-11
|
||||
{
|
||||
Notice::policy = table();
|
||||
}
|
||||
|
||||
event Notice::begin_suppression(n: Notice::Info)
|
||||
{
|
||||
|
|
|
|||
|
|
@ -92,6 +92,7 @@ type icmp_conn: record {
|
|||
itype: count; ##< The ICMP type of the packet that triggered the instantiation of the record.
|
||||
icode: count; ##< The ICMP code of the packet that triggered the instantiation of the record.
|
||||
len: count; ##< The length of the ICMP payload of the packet that triggered the instantiation of the record.
|
||||
hlim: count; ##< The encapsulating IP header's Hop Limit value.
|
||||
v6: bool; ##< True if it's an ICMPv6 packet.
|
||||
};
|
||||
|
||||
|
|
@ -2336,6 +2337,11 @@ type bt_tracker_headers: table[string] of string;
|
|||
## BPF filter the user has set via the -f command line options. Empty if none.
|
||||
const cmd_line_bpf_filter = "" &redef;
|
||||
|
||||
## The maximum number of open files to keep cached at a given time.
|
||||
## If set to zero, this is automatically determined by inspecting
|
||||
## the current/maximum limit on open files for the process.
|
||||
const max_files_in_cache = 0 &redef;
|
||||
|
||||
## Deprecated.
|
||||
const log_rotate_interval = 0 sec &redef;
|
||||
|
||||
|
|
|
|||
|
|
@ -6,6 +6,7 @@
|
|||
@load ./utils-commands
|
||||
@load base/utils/paths
|
||||
@load base/utils/numbers
|
||||
@load base/utils/addrs
|
||||
|
||||
module FTP;
|
||||
|
||||
|
|
@ -22,7 +23,7 @@ export {
|
|||
const default_capture_password = F &redef;
|
||||
|
||||
## User IDs that can be considered "anonymous".
|
||||
const guest_ids = { "anonymous", "ftp", "guest" } &redef;
|
||||
const guest_ids = { "anonymous", "ftp", "ftpuser", "guest" } &redef;
|
||||
|
||||
type Info: record {
|
||||
## Time when the command was sent.
|
||||
|
|
@ -160,17 +161,16 @@ function ftp_message(s: Info)
|
|||
# or it's a deliberately logged command.
|
||||
if ( |s$tags| > 0 || (s?$cmdarg && s$cmdarg$cmd in logged_commands) )
|
||||
{
|
||||
if ( s?$password && to_lower(s$user) !in guest_ids )
|
||||
if ( s?$password &&
|
||||
! s$capture_password &&
|
||||
to_lower(s$user) !in guest_ids )
|
||||
{
|
||||
s$password = "<hidden>";
|
||||
}
|
||||
|
||||
local arg = s$cmdarg$arg;
|
||||
if ( s$cmdarg$cmd in file_cmds )
|
||||
{
|
||||
if ( is_v4_addr(s$id$resp_h) )
|
||||
arg = fmt("ftp://%s%s", s$id$resp_h, build_path_compressed(s$cwd, arg));
|
||||
else
|
||||
arg = fmt("ftp://[%s]%s", s$id$resp_h, build_path_compressed(s$cwd, arg));
|
||||
}
|
||||
arg = fmt("ftp://%s%s", addr_to_uri(s$id$resp_h), build_path_compressed(s$cwd, arg));
|
||||
|
||||
s$ts=s$cmdarg$ts;
|
||||
s$command=s$cmdarg$cmd;
|
||||
|
|
|
|||
|
|
@ -1,6 +1,7 @@
|
|||
##! Utilities specific for HTTP processing.
|
||||
|
||||
@load ./main
|
||||
@load base/utils/addrs
|
||||
|
||||
module HTTP;
|
||||
|
||||
|
|
@ -51,7 +52,7 @@ function extract_keys(data: string, kv_splitter: pattern): string_vec
|
|||
function build_url(rec: Info): string
|
||||
{
|
||||
local uri = rec?$uri ? rec$uri : "/<missed_request>";
|
||||
local host = rec?$host ? rec$host : fmt("%s", rec$id$resp_h);
|
||||
local host = rec?$host ? rec$host : addr_to_uri(rec$id$resp_h);
|
||||
if ( rec$id$resp_p != 80/tcp )
|
||||
host = fmt("%s:%s", host, rec$id$resp_p);
|
||||
return fmt("%s%s", host, uri);
|
||||
|
|
|
|||
|
|
@ -77,8 +77,12 @@ export {
|
|||
[12] = "srp",
|
||||
[13] = "signature_algorithms",
|
||||
[14] = "use_srtp",
|
||||
[15] = "heartbeat",
|
||||
[35] = "SessionTicket TLS",
|
||||
[40] = "extended_random",
|
||||
[13172] = "next_protocol_negotiation",
|
||||
[13175] = "origin_bound_certificates",
|
||||
[13180] = "encrypted_client_certificates",
|
||||
[65281] = "renegotiation_info"
|
||||
} &default=function(i: count):string { return fmt("unknown-%d", i); };
|
||||
|
||||
|
|
|
|||
|
|
@ -24,6 +24,8 @@ export {
|
|||
session_id: string &log &optional;
|
||||
## Subject of the X.509 certificate offered by the server.
|
||||
subject: string &log &optional;
|
||||
## Subject of the signer of the X.509 certificate offered by the server.
|
||||
issuer_subject: string &log &optional;
|
||||
## NotValidBefore field value from the server certificate.
|
||||
not_valid_before: time &log &optional;
|
||||
## NotValidAfter field value from the serve certificate.
|
||||
|
|
@ -146,6 +148,7 @@ event x509_certificate(c: connection, is_orig: bool, cert: X509, chain_idx: coun
|
|||
|
||||
# Also save other certificate information about the primary cert.
|
||||
c$ssl$subject = cert$subject;
|
||||
c$ssl$issuer_subject = cert$issuer;
|
||||
c$ssl$not_valid_before = cert$not_valid_before;
|
||||
c$ssl$not_valid_after = cert$not_valid_after;
|
||||
}
|
||||
|
|
|
|||
|
|
@ -98,3 +98,18 @@ function find_ip_addresses(input: string): string_array
|
|||
}
|
||||
return output;
|
||||
}
|
||||
|
||||
## Returns the string representation of an IP address suitable for inclusion
|
||||
## in a URI. For IPv4, this does no special formatting, but for IPv6, the
|
||||
## address is included in square brackets.
|
||||
##
|
||||
## a: the address to make suitable for URI inclusion.
|
||||
##
|
||||
## Returns: the string representation of *a* suitable for URI inclusion.
|
||||
function addr_to_uri(a: addr): string
|
||||
{
|
||||
if ( is_v4_addr(a) )
|
||||
return fmt("%s", a);
|
||||
else
|
||||
return fmt("[%s]", a);
|
||||
}
|
||||
|
|
|
|||
|
|
@ -1,10 +1,11 @@
|
|||
@load ./addrs
|
||||
|
||||
## This function can be used to generate a consistent filename for when
|
||||
## contents of a file, stream, or connection are being extracted to disk.
|
||||
function generate_extraction_filename(prefix: string, c: connection, suffix: string): string
|
||||
{
|
||||
local conn_info = fmt("%s:%d-%s:%d",
|
||||
c$id$orig_h, c$id$orig_p, c$id$resp_h, c$id$resp_p);
|
||||
local conn_info = fmt("%s:%d-%s:%d", addr_to_uri(c$id$orig_h), c$id$orig_p,
|
||||
addr_to_uri(c$id$resp_h), c$id$resp_p);
|
||||
|
||||
if ( prefix != "" )
|
||||
conn_info = fmt("%s_%s", prefix, conn_info);
|
||||
|
|
|
|||
|
|
@ -10,15 +10,19 @@ export {
|
|||
const private_address_space: set[subnet] = {
|
||||
10.0.0.0/8,
|
||||
192.168.0.0/16,
|
||||
172.16.0.0/12,
|
||||
100.64.0.0/10, # RFC6598 Carrier Grade NAT
|
||||
127.0.0.0/8,
|
||||
172.16.0.0/12
|
||||
[fe80::]/10,
|
||||
[::1]/128,
|
||||
} &redef;
|
||||
|
||||
## Networks that are considered "local".
|
||||
const local_nets: set[subnet] &redef;
|
||||
|
||||
## This is used for retrieving the subnet when you multiple
|
||||
## :bro:id:`Site::local_nets`. A membership query can be done with an
|
||||
## This is used for retrieving the subnet when using multiple entries in
|
||||
## :bro:id:`Site::local_nets`. It's populated automatically from there.
|
||||
## A membership query can be done with an
|
||||
## :bro:type:`addr` and the table will yield the subnet it was found
|
||||
## within.
|
||||
global local_nets_table: table[subnet] of subnet = {};
|
||||
|
|
@ -40,27 +44,33 @@ export {
|
|||
|
||||
## Function that returns true if an address corresponds to one of
|
||||
## the local networks, false if not.
|
||||
## The function inspects :bro:id:`Site::local_nets`.
|
||||
global is_local_addr: function(a: addr): bool;
|
||||
|
||||
## Function that returns true if an address corresponds to one of
|
||||
## the neighbor networks, false if not.
|
||||
## The function inspects :bro:id:`Site::neighbor_nets`.
|
||||
global is_neighbor_addr: function(a: addr): bool;
|
||||
|
||||
## Function that returns true if an address corresponds to one of
|
||||
## the private/unrouted networks, false if not.
|
||||
## The function inspects :bro:id:`Site::private_address_space`.
|
||||
global is_private_addr: function(a: addr): bool;
|
||||
|
||||
## Function that returns true if a host name is within a local
|
||||
## DNS zone.
|
||||
## The function inspects :bro:id:`Site::local_zones`.
|
||||
global is_local_name: function(name: string): bool;
|
||||
|
||||
## Function that returns true if a host name is within a neighbor
|
||||
## DNS zone.
|
||||
## The function inspects :bro:id:`Site::neighbor_zones`.
|
||||
global is_neighbor_name: function(name: string): bool;
|
||||
|
||||
## Function that returns a common separated list of email addresses
|
||||
## that are considered administrators for the IP address provided as
|
||||
## an argument.
|
||||
## The function inspects :bro:id:`Site::local_admins`.
|
||||
global get_emails: function(a: addr): string;
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -34,7 +34,7 @@
|
|||
#include "Portmap.h"
|
||||
#include "POP3.h"
|
||||
#include "SSH.h"
|
||||
#include "SSL-binpac.h"
|
||||
#include "SSL.h"
|
||||
#include "Syslog-binpac.h"
|
||||
#include "ConnSizeAnalyzer.h"
|
||||
|
||||
|
|
@ -121,8 +121,8 @@ const Analyzer::Config Analyzer::analyzer_configs[] = {
|
|||
HTTP_Analyzer_binpac::InstantiateAnalyzer,
|
||||
HTTP_Analyzer_binpac::Available, 0, false },
|
||||
{ AnalyzerTag::SSL, "SSL",
|
||||
SSL_Analyzer_binpac::InstantiateAnalyzer,
|
||||
SSL_Analyzer_binpac::Available, 0, false },
|
||||
SSL_Analyzer::InstantiateAnalyzer,
|
||||
SSL_Analyzer::Available, 0, false },
|
||||
{ AnalyzerTag::SYSLOG_BINPAC, "SYSLOG_BINPAC",
|
||||
Syslog_Analyzer_binpac::InstantiateAnalyzer,
|
||||
Syslog_Analyzer_binpac::Available, 0, false },
|
||||
|
|
|
|||
|
|
@ -5,7 +5,6 @@
|
|||
|
||||
#include "util.h"
|
||||
#include "net_util.h"
|
||||
#include "md5.h"
|
||||
#include "Anon.h"
|
||||
#include "Val.h"
|
||||
#include "NetVar.h"
|
||||
|
|
|
|||
|
|
@ -248,7 +248,6 @@ add_custom_command(OUTPUT ${CMAKE_CURRENT_BINARY_DIR}/DebugCmdConstants.h
|
|||
WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}
|
||||
)
|
||||
|
||||
set(dns_SRCS nb_dns.c)
|
||||
set_source_files_properties(nb_dns.c PROPERTIES COMPILE_FLAGS
|
||||
-fno-strict-aliasing)
|
||||
|
||||
|
|
@ -377,7 +376,7 @@ set(bro_SRCS
|
|||
SMB.cc
|
||||
SMTP.cc
|
||||
SSH.cc
|
||||
SSL-binpac.cc
|
||||
SSL.cc
|
||||
Scope.cc
|
||||
SerializationFormat.cc
|
||||
SerialObj.cc
|
||||
|
|
@ -404,7 +403,6 @@ set(bro_SRCS
|
|||
bsd-getopt-long.c
|
||||
bro_inet_ntop.c
|
||||
cq.c
|
||||
md5.c
|
||||
patricia.c
|
||||
setsignal.c
|
||||
PacketDumper.cc
|
||||
|
|
@ -420,6 +418,7 @@ set(bro_SRCS
|
|||
logging/WriterBackend.cc
|
||||
logging/WriterFrontend.cc
|
||||
logging/writers/Ascii.cc
|
||||
logging/writers/DataSeries.cc
|
||||
logging/writers/None.cc
|
||||
|
||||
input/Manager.cc
|
||||
|
|
@ -429,8 +428,8 @@ set(bro_SRCS
|
|||
input/readers/Raw.cc
|
||||
input/readers/Benchmark.cc
|
||||
|
||||
${dns_SRCS}
|
||||
${openssl_SRCS}
|
||||
nb_dns.c
|
||||
digest.h
|
||||
)
|
||||
|
||||
collect_headers(bro_HEADERS ${bro_SRCS})
|
||||
|
|
|
|||
15
src/DFA.cc
15
src/DFA.cc
|
|
@ -2,9 +2,10 @@
|
|||
|
||||
#include "config.h"
|
||||
|
||||
#include <openssl/md5.h>
|
||||
|
||||
#include "EquivClass.h"
|
||||
#include "DFA.h"
|
||||
#include "md5.h"
|
||||
|
||||
int dfa_state_cache_size = 10000;
|
||||
|
||||
|
|
@ -312,8 +313,8 @@ DFA_State* DFA_State_Cache::Lookup(const NFA_state_list& nfas,
|
|||
{
|
||||
// We assume that state ID's don't exceed 10 digits, plus
|
||||
// we allow one more character for the delimiter.
|
||||
md5_byte_t id_tag[nfas.length() * 11 + 1];
|
||||
md5_byte_t* p = id_tag;
|
||||
u_char id_tag[nfas.length() * 11 + 1];
|
||||
u_char* p = id_tag;
|
||||
|
||||
for ( int i = 0; i < nfas.length(); ++i )
|
||||
{
|
||||
|
|
@ -335,12 +336,8 @@ DFA_State* DFA_State_Cache::Lookup(const NFA_state_list& nfas,
|
|||
|
||||
// We use the short MD5 instead of the full string for the
|
||||
// HashKey because the data is copied into the key.
|
||||
md5_state_t state;
|
||||
md5_byte_t digest[16];
|
||||
|
||||
md5_init(&state);
|
||||
md5_append(&state, id_tag, p - id_tag);
|
||||
md5_finish(&state, digest);
|
||||
u_char digest[16];
|
||||
MD5(id_tag, p - id_tag, digest);
|
||||
|
||||
*hash = new HashKey(&digest, sizeof(digest));
|
||||
CacheEntry* e = states.Lookup(*hash);
|
||||
|
|
|
|||
24
src/File.cc
24
src/File.cc
|
|
@ -74,9 +74,8 @@ void RotateTimer::Dispatch(double t, int is_expire)
|
|||
|
||||
// The following could in principle be part of a "file manager" object.
|
||||
|
||||
#define MAX_FILE_CACHE_SIZE 32
|
||||
#define MAX_FILE_CACHE_SIZE 512
|
||||
static int num_files_in_cache = 0;
|
||||
static int max_files_in_cache = 0;
|
||||
static BroFile* head = 0;
|
||||
static BroFile* tail = 0;
|
||||
|
||||
|
|
@ -87,12 +86,9 @@ double BroFile::default_rotation_size = 0;
|
|||
// that we should use for the cache.
|
||||
static int maximize_num_fds()
|
||||
{
|
||||
#ifdef NO_HAVE_SETRLIMIT
|
||||
return MAX_FILE_CACHE_SIZE;
|
||||
#else
|
||||
struct rlimit rl;
|
||||
if ( getrlimit(RLIMIT_NOFILE, &rl) < 0 )
|
||||
reporter->InternalError("maximize_num_fds(): getrlimit failed");
|
||||
reporter->FatalError("maximize_num_fds(): getrlimit failed");
|
||||
|
||||
if ( rl.rlim_max == RLIM_INFINITY )
|
||||
{
|
||||
|
|
@ -108,10 +104,9 @@ static int maximize_num_fds()
|
|||
rl.rlim_cur = rl.rlim_max;
|
||||
|
||||
if ( setrlimit(RLIMIT_NOFILE, &rl) < 0 )
|
||||
reporter->InternalError("maximize_num_fds(): setrlimit failed");
|
||||
reporter->FatalError("maximize_num_fds(): setrlimit failed");
|
||||
|
||||
return rl.rlim_cur / 2;
|
||||
#endif
|
||||
}
|
||||
|
||||
|
||||
|
|
@ -172,7 +167,7 @@ const char* BroFile::Name() const
|
|||
return 0;
|
||||
}
|
||||
|
||||
bool BroFile::Open(FILE* file)
|
||||
bool BroFile::Open(FILE* file, const char* mode)
|
||||
{
|
||||
open_time = network_time ? network_time : current_time();
|
||||
|
||||
|
|
@ -196,7 +191,12 @@ bool BroFile::Open(FILE* file)
|
|||
InstallRotateTimer();
|
||||
|
||||
if ( ! f )
|
||||
f = fopen(name, access);
|
||||
{
|
||||
if ( ! mode )
|
||||
f = fopen(name, access);
|
||||
else
|
||||
f = fopen(name, mode);
|
||||
}
|
||||
|
||||
SetBuf(buffered);
|
||||
|
||||
|
|
@ -846,8 +846,8 @@ BroFile* BroFile::Unserialize(UnserialInfo* info)
|
|||
}
|
||||
}
|
||||
|
||||
// Otherwise, open.
|
||||
if ( ! file->Open() )
|
||||
// Otherwise, open, but don't clobber.
|
||||
if ( ! file->Open(0, "a") )
|
||||
{
|
||||
info->s->Error(fmt("cannot open %s: %s",
|
||||
file->name, strerror(errno)));
|
||||
|
|
|
|||
|
|
@ -87,7 +87,13 @@ protected:
|
|||
|
||||
BroFile() { Init(); }
|
||||
void Init();
|
||||
bool Open(FILE* f = 0); // if file is given, it's an open file to use
|
||||
|
||||
/**
|
||||
* If file is given, it's an open file to use already.
|
||||
* If file is not given and mode is, the filename will be opened with that
|
||||
* access mode.
|
||||
*/
|
||||
bool Open(FILE* f = 0, const char* mode = 0);
|
||||
|
||||
BroFile* Prev() { return prev; }
|
||||
BroFile* Next() { return next; }
|
||||
|
|
|
|||
|
|
@ -29,7 +29,6 @@
|
|||
|
||||
#include <algorithm>
|
||||
|
||||
#include "md5.h"
|
||||
#include "Base64.h"
|
||||
#include "Stmt.h"
|
||||
#include "Scope.h"
|
||||
|
|
|
|||
71
src/ICMP.cc
71
src/ICMP.cc
|
|
@ -131,7 +131,7 @@ void ICMP_Analyzer::NextICMP4(double t, const struct icmp* icmpp, int len, int c
|
|||
break;
|
||||
|
||||
default:
|
||||
ICMPEvent(icmp_sent, icmpp, len, 0);
|
||||
ICMPEvent(icmp_sent, icmpp, len, 0, ip_hdr);
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
|
@ -181,23 +181,31 @@ void ICMP_Analyzer::NextICMP6(double t, const struct icmp* icmpp, int len, int c
|
|||
case MLD_LISTENER_REDUCTION:
|
||||
#endif
|
||||
default:
|
||||
ICMPEvent(icmp_sent, icmpp, len, 1);
|
||||
// Error messages (i.e., ICMPv6 type < 128) all have
|
||||
// the same structure for their context, and are
|
||||
// handled by the same function.
|
||||
if ( icmpp->icmp_type < 128 )
|
||||
Context6(t, icmpp, len, caplen, data, ip_hdr);
|
||||
else
|
||||
ICMPEvent(icmp_sent, icmpp, len, 1, ip_hdr);
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
void ICMP_Analyzer::ICMPEvent(EventHandlerPtr f, const struct icmp* icmpp, int len, int icmpv6)
|
||||
void ICMP_Analyzer::ICMPEvent(EventHandlerPtr f, const struct icmp* icmpp,
|
||||
int len, int icmpv6, const IP_Hdr* ip_hdr)
|
||||
{
|
||||
if ( ! f )
|
||||
return;
|
||||
|
||||
val_list* vl = new val_list;
|
||||
vl->append(BuildConnVal());
|
||||
vl->append(BuildICMPVal(icmpp, len, icmpv6));
|
||||
vl->append(BuildICMPVal(icmpp, len, icmpv6, ip_hdr));
|
||||
ConnectionEvent(f, vl);
|
||||
}
|
||||
|
||||
RecordVal* ICMP_Analyzer::BuildICMPVal(const struct icmp* icmpp, int len, int icmpv6)
|
||||
RecordVal* ICMP_Analyzer::BuildICMPVal(const struct icmp* icmpp, int len,
|
||||
int icmpv6, const IP_Hdr* ip_hdr)
|
||||
{
|
||||
if ( ! icmp_conn_val )
|
||||
{
|
||||
|
|
@ -208,7 +216,8 @@ RecordVal* ICMP_Analyzer::BuildICMPVal(const struct icmp* icmpp, int len, int ic
|
|||
icmp_conn_val->Assign(2, new Val(icmpp->icmp_type, TYPE_COUNT));
|
||||
icmp_conn_val->Assign(3, new Val(icmpp->icmp_code, TYPE_COUNT));
|
||||
icmp_conn_val->Assign(4, new Val(len, TYPE_COUNT));
|
||||
icmp_conn_val->Assign(5, new Val(icmpv6, TYPE_BOOL));
|
||||
icmp_conn_val->Assign(5, new Val(ip_hdr->TTL(), TYPE_COUNT));
|
||||
icmp_conn_val->Assign(6, new Val(icmpv6, TYPE_BOOL));
|
||||
}
|
||||
|
||||
Ref(icmp_conn_val);
|
||||
|
|
@ -494,7 +503,7 @@ void ICMP_Analyzer::Echo(double t, const struct icmp* icmpp, int len,
|
|||
|
||||
val_list* vl = new val_list;
|
||||
vl->append(BuildConnVal());
|
||||
vl->append(BuildICMPVal(icmpp, len, ip_hdr->NextProto() != IPPROTO_ICMP));
|
||||
vl->append(BuildICMPVal(icmpp, len, ip_hdr->NextProto() != IPPROTO_ICMP, ip_hdr));
|
||||
vl->append(new Val(iid, TYPE_COUNT));
|
||||
vl->append(new Val(iseq, TYPE_COUNT));
|
||||
vl->append(new StringVal(payload));
|
||||
|
|
@ -504,7 +513,7 @@ void ICMP_Analyzer::Echo(double t, const struct icmp* icmpp, int len,
|
|||
|
||||
|
||||
void ICMP_Analyzer::RouterAdvert(double t, const struct icmp* icmpp, int len,
|
||||
int caplen, const u_char*& data, const IP_Hdr* /*ip_hdr*/)
|
||||
int caplen, const u_char*& data, const IP_Hdr* ip_hdr)
|
||||
{
|
||||
EventHandlerPtr f = icmp_router_advertisement;
|
||||
uint32 reachable, retrans;
|
||||
|
|
@ -514,19 +523,24 @@ void ICMP_Analyzer::RouterAdvert(double t, const struct icmp* icmpp, int len,
|
|||
|
||||
val_list* vl = new val_list;
|
||||
vl->append(BuildConnVal());
|
||||
vl->append(BuildICMPVal(icmpp, len, 1));
|
||||
vl->append(new Val(icmpp->icmp_num_addrs, TYPE_COUNT));
|
||||
vl->append(new Val(icmpp->icmp_wpa & 0x80, TYPE_BOOL));
|
||||
vl->append(new Val(htons(icmpp->icmp_lifetime), TYPE_COUNT));
|
||||
vl->append(new Val(reachable, TYPE_INTERVAL));
|
||||
vl->append(new Val(retrans, TYPE_INTERVAL));
|
||||
vl->append(BuildICMPVal(icmpp, len, 1, ip_hdr));
|
||||
vl->append(new Val(icmpp->icmp_num_addrs, TYPE_COUNT)); // Cur Hop Limit
|
||||
vl->append(new Val(icmpp->icmp_wpa & 0x80, TYPE_BOOL)); // Managed
|
||||
vl->append(new Val(icmpp->icmp_wpa & 0x40, TYPE_BOOL)); // Other
|
||||
vl->append(new Val(icmpp->icmp_wpa & 0x20, TYPE_BOOL)); // Home Agent
|
||||
vl->append(new Val((icmpp->icmp_wpa & 0x18)>>3, TYPE_COUNT)); // Pref
|
||||
vl->append(new Val(icmpp->icmp_wpa & 0x04, TYPE_BOOL)); // Proxy
|
||||
vl->append(new Val(icmpp->icmp_wpa & 0x02, TYPE_COUNT)); // Reserved
|
||||
vl->append(new IntervalVal((double)ntohs(icmpp->icmp_lifetime), Seconds));
|
||||
vl->append(new IntervalVal((double)ntohl(reachable), Milliseconds));
|
||||
vl->append(new IntervalVal((double)ntohl(retrans), Milliseconds));
|
||||
|
||||
ConnectionEvent(f, vl);
|
||||
}
|
||||
|
||||
|
||||
void ICMP_Analyzer::NeighborAdvert(double t, const struct icmp* icmpp, int len,
|
||||
int caplen, const u_char*& data, const IP_Hdr* /*ip_hdr*/)
|
||||
int caplen, const u_char*& data, const IP_Hdr* ip_hdr)
|
||||
{
|
||||
EventHandlerPtr f = icmp_neighbor_advertisement;
|
||||
in6_addr tgtaddr;
|
||||
|
|
@ -535,7 +549,10 @@ void ICMP_Analyzer::NeighborAdvert(double t, const struct icmp* icmpp, int len,
|
|||
|
||||
val_list* vl = new val_list;
|
||||
vl->append(BuildConnVal());
|
||||
vl->append(BuildICMPVal(icmpp, len, 1));
|
||||
vl->append(BuildICMPVal(icmpp, len, 1, ip_hdr));
|
||||
vl->append(new Val(icmpp->icmp_num_addrs & 0x80, TYPE_BOOL)); // Router
|
||||
vl->append(new Val(icmpp->icmp_num_addrs & 0x40, TYPE_BOOL)); // Solicited
|
||||
vl->append(new Val(icmpp->icmp_num_addrs & 0x20, TYPE_BOOL)); // Override
|
||||
vl->append(new AddrVal(IPAddr(tgtaddr)));
|
||||
|
||||
ConnectionEvent(f, vl);
|
||||
|
|
@ -543,7 +560,7 @@ void ICMP_Analyzer::NeighborAdvert(double t, const struct icmp* icmpp, int len,
|
|||
|
||||
|
||||
void ICMP_Analyzer::NeighborSolicit(double t, const struct icmp* icmpp, int len,
|
||||
int caplen, const u_char*& data, const IP_Hdr* /*ip_hdr*/)
|
||||
int caplen, const u_char*& data, const IP_Hdr* ip_hdr)
|
||||
{
|
||||
EventHandlerPtr f = icmp_neighbor_solicitation;
|
||||
in6_addr tgtaddr;
|
||||
|
|
@ -552,7 +569,7 @@ void ICMP_Analyzer::NeighborSolicit(double t, const struct icmp* icmpp, int len,
|
|||
|
||||
val_list* vl = new val_list;
|
||||
vl->append(BuildConnVal());
|
||||
vl->append(BuildICMPVal(icmpp, len, 1));
|
||||
vl->append(BuildICMPVal(icmpp, len, 1, ip_hdr));
|
||||
vl->append(new AddrVal(IPAddr(tgtaddr)));
|
||||
|
||||
ConnectionEvent(f, vl);
|
||||
|
|
@ -560,7 +577,7 @@ void ICMP_Analyzer::NeighborSolicit(double t, const struct icmp* icmpp, int len,
|
|||
|
||||
|
||||
void ICMP_Analyzer::Redirect(double t, const struct icmp* icmpp, int len,
|
||||
int caplen, const u_char*& data, const IP_Hdr* /*ip_hdr*/)
|
||||
int caplen, const u_char*& data, const IP_Hdr* ip_hdr)
|
||||
{
|
||||
EventHandlerPtr f = icmp_redirect;
|
||||
in6_addr tgtaddr, dstaddr;
|
||||
|
|
@ -570,7 +587,7 @@ void ICMP_Analyzer::Redirect(double t, const struct icmp* icmpp, int len,
|
|||
|
||||
val_list* vl = new val_list;
|
||||
vl->append(BuildConnVal());
|
||||
vl->append(BuildICMPVal(icmpp, len, 1));
|
||||
vl->append(BuildICMPVal(icmpp, len, 1, ip_hdr));
|
||||
vl->append(new AddrVal(IPAddr(tgtaddr)));
|
||||
vl->append(new AddrVal(IPAddr(dstaddr)));
|
||||
|
||||
|
|
@ -579,7 +596,7 @@ void ICMP_Analyzer::Redirect(double t, const struct icmp* icmpp, int len,
|
|||
|
||||
|
||||
void ICMP_Analyzer::Router(double t, const struct icmp* icmpp, int len,
|
||||
int caplen, const u_char*& data, const IP_Hdr* /*ip_hdr*/)
|
||||
int caplen, const u_char*& data, const IP_Hdr* ip_hdr)
|
||||
{
|
||||
EventHandlerPtr f = 0;
|
||||
|
||||
|
|
@ -590,13 +607,13 @@ void ICMP_Analyzer::Router(double t, const struct icmp* icmpp, int len,
|
|||
break;
|
||||
case ICMP6_ROUTER_RENUMBERING:
|
||||
default:
|
||||
ICMPEvent(icmp_sent, icmpp, len, 1);
|
||||
ICMPEvent(icmp_sent, icmpp, len, 1, ip_hdr);
|
||||
return;
|
||||
}
|
||||
|
||||
val_list* vl = new val_list;
|
||||
vl->append(BuildConnVal());
|
||||
vl->append(BuildICMPVal(icmpp, len, 1));
|
||||
vl->append(BuildICMPVal(icmpp, len, 1, ip_hdr));
|
||||
|
||||
ConnectionEvent(f, vl);
|
||||
}
|
||||
|
|
@ -622,7 +639,7 @@ void ICMP_Analyzer::Context4(double t, const struct icmp* icmpp,
|
|||
{
|
||||
val_list* vl = new val_list;
|
||||
vl->append(BuildConnVal());
|
||||
vl->append(BuildICMPVal(icmpp, len, 0));
|
||||
vl->append(BuildICMPVal(icmpp, len, 0, ip_hdr));
|
||||
vl->append(new Val(icmpp->icmp_code, TYPE_COUNT));
|
||||
vl->append(ExtractICMP4Context(caplen, data));
|
||||
ConnectionEvent(f, vl);
|
||||
|
|
@ -652,13 +669,17 @@ void ICMP_Analyzer::Context6(double t, const struct icmp* icmpp,
|
|||
case ICMP6_PACKET_TOO_BIG:
|
||||
f = icmp_packet_too_big;
|
||||
break;
|
||||
|
||||
default:
|
||||
f = icmp_error_message;
|
||||
break;
|
||||
}
|
||||
|
||||
if ( f )
|
||||
{
|
||||
val_list* vl = new val_list;
|
||||
vl->append(BuildConnVal());
|
||||
vl->append(BuildICMPVal(icmpp, len, 1));
|
||||
vl->append(BuildICMPVal(icmpp, len, 1, ip_hdr));
|
||||
vl->append(new Val(icmpp->icmp_code, TYPE_COUNT));
|
||||
vl->append(ExtractICMP6Context(caplen, data));
|
||||
ConnectionEvent(f, vl);
|
||||
|
|
|
|||
|
|
@ -33,7 +33,8 @@ protected:
|
|||
virtual bool IsReuse(double t, const u_char* pkt);
|
||||
virtual unsigned int MemoryAllocation() const;
|
||||
|
||||
void ICMPEvent(EventHandlerPtr f, const struct icmp* icmpp, int len, int icmpv6);
|
||||
void ICMPEvent(EventHandlerPtr f, const struct icmp* icmpp, int len,
|
||||
int icmpv6, const IP_Hdr* ip_hdr);
|
||||
|
||||
void Echo(double t, const struct icmp* icmpp, int len,
|
||||
int caplen, const u_char*& data, const IP_Hdr* ip_hdr);
|
||||
|
|
@ -52,7 +53,8 @@ protected:
|
|||
|
||||
void Describe(ODesc* d) const;
|
||||
|
||||
RecordVal* BuildICMPVal(const struct icmp* icmpp, int len, int icmpv6);
|
||||
RecordVal* BuildICMPVal(const struct icmp* icmpp, int len, int icmpv6,
|
||||
const IP_Hdr* ip_hdr);
|
||||
|
||||
void NextICMP4(double t, const struct icmp* icmpp, int len, int caplen,
|
||||
const u_char*& data, const IP_Hdr* ip_hdr );
|
||||
|
|
|
|||
|
|
@ -4,6 +4,7 @@
|
|||
#include "MIME.h"
|
||||
#include "Event.h"
|
||||
#include "Reporter.h"
|
||||
#include "digest.h"
|
||||
|
||||
// Here are a few things to do:
|
||||
//
|
||||
|
|
@ -1008,7 +1009,7 @@ void MIME_Mail::Done()
|
|||
if ( compute_content_hash && mime_content_hash )
|
||||
{
|
||||
u_char* digest = new u_char[16];
|
||||
md5_finish(&md5_hash, digest);
|
||||
md5_final(&md5_hash, digest);
|
||||
|
||||
val_list* vl = new val_list;
|
||||
vl->append(analyzer->BuildConnVal());
|
||||
|
|
@ -1096,7 +1097,7 @@ void MIME_Mail::SubmitData(int len, const char* buf)
|
|||
if ( compute_content_hash )
|
||||
{
|
||||
content_hash_length += len;
|
||||
md5_append(&md5_hash, (const u_char*) buf, len);
|
||||
md5_update(&md5_hash, (const u_char*) buf, len);
|
||||
}
|
||||
|
||||
if ( mime_entity_data || mime_all_data )
|
||||
|
|
|
|||
|
|
@ -2,13 +2,12 @@
|
|||
#define mime_h
|
||||
|
||||
#include <assert.h>
|
||||
|
||||
#include <openssl/md5.h>
|
||||
#include <stdio.h>
|
||||
#include <vector>
|
||||
#include <queue>
|
||||
using namespace std;
|
||||
|
||||
#include "md5.h"
|
||||
#include "Base64.h"
|
||||
#include "BroString.h"
|
||||
#include "Analyzer.h"
|
||||
|
|
@ -248,7 +247,7 @@ protected:
|
|||
int buffer_offset;
|
||||
int compute_content_hash;
|
||||
int content_hash_length;
|
||||
md5_state_t md5_hash;
|
||||
MD5_CTX md5_hash;
|
||||
vector<const BroString*> entity_content;
|
||||
vector<const BroString*> all_content;
|
||||
|
||||
|
|
|
|||
|
|
@ -167,6 +167,7 @@ TableVal* preserve_orig_addr;
|
|||
TableVal* preserve_resp_addr;
|
||||
TableVal* preserve_other_addr;
|
||||
|
||||
int max_files_in_cache;
|
||||
double log_rotate_interval;
|
||||
double log_max_size;
|
||||
RecordType* rotate_info;
|
||||
|
|
@ -258,6 +259,7 @@ void init_general_global_var()
|
|||
state_dir = internal_val("state_dir")->AsStringVal();
|
||||
state_write_delay = opt_internal_double("state_write_delay");
|
||||
|
||||
max_files_in_cache = opt_internal_int("max_files_in_cache");
|
||||
log_rotate_interval = opt_internal_double("log_rotate_interval");
|
||||
log_max_size = opt_internal_double("log_max_size");
|
||||
rotate_info = internal_type("rotate_info")->AsRecordType();
|
||||
|
|
|
|||
|
|
@ -170,6 +170,7 @@ extern double connection_status_update_interval;
|
|||
extern StringVal* state_dir;
|
||||
extern double state_write_delay;
|
||||
|
||||
extern int max_files_in_cache;
|
||||
extern double log_rotate_interval;
|
||||
extern double log_max_size;
|
||||
extern RecordType* rotate_info;
|
||||
|
|
|
|||
|
|
@ -149,7 +149,7 @@ void Reporter::WeirdHelper(EventHandlerPtr event, Val* conn_val, const char* add
|
|||
|
||||
va_list ap;
|
||||
va_start(ap, fmt_name);
|
||||
DoLog("weird", event, stderr, 0, vl, false, false, 0, fmt_name, ap);
|
||||
DoLog("weird", event, 0, 0, vl, false, false, 0, fmt_name, ap);
|
||||
va_end(ap);
|
||||
|
||||
delete vl;
|
||||
|
|
@ -163,7 +163,7 @@ void Reporter::WeirdFlowHelper(const IPAddr& orig, const IPAddr& resp, const cha
|
|||
|
||||
va_list ap;
|
||||
va_start(ap, fmt_name);
|
||||
DoLog("weird", flow_weird, stderr, 0, vl, false, false, 0, fmt_name, ap);
|
||||
DoLog("weird", flow_weird, 0, 0, vl, false, false, 0, fmt_name, ap);
|
||||
va_end(ap);
|
||||
|
||||
delete vl;
|
||||
|
|
@ -326,7 +326,8 @@ void Reporter::DoLog(const char* prefix, EventHandlerPtr event, FILE* out, Conne
|
|||
s += buffer;
|
||||
s += "\n";
|
||||
|
||||
fprintf(out, "%s", s.c_str());
|
||||
if ( out )
|
||||
fprintf(out, "%s", s.c_str());
|
||||
|
||||
if ( addl )
|
||||
{
|
||||
|
|
|
|||
|
|
@ -1,21 +1,21 @@
|
|||
#include "SSL-binpac.h"
|
||||
#include "SSL.h"
|
||||
#include "TCP_Reassembler.h"
|
||||
#include "Reporter.h"
|
||||
#include "util.h"
|
||||
|
||||
SSL_Analyzer_binpac::SSL_Analyzer_binpac(Connection* c)
|
||||
SSL_Analyzer::SSL_Analyzer(Connection* c)
|
||||
: TCP_ApplicationAnalyzer(AnalyzerTag::SSL, c)
|
||||
{
|
||||
interp = new binpac::SSL::SSL_Conn(this);
|
||||
had_gap = false;
|
||||
}
|
||||
|
||||
SSL_Analyzer_binpac::~SSL_Analyzer_binpac()
|
||||
SSL_Analyzer::~SSL_Analyzer()
|
||||
{
|
||||
delete interp;
|
||||
}
|
||||
|
||||
void SSL_Analyzer_binpac::Done()
|
||||
void SSL_Analyzer::Done()
|
||||
{
|
||||
TCP_ApplicationAnalyzer::Done();
|
||||
|
||||
|
|
@ -23,23 +23,22 @@ void SSL_Analyzer_binpac::Done()
|
|||
interp->FlowEOF(false);
|
||||
}
|
||||
|
||||
void SSL_Analyzer_binpac::EndpointEOF(TCP_Reassembler* endp)
|
||||
void SSL_Analyzer::EndpointEOF(TCP_Reassembler* endp)
|
||||
{
|
||||
TCP_ApplicationAnalyzer::EndpointEOF(endp);
|
||||
interp->FlowEOF(endp->IsOrig());
|
||||
}
|
||||
|
||||
void SSL_Analyzer_binpac::DeliverStream(int len, const u_char* data, bool orig)
|
||||
void SSL_Analyzer::DeliverStream(int len, const u_char* data, bool orig)
|
||||
{
|
||||
TCP_ApplicationAnalyzer::DeliverStream(len, data, orig);
|
||||
|
||||
assert(TCP());
|
||||
|
||||
if ( TCP()->IsPartial() )
|
||||
return;
|
||||
|
||||
if ( had_gap )
|
||||
// XXX: If only one side had a content gap, we could still try to
|
||||
// If only one side had a content gap, we could still try to
|
||||
// deliver data to the other side if the script layer can handle this.
|
||||
return;
|
||||
|
||||
|
|
@ -53,7 +52,7 @@ void SSL_Analyzer_binpac::DeliverStream(int len, const u_char* data, bool orig)
|
|||
}
|
||||
}
|
||||
|
||||
void SSL_Analyzer_binpac::Undelivered(int seq, int len, bool orig)
|
||||
void SSL_Analyzer::Undelivered(int seq, int len, bool orig)
|
||||
{
|
||||
TCP_ApplicationAnalyzer::Undelivered(seq, len, orig);
|
||||
had_gap = true;
|
||||
|
|
@ -1,14 +1,13 @@
|
|||
#ifndef ssl_binpac_h
|
||||
#define ssl_binpac_h
|
||||
#ifndef ssl_h
|
||||
#define ssl_h
|
||||
|
||||
#include "TCP.h"
|
||||
|
||||
#include "ssl_pac.h"
|
||||
|
||||
class SSL_Analyzer_binpac : public TCP_ApplicationAnalyzer {
|
||||
class SSL_Analyzer : public TCP_ApplicationAnalyzer {
|
||||
public:
|
||||
SSL_Analyzer_binpac(Connection* conn);
|
||||
virtual ~SSL_Analyzer_binpac();
|
||||
SSL_Analyzer(Connection* conn);
|
||||
virtual ~SSL_Analyzer();
|
||||
|
||||
// Overriden from Analyzer.
|
||||
virtual void Done();
|
||||
|
|
@ -19,7 +18,7 @@ public:
|
|||
virtual void EndpointEOF(TCP_Reassembler* endp);
|
||||
|
||||
static Analyzer* InstantiateAnalyzer(Connection* conn)
|
||||
{ return new SSL_Analyzer_binpac(conn); }
|
||||
{ return new SSL_Analyzer(conn); }
|
||||
|
||||
static bool Available()
|
||||
{
|
||||
|
|
@ -15,10 +15,9 @@
|
|||
|
||||
extern int generate_documentation;
|
||||
|
||||
// Note: This function must be thread-safe.
|
||||
const char* type_name(TypeTag t)
|
||||
{
|
||||
static char errbuf[512];
|
||||
|
||||
static const char* type_names[int(NUM_TYPES)] = {
|
||||
"void",
|
||||
"bool", "int", "count", "counter",
|
||||
|
|
@ -37,10 +36,7 @@ const char* type_name(TypeTag t)
|
|||
};
|
||||
|
||||
if ( int(t) >= NUM_TYPES )
|
||||
{
|
||||
snprintf(errbuf, sizeof(errbuf), "%d: not a type tag", int(t));
|
||||
return errbuf;
|
||||
}
|
||||
return "type_name(): not a type tag";
|
||||
|
||||
return type_names[int(t)];
|
||||
}
|
||||
|
|
|
|||
301
src/bro.bif
301
src/bro.bif
|
|
@ -6,13 +6,13 @@
|
|||
|
||||
%%{ // C segment
|
||||
#include <math.h>
|
||||
|
||||
#include <vector>
|
||||
#include <algorithm>
|
||||
#include <cmath>
|
||||
#include <sys/stat.h>
|
||||
#include <cstdio>
|
||||
|
||||
#include "digest.h"
|
||||
#include "Reporter.h"
|
||||
#include "IPAddr.h"
|
||||
|
||||
|
|
@ -530,7 +530,7 @@ function piped_exec%(program: string, to_write: string%): bool
|
|||
%%{
|
||||
static void hash_md5_val(val_list& vlist, unsigned char digest[16])
|
||||
{
|
||||
md5_state_s h;
|
||||
MD5_CTX h;
|
||||
|
||||
md5_init(&h);
|
||||
loop_over_list(vlist, i)
|
||||
|
|
@ -539,16 +539,16 @@ static void hash_md5_val(val_list& vlist, unsigned char digest[16])
|
|||
if ( v->Type()->Tag() == TYPE_STRING )
|
||||
{
|
||||
const BroString* str = v->AsString();
|
||||
md5_append(&h, str->Bytes(), str->Len());
|
||||
md5_update(&h, str->Bytes(), str->Len());
|
||||
}
|
||||
else
|
||||
{
|
||||
ODesc d(DESC_BINARY);
|
||||
v->Describe(&d);
|
||||
md5_append(&h, (const md5_byte_t *) d.Bytes(), d.Len());
|
||||
md5_update(&h, (const u_char *) d.Bytes(), d.Len());
|
||||
}
|
||||
}
|
||||
md5_finish(&h, digest);
|
||||
md5_final(&h, digest);
|
||||
}
|
||||
|
||||
static void hmac_md5_val(val_list& vlist, unsigned char digest[16])
|
||||
|
|
@ -556,7 +556,53 @@ static void hmac_md5_val(val_list& vlist, unsigned char digest[16])
|
|||
hash_md5_val(vlist, digest);
|
||||
for ( int i = 0; i < 16; ++i )
|
||||
digest[i] = digest[i] ^ shared_hmac_md5_key[i];
|
||||
hash_md5(16, digest, digest);
|
||||
MD5(digest, 16, digest);
|
||||
}
|
||||
|
||||
static void hash_sha1_val(val_list& vlist, unsigned char digest[20])
|
||||
{
|
||||
SHA_CTX h;
|
||||
|
||||
sha1_init(&h);
|
||||
loop_over_list(vlist, i)
|
||||
{
|
||||
Val* v = vlist[i];
|
||||
if ( v->Type()->Tag() == TYPE_STRING )
|
||||
{
|
||||
const BroString* str = v->AsString();
|
||||
sha1_update(&h, str->Bytes(), str->Len());
|
||||
}
|
||||
else
|
||||
{
|
||||
ODesc d(DESC_BINARY);
|
||||
v->Describe(&d);
|
||||
sha1_update(&h, (const u_char *) d.Bytes(), d.Len());
|
||||
}
|
||||
}
|
||||
sha1_final(&h, digest);
|
||||
}
|
||||
|
||||
static void hash_sha256_val(val_list& vlist, unsigned char digest[32])
|
||||
{
|
||||
SHA256_CTX h;
|
||||
|
||||
sha256_init(&h);
|
||||
loop_over_list(vlist, i)
|
||||
{
|
||||
Val* v = vlist[i];
|
||||
if ( v->Type()->Tag() == TYPE_STRING )
|
||||
{
|
||||
const BroString* str = v->AsString();
|
||||
sha256_update(&h, str->Bytes(), str->Len());
|
||||
}
|
||||
else
|
||||
{
|
||||
ODesc d(DESC_BINARY);
|
||||
v->Describe(&d);
|
||||
sha256_update(&h, (const u_char *) d.Bytes(), d.Len());
|
||||
}
|
||||
}
|
||||
sha256_final(&h, digest);
|
||||
}
|
||||
%%}
|
||||
|
||||
|
|
@ -565,6 +611,8 @@ static void hmac_md5_val(val_list& vlist, unsigned char digest[16])
|
|||
## Returns: The MD5 hash value of the concatenated arguments.
|
||||
##
|
||||
## .. bro:see:: md5_hmac md5_hash_init md5_hash_update md5_hash_finish
|
||||
## sha1_hash sha1_hash_init sha1_hash_update sha1_hash_finish
|
||||
## sha256_hash sha256_hash_init sha256_hash_update sha256_hash_finish
|
||||
##
|
||||
## .. note::
|
||||
##
|
||||
|
|
@ -578,6 +626,46 @@ function md5_hash%(...%): string
|
|||
return new StringVal(md5_digest_print(digest));
|
||||
%}
|
||||
|
||||
## Computes the SHA1 hash value of the provided list of arguments.
|
||||
##
|
||||
## Returns: The SHA1 hash value of the concatenated arguments.
|
||||
##
|
||||
## .. bro:see:: md5_hash md5_hmac md5_hash_init md5_hash_update md5_hash_finish
|
||||
## sha1_hash_init sha1_hash_update sha1_hash_finish
|
||||
## sha256_hash sha256_hash_init sha256_hash_update sha256_hash_finish
|
||||
##
|
||||
## .. note::
|
||||
##
|
||||
## This function performs a one-shot computation of its arguments.
|
||||
## For incremental hash computation, see :bro:id:`sha1_hash_init` and
|
||||
## friends.
|
||||
function sha1_hash%(...%): string
|
||||
%{
|
||||
unsigned char digest[20];
|
||||
hash_sha1_val(@ARG@, digest);
|
||||
return new StringVal(sha1_digest_print(digest));
|
||||
%}
|
||||
|
||||
## Computes the SHA256 hash value of the provided list of arguments.
|
||||
##
|
||||
## Returns: The SHA256 hash value of the concatenated arguments.
|
||||
##
|
||||
## .. bro:see:: md5_hash md5_hmac md5_hash_init md5_hash_update md5_hash_finish
|
||||
## sha1_hash sha1_hash_init sha1_hash_update sha1_hash_finish
|
||||
## sha256_hash_init sha256_hash_update sha256_hash_finish
|
||||
##
|
||||
## .. note::
|
||||
##
|
||||
## This function performs a one-shot computation of its arguments.
|
||||
## For incremental hash computation, see :bro:id:`sha256_hash_init` and
|
||||
## friends.
|
||||
function sha256_hash%(...%): string
|
||||
%{
|
||||
unsigned char digest[32];
|
||||
hash_sha256_val(@ARG@, digest);
|
||||
return new StringVal(sha256_digest_print(digest));
|
||||
%}
|
||||
|
||||
## Computes an HMAC-MD5 hash value of the provided list of arguments. The HMAC
|
||||
## secret key is generated from available entropy when Bro starts up, or it can
|
||||
## be specified for repeatability using the ``-K`` command line flag.
|
||||
|
|
@ -585,6 +673,8 @@ function md5_hash%(...%): string
|
|||
## Returns: The HMAC-MD5 hash value of the concatenated arguments.
|
||||
##
|
||||
## .. bro:see:: md5_hash md5_hash_init md5_hash_update md5_hash_finish
|
||||
## sha1_hash sha1_hash_init sha1_hash_update sha1_hash_finish
|
||||
## sha256_hash sha256_hash_init sha256_hash_update sha256_hash_finish
|
||||
function md5_hmac%(...%): string
|
||||
%{
|
||||
unsigned char digest[16];
|
||||
|
|
@ -593,7 +683,9 @@ function md5_hmac%(...%): string
|
|||
%}
|
||||
|
||||
%%{
|
||||
static map<BroString, md5_state_s> md5_states;
|
||||
static map<BroString, MD5_CTX> md5_states;
|
||||
static map<BroString, SHA_CTX> sha1_states;
|
||||
static map<BroString, SHA256_CTX> sha256_states;
|
||||
|
||||
BroString* convert_index_to_string(Val* index)
|
||||
{
|
||||
|
|
@ -618,7 +710,9 @@ BroString* convert_index_to_string(Val* index)
|
|||
##
|
||||
## index: The unique identifier to associate with this hash computation.
|
||||
##
|
||||
## .. bro:see:: md5_hash md5_hmac md5_hash_update md5_hash_finish
|
||||
## .. bro:see:: md5_hmac md5_hash md5_hash_update md5_hash_finish
|
||||
## sha1_hash sha1_hash_init sha1_hash_update sha1_hash_finish
|
||||
## sha256_hash sha256_hash_init sha256_hash_update sha256_hash_finish
|
||||
function md5_hash_init%(index: any%): bool
|
||||
%{
|
||||
BroString* s = convert_index_to_string(index);
|
||||
|
|
@ -626,7 +720,7 @@ function md5_hash_init%(index: any%): bool
|
|||
|
||||
if ( md5_states.count(*s) < 1 )
|
||||
{
|
||||
md5_state_s h;
|
||||
MD5_CTX h;
|
||||
md5_init(&h);
|
||||
md5_states[*s] = h;
|
||||
status = 1;
|
||||
|
|
@ -636,6 +730,75 @@ function md5_hash_init%(index: any%): bool
|
|||
return new Val(status, TYPE_BOOL);
|
||||
%}
|
||||
|
||||
## Initializes SHA1 state to enable incremental hash computation. After
|
||||
## initializing the SHA1 state with this function, you can feed data to
|
||||
## :bro:id:`sha1_hash_update` and finally need to call
|
||||
## :bro:id:`sha1_hash_finish` to finish the computation and get the final hash
|
||||
## value.
|
||||
##
|
||||
## For example, when computing incremental SHA1 values of transferred files in
|
||||
## multiple concurrent HTTP connections, one would call ``sha1_hash_init(c$id)``
|
||||
## once before invoking ``sha1_hash_update(c$id, some_more_data)`` in the
|
||||
## :bro:id:`http_entity_data` event handler. When all data has arrived, a call
|
||||
## to :bro:id:`sha1_hash_finish` returns the final hash value.
|
||||
##
|
||||
## index: The unique identifier to associate with this hash computation.
|
||||
##
|
||||
## .. bro:see:: md5_hmac md5_hash md5_hash_init md5_hash_update md5_hash_finish
|
||||
## sha1_hash sha1_hash_update sha1_hash_finish
|
||||
## sha256_hash sha256_hash_init sha256_hash_update sha256_hash_finish
|
||||
function sha1_hash_init%(index: any%): bool
|
||||
%{
|
||||
BroString* s = convert_index_to_string(index);
|
||||
int status = 0;
|
||||
|
||||
if ( sha1_states.count(*s) < 1 )
|
||||
{
|
||||
SHA_CTX h;
|
||||
sha1_init(&h);
|
||||
sha1_states[*s] = h;
|
||||
status = 1;
|
||||
}
|
||||
|
||||
delete s;
|
||||
return new Val(status, TYPE_BOOL);
|
||||
%}
|
||||
|
||||
## Initializes SHA256 state to enable incremental hash computation. After
|
||||
## initializing the SHA256 state with this function, you can feed data to
|
||||
## :bro:id:`sha256_hash_update` and finally need to call
|
||||
## :bro:id:`sha256_hash_finish` to finish the computation and get the final hash
|
||||
## value.
|
||||
##
|
||||
## For example, when computing incremental SHA256 values of transferred files in
|
||||
## multiple concurrent HTTP connections, one would call
|
||||
## ``sha256_hash_init(c$id)`` once before invoking
|
||||
## ``sha256_hash_update(c$id, some_more_data)`` in the
|
||||
## :bro:id:`http_entity_data` event handler. When all data has arrived, a call
|
||||
## to :bro:id:`sha256_hash_finish` returns the final hash value.
|
||||
##
|
||||
## index: The unique identifier to associate with this hash computation.
|
||||
##
|
||||
## .. bro:see:: md5_hmac md5_hash md5_hash_init md5_hash_update md5_hash_finish
|
||||
## sha1_hash sha1_hash_init sha1_hash_update sha1_hash_finish
|
||||
## sha256_hash sha256_hash_update sha256_hash_finish
|
||||
function sha256_hash_init%(index: any%): bool
|
||||
%{
|
||||
BroString* s = convert_index_to_string(index);
|
||||
int status = 0;
|
||||
|
||||
if ( sha256_states.count(*s) < 1 )
|
||||
{
|
||||
SHA256_CTX h;
|
||||
sha256_init(&h);
|
||||
sha256_states[*s] = h;
|
||||
status = 1;
|
||||
}
|
||||
|
||||
delete s;
|
||||
return new Val(status, TYPE_BOOL);
|
||||
%}
|
||||
|
||||
## Update the MD5 value associated with a given index. It is required to
|
||||
## call :bro:id:`md5_hash_init` once before calling this
|
||||
## function.
|
||||
|
|
@ -644,7 +807,9 @@ function md5_hash_init%(index: any%): bool
|
|||
##
|
||||
## data: The data to add to the hash computation.
|
||||
##
|
||||
## .. bro:see:: md5_hash md5_hmac md5_hash_init md5_hash_finish
|
||||
## .. bro:see:: md5_hmac md5_hash md5_hash_init md5_hash_finish
|
||||
## sha1_hash sha1_hash_init sha1_hash_update sha1_hash_finish
|
||||
## sha256_hash sha256_hash_init sha256_hash_update sha256_hash_finish
|
||||
function md5_hash_update%(index: any, data: string%): bool
|
||||
%{
|
||||
BroString* s = convert_index_to_string(index);
|
||||
|
|
@ -652,7 +817,59 @@ function md5_hash_update%(index: any, data: string%): bool
|
|||
|
||||
if ( md5_states.count(*s) > 0 )
|
||||
{
|
||||
md5_append(&md5_states[*s], data->Bytes(), data->Len());
|
||||
md5_update(&md5_states[*s], data->Bytes(), data->Len());
|
||||
status = 1;
|
||||
}
|
||||
|
||||
delete s;
|
||||
return new Val(status, TYPE_BOOL);
|
||||
%}
|
||||
|
||||
## Update the SHA1 value associated with a given index. It is required to
|
||||
## call :bro:id:`sha1_hash_init` once before calling this
|
||||
## function.
|
||||
##
|
||||
## index: The unique identifier to associate with this hash computation.
|
||||
##
|
||||
## data: The data to add to the hash computation.
|
||||
##
|
||||
## .. bro:see:: md5_hmac md5_hash md5_hash_init md5_hash_update md5_hash_finish
|
||||
## sha1_hash sha1_hash_init sha1_hash_finish
|
||||
## sha256_hash sha256_hash_init sha256_hash_update sha256_hash_finish
|
||||
function sha1_hash_update%(index: any, data: string%): bool
|
||||
%{
|
||||
BroString* s = convert_index_to_string(index);
|
||||
int status = 0;
|
||||
|
||||
if ( sha1_states.count(*s) > 0 )
|
||||
{
|
||||
sha1_update(&sha1_states[*s], data->Bytes(), data->Len());
|
||||
status = 1;
|
||||
}
|
||||
|
||||
delete s;
|
||||
return new Val(status, TYPE_BOOL);
|
||||
%}
|
||||
|
||||
## Update the SHA256 value associated with a given index. It is required to
|
||||
## call :bro:id:`sha256_hash_init` once before calling this
|
||||
## function.
|
||||
##
|
||||
## index: The unique identifier to associate with this hash computation.
|
||||
##
|
||||
## data: The data to add to the hash computation.
|
||||
##
|
||||
## .. bro:see:: md5_hmac md5_hash md5_hash_init md5_hash_update md5_hash_finish
|
||||
## sha1_hash sha1_hash_init sha1_hash_update sha1_hash_finish
|
||||
## sha256_hash sha256_hash_init sha256_hash_finish
|
||||
function sha256_hash_update%(index: any, data: string%): bool
|
||||
%{
|
||||
BroString* s = convert_index_to_string(index);
|
||||
int status = 0;
|
||||
|
||||
if ( sha256_states.count(*s) > 0 )
|
||||
{
|
||||
sha256_update(&sha256_states[*s], data->Bytes(), data->Len());
|
||||
status = 1;
|
||||
}
|
||||
|
||||
|
|
@ -666,7 +883,9 @@ function md5_hash_update%(index: any, data: string%): bool
|
|||
##
|
||||
## Returns: The hash value associated with the computation at *index*.
|
||||
##
|
||||
## .. bro:see:: md5_hash md5_hmac md5_hash_init md5_hash_update
|
||||
## .. bro:see:: md5_hmac md5_hash md5_hash_init md5_hash_update
|
||||
## sha1_hash sha1_hash_init sha1_hash_update sha1_hash_finish
|
||||
## sha256_hash sha256_hash_init sha256_hash_update sha256_hash_finish
|
||||
function md5_hash_finish%(index: any%): string
|
||||
%{
|
||||
BroString* s = convert_index_to_string(index);
|
||||
|
|
@ -675,7 +894,7 @@ function md5_hash_finish%(index: any%): string
|
|||
if ( md5_states.count(*s) > 0 )
|
||||
{
|
||||
unsigned char digest[16];
|
||||
md5_finish(&md5_states[*s], digest);
|
||||
md5_final(&md5_states[*s], digest);
|
||||
md5_states.erase(*s);
|
||||
printable_digest = new StringVal(md5_digest_print(digest));
|
||||
}
|
||||
|
|
@ -686,6 +905,62 @@ function md5_hash_finish%(index: any%): string
|
|||
return printable_digest;
|
||||
%}
|
||||
|
||||
## Returns the final SHA1 digest of an incremental hash computation.
|
||||
##
|
||||
## index: The unique identifier of this hash computation.
|
||||
##
|
||||
## Returns: The hash value associated with the computation at *index*.
|
||||
##
|
||||
## .. bro:see:: md5_hmac md5_hash md5_hash_init md5_hash_update md5_hash_finish
|
||||
## sha1_hash sha1_hash_init sha1_hash_update
|
||||
## sha256_hash sha256_hash_init sha256_hash_update sha256_hash_finish
|
||||
function sha1_hash_finish%(index: any%): string
|
||||
%{
|
||||
BroString* s = convert_index_to_string(index);
|
||||
StringVal* printable_digest;
|
||||
|
||||
if ( sha1_states.count(*s) > 0 )
|
||||
{
|
||||
unsigned char digest[20];
|
||||
sha1_final(&sha1_states[*s], digest);
|
||||
sha1_states.erase(*s);
|
||||
printable_digest = new StringVal(sha1_digest_print(digest));
|
||||
}
|
||||
else
|
||||
printable_digest = new StringVal("");
|
||||
|
||||
delete s;
|
||||
return printable_digest;
|
||||
%}
|
||||
|
||||
## Returns the final SHA256 digest of an incremental hash computation.
|
||||
##
|
||||
## index: The unique identifier of this hash computation.
|
||||
##
|
||||
## Returns: The hash value associated with the computation at *index*.
|
||||
##
|
||||
## .. bro:see:: md5_hmac md5_hash md5_hash_init md5_hash_update md5_hash_finish
|
||||
## sha1_hash sha1_hash_init sha1_hash_update sha1_hash_finish
|
||||
## sha256_hash sha256_hash_init sha256_hash_update
|
||||
function sha256_hash_finish%(index: any%): string
|
||||
%{
|
||||
BroString* s = convert_index_to_string(index);
|
||||
StringVal* printable_digest;
|
||||
|
||||
if ( sha256_states.count(*s) > 0 )
|
||||
{
|
||||
unsigned char digest[32];
|
||||
sha256_final(&sha256_states[*s], digest);
|
||||
sha256_states.erase(*s);
|
||||
printable_digest = new StringVal(sha256_digest_print(digest));
|
||||
}
|
||||
else
|
||||
printable_digest = new StringVal("");
|
||||
|
||||
delete s;
|
||||
return printable_digest;
|
||||
%}
|
||||
|
||||
## Generates a random number.
|
||||
##
|
||||
## max: The maximum value the random number.
|
||||
|
|
|
|||
92
src/digest.h
Normal file
92
src/digest.h
Normal file
|
|
@ -0,0 +1,92 @@
|
|||
// See the file "COPYING" in the main distribution directory for copyright.
|
||||
|
||||
/**
|
||||
* Wrapper and helper functions for MD5/SHA digest algorithms.
|
||||
*/
|
||||
|
||||
#ifndef bro_digest_h
|
||||
#define bro_digest_h
|
||||
|
||||
#include <openssl/md5.h>
|
||||
#include <openssl/sha.h>
|
||||
|
||||
#include "Reporter.h"
|
||||
|
||||
static inline const char* digest_print(const u_char* digest, size_t n)
|
||||
{
|
||||
static char buf[256]; // big enough for any of md5/sha1/sha256
|
||||
for ( size_t i = 0; i < n; ++i )
|
||||
snprintf(buf + i * 2, 3, "%02x", digest[i]);
|
||||
return buf;
|
||||
}
|
||||
|
||||
inline const char* md5_digest_print(const u_char digest[MD5_DIGEST_LENGTH])
|
||||
{
|
||||
return digest_print(digest, MD5_DIGEST_LENGTH);
|
||||
}
|
||||
|
||||
inline const char* sha1_digest_print(const u_char digest[SHA_DIGEST_LENGTH])
|
||||
{
|
||||
return digest_print(digest, SHA_DIGEST_LENGTH);
|
||||
}
|
||||
|
||||
inline const char* sha256_digest_print(const u_char digest[SHA256_DIGEST_LENGTH])
|
||||
{
|
||||
return digest_print(digest, SHA256_DIGEST_LENGTH);
|
||||
}
|
||||
|
||||
inline void md5_init(MD5_CTX* c)
|
||||
{
|
||||
if ( ! MD5_Init(c) )
|
||||
reporter->InternalError("MD5_Init failed");
|
||||
}
|
||||
|
||||
inline void md5_update(MD5_CTX* c, const void* data, unsigned long len)
|
||||
{
|
||||
if ( ! MD5_Update(c, data, len) )
|
||||
reporter->InternalError("MD5_Update failed");
|
||||
}
|
||||
|
||||
inline void md5_final(MD5_CTX* c, u_char md[MD5_DIGEST_LENGTH])
|
||||
{
|
||||
if ( ! MD5_Final(md, c) )
|
||||
reporter->InternalError("MD5_Final failed");
|
||||
}
|
||||
|
||||
inline void sha1_init(SHA_CTX* c)
|
||||
{
|
||||
if ( ! SHA1_Init(c) )
|
||||
reporter->InternalError("SHA_Init failed");
|
||||
}
|
||||
|
||||
inline void sha1_update(SHA_CTX* c, const void* data, unsigned long len)
|
||||
{
|
||||
if ( ! SHA1_Update(c, data, len) )
|
||||
reporter->InternalError("SHA_Update failed");
|
||||
}
|
||||
|
||||
inline void sha1_final(SHA_CTX* c, u_char md[SHA_DIGEST_LENGTH])
|
||||
{
|
||||
if ( ! SHA1_Final(md, c) )
|
||||
reporter->InternalError("SHA_Final failed");
|
||||
}
|
||||
|
||||
inline void sha256_init(SHA256_CTX* c)
|
||||
{
|
||||
if ( ! SHA256_Init(c) )
|
||||
reporter->InternalError("SHA256_Init failed");
|
||||
}
|
||||
|
||||
inline void sha256_update(SHA256_CTX* c, const void* data, unsigned long len)
|
||||
{
|
||||
if ( ! SHA256_Update(c, data, len) )
|
||||
reporter->InternalError("SHA256_Update failed");
|
||||
}
|
||||
|
||||
inline void sha256_final(SHA256_CTX* c, u_char md[SHA256_DIGEST_LENGTH])
|
||||
{
|
||||
if ( ! SHA256_Final(md, c) )
|
||||
reporter->InternalError("SHA256_Final failed");
|
||||
}
|
||||
|
||||
#endif //bro_digest_h
|
||||
126
src/event.bif
126
src/event.bif
|
|
@ -762,10 +762,9 @@ event udp_contents%(u: connection, is_orig: bool, contents: string%);
|
|||
## .. bro:see:: udp_contents udp_reply udp_request
|
||||
event udp_session_done%(u: connection%);
|
||||
|
||||
## Generated for all ICMP messages that are not handled separetely with dedicated
|
||||
## ICMP events. Bro's ICMP analyzer handles a number of ICMP messages directly
|
||||
## with dedicated events. This handlers acts as a fallback for those it doesn't.
|
||||
## The *icmp* record provides more information about the message.
|
||||
## Generated for all ICMP messages that are not handled separately with dedicated
|
||||
## ICMP events. Bro's ICMP analyzer handles a number of ICMP messages directly
|
||||
## with dedicated events. This event acts as a fallback for those it doesn't.
|
||||
##
|
||||
## See `Wikipedia
|
||||
## <http://en.wikipedia.org/wiki/Internet_Control_Message_Protocol>`__ for more
|
||||
|
|
@ -776,8 +775,7 @@ event udp_session_done%(u: connection%);
|
|||
## icmp: Additional ICMP-specific information augmenting the standard
|
||||
## connection record *c*.
|
||||
##
|
||||
## .. bro:see:: icmp_echo_reply icmp_echo_request icmp_redirect
|
||||
## icmp_time_exceeded icmp_unreachable
|
||||
## .. bro:see:: icmp_error_message
|
||||
event icmp_sent%(c: connection, icmp: icmp_conn%);
|
||||
|
||||
## Generated for ICMP *echo request* messages.
|
||||
|
|
@ -798,8 +796,7 @@ event icmp_sent%(c: connection, icmp: icmp_conn%);
|
|||
## payload: The message-specific data of the packet payload, i.e., everything after
|
||||
## the first 8 bytes of the ICMP header.
|
||||
##
|
||||
## .. bro:see:: icmp_echo_reply icmp_redirect icmp_sent
|
||||
## icmp_time_exceeded icmp_unreachable
|
||||
## .. bro:see:: icmp_echo_reply
|
||||
event icmp_echo_request%(c: connection, icmp: icmp_conn, id: count, seq: count, payload: string%);
|
||||
|
||||
## Generated for ICMP *echo reply* messages.
|
||||
|
|
@ -820,26 +817,30 @@ event icmp_echo_request%(c: connection, icmp: icmp_conn, id: count, seq: count,
|
|||
## payload: The message-specific data of the packet payload, i.e., everything after
|
||||
## the first 8 bytes of the ICMP header.
|
||||
##
|
||||
## .. bro:see:: icmp_echo_request icmp_redirect icmp_sent
|
||||
## icmp_time_exceeded icmp_unreachable
|
||||
## .. bro:see:: icmp_echo_request
|
||||
event icmp_echo_reply%(c: connection, icmp: icmp_conn, id: count, seq: count, payload: string%);
|
||||
|
||||
## Generated for all ICMP error messages that are not handled separately with dedicated
|
||||
## ICMP events. Bro's ICMP analyzer handles a number of ICMP messages directly
|
||||
## with dedicated events. This handler acts as a fallback for those it doesn't.
|
||||
## The *icmp* record provides more information about the message.
|
||||
## Generated for all ICMPv6 error messages that are not handled
|
||||
## separately with dedicated events. Bro's ICMP analyzer handles a number
|
||||
## of ICMP error messages directly with dedicated events. This event acts
|
||||
## as a fallback for those it doesn't.
|
||||
##
|
||||
## See `Wikipedia
|
||||
## <http://en.wikipedia.org/wiki/Internet_Control_Message_Protocol>`__ for more
|
||||
## information about the ICMP protocol.
|
||||
## <http://en.wikipedia.org/wiki/ICMPv6>`__ for more
|
||||
## information about the ICMPv6 protocol.
|
||||
##
|
||||
## c: The connection record for the corresponding ICMP flow.
|
||||
##
|
||||
## icmp: Additional ICMP-specific information augmenting the standard
|
||||
## connection record *c*.
|
||||
##
|
||||
## .. bro:see:: icmp_echo_reply icmp_echo_request icmp_redirect
|
||||
## icmp_time_exceeded icmp_unreachable
|
||||
## code: The ICMP code of the error message.
|
||||
##
|
||||
## context: A record with specifics of the original packet that the message refers
|
||||
## to.
|
||||
##
|
||||
## .. bro:see:: icmp_unreachable icmp_packet_too_big
|
||||
## icmp_time_exceeded icmp_parameter_problem
|
||||
event icmp_error_message%(c: connection, icmp: icmp_conn, code: count, context: icmp_context%);
|
||||
|
||||
## Generated for ICMP *destination unreachable* messages.
|
||||
|
|
@ -861,15 +862,15 @@ event icmp_error_message%(c: connection, icmp: icmp_conn, code: count, context:
|
|||
## that if the *unreachable* includes only a partial IP header for some reason, no
|
||||
## fields of *context* will be filled out.
|
||||
##
|
||||
## .. bro:see:: icmp_echo_reply icmp_echo_request icmp_redirect icmp_sent
|
||||
## icmp_time_exceeded
|
||||
## .. bro:see:: icmp_error_message icmp_packet_too_big
|
||||
## icmp_time_exceeded icmp_parameter_problem
|
||||
event icmp_unreachable%(c: connection, icmp: icmp_conn, code: count, context: icmp_context%);
|
||||
|
||||
## Generated for ICMP *packet too big* messages.
|
||||
## Generated for ICMPv6 *packet too big* messages.
|
||||
##
|
||||
## See `Wikipedia
|
||||
## <http://en.wikipedia.org/wiki/Internet_Control_Message_Protocol>`__ for more
|
||||
## information about the ICMP protocol.
|
||||
## <http://en.wikipedia.org/wiki/ICMPv6>`__ for more
|
||||
## information about the ICMPv6 protocol.
|
||||
##
|
||||
## c: The connection record for the corresponding ICMP flow.
|
||||
##
|
||||
|
|
@ -884,8 +885,8 @@ event icmp_unreachable%(c: connection, icmp: icmp_conn, code: count, context: ic
|
|||
## that if the *too big* includes only a partial IP header for some reason, no
|
||||
## fields of *context* will be filled out.
|
||||
##
|
||||
## .. bro:see:: icmp_echo_reply icmp_echo_request icmp_redirect icmp_sent
|
||||
## icmp_time_exceeded
|
||||
## .. bro:see:: icmp_error_message icmp_unreachable
|
||||
## icmp_time_exceeded icmp_parameter_problem
|
||||
event icmp_packet_too_big%(c: connection, icmp: icmp_conn, code: count, context: icmp_context%);
|
||||
|
||||
## Generated for ICMP *time exceeded* messages.
|
||||
|
|
@ -907,15 +908,15 @@ event icmp_packet_too_big%(c: connection, icmp: icmp_conn, code: count, context:
|
|||
## if the *exceeded* includes only a partial IP header for some reason, no fields
|
||||
## of *context* will be filled out.
|
||||
##
|
||||
## .. bro:see:: icmp_echo_reply icmp_echo_request icmp_redirect icmp_sent
|
||||
## icmp_unreachable
|
||||
## .. bro:see:: icmp_error_message icmp_unreachable icmp_packet_too_big
|
||||
## icmp_parameter_problem
|
||||
event icmp_time_exceeded%(c: connection, icmp: icmp_conn, code: count, context: icmp_context%);
|
||||
|
||||
## Generated for ICMP *parameter problem* messages.
|
||||
## Generated for ICMPv6 *parameter problem* messages.
|
||||
##
|
||||
## See `Wikipedia
|
||||
## <http://en.wikipedia.org/wiki/Internet_Control_Message_Protocol>`__ for more
|
||||
## information about the ICMP protocol.
|
||||
## <http://en.wikipedia.org/wiki/ICMPv6>`__ for more
|
||||
## information about the ICMPv6 protocol.
|
||||
##
|
||||
## c: The connection record for the corresponding ICMP flow.
|
||||
##
|
||||
|
|
@ -930,8 +931,8 @@ event icmp_time_exceeded%(c: connection, icmp: icmp_conn, code: count, context:
|
|||
## if the *parameter problem* includes only a partial IP header for some reason, no fields
|
||||
## of *context* will be filled out.
|
||||
##
|
||||
## .. bro:see:: icmp_echo_reply icmp_echo_request icmp_redirect icmp_sent
|
||||
## icmp_unreachable
|
||||
## .. bro:see:: icmp_error_message icmp_unreachable icmp_packet_too_big
|
||||
## icmp_time_exceeded
|
||||
event icmp_parameter_problem%(c: connection, icmp: icmp_conn, code: count, context: icmp_context%);
|
||||
|
||||
## Generated for ICMP *router solicitation* messages.
|
||||
|
|
@ -945,8 +946,8 @@ event icmp_parameter_problem%(c: connection, icmp: icmp_conn, code: count, conte
|
|||
## icmp: Additional ICMP-specific information augmenting the standard connection
|
||||
## record *c*.
|
||||
##
|
||||
## .. bro:see:: icmp_echo_reply icmp_echo_request icmp_sent
|
||||
## icmp_time_exceeded icmp_unreachable
|
||||
## .. bro:see:: icmp_router_advertisement
|
||||
## icmp_neighbor_solicitation icmp_neighbor_advertisement icmp_redirect
|
||||
event icmp_router_solicitation%(c: connection, icmp: icmp_conn%);
|
||||
|
||||
## Generated for ICMP *router advertisement* messages.
|
||||
|
|
@ -960,9 +961,30 @@ event icmp_router_solicitation%(c: connection, icmp: icmp_conn%);
|
|||
## icmp: Additional ICMP-specific information augmenting the standard connection
|
||||
## record *c*.
|
||||
##
|
||||
## .. bro:see:: icmp_echo_reply icmp_echo_request icmp_sent
|
||||
## icmp_time_exceeded icmp_unreachable
|
||||
event icmp_router_advertisement%(c: connection, icmp: icmp_conn, hop_limit: count, managed: bool, router_lifetime: count, reachable_time: interval, retrans_timer: interval%);
|
||||
## cur_hop_limit: The default value that should be placed in Hop Count field
|
||||
## for outgoing IP packets.
|
||||
##
|
||||
## managed: Managed address configuration flag, :rfc:`4861`.
|
||||
##
|
||||
## other: Other stateful configuration flag, :rfc:`4861`.
|
||||
##
|
||||
## home_agent: Mobile IPv6 home agent flag, :rfc:`3775`.
|
||||
##
|
||||
## pref: Router selection preferences, :rfc:`4191`.
|
||||
##
|
||||
## proxy: Neighbor discovery proxy flag, :rfc:`4389`.
|
||||
##
|
||||
## rsv: Remaining two reserved bits of router advertisement flags.
|
||||
##
|
||||
## router_lifetime: How long this router should be used as a default router.
|
||||
##
|
||||
## reachable_time: How long a neighbor should be considered reachable.
|
||||
##
|
||||
## retrans_timer: How long a host should wait before retransmitting.
|
||||
##
|
||||
## .. bro:see:: icmp_router_solicitation
|
||||
## icmp_neighbor_solicitation icmp_neighbor_advertisement icmp_redirect
|
||||
event icmp_router_advertisement%(c: connection, icmp: icmp_conn, cur_hop_limit: count, managed: bool, other: bool, home_agent: bool, pref: count, proxy: bool, rsv: count, router_lifetime: interval, reachable_time: interval, retrans_timer: interval%);
|
||||
|
||||
## Generated for ICMP *neighbor solicitation* messages.
|
||||
##
|
||||
|
|
@ -975,8 +997,10 @@ event icmp_router_advertisement%(c: connection, icmp: icmp_conn, hop_limit: coun
|
|||
## icmp: Additional ICMP-specific information augmenting the standard connection
|
||||
## record *c*.
|
||||
##
|
||||
## .. bro:see:: icmp_echo_reply icmp_echo_request icmp_sent
|
||||
## icmp_time_exceeded icmp_unreachable
|
||||
## tgt: The IP address of the target of the solicitation.
|
||||
##
|
||||
## .. bro:see:: icmp_router_solicitation icmp_router_advertisement
|
||||
## icmp_neighbor_advertisement icmp_redirect
|
||||
event icmp_neighbor_solicitation%(c: connection, icmp: icmp_conn, tgt:addr%);
|
||||
|
||||
## Generated for ICMP *neighbor advertisement* messages.
|
||||
|
|
@ -990,9 +1014,18 @@ event icmp_neighbor_solicitation%(c: connection, icmp: icmp_conn, tgt:addr%);
|
|||
## icmp: Additional ICMP-specific information augmenting the standard connection
|
||||
## record *c*.
|
||||
##
|
||||
## .. bro:see:: icmp_echo_reply icmp_echo_request icmp_sent
|
||||
## icmp_time_exceeded icmp_unreachable
|
||||
event icmp_neighbor_advertisement%(c: connection, icmp: icmp_conn, tgt:addr%);
|
||||
## router: Flag indicating the sender is a router.
|
||||
##
|
||||
## solicited: Flag indicating advertisement is in response to a solicitation.
|
||||
##
|
||||
## override: Flag indicating advertisement should override existing caches.
|
||||
##
|
||||
## tgt: the Target Address in the soliciting message or the address whose
|
||||
## link-layer address has changed for unsolicited adverts.
|
||||
##
|
||||
## .. bro:see:: icmp_router_solicitation icmp_router_advertisement
|
||||
## icmp_neighbor_solicitation icmp_redirect
|
||||
event icmp_neighbor_advertisement%(c: connection, icmp: icmp_conn, router: bool, solicited: bool, override: bool, tgt:addr%);
|
||||
|
||||
## Generated for ICMP *redirect* messages.
|
||||
##
|
||||
|
|
@ -1005,10 +1038,13 @@ event icmp_neighbor_advertisement%(c: connection, icmp: icmp_conn, tgt:addr%);
|
|||
## icmp: Additional ICMP-specific information augmenting the standard connection
|
||||
## record *c*.
|
||||
##
|
||||
## a: The new destination address the message is redirecting to.
|
||||
## tgt: The address that is supposed to be a better first hop to use for
|
||||
## ICMP Destination Address.
|
||||
##
|
||||
## .. bro:see:: icmp_echo_reply icmp_echo_request icmp_sent
|
||||
## icmp_time_exceeded icmp_unreachable
|
||||
## dest: The address of the destination which is redirected to the target.
|
||||
##
|
||||
## .. bro:see:: icmp_router_solicitation icmp_router_advertisement
|
||||
## icmp_neighbor_solicitation icmp_neighbor_advertisement
|
||||
event icmp_redirect%(c: connection, icmp: icmp_conn, tgt: addr, dest: addr%);
|
||||
|
||||
## Generated when a TCP connection terminated, passing on statistics about the
|
||||
|
|
|
|||
|
|
@ -641,7 +641,7 @@ bool Manager::RemoveStream(const string &name) {
|
|||
|
||||
i->removed = true;
|
||||
|
||||
i->reader->Finish();
|
||||
i->reader->Close();
|
||||
|
||||
#ifdef DEBUG
|
||||
DBG_LOG(DBG_INPUT, "Successfully queued removal of stream %s",
|
||||
|
|
|
|||
|
|
@ -82,7 +82,7 @@ protected:
|
|||
friend class SendEventMessage;
|
||||
friend class SendEntryMessage;
|
||||
friend class EndCurrentSendMessage;
|
||||
friend class ReaderFinishedMessage;
|
||||
friend class ReaderClosedMessage;
|
||||
|
||||
// For readers to write to input stream in direct mode (reporting new/deleted values directly)
|
||||
// Functions take ownership of threading::Value fields
|
||||
|
|
|
|||
|
|
@ -100,10 +100,10 @@ public:
|
|||
private:
|
||||
};
|
||||
|
||||
class ReaderFinishedMessage : public threading::OutputMessage<ReaderFrontend> {
|
||||
class ReaderClosedMessage : public threading::OutputMessage<ReaderFrontend> {
|
||||
public:
|
||||
ReaderFinishedMessage(ReaderFrontend* reader)
|
||||
: threading::OutputMessage<ReaderFrontend>("ReaderFinished", reader) {}
|
||||
ReaderClosedMessage(ReaderFrontend* reader)
|
||||
: threading::OutputMessage<ReaderFrontend>("ReaderClosed", reader) {}
|
||||
|
||||
virtual bool Process() {
|
||||
return input_mgr->RemoveStreamContinuation(Object());
|
||||
|
|
@ -190,12 +190,12 @@ bool ReaderBackend::Init(string arg_source, int mode, const int arg_num_fields,
|
|||
return success;
|
||||
}
|
||||
|
||||
void ReaderBackend::Finish()
|
||||
void ReaderBackend::Close()
|
||||
{
|
||||
DoFinish();
|
||||
DoClose();
|
||||
disabled = true;
|
||||
DisableFrontend();
|
||||
SendOut(new ReaderFinishedMessage(frontend));
|
||||
SendOut(new ReaderClosedMessage(frontend));
|
||||
|
||||
if ( fields != 0 ) {
|
||||
|
||||
|
|
|
|||
|
|
@ -68,7 +68,7 @@ public:
|
|||
*
|
||||
* @return False if an error occured.
|
||||
*/
|
||||
void Finish();
|
||||
void Close();
|
||||
|
||||
/**
|
||||
* Force trigger an update of the input stream.
|
||||
|
|
@ -112,7 +112,7 @@ protected:
|
|||
* during shutdown, an implementation should also call Error() to indicate what
|
||||
* happened.
|
||||
*/
|
||||
virtual void DoFinish() = 0;
|
||||
virtual void DoClose() = 0;
|
||||
|
||||
/**
|
||||
* Reader-specific method implementing the forced update trigger
|
||||
|
|
|
|||
|
|
@ -35,14 +35,14 @@ public:
|
|||
virtual bool Process() { return Object()->Update(); }
|
||||
};
|
||||
|
||||
class FinishMessage : public threading::InputMessage<ReaderBackend>
|
||||
class CloseMessage : public threading::InputMessage<ReaderBackend>
|
||||
{
|
||||
public:
|
||||
FinishMessage(ReaderBackend* backend)
|
||||
: threading::InputMessage<ReaderBackend>("Finish", backend)
|
||||
CloseMessage(ReaderBackend* backend)
|
||||
: threading::InputMessage<ReaderBackend>("Close", backend)
|
||||
{ }
|
||||
|
||||
virtual bool Process() { Object()->Finish(); return true; }
|
||||
virtual bool Process() { Object()->Close(); return true; }
|
||||
};
|
||||
|
||||
|
||||
|
|
@ -83,7 +83,7 @@ void ReaderFrontend::Update() {
|
|||
backend->SendIn(new UpdateMessage(backend));
|
||||
}
|
||||
|
||||
void ReaderFrontend::Finish() {
|
||||
void ReaderFrontend::Close() {
|
||||
if ( disabled )
|
||||
return;
|
||||
|
||||
|
|
@ -92,7 +92,7 @@ void ReaderFrontend::Finish() {
|
|||
return;
|
||||
}
|
||||
|
||||
backend->SendIn(new FinishMessage(backend));
|
||||
backend->SendIn(new CloseMessage(backend));
|
||||
}
|
||||
|
||||
string ReaderFrontend::Name() const
|
||||
|
|
|
|||
|
|
@ -68,7 +68,7 @@ public:
|
|||
* the corresponding message there.
|
||||
* This method must only be called from the main thread.
|
||||
*/
|
||||
void Finish();
|
||||
void Close();
|
||||
|
||||
/**
|
||||
* Disables the reader frontend. From now on, all method calls that
|
||||
|
|
|
|||
|
|
@ -72,10 +72,10 @@ Ascii::Ascii(ReaderFrontend *frontend) : ReaderBackend(frontend)
|
|||
|
||||
Ascii::~Ascii()
|
||||
{
|
||||
DoFinish();
|
||||
DoClose();
|
||||
}
|
||||
|
||||
void Ascii::DoFinish()
|
||||
void Ascii::DoClose()
|
||||
{
|
||||
if ( file != 0 ) {
|
||||
file->close();
|
||||
|
|
|
|||
|
|
@ -42,7 +42,7 @@ protected:
|
|||
|
||||
virtual bool DoInit(string path, int mode, int arg_num_fields, const threading::Field* const* fields);
|
||||
|
||||
virtual void DoFinish();
|
||||
virtual void DoClose();
|
||||
|
||||
virtual bool DoUpdate();
|
||||
|
||||
|
|
|
|||
|
|
@ -36,10 +36,10 @@ Benchmark::Benchmark(ReaderFrontend *frontend) : ReaderBackend(frontend)
|
|||
|
||||
Benchmark::~Benchmark()
|
||||
{
|
||||
DoFinish();
|
||||
DoClose();
|
||||
}
|
||||
|
||||
void Benchmark::DoFinish()
|
||||
void Benchmark::DoClose()
|
||||
{
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -19,7 +19,7 @@ protected:
|
|||
|
||||
virtual bool DoInit(string path, int mode, int arg_num_fields, const threading::Field* const* fields);
|
||||
|
||||
virtual void DoFinish();
|
||||
virtual void DoClose();
|
||||
|
||||
virtual bool DoUpdate();
|
||||
|
||||
|
|
|
|||
|
|
@ -39,10 +39,10 @@ Raw::Raw(ReaderFrontend *frontend) : ReaderBackend(frontend)
|
|||
|
||||
Raw::~Raw()
|
||||
{
|
||||
DoFinish();
|
||||
DoClose();
|
||||
}
|
||||
|
||||
void Raw::DoFinish()
|
||||
void Raw::DoClose()
|
||||
{
|
||||
if ( file != 0 ) {
|
||||
Close();
|
||||
|
|
|
|||
|
|
@ -21,7 +21,7 @@ protected:
|
|||
|
||||
virtual bool DoInit(string path, int mode, int arg_num_fields, const threading::Field* const* fields);
|
||||
|
||||
virtual void DoFinish();
|
||||
virtual void DoClose();
|
||||
|
||||
virtual bool DoUpdate();
|
||||
|
||||
|
|
|
|||
|
|
@ -72,3 +72,12 @@ const set_separator: string;
|
|||
const empty_field: string;
|
||||
const unset_field: string;
|
||||
|
||||
# Options for the DataSeries writer.
|
||||
|
||||
module LogDataSeries;
|
||||
|
||||
const compression: string;
|
||||
const extent_size: count;
|
||||
const dump_schema: bool;
|
||||
const use_integer_for_time: bool;
|
||||
const num_threads: count;
|
||||
|
|
|
|||
|
|
@ -7,6 +7,7 @@
|
|||
#include "../NetVar.h"
|
||||
#include "../Net.h"
|
||||
|
||||
#include "threading/Manager.h"
|
||||
#include "threading/SerialTypes.h"
|
||||
|
||||
#include "Manager.h"
|
||||
|
|
@ -16,9 +17,11 @@
|
|||
#include "writers/Ascii.h"
|
||||
#include "writers/None.h"
|
||||
|
||||
#ifdef USE_DATASERIES
|
||||
#include "writers/DataSeries.h"
|
||||
#endif
|
||||
|
||||
using namespace logging;
|
||||
using threading::Value;
|
||||
using threading::Field;
|
||||
|
||||
// Structure describing a log writer type.
|
||||
struct WriterDefinition {
|
||||
|
|
@ -32,6 +35,9 @@ struct WriterDefinition {
|
|||
WriterDefinition log_writers[] = {
|
||||
{ BifEnum::Log::WRITER_NONE, "None", 0, writer::None::Instantiate },
|
||||
{ BifEnum::Log::WRITER_ASCII, "Ascii", 0, writer::Ascii::Instantiate },
|
||||
#ifdef USE_DATASERIES
|
||||
{ BifEnum::Log::WRITER_DATASERIES, "DataSeries", 0, writer::DataSeries::Instantiate },
|
||||
#endif
|
||||
|
||||
// End marker, don't touch.
|
||||
{ BifEnum::Log::WRITER_DEFAULT, "None", 0, (WriterBackend* (*)(WriterFrontend* frontend))0 }
|
||||
|
|
@ -51,7 +57,7 @@ struct Manager::Filter {
|
|||
Func* postprocessor;
|
||||
|
||||
int num_fields;
|
||||
Field** fields;
|
||||
threading::Field** fields;
|
||||
|
||||
// Vector indexed by field number. Each element is a list of record
|
||||
// indices defining a path leading to the value across potential
|
||||
|
|
@ -119,6 +125,7 @@ Manager::Stream::~Stream()
|
|||
|
||||
Manager::Manager()
|
||||
{
|
||||
rotations_pending = 0;
|
||||
}
|
||||
|
||||
Manager::~Manager()
|
||||
|
|
@ -127,6 +134,16 @@ Manager::~Manager()
|
|||
delete *s;
|
||||
}
|
||||
|
||||
list<string> Manager::SupportedFormats()
|
||||
{
|
||||
list<string> formats;
|
||||
|
||||
for ( WriterDefinition* ld = log_writers; ld->type != BifEnum::Log::WRITER_DEFAULT; ++ld )
|
||||
formats.push_back(ld->name);
|
||||
|
||||
return formats;
|
||||
}
|
||||
|
||||
WriterBackend* Manager::CreateBackend(WriterFrontend* frontend, bro_int_t type)
|
||||
{
|
||||
WriterDefinition* ld = log_writers;
|
||||
|
|
@ -135,7 +152,7 @@ WriterBackend* Manager::CreateBackend(WriterFrontend* frontend, bro_int_t type)
|
|||
{
|
||||
if ( ld->type == BifEnum::Log::WRITER_DEFAULT )
|
||||
{
|
||||
reporter->Error("unknow writer when creating writer");
|
||||
reporter->Error("unknown writer type requested");
|
||||
return 0;
|
||||
}
|
||||
|
||||
|
|
@ -159,10 +176,8 @@ WriterBackend* Manager::CreateBackend(WriterFrontend* frontend, bro_int_t type)
|
|||
// function.
|
||||
ld->factory = 0;
|
||||
|
||||
DBG_LOG(DBG_LOGGING, "failed to init writer class %s",
|
||||
ld->name);
|
||||
|
||||
return false;
|
||||
reporter->Error("initialization of writer %s failed", ld->name);
|
||||
return 0;
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -449,9 +464,9 @@ bool Manager::TraverseRecord(Stream* stream, Filter* filter, RecordType* rt,
|
|||
|
||||
filter->indices.push_back(new_indices);
|
||||
|
||||
filter->fields = (Field**)
|
||||
filter->fields = (threading::Field**)
|
||||
realloc(filter->fields,
|
||||
sizeof(Field) * ++filter->num_fields);
|
||||
sizeof(threading::Field) * ++filter->num_fields);
|
||||
|
||||
if ( ! filter->fields )
|
||||
{
|
||||
|
|
@ -459,7 +474,7 @@ bool Manager::TraverseRecord(Stream* stream, Filter* filter, RecordType* rt,
|
|||
return false;
|
||||
}
|
||||
|
||||
Field* field = new Field();
|
||||
threading::Field* field = new threading::Field();
|
||||
field->name = new_path;
|
||||
field->type = t->Tag();
|
||||
|
||||
|
|
@ -571,7 +586,7 @@ bool Manager::AddFilter(EnumVal* id, RecordVal* fval)
|
|||
|
||||
for ( int i = 0; i < filter->num_fields; i++ )
|
||||
{
|
||||
Field* field = filter->fields[i];
|
||||
threading::Field* field = filter->fields[i];
|
||||
DBG_LOG(DBG_LOGGING, " field %10s: %s",
|
||||
field->name.c_str(), type_name(field->type));
|
||||
}
|
||||
|
|
@ -743,10 +758,10 @@ bool Manager::Write(EnumVal* id, RecordVal* columns)
|
|||
|
||||
// Copy the fields for WriterFrontend::Init() as it
|
||||
// will take ownership.
|
||||
Field** arg_fields = new Field*[filter->num_fields];
|
||||
threading::Field** arg_fields = new threading::Field*[filter->num_fields];
|
||||
|
||||
for ( int j = 0; j < filter->num_fields; ++j )
|
||||
arg_fields[j] = new Field(*filter->fields[j]);
|
||||
arg_fields[j] = new threading::Field(*filter->fields[j]);
|
||||
|
||||
writer = CreateWriter(stream->id, filter->writer,
|
||||
path, filter->num_fields,
|
||||
|
|
@ -897,10 +912,10 @@ threading::Value* Manager::ValToLogVal(Val* val, BroType* ty)
|
|||
return lval;
|
||||
}
|
||||
|
||||
Value** Manager::RecordToFilterVals(Stream* stream, Filter* filter,
|
||||
threading::Value** Manager::RecordToFilterVals(Stream* stream, Filter* filter,
|
||||
RecordVal* columns)
|
||||
{
|
||||
Value** vals = new Value*[filter->num_fields];
|
||||
threading::Value** vals = new threading::Value*[filter->num_fields];
|
||||
|
||||
for ( int i = 0; i < filter->num_fields; ++i )
|
||||
{
|
||||
|
|
@ -919,7 +934,7 @@ Value** Manager::RecordToFilterVals(Stream* stream, Filter* filter,
|
|||
if ( ! val )
|
||||
{
|
||||
// Value, or any of its parents, is not set.
|
||||
vals[i] = new Value(filter->fields[i]->type, false);
|
||||
vals[i] = new threading::Value(filter->fields[i]->type, false);
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
|
@ -932,7 +947,7 @@ Value** Manager::RecordToFilterVals(Stream* stream, Filter* filter,
|
|||
}
|
||||
|
||||
WriterFrontend* Manager::CreateWriter(EnumVal* id, EnumVal* writer, string path,
|
||||
int num_fields, const Field* const* fields, bool local, bool remote)
|
||||
int num_fields, const threading::Field* const* fields, bool local, bool remote)
|
||||
{
|
||||
Stream* stream = FindStream(id);
|
||||
|
||||
|
|
@ -996,7 +1011,7 @@ WriterFrontend* Manager::CreateWriter(EnumVal* id, EnumVal* writer, string path,
|
|||
return writer_obj;
|
||||
}
|
||||
|
||||
void Manager::DeleteVals(int num_fields, Value** vals)
|
||||
void Manager::DeleteVals(int num_fields, threading::Value** vals)
|
||||
{
|
||||
// Note this code is duplicated in WriterBackend::DeleteVals().
|
||||
for ( int i = 0; i < num_fields; i++ )
|
||||
|
|
@ -1006,7 +1021,7 @@ void Manager::DeleteVals(int num_fields, Value** vals)
|
|||
}
|
||||
|
||||
bool Manager::Write(EnumVal* id, EnumVal* writer, string path, int num_fields,
|
||||
Value** vals)
|
||||
threading::Value** vals)
|
||||
{
|
||||
Stream* stream = FindStream(id);
|
||||
|
||||
|
|
@ -1113,10 +1128,19 @@ bool Manager::Flush(EnumVal* id)
|
|||
|
||||
void Manager::Terminate()
|
||||
{
|
||||
// Make sure we process all the pending rotations.
|
||||
while ( rotations_pending )
|
||||
{
|
||||
thread_mgr->ForceProcessing(); // A blatant layering violation ...
|
||||
usleep(1000);
|
||||
}
|
||||
|
||||
for ( vector<Stream *>::iterator s = streams.begin(); s != streams.end(); ++s )
|
||||
{
|
||||
if ( *s )
|
||||
Flush((*s)->id);
|
||||
if ( ! *s )
|
||||
continue;
|
||||
|
||||
Flush((*s)->id);
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -1219,11 +1243,19 @@ void Manager::Rotate(WriterInfo* winfo)
|
|||
|
||||
// Trigger the rotation.
|
||||
winfo->writer->Rotate(tmp, winfo->open_time, network_time, terminating);
|
||||
|
||||
++rotations_pending;
|
||||
}
|
||||
|
||||
bool Manager::FinishedRotation(WriterFrontend* writer, string new_name, string old_name,
|
||||
double open, double close, bool terminating)
|
||||
{
|
||||
--rotations_pending;
|
||||
|
||||
if ( ! writer )
|
||||
// Writer didn't produce local output.
|
||||
return true;
|
||||
|
||||
DBG_LOG(DBG_LOGGING, "Finished rotating %s at %.6f, new name %s",
|
||||
writer->Path().c_str(), network_time, new_name.c_str());
|
||||
|
||||
|
|
|
|||
|
|
@ -15,7 +15,6 @@ class RotationTimer;
|
|||
|
||||
namespace logging {
|
||||
|
||||
|
||||
class WriterBackend;
|
||||
class WriterFrontend;
|
||||
class RotationFinishedMessage;
|
||||
|
|
@ -145,6 +144,11 @@ public:
|
|||
*/
|
||||
void Terminate();
|
||||
|
||||
/**
|
||||
* Returns a list of supported output formats.
|
||||
*/
|
||||
static list<string> SupportedFormats();
|
||||
|
||||
protected:
|
||||
friend class WriterFrontend;
|
||||
friend class RotationFinishedMessage;
|
||||
|
|
@ -196,6 +200,7 @@ private:
|
|||
WriterInfo* FindWriter(WriterFrontend* writer);
|
||||
|
||||
vector<Stream *> streams; // Indexed by stream enum.
|
||||
int rotations_pending; // Number of rotations not yet finished.
|
||||
};
|
||||
|
||||
}
|
||||
|
|
|
|||
|
|
@ -1,6 +1,7 @@
|
|||
// See the file "COPYING" in the main distribution directory for copyright.
|
||||
|
||||
#include "util.h"
|
||||
#include "bro_inet_ntop.h"
|
||||
#include "threading/SerialTypes.h"
|
||||
|
||||
#include "WriterBackend.h"
|
||||
|
|
@ -222,17 +223,6 @@ bool WriterBackend::Flush()
|
|||
return true;
|
||||
}
|
||||
|
||||
bool WriterBackend::Finish()
|
||||
{
|
||||
if ( ! DoFlush() )
|
||||
{
|
||||
DisableFrontend();
|
||||
return false;
|
||||
}
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
bool WriterBackend::DoHeartbeat(double network_time, double current_time)
|
||||
{
|
||||
MsgThread::DoHeartbeat(network_time, current_time);
|
||||
|
|
@ -248,7 +238,7 @@ string WriterBackend::Render(const threading::Value::addr_t& addr) const
|
|||
{
|
||||
char s[INET_ADDRSTRLEN];
|
||||
|
||||
if ( inet_ntop(AF_INET, &addr.in.in4, s, INET_ADDRSTRLEN) == NULL )
|
||||
if ( ! bro_inet_ntop(AF_INET, &addr.in.in4, s, INET_ADDRSTRLEN) )
|
||||
return "<bad IPv4 address conversion>";
|
||||
else
|
||||
return s;
|
||||
|
|
@ -257,7 +247,7 @@ string WriterBackend::Render(const threading::Value::addr_t& addr) const
|
|||
{
|
||||
char s[INET6_ADDRSTRLEN];
|
||||
|
||||
if ( inet_ntop(AF_INET6, &addr.in.in6, s, INET6_ADDRSTRLEN) == NULL )
|
||||
if ( ! bro_inet_ntop(AF_INET6, &addr.in.in6, s, INET6_ADDRSTRLEN) )
|
||||
return "<bad IPv6 address conversion>";
|
||||
else
|
||||
return s;
|
||||
|
|
@ -278,4 +268,9 @@ string WriterBackend::Render(const threading::Value::subnet_t& subnet) const
|
|||
return s;
|
||||
}
|
||||
|
||||
|
||||
string WriterBackend::Render(double d) const
|
||||
{
|
||||
char buf[256];
|
||||
modp_dtoa(d, buf, 6);
|
||||
return buf;
|
||||
}
|
||||
|
|
|
|||
|
|
@ -101,15 +101,6 @@ public:
|
|||
*/
|
||||
bool Rotate(string rotated_path, double open, double close, bool terminating);
|
||||
|
||||
/**
|
||||
* Finishes writing to this logger in a regularl fashion. Must not be
|
||||
* called if an error has been indicated earlier. After calling this,
|
||||
* no further writing must be performed.
|
||||
*
|
||||
* @return False if an error occured.
|
||||
*/
|
||||
bool Finish();
|
||||
|
||||
/**
|
||||
* Disables the frontend that has instantiated this backend. Once
|
||||
* disabled,the frontend will not send any further message over.
|
||||
|
|
@ -174,7 +165,17 @@ public:
|
|||
*/
|
||||
string Render(const threading::Value::subnet_t& subnet) const;
|
||||
|
||||
/** Helper method to render a double in Bro's standard precision.
|
||||
*
|
||||
* @param d The double.
|
||||
*
|
||||
* @return An ASCII representation of the double.
|
||||
*/
|
||||
string Render(double d) const;
|
||||
|
||||
protected:
|
||||
friend class FinishMessage;
|
||||
|
||||
/**
|
||||
* Writer-specific intialization method.
|
||||
*
|
||||
|
|
@ -272,26 +273,18 @@ protected:
|
|||
bool terminating) = 0;
|
||||
|
||||
/**
|
||||
* Writer-specific method implementing log output finalization at
|
||||
* termination. Not called when any of the other methods has
|
||||
* previously signaled an error, i.e., executing this method signals
|
||||
* a regular shutdown of the writer.
|
||||
* Writer-specific method called just before the threading system is
|
||||
* going to shutdown.
|
||||
*
|
||||
* A writer implementation must override this method but it can just
|
||||
* ignore calls if flushing doesn't align with its semantics.
|
||||
*
|
||||
* If the method returns false, it will be assumed that a fatal error
|
||||
* has occured that prevents the writer from further operation; it
|
||||
* will then be disabled and eventually deleted. When returning
|
||||
* false, an implementation should also call Error() to indicate what
|
||||
* happened.
|
||||
* This method can be overridden but one must call
|
||||
* WriterBackend::DoFinish().
|
||||
*/
|
||||
virtual bool DoFinish() = 0;
|
||||
virtual bool DoFinish() { return MsgThread::DoFinish(); }
|
||||
|
||||
/**
|
||||
* Triggered by regular heartbeat messages from the main thread.
|
||||
*
|
||||
* This method can be overridden but once must call
|
||||
* This method can be overridden but one must call
|
||||
* WriterBackend::DoHeartbeat().
|
||||
*/
|
||||
virtual bool DoHeartbeat(double network_time, double current_time);
|
||||
|
|
|
|||
|
|
@ -90,7 +90,7 @@ public:
|
|||
FinishMessage(WriterBackend* backend)
|
||||
: threading::InputMessage<WriterBackend>("Finish", backend) {}
|
||||
|
||||
virtual bool Process() { return Object()->Finish(); }
|
||||
virtual bool Process() { return Object()->DoFinish(); }
|
||||
};
|
||||
|
||||
}
|
||||
|
|
@ -117,8 +117,9 @@ WriterFrontend::WriterFrontend(EnumVal* arg_stream, EnumVal* arg_writer, bool ar
|
|||
if ( local )
|
||||
{
|
||||
backend = log_mgr->CreateBackend(this, writer->AsEnum());
|
||||
assert(backend);
|
||||
backend->Start();
|
||||
|
||||
if ( backend )
|
||||
backend->Start();
|
||||
}
|
||||
|
||||
else
|
||||
|
|
@ -256,6 +257,10 @@ void WriterFrontend::Rotate(string rotated_path, double open, double close, bool
|
|||
|
||||
if ( backend )
|
||||
backend->SendIn(new RotateMessage(backend, this, rotated_path, open, close, terminating));
|
||||
else
|
||||
// Still signal log manager that we're done, but signal that
|
||||
// nothing happened by setting the writer to zeri.
|
||||
log_mgr->FinishedRotation(0, "", rotated_path, open, close, terminating);
|
||||
}
|
||||
|
||||
void WriterFrontend::Finish()
|
||||
|
|
|
|||
|
|
@ -69,8 +69,7 @@ bool Ascii::WriteHeaderField(const string& key, const string& val)
|
|||
return (fwrite(str.c_str(), str.length(), 1, file) == 1);
|
||||
}
|
||||
|
||||
bool Ascii::DoInit(string path, int num_fields,
|
||||
const Field* const * fields)
|
||||
bool Ascii::DoInit(string path, int num_fields, const Field* const * fields)
|
||||
{
|
||||
if ( output_to_stdout )
|
||||
path = "/dev/stdout";
|
||||
|
|
@ -87,6 +86,9 @@ bool Ascii::DoInit(string path, int num_fields,
|
|||
|
||||
if ( include_header )
|
||||
{
|
||||
string names;
|
||||
string types;
|
||||
|
||||
string str = string(header_prefix, header_prefix_len)
|
||||
+ "separator " // Always use space as separator here.
|
||||
+ get_escaped_string(string(separator, separator_len), false)
|
||||
|
|
@ -104,9 +106,6 @@ bool Ascii::DoInit(string path, int num_fields,
|
|||
WriteHeaderField("path", get_escaped_string(path, false))) )
|
||||
goto write_error;
|
||||
|
||||
string names;
|
||||
string types;
|
||||
|
||||
for ( int i = 0; i < num_fields; ++i )
|
||||
{
|
||||
if ( i > 0 )
|
||||
|
|
@ -115,15 +114,8 @@ bool Ascii::DoInit(string path, int num_fields,
|
|||
types += string(separator, separator_len);
|
||||
}
|
||||
|
||||
const Field* field = fields[i];
|
||||
names += field->name;
|
||||
types += type_name(field->type);
|
||||
if ( (field->type == TYPE_TABLE) || (field->type == TYPE_VECTOR) )
|
||||
{
|
||||
types += "[";
|
||||
types += type_name(field->subtype);
|
||||
types += "]";
|
||||
}
|
||||
names += fields[i]->name;
|
||||
types += fields[i]->TypeName();
|
||||
}
|
||||
|
||||
if ( ! (WriteHeaderField("fields", names)
|
||||
|
|
@ -146,7 +138,7 @@ bool Ascii::DoFlush()
|
|||
|
||||
bool Ascii::DoFinish()
|
||||
{
|
||||
return true;
|
||||
return WriterBackend::DoFinish();
|
||||
}
|
||||
|
||||
bool Ascii::DoWriteOne(ODesc* desc, Value* val, const Field* field)
|
||||
|
|
@ -184,15 +176,19 @@ bool Ascii::DoWriteOne(ODesc* desc, Value* val, const Field* field)
|
|||
desc->Add(Render(val->val.addr_val));
|
||||
break;
|
||||
|
||||
case TYPE_TIME:
|
||||
case TYPE_INTERVAL:
|
||||
char buf[256];
|
||||
modp_dtoa(val->val.double_val, buf, 6);
|
||||
desc->Add(buf);
|
||||
case TYPE_DOUBLE:
|
||||
// Rendering via Add() truncates trailing 0s after the
|
||||
// decimal point. The difference with TIME/INTERVAL is mainly
|
||||
// to keep the log format consistent.
|
||||
desc->Add(val->val.double_val);
|
||||
break;
|
||||
|
||||
case TYPE_DOUBLE:
|
||||
desc->Add(val->val.double_val);
|
||||
case TYPE_INTERVAL:
|
||||
case TYPE_TIME:
|
||||
// Rendering via Render() keeps trailing 0s after the decimal
|
||||
// point. The difference with DOUBLEis mainly to keep the log
|
||||
// format consistent.
|
||||
desc->Add(Render(val->val.double_val));
|
||||
break;
|
||||
|
||||
case TYPE_ENUM:
|
||||
|
|
|
|||
423
src/logging/writers/DataSeries.cc
Normal file
423
src/logging/writers/DataSeries.cc
Normal file
|
|
@ -0,0 +1,423 @@
|
|||
// See the file "COPYING" in the main distribution directory for copyright.
|
||||
|
||||
#include "config.h"
|
||||
|
||||
#ifdef USE_DATASERIES
|
||||
|
||||
#include <map>
|
||||
#include <string>
|
||||
#include <errno.h>
|
||||
|
||||
#include <DataSeries/GeneralField.hpp>
|
||||
|
||||
#include "NetVar.h"
|
||||
#include "threading/SerialTypes.h"
|
||||
|
||||
#include "DataSeries.h"
|
||||
|
||||
using namespace logging;
|
||||
using namespace writer;
|
||||
|
||||
std::string DataSeries::LogValueToString(threading::Value *val)
|
||||
{
|
||||
// In some cases, no value is attached. If this is the case, return
|
||||
// an empty string.
|
||||
if( ! val->present )
|
||||
return "";
|
||||
|
||||
switch(val->type) {
|
||||
case TYPE_BOOL:
|
||||
return (val->val.int_val ? "true" : "false");
|
||||
|
||||
case TYPE_INT:
|
||||
{
|
||||
std::ostringstream ostr;
|
||||
ostr << val->val.int_val;
|
||||
return ostr.str();
|
||||
}
|
||||
|
||||
case TYPE_COUNT:
|
||||
case TYPE_COUNTER:
|
||||
case TYPE_PORT:
|
||||
{
|
||||
std::ostringstream ostr;
|
||||
ostr << val->val.uint_val;
|
||||
return ostr.str();
|
||||
}
|
||||
|
||||
case TYPE_SUBNET:
|
||||
return Render(val->val.subnet_val);
|
||||
|
||||
case TYPE_ADDR:
|
||||
return Render(val->val.addr_val);
|
||||
|
||||
// Note: These two cases are relatively special. We need to convert
|
||||
// these values into their integer equivalents to maximize precision.
|
||||
// At the moment, there won't be a noticeable effect (Bro uses the
|
||||
// double format everywhere internally, so we've already lost the
|
||||
// precision we'd gain here), but timestamps may eventually switch to
|
||||
// this representation within Bro.
|
||||
//
|
||||
// In the near-term, this *should* lead to better pack_relative (and
|
||||
// thus smaller output files).
|
||||
case TYPE_TIME:
|
||||
case TYPE_INTERVAL:
|
||||
if ( ds_use_integer_for_time )
|
||||
{
|
||||
std::ostringstream ostr;
|
||||
ostr << (uint64_t)(DataSeries::TIME_SCALE * val->val.double_val);
|
||||
return ostr.str();
|
||||
}
|
||||
else
|
||||
return Render(val->val.double_val);
|
||||
|
||||
case TYPE_DOUBLE:
|
||||
return Render(val->val.double_val);
|
||||
|
||||
case TYPE_ENUM:
|
||||
case TYPE_STRING:
|
||||
case TYPE_FILE:
|
||||
case TYPE_FUNC:
|
||||
if ( ! val->val.string_val->size() )
|
||||
return "";
|
||||
|
||||
return string(val->val.string_val->data(), val->val.string_val->size());
|
||||
|
||||
case TYPE_TABLE:
|
||||
{
|
||||
if ( ! val->val.set_val.size )
|
||||
return "";
|
||||
|
||||
string tmpString = "";
|
||||
|
||||
for ( int j = 0; j < val->val.set_val.size; j++ )
|
||||
{
|
||||
if ( j > 0 )
|
||||
tmpString += ds_set_separator;
|
||||
|
||||
tmpString += LogValueToString(val->val.set_val.vals[j]);
|
||||
}
|
||||
|
||||
return tmpString;
|
||||
}
|
||||
|
||||
case TYPE_VECTOR:
|
||||
{
|
||||
if ( ! val->val.vector_val.size )
|
||||
return "";
|
||||
|
||||
string tmpString = "";
|
||||
|
||||
for ( int j = 0; j < val->val.vector_val.size; j++ )
|
||||
{
|
||||
if ( j > 0 )
|
||||
tmpString += ds_set_separator;
|
||||
|
||||
tmpString += LogValueToString(val->val.vector_val.vals[j]);
|
||||
}
|
||||
|
||||
return tmpString;
|
||||
}
|
||||
|
||||
default:
|
||||
InternalError(Fmt("unknown type %s in DataSeries::LogValueToString", type_name(val->type)));
|
||||
return "cannot be reached";
|
||||
}
|
||||
}
|
||||
|
||||
string DataSeries::GetDSFieldType(const threading::Field *field)
|
||||
{
|
||||
switch(field->type) {
|
||||
case TYPE_BOOL:
|
||||
return "bool";
|
||||
|
||||
case TYPE_COUNT:
|
||||
case TYPE_COUNTER:
|
||||
case TYPE_PORT:
|
||||
case TYPE_INT:
|
||||
return "int64";
|
||||
|
||||
case TYPE_DOUBLE:
|
||||
return "double";
|
||||
|
||||
case TYPE_TIME:
|
||||
case TYPE_INTERVAL:
|
||||
return ds_use_integer_for_time ? "int64" : "double";
|
||||
|
||||
case TYPE_SUBNET:
|
||||
case TYPE_ADDR:
|
||||
case TYPE_ENUM:
|
||||
case TYPE_STRING:
|
||||
case TYPE_FILE:
|
||||
case TYPE_TABLE:
|
||||
case TYPE_VECTOR:
|
||||
case TYPE_FUNC:
|
||||
return "variable32";
|
||||
|
||||
default:
|
||||
InternalError(Fmt("unknown type %s in DataSeries::GetDSFieldType", type_name(field->type)));
|
||||
return "cannot be reached";
|
||||
}
|
||||
}
|
||||
|
||||
string DataSeries::BuildDSSchemaFromFieldTypes(const vector<SchemaValue>& vals, string sTitle)
|
||||
{
|
||||
if( ! sTitle.size() )
|
||||
sTitle = "GenericBroStream";
|
||||
|
||||
string xmlschema = "<ExtentType name=\""
|
||||
+ sTitle
|
||||
+ "\" version=\"1.0\" namespace=\"bro-ids.org\">\n";
|
||||
|
||||
for( size_t i = 0; i < vals.size(); ++i )
|
||||
{
|
||||
xmlschema += "\t<field type=\""
|
||||
+ vals[i].ds_type
|
||||
+ "\" name=\""
|
||||
+ vals[i].field_name
|
||||
+ "\" " + vals[i].field_options
|
||||
+ "/>\n";
|
||||
}
|
||||
|
||||
xmlschema += "</ExtentType>\n";
|
||||
|
||||
for( size_t i = 0; i < vals.size(); ++i )
|
||||
{
|
||||
xmlschema += "<!-- " + vals[i].field_name
|
||||
+ " : " + vals[i].bro_type
|
||||
+ " -->\n";
|
||||
}
|
||||
|
||||
return xmlschema;
|
||||
}
|
||||
|
||||
std::string DataSeries::GetDSOptionsForType(const threading::Field *field)
|
||||
{
|
||||
switch( field->type ) {
|
||||
case TYPE_TIME:
|
||||
case TYPE_INTERVAL:
|
||||
{
|
||||
std::string s;
|
||||
s += "pack_relative=\"" + std::string(field->name) + "\"";
|
||||
|
||||
if ( ! ds_use_integer_for_time )
|
||||
s += " pack_scale=\"1e-6\" print_format=\"%.6f\" pack_scale_warn=\"no\"";
|
||||
else
|
||||
s += string(" units=\"") + TIME_UNIT() + "\" epoch=\"unix\"";
|
||||
|
||||
return s;
|
||||
}
|
||||
|
||||
case TYPE_SUBNET:
|
||||
case TYPE_ADDR:
|
||||
case TYPE_ENUM:
|
||||
case TYPE_STRING:
|
||||
case TYPE_FILE:
|
||||
case TYPE_TABLE:
|
||||
case TYPE_VECTOR:
|
||||
return "pack_unique=\"yes\"";
|
||||
|
||||
default:
|
||||
return "";
|
||||
}
|
||||
}
|
||||
|
||||
DataSeries::DataSeries(WriterFrontend* frontend) : WriterBackend(frontend)
|
||||
{
|
||||
ds_compression = string((const char *)BifConst::LogDataSeries::compression->Bytes(),
|
||||
BifConst::LogDataSeries::compression->Len());
|
||||
ds_dump_schema = BifConst::LogDataSeries::dump_schema;
|
||||
ds_extent_size = BifConst::LogDataSeries::extent_size;
|
||||
ds_num_threads = BifConst::LogDataSeries::num_threads;
|
||||
ds_use_integer_for_time = BifConst::LogDataSeries::use_integer_for_time;
|
||||
ds_set_separator = ",";
|
||||
}
|
||||
|
||||
DataSeries::~DataSeries()
|
||||
{
|
||||
}
|
||||
|
||||
bool DataSeries::OpenLog(string path)
|
||||
{
|
||||
log_file = new DataSeriesSink(path + ".ds", compress_type);
|
||||
log_file->writeExtentLibrary(log_types);
|
||||
|
||||
for( size_t i = 0; i < schema_list.size(); ++i )
|
||||
extents.insert(std::make_pair(schema_list[i].field_name,
|
||||
GeneralField::create(log_series, schema_list[i].field_name)));
|
||||
|
||||
if ( ds_extent_size < ROW_MIN )
|
||||
{
|
||||
Warning(Fmt("%d is not a valid value for 'rows'. Using min of %d instead", (int)ds_extent_size, (int)ROW_MIN));
|
||||
ds_extent_size = ROW_MIN;
|
||||
}
|
||||
|
||||
else if( ds_extent_size > ROW_MAX )
|
||||
{
|
||||
Warning(Fmt("%d is not a valid value for 'rows'. Using max of %d instead", (int)ds_extent_size, (int)ROW_MAX));
|
||||
ds_extent_size = ROW_MAX;
|
||||
}
|
||||
|
||||
log_output = new OutputModule(*log_file, log_series, log_type, ds_extent_size);
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
bool DataSeries::DoInit(string path, int num_fields, const threading::Field* const * fields)
|
||||
{
|
||||
// We first construct an XML schema thing (and, if ds_dump_schema is
|
||||
// set, dump it to path + ".ds.xml"). Assuming that goes well, we
|
||||
// use that schema to build our output logfile and prepare it to be
|
||||
// written to.
|
||||
|
||||
// Note: compressor count must be set *BEFORE* DataSeriesSink is
|
||||
// instantiated.
|
||||
if( ds_num_threads < THREAD_MIN && ds_num_threads != 0 )
|
||||
{
|
||||
Warning(Fmt("%d is too few threads! Using %d instead", (int)ds_num_threads, (int)THREAD_MIN));
|
||||
ds_num_threads = THREAD_MIN;
|
||||
}
|
||||
|
||||
if( ds_num_threads > THREAD_MAX )
|
||||
{
|
||||
Warning(Fmt("%d is too many threads! Dropping back to %d", (int)ds_num_threads, (int)THREAD_MAX));
|
||||
ds_num_threads = THREAD_MAX;
|
||||
}
|
||||
|
||||
if( ds_num_threads > 0 )
|
||||
DataSeriesSink::setCompressorCount(ds_num_threads);
|
||||
|
||||
for ( int i = 0; i < num_fields; i++ )
|
||||
{
|
||||
const threading::Field* field = fields[i];
|
||||
SchemaValue val;
|
||||
val.ds_type = GetDSFieldType(field);
|
||||
val.field_name = string(field->name);
|
||||
val.field_options = GetDSOptionsForType(field);
|
||||
val.bro_type = field->TypeName();
|
||||
schema_list.push_back(val);
|
||||
}
|
||||
|
||||
string schema = BuildDSSchemaFromFieldTypes(schema_list, path);
|
||||
|
||||
if( ds_dump_schema )
|
||||
{
|
||||
FILE* pFile = fopen ( string(path + ".ds.xml").c_str() , "wb" );
|
||||
|
||||
if( pFile )
|
||||
{
|
||||
fwrite(schema.c_str(), 1, schema.length(), pFile);
|
||||
fclose(pFile);
|
||||
}
|
||||
|
||||
else
|
||||
Error(Fmt("cannot dump schema: %s", strerror(errno)));
|
||||
}
|
||||
|
||||
compress_type = Extent::compress_all;
|
||||
|
||||
if( ds_compression == "lzf" )
|
||||
compress_type = Extent::compress_lzf;
|
||||
|
||||
else if( ds_compression == "lzo" )
|
||||
compress_type = Extent::compress_lzo;
|
||||
|
||||
else if( ds_compression == "gz" )
|
||||
compress_type = Extent::compress_gz;
|
||||
|
||||
else if( ds_compression == "bz2" )
|
||||
compress_type = Extent::compress_bz2;
|
||||
|
||||
else if( ds_compression == "none" )
|
||||
compress_type = Extent::compress_none;
|
||||
|
||||
else if( ds_compression == "any" )
|
||||
compress_type = Extent::compress_all;
|
||||
|
||||
else
|
||||
Warning(Fmt("%s is not a valid compression type. Valid types are: 'lzf', 'lzo', 'gz', 'bz2', 'none', 'any'. Defaulting to 'any'", ds_compression.c_str()));
|
||||
|
||||
log_type = log_types.registerTypePtr(schema);
|
||||
log_series.setType(log_type);
|
||||
|
||||
return OpenLog(path);
|
||||
}
|
||||
|
||||
bool DataSeries::DoFlush()
|
||||
{
|
||||
// Flushing is handled by DataSeries automatically, so this function
|
||||
// doesn't do anything.
|
||||
return true;
|
||||
}
|
||||
|
||||
void DataSeries::CloseLog()
|
||||
{
|
||||
for( ExtentIterator iter = extents.begin(); iter != extents.end(); ++iter )
|
||||
delete iter->second;
|
||||
|
||||
extents.clear();
|
||||
|
||||
// Don't delete the file before you delete the output, or bad things
|
||||
// will happen.
|
||||
delete log_output;
|
||||
delete log_file;
|
||||
|
||||
log_output = 0;
|
||||
log_file = 0;
|
||||
}
|
||||
|
||||
bool DataSeries::DoFinish()
|
||||
{
|
||||
CloseLog();
|
||||
|
||||
return WriterBackend::DoFinish();
|
||||
}
|
||||
|
||||
bool DataSeries::DoWrite(int num_fields, const threading::Field* const * fields,
|
||||
threading::Value** vals)
|
||||
{
|
||||
log_output->newRecord();
|
||||
|
||||
for( size_t i = 0; i < (size_t)num_fields; ++i )
|
||||
{
|
||||
ExtentIterator iter = extents.find(fields[i]->name);
|
||||
assert(iter != extents.end());
|
||||
|
||||
if( iter != extents.end() )
|
||||
{
|
||||
GeneralField *cField = iter->second;
|
||||
|
||||
if( vals[i]->present )
|
||||
cField->set(LogValueToString(vals[i]));
|
||||
}
|
||||
}
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
bool DataSeries::DoRotate(string rotated_path, double open, double close, bool terminating)
|
||||
{
|
||||
// Note that if DS files are rotated too often, the aggregate log
|
||||
// size will be (much) larger.
|
||||
CloseLog();
|
||||
|
||||
string dsname = Path() + ".ds";
|
||||
string nname = rotated_path + ".ds";
|
||||
rename(dsname.c_str(), nname.c_str());
|
||||
|
||||
if ( ! FinishedRotation(nname, dsname, open, close, terminating) )
|
||||
{
|
||||
Error(Fmt("error rotating %s to %s", dsname.c_str(), nname.c_str()));
|
||||
return false;
|
||||
}
|
||||
|
||||
return OpenLog(Path());
|
||||
}
|
||||
|
||||
bool DataSeries::DoSetBuf(bool enabled)
|
||||
{
|
||||
// DataSeries is *always* buffered to some degree. This option is ignored.
|
||||
return true;
|
||||
}
|
||||
|
||||
#endif /* USE_DATASERIES */
|
||||
124
src/logging/writers/DataSeries.h
Normal file
124
src/logging/writers/DataSeries.h
Normal file
|
|
@ -0,0 +1,124 @@
|
|||
// See the file "COPYING" in the main distribution directory for copyright.
|
||||
//
|
||||
// A binary log writer producing DataSeries output. See doc/data-series.rst
|
||||
// for more information.
|
||||
|
||||
#ifndef LOGGING_WRITER_DATA_SERIES_H
|
||||
#define LOGGING_WRITER_DATA_SERIES_H
|
||||
|
||||
#include <DataSeries/ExtentType.hpp>
|
||||
#include <DataSeries/DataSeriesFile.hpp>
|
||||
#include <DataSeries/DataSeriesModule.hpp>
|
||||
#include <DataSeries/GeneralField.hpp>
|
||||
|
||||
#include "../WriterBackend.h"
|
||||
|
||||
namespace logging { namespace writer {
|
||||
|
||||
class DataSeries : public WriterBackend {
|
||||
public:
|
||||
DataSeries(WriterFrontend* frontend);
|
||||
~DataSeries();
|
||||
|
||||
static WriterBackend* Instantiate(WriterFrontend* frontend)
|
||||
{ return new DataSeries(frontend); }
|
||||
|
||||
protected:
|
||||
// Overidden from WriterBackend.
|
||||
|
||||
virtual bool DoInit(string path, int num_fields,
|
||||
const threading::Field* const * fields);
|
||||
|
||||
virtual bool DoWrite(int num_fields, const threading::Field* const* fields,
|
||||
threading::Value** vals);
|
||||
virtual bool DoSetBuf(bool enabled);
|
||||
virtual bool DoRotate(string rotated_path, double open,
|
||||
double close, bool terminating);
|
||||
virtual bool DoFlush();
|
||||
virtual bool DoFinish();
|
||||
|
||||
private:
|
||||
static const size_t ROW_MIN = 2048; // Minimum extent size.
|
||||
static const size_t ROW_MAX = (1024 * 1024 * 100); // Maximum extent size.
|
||||
static const size_t THREAD_MIN = 1; // Minimum number of compression threads that DataSeries may spawn.
|
||||
static const size_t THREAD_MAX = 128; // Maximum number of compression threads that DataSeries may spawn.
|
||||
static const size_t TIME_SCALE = 1000000; // Fixed-point multiplier for time values when converted to integers.
|
||||
const char* TIME_UNIT() { return "microseconds"; } // DS name for time resolution when converted to integers. Must match TIME_SCALE.
|
||||
|
||||
struct SchemaValue
|
||||
{
|
||||
string ds_type;
|
||||
string bro_type;
|
||||
string field_name;
|
||||
string field_options;
|
||||
};
|
||||
|
||||
/**
|
||||
* Turns a log value into a std::string. Uses an ostringstream to do the
|
||||
* heavy lifting, but still need to switch on the type to know which value
|
||||
* in the union to give to the string string for processing.
|
||||
*
|
||||
* @param val The value we wish to convert to a string
|
||||
* @return the string value of val
|
||||
*/
|
||||
std::string LogValueToString(threading::Value *val);
|
||||
|
||||
/**
|
||||
* Takes a field type and converts it to a relevant DataSeries type.
|
||||
*
|
||||
* @param field We extract the type from this and convert it into a relevant DS type.
|
||||
* @return String representation of type that DataSeries can understand.
|
||||
*/
|
||||
string GetDSFieldType(const threading::Field *field);
|
||||
|
||||
/**
|
||||
* Are there any options we should put into the XML schema?
|
||||
*
|
||||
* @param field We extract the type from this and return any options that make sense for that type.
|
||||
* @return Options that can be added directly to the XML (e.g. "pack_relative=\"yes\"")
|
||||
*/
|
||||
std::string GetDSOptionsForType(const threading::Field *field);
|
||||
|
||||
/**
|
||||
* Takes a list of types, a list of names, and a title, and uses it to construct a valid DataSeries XML schema
|
||||
* thing, which is then returned as a std::string
|
||||
*
|
||||
* @param opts std::vector of strings containing a list of options to be appended to each field (e.g. "pack_relative=yes")
|
||||
* @param sTitle Name of this schema. Ideally, these schemas would be aggregated and re-used.
|
||||
*/
|
||||
string BuildDSSchemaFromFieldTypes(const vector<SchemaValue>& vals, string sTitle);
|
||||
|
||||
/** Closes the currently open file. */
|
||||
void CloseLog();
|
||||
|
||||
/** Opens a new file. */
|
||||
bool OpenLog(string path);
|
||||
|
||||
typedef std::map<string, GeneralField *> ExtentMap;
|
||||
typedef ExtentMap::iterator ExtentIterator;
|
||||
|
||||
// Internal DataSeries structures we need to keep track of.
|
||||
vector<SchemaValue> schema_list;
|
||||
ExtentTypeLibrary log_types;
|
||||
ExtentType::Ptr log_type;
|
||||
ExtentSeries log_series;
|
||||
ExtentMap extents;
|
||||
int compress_type;
|
||||
|
||||
DataSeriesSink* log_file;
|
||||
OutputModule* log_output;
|
||||
|
||||
// Options set from the script-level.
|
||||
uint64 ds_extent_size;
|
||||
uint64 ds_num_threads;
|
||||
string ds_compression;
|
||||
bool ds_dump_schema;
|
||||
bool ds_use_integer_for_time;
|
||||
string ds_set_separator;
|
||||
};
|
||||
|
||||
}
|
||||
}
|
||||
|
||||
#endif
|
||||
|
||||
26
src/main.cc
26
src/main.cc
|
|
@ -18,6 +18,8 @@ extern "C" {
|
|||
}
|
||||
#endif
|
||||
|
||||
#include <openssl/md5.h>
|
||||
|
||||
extern "C" void OPENSSL_add_all_algorithms_conf(void);
|
||||
|
||||
#include "bsd-getopt-long.h"
|
||||
|
|
@ -204,6 +206,27 @@ void usage()
|
|||
fprintf(stderr, " $BRO_LOG_SUFFIX | ASCII log file extension (.%s)\n", logging::writer::Ascii::LogExt().c_str());
|
||||
fprintf(stderr, " $BRO_PROFILER_FILE | Output file for script execution statistics (not set)\n");
|
||||
|
||||
fprintf(stderr, "\n");
|
||||
fprintf(stderr, " Supported log formats: ");
|
||||
|
||||
bool first = true;
|
||||
list<string> fmts = logging::Manager::SupportedFormats();
|
||||
|
||||
for ( list<string>::const_iterator i = fmts.begin(); i != fmts.end(); ++i )
|
||||
{
|
||||
if ( *i == "None" )
|
||||
// Skip, it's uninteresting.
|
||||
continue;
|
||||
|
||||
if ( ! first )
|
||||
fprintf(stderr, ",");
|
||||
|
||||
fprintf(stderr, "%s", (*i).c_str());
|
||||
first = false;
|
||||
}
|
||||
|
||||
fprintf(stderr, "\n");
|
||||
|
||||
exit(1);
|
||||
}
|
||||
|
||||
|
|
@ -574,8 +597,7 @@ int main(int argc, char** argv)
|
|||
break;
|
||||
|
||||
case 'K':
|
||||
hash_md5(strlen(optarg), (const u_char*) optarg,
|
||||
shared_hmac_md5_key);
|
||||
MD5((const u_char*) optarg, strlen(optarg), shared_hmac_md5_key);
|
||||
hmac_key_set = 1;
|
||||
break;
|
||||
|
||||
|
|
|
|||
380
src/md5.c
380
src/md5.c
|
|
@ -1,380 +0,0 @@
|
|||
/*
|
||||
Copyright (C) 1999, 2000, 2002 Aladdin Enterprises. All rights reserved.
|
||||
|
||||
This software is provided 'as-is', without any express or implied
|
||||
warranty. In no event will the authors be held liable for any damages
|
||||
arising from the use of this software.
|
||||
|
||||
Permission is granted to anyone to use this software for any purpose,
|
||||
including commercial applications, and to alter it and redistribute it
|
||||
freely, subject to the following restrictions:
|
||||
|
||||
1. The origin of this software must not be misrepresented; you must not
|
||||
claim that you wrote the original software. If you use this software
|
||||
in a product, an acknowledgment in the product documentation would be
|
||||
appreciated but is not required.
|
||||
2. Altered source versions must be plainly marked as such, and must not be
|
||||
misrepresented as being the original software.
|
||||
3. This notice may not be removed or altered from any source distribution.
|
||||
|
||||
L. Peter Deutsch
|
||||
ghost@aladdin.com
|
||||
|
||||
*/
|
||||
/*
|
||||
Independent implementation of MD5 (RFC 1321).
|
||||
|
||||
This code implements the MD5 Algorithm defined in RFC 1321, whose
|
||||
text is available at
|
||||
http://www.ietf.org/rfc/rfc1321.txt
|
||||
The code is derived from the text of the RFC, including the test suite
|
||||
(section A.5) but excluding the rest of Appendix A. It does not include
|
||||
any code or documentation that is identified in the RFC as being
|
||||
copyrighted.
|
||||
|
||||
The original and principal author of md5.c is L. Peter Deutsch
|
||||
<ghost@aladdin.com>. Other authors are noted in the change history
|
||||
that follows (in reverse chronological order):
|
||||
|
||||
2002-04-13 lpd Clarified derivation from RFC 1321; now handles byte order
|
||||
either statically or dynamically; added missing #include <string.h>
|
||||
in library.
|
||||
2002-03-11 lpd Corrected argument list for main(), and added int return
|
||||
type, in test program and T value program.
|
||||
2002-02-21 lpd Added missing #include <stdio.h> in test program.
|
||||
2000-07-03 lpd Patched to eliminate warnings about "constant is
|
||||
unsigned in ANSI C, signed in traditional"; made test program
|
||||
self-checking.
|
||||
1999-11-04 lpd Edited comments slightly for automatic TOC extraction.
|
||||
1999-10-18 lpd Fixed typo in header comment (ansi2knr rather than md5).
|
||||
1999-05-03 lpd Original version.
|
||||
*/
|
||||
|
||||
#include "md5.h"
|
||||
#include <string.h>
|
||||
|
||||
#undef BYTE_ORDER /* 1 = big-endian, -1 = little-endian, 0 = unknown */
|
||||
#ifdef ARCH_IS_BIG_ENDIAN
|
||||
# define BYTE_ORDER (ARCH_IS_BIG_ENDIAN ? 1 : -1)
|
||||
#else
|
||||
# define BYTE_ORDER 0
|
||||
#endif
|
||||
|
||||
#define T_MASK ((md5_word_t)~0)
|
||||
#define T1 /* 0xd76aa478 */ (T_MASK ^ 0x28955b87)
|
||||
#define T2 /* 0xe8c7b756 */ (T_MASK ^ 0x173848a9)
|
||||
#define T3 0x242070db
|
||||
#define T4 /* 0xc1bdceee */ (T_MASK ^ 0x3e423111)
|
||||
#define T5 /* 0xf57c0faf */ (T_MASK ^ 0x0a83f050)
|
||||
#define T6 0x4787c62a
|
||||
#define T7 /* 0xa8304613 */ (T_MASK ^ 0x57cfb9ec)
|
||||
#define T8 /* 0xfd469501 */ (T_MASK ^ 0x02b96afe)
|
||||
#define T9 0x698098d8
|
||||
#define T10 /* 0x8b44f7af */ (T_MASK ^ 0x74bb0850)
|
||||
#define T11 /* 0xffff5bb1 */ (T_MASK ^ 0x0000a44e)
|
||||
#define T12 /* 0x895cd7be */ (T_MASK ^ 0x76a32841)
|
||||
#define T13 0x6b901122
|
||||
#define T14 /* 0xfd987193 */ (T_MASK ^ 0x02678e6c)
|
||||
#define T15 /* 0xa679438e */ (T_MASK ^ 0x5986bc71)
|
||||
#define T16 0x49b40821
|
||||
#define T17 /* 0xf61e2562 */ (T_MASK ^ 0x09e1da9d)
|
||||
#define T18 /* 0xc040b340 */ (T_MASK ^ 0x3fbf4cbf)
|
||||
#define T19 0x265e5a51
|
||||
#define T20 /* 0xe9b6c7aa */ (T_MASK ^ 0x16493855)
|
||||
#define T21 /* 0xd62f105d */ (T_MASK ^ 0x29d0efa2)
|
||||
#define T22 0x02441453
|
||||
#define T23 /* 0xd8a1e681 */ (T_MASK ^ 0x275e197e)
|
||||
#define T24 /* 0xe7d3fbc8 */ (T_MASK ^ 0x182c0437)
|
||||
#define T25 0x21e1cde6
|
||||
#define T26 /* 0xc33707d6 */ (T_MASK ^ 0x3cc8f829)
|
||||
#define T27 /* 0xf4d50d87 */ (T_MASK ^ 0x0b2af278)
|
||||
#define T28 0x455a14ed
|
||||
#define T29 /* 0xa9e3e905 */ (T_MASK ^ 0x561c16fa)
|
||||
#define T30 /* 0xfcefa3f8 */ (T_MASK ^ 0x03105c07)
|
||||
#define T31 0x676f02d9
|
||||
#define T32 /* 0x8d2a4c8a */ (T_MASK ^ 0x72d5b375)
|
||||
#define T33 /* 0xfffa3942 */ (T_MASK ^ 0x0005c6bd)
|
||||
#define T34 /* 0x8771f681 */ (T_MASK ^ 0x788e097e)
|
||||
#define T35 0x6d9d6122
|
||||
#define T36 /* 0xfde5380c */ (T_MASK ^ 0x021ac7f3)
|
||||
#define T37 /* 0xa4beea44 */ (T_MASK ^ 0x5b4115bb)
|
||||
#define T38 0x4bdecfa9
|
||||
#define T39 /* 0xf6bb4b60 */ (T_MASK ^ 0x0944b49f)
|
||||
#define T40 /* 0xbebfbc70 */ (T_MASK ^ 0x4140438f)
|
||||
#define T41 0x289b7ec6
|
||||
#define T42 /* 0xeaa127fa */ (T_MASK ^ 0x155ed805)
|
||||
#define T43 /* 0xd4ef3085 */ (T_MASK ^ 0x2b10cf7a)
|
||||
#define T44 0x04881d05
|
||||
#define T45 /* 0xd9d4d039 */ (T_MASK ^ 0x262b2fc6)
|
||||
#define T46 /* 0xe6db99e5 */ (T_MASK ^ 0x1924661a)
|
||||
#define T47 0x1fa27cf8
|
||||
#define T48 /* 0xc4ac5665 */ (T_MASK ^ 0x3b53a99a)
|
||||
#define T49 /* 0xf4292244 */ (T_MASK ^ 0x0bd6ddbb)
|
||||
#define T50 0x432aff97
|
||||
#define T51 /* 0xab9423a7 */ (T_MASK ^ 0x546bdc58)
|
||||
#define T52 /* 0xfc93a039 */ (T_MASK ^ 0x036c5fc6)
|
||||
#define T53 0x655b59c3
|
||||
#define T54 /* 0x8f0ccc92 */ (T_MASK ^ 0x70f3336d)
|
||||
#define T55 /* 0xffeff47d */ (T_MASK ^ 0x00100b82)
|
||||
#define T56 /* 0x85845dd1 */ (T_MASK ^ 0x7a7ba22e)
|
||||
#define T57 0x6fa87e4f
|
||||
#define T58 /* 0xfe2ce6e0 */ (T_MASK ^ 0x01d3191f)
|
||||
#define T59 /* 0xa3014314 */ (T_MASK ^ 0x5cfebceb)
|
||||
#define T60 0x4e0811a1
|
||||
#define T61 /* 0xf7537e82 */ (T_MASK ^ 0x08ac817d)
|
||||
#define T62 /* 0xbd3af235 */ (T_MASK ^ 0x42c50dca)
|
||||
#define T63 0x2ad7d2bb
|
||||
#define T64 /* 0xeb86d391 */ (T_MASK ^ 0x14792c6e)
|
||||
|
||||
|
||||
static void
|
||||
md5_process(md5_state_t *pms, const md5_byte_t *data /*[64]*/)
|
||||
{
|
||||
md5_word_t
|
||||
a = pms->abcd[0], b = pms->abcd[1],
|
||||
c = pms->abcd[2], d = pms->abcd[3];
|
||||
md5_word_t t;
|
||||
#if BYTE_ORDER > 0
|
||||
/* Define storage only for big-endian CPUs. */
|
||||
md5_word_t X[16];
|
||||
#else
|
||||
/* Define storage for little-endian or both types of CPUs. */
|
||||
md5_word_t xbuf[16];
|
||||
const md5_word_t *X;
|
||||
#endif
|
||||
|
||||
{
|
||||
#if BYTE_ORDER == 0
|
||||
/*
|
||||
* Determine dynamically whether this is a big-endian or
|
||||
* little-endian machine, since we can use a more efficient
|
||||
* algorithm on the latter.
|
||||
*/
|
||||
static const int w = 1;
|
||||
|
||||
if (*((const md5_byte_t *)&w)) /* dynamic little-endian */
|
||||
#endif
|
||||
#if BYTE_ORDER <= 0 /* little-endian */
|
||||
{
|
||||
/*
|
||||
* On little-endian machines, we can process properly aligned
|
||||
* data without copying it.
|
||||
*/
|
||||
if (!((data - (const md5_byte_t *)0) & 3)) {
|
||||
/* data are properly aligned */
|
||||
X = (const md5_word_t *)data;
|
||||
} else {
|
||||
/* not aligned */
|
||||
memcpy(xbuf, data, 64);
|
||||
X = xbuf;
|
||||
}
|
||||
}
|
||||
#endif
|
||||
#if BYTE_ORDER == 0
|
||||
else /* dynamic big-endian */
|
||||
#endif
|
||||
#if BYTE_ORDER >= 0 /* big-endian */
|
||||
{
|
||||
/*
|
||||
* On big-endian machines, we must arrange the bytes in the
|
||||
* right order.
|
||||
*/
|
||||
const md5_byte_t *xp = data;
|
||||
int i;
|
||||
|
||||
# if BYTE_ORDER == 0
|
||||
X = xbuf; /* (dynamic only) */
|
||||
# else
|
||||
# define xbuf X /* (static only) */
|
||||
# endif
|
||||
for (i = 0; i < 16; ++i, xp += 4)
|
||||
xbuf[i] = xp[0] + (xp[1] << 8) + (xp[2] << 16) + (xp[3] << 24);
|
||||
}
|
||||
#endif
|
||||
}
|
||||
|
||||
#define ROTATE_LEFT(x, n) (((x) << (n)) | ((x) >> (32 - (n))))
|
||||
|
||||
/* Round 1. */
|
||||
/* Let [abcd k s i] denote the operation
|
||||
a = b + ((a + F(b,c,d) + X[k] + T[i]) <<< s). */
|
||||
#define F(x, y, z) (((x) & (y)) | (~(x) & (z)))
|
||||
#define SET(a, b, c, d, k, s, Ti)\
|
||||
t = a + F(b,c,d) + X[k] + Ti;\
|
||||
a = ROTATE_LEFT(t, s) + b
|
||||
/* Do the following 16 operations. */
|
||||
SET(a, b, c, d, 0, 7, T1);
|
||||
SET(d, a, b, c, 1, 12, T2);
|
||||
SET(c, d, a, b, 2, 17, T3);
|
||||
SET(b, c, d, a, 3, 22, T4);
|
||||
SET(a, b, c, d, 4, 7, T5);
|
||||
SET(d, a, b, c, 5, 12, T6);
|
||||
SET(c, d, a, b, 6, 17, T7);
|
||||
SET(b, c, d, a, 7, 22, T8);
|
||||
SET(a, b, c, d, 8, 7, T9);
|
||||
SET(d, a, b, c, 9, 12, T10);
|
||||
SET(c, d, a, b, 10, 17, T11);
|
||||
SET(b, c, d, a, 11, 22, T12);
|
||||
SET(a, b, c, d, 12, 7, T13);
|
||||
SET(d, a, b, c, 13, 12, T14);
|
||||
SET(c, d, a, b, 14, 17, T15);
|
||||
SET(b, c, d, a, 15, 22, T16);
|
||||
#undef SET
|
||||
|
||||
/* Round 2. */
|
||||
/* Let [abcd k s i] denote the operation
|
||||
a = b + ((a + G(b,c,d) + X[k] + T[i]) <<< s). */
|
||||
#define G(x, y, z) (((x) & (z)) | ((y) & ~(z)))
|
||||
#define SET(a, b, c, d, k, s, Ti)\
|
||||
t = a + G(b,c,d) + X[k] + Ti;\
|
||||
a = ROTATE_LEFT(t, s) + b
|
||||
/* Do the following 16 operations. */
|
||||
SET(a, b, c, d, 1, 5, T17);
|
||||
SET(d, a, b, c, 6, 9, T18);
|
||||
SET(c, d, a, b, 11, 14, T19);
|
||||
SET(b, c, d, a, 0, 20, T20);
|
||||
SET(a, b, c, d, 5, 5, T21);
|
||||
SET(d, a, b, c, 10, 9, T22);
|
||||
SET(c, d, a, b, 15, 14, T23);
|
||||
SET(b, c, d, a, 4, 20, T24);
|
||||
SET(a, b, c, d, 9, 5, T25);
|
||||
SET(d, a, b, c, 14, 9, T26);
|
||||
SET(c, d, a, b, 3, 14, T27);
|
||||
SET(b, c, d, a, 8, 20, T28);
|
||||
SET(a, b, c, d, 13, 5, T29);
|
||||
SET(d, a, b, c, 2, 9, T30);
|
||||
SET(c, d, a, b, 7, 14, T31);
|
||||
SET(b, c, d, a, 12, 20, T32);
|
||||
#undef SET
|
||||
|
||||
/* Round 3. */
|
||||
/* Let [abcd k s t] denote the operation
|
||||
a = b + ((a + H(b,c,d) + X[k] + T[i]) <<< s). */
|
||||
#define H(x, y, z) ((x) ^ (y) ^ (z))
|
||||
#define SET(a, b, c, d, k, s, Ti)\
|
||||
t = a + H(b,c,d) + X[k] + Ti;\
|
||||
a = ROTATE_LEFT(t, s) + b
|
||||
/* Do the following 16 operations. */
|
||||
SET(a, b, c, d, 5, 4, T33);
|
||||
SET(d, a, b, c, 8, 11, T34);
|
||||
SET(c, d, a, b, 11, 16, T35);
|
||||
SET(b, c, d, a, 14, 23, T36);
|
||||
SET(a, b, c, d, 1, 4, T37);
|
||||
SET(d, a, b, c, 4, 11, T38);
|
||||
SET(c, d, a, b, 7, 16, T39);
|
||||
SET(b, c, d, a, 10, 23, T40);
|
||||
SET(a, b, c, d, 13, 4, T41);
|
||||
SET(d, a, b, c, 0, 11, T42);
|
||||
SET(c, d, a, b, 3, 16, T43);
|
||||
SET(b, c, d, a, 6, 23, T44);
|
||||
SET(a, b, c, d, 9, 4, T45);
|
||||
SET(d, a, b, c, 12, 11, T46);
|
||||
SET(c, d, a, b, 15, 16, T47);
|
||||
SET(b, c, d, a, 2, 23, T48);
|
||||
#undef SET
|
||||
|
||||
/* Round 4. */
|
||||
/* Let [abcd k s t] denote the operation
|
||||
a = b + ((a + I(b,c,d) + X[k] + T[i]) <<< s). */
|
||||
#define I(x, y, z) ((y) ^ ((x) | ~(z)))
|
||||
#define SET(a, b, c, d, k, s, Ti)\
|
||||
t = a + I(b,c,d) + X[k] + Ti;\
|
||||
a = ROTATE_LEFT(t, s) + b
|
||||
/* Do the following 16 operations. */
|
||||
SET(a, b, c, d, 0, 6, T49);
|
||||
SET(d, a, b, c, 7, 10, T50);
|
||||
SET(c, d, a, b, 14, 15, T51);
|
||||
SET(b, c, d, a, 5, 21, T52);
|
||||
SET(a, b, c, d, 12, 6, T53);
|
||||
SET(d, a, b, c, 3, 10, T54);
|
||||
SET(c, d, a, b, 10, 15, T55);
|
||||
SET(b, c, d, a, 1, 21, T56);
|
||||
SET(a, b, c, d, 8, 6, T57);
|
||||
SET(d, a, b, c, 15, 10, T58);
|
||||
SET(c, d, a, b, 6, 15, T59);
|
||||
SET(b, c, d, a, 13, 21, T60);
|
||||
SET(a, b, c, d, 4, 6, T61);
|
||||
SET(d, a, b, c, 11, 10, T62);
|
||||
SET(c, d, a, b, 2, 15, T63);
|
||||
SET(b, c, d, a, 9, 21, T64);
|
||||
#undef SET
|
||||
|
||||
/* Then perform the following additions. (That is increment each
|
||||
of the four registers by the value it had before this block
|
||||
was started.) */
|
||||
pms->abcd[0] += a;
|
||||
pms->abcd[1] += b;
|
||||
pms->abcd[2] += c;
|
||||
pms->abcd[3] += d;
|
||||
}
|
||||
|
||||
void
|
||||
md5_init(md5_state_t *pms)
|
||||
{
|
||||
pms->count[0] = pms->count[1] = 0;
|
||||
pms->abcd[0] = 0x67452301;
|
||||
pms->abcd[1] = /*0xefcdab89*/ T_MASK ^ 0x10325476;
|
||||
pms->abcd[2] = /*0x98badcfe*/ T_MASK ^ 0x67452301;
|
||||
pms->abcd[3] = 0x10325476;
|
||||
}
|
||||
|
||||
void
|
||||
md5_append(md5_state_t *pms, const md5_byte_t *data, int nbytes)
|
||||
{
|
||||
const md5_byte_t *p = data;
|
||||
int left = nbytes;
|
||||
int offset = (pms->count[0] >> 3) & 63;
|
||||
md5_word_t nbits = (md5_word_t)(nbytes << 3);
|
||||
|
||||
if (nbytes <= 0)
|
||||
return;
|
||||
|
||||
/* Update the message length. */
|
||||
pms->count[1] += nbytes >> 29;
|
||||
pms->count[0] += nbits;
|
||||
if (pms->count[0] < nbits)
|
||||
pms->count[1]++;
|
||||
|
||||
/* Process an initial partial block. */
|
||||
if (offset) {
|
||||
int copy = (offset + nbytes > 64 ? 64 - offset : nbytes);
|
||||
|
||||
memcpy(pms->buf + offset, p, copy);
|
||||
if (offset + copy < 64)
|
||||
return;
|
||||
p += copy;
|
||||
left -= copy;
|
||||
md5_process(pms, pms->buf);
|
||||
}
|
||||
|
||||
/* Process full blocks. */
|
||||
for (; left >= 64; p += 64, left -= 64)
|
||||
md5_process(pms, p);
|
||||
|
||||
/* Process a final partial block. */
|
||||
if (left)
|
||||
memcpy(pms->buf, p, left);
|
||||
}
|
||||
|
||||
void
|
||||
md5_finish(md5_state_t *pms, md5_byte_t digest[16])
|
||||
{
|
||||
static const md5_byte_t pad[64] = {
|
||||
0x80, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
|
||||
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
|
||||
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
|
||||
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
|
||||
};
|
||||
md5_byte_t data[8];
|
||||
int i;
|
||||
|
||||
/* Save the length before padding. */
|
||||
for (i = 0; i < 8; ++i)
|
||||
data[i] = (md5_byte_t)(pms->count[i >> 2] >> ((i & 3) << 3));
|
||||
/* Pad to 56 bytes mod 64. */
|
||||
md5_append(pms, pad, ((55 - (pms->count[0] >> 3)) & 63) + 1);
|
||||
/* Append the length. */
|
||||
md5_append(pms, data, 8);
|
||||
for (i = 0; i < 16; ++i)
|
||||
digest[i] = (md5_byte_t)(pms->abcd[i >> 2] >> ((i & 3) << 3));
|
||||
}
|
||||
90
src/md5.h
90
src/md5.h
|
|
@ -1,90 +0,0 @@
|
|||
/*
|
||||
Copyright (C) 1999, 2002 Aladdin Enterprises. All rights reserved.
|
||||
|
||||
This software is provided 'as-is', without any express or implied
|
||||
warranty. In no event will the authors be held liable for any damages
|
||||
arising from the use of this software.
|
||||
|
||||
Permission is granted to anyone to use this software for any purpose,
|
||||
including commercial applications, and to alter it and redistribute it
|
||||
freely, subject to the following restrictions:
|
||||
|
||||
1. The origin of this software must not be misrepresented; you must not
|
||||
claim that you wrote the original software. If you use this software
|
||||
in a product, an acknowledgment in the product documentation would be
|
||||
appreciated but is not required.
|
||||
2. Altered source versions must be plainly marked as such, and must not be
|
||||
misrepresented as being the original software.
|
||||
3. This notice may not be removed or altered from any source distribution.
|
||||
|
||||
L. Peter Deutsch
|
||||
ghost@aladdin.com
|
||||
|
||||
*/
|
||||
/*
|
||||
Independent implementation of MD5 (RFC 1321).
|
||||
|
||||
This code implements the MD5 Algorithm defined in RFC 1321, whose
|
||||
text is available at
|
||||
http://www.ietf.org/rfc/rfc1321.txt
|
||||
The code is derived from the text of the RFC, including the test suite
|
||||
(section A.5) but excluding the rest of Appendix A. It does not include
|
||||
any code or documentation that is identified in the RFC as being
|
||||
copyrighted.
|
||||
|
||||
The original and principal author of md5.h is L. Peter Deutsch
|
||||
<ghost@aladdin.com>. Other authors are noted in the change history
|
||||
that follows (in reverse chronological order):
|
||||
|
||||
2002-04-13 lpd Removed support for non-ANSI compilers; removed
|
||||
references to Ghostscript; clarified derivation from RFC 1321;
|
||||
now handles byte order either statically or dynamically.
|
||||
1999-11-04 lpd Edited comments slightly for automatic TOC extraction.
|
||||
1999-10-18 lpd Fixed typo in header comment (ansi2knr rather than md5);
|
||||
added conditionalization for C++ compilation from Martin
|
||||
Purschke <purschke@bnl.gov>.
|
||||
1999-05-03 lpd Original version.
|
||||
*/
|
||||
|
||||
#ifndef md5_INCLUDED
|
||||
# define md5_INCLUDED
|
||||
|
||||
/*
|
||||
* This package supports both compile-time and run-time determination of CPU
|
||||
* byte order. If ARCH_IS_BIG_ENDIAN is defined as 0, the code will be
|
||||
* compiled to run only on little-endian CPUs; if ARCH_IS_BIG_ENDIAN is
|
||||
* defined as non-zero, the code will be compiled to run only on big-endian
|
||||
* CPUs; if ARCH_IS_BIG_ENDIAN is not defined, the code will be compiled to
|
||||
* run on either big- or little-endian CPUs, but will run slightly less
|
||||
* efficiently on either one than if ARCH_IS_BIG_ENDIAN is defined.
|
||||
*/
|
||||
|
||||
typedef unsigned char md5_byte_t; /* 8-bit byte */
|
||||
typedef unsigned int md5_word_t; /* 32-bit word */
|
||||
|
||||
/* Define the state of the MD5 Algorithm. */
|
||||
typedef struct md5_state_s {
|
||||
md5_word_t count[2]; /* message length in bits, lsw first */
|
||||
md5_word_t abcd[4]; /* digest buffer */
|
||||
md5_byte_t buf[64]; /* accumulate block */
|
||||
} md5_state_t;
|
||||
|
||||
#ifdef __cplusplus
|
||||
extern "C"
|
||||
{
|
||||
#endif
|
||||
|
||||
/* Initialize the algorithm. */
|
||||
void md5_init(md5_state_t *pms);
|
||||
|
||||
/* Append a string to the message. */
|
||||
void md5_append(md5_state_t *pms, const md5_byte_t *data, int nbytes);
|
||||
|
||||
/* Finish the message and return the digest. */
|
||||
void md5_finish(md5_state_t *pms, md5_byte_t digest[16]);
|
||||
|
||||
#ifdef __cplusplus
|
||||
} /* end extern "C" */
|
||||
#endif
|
||||
|
||||
#endif /* md5_INCLUDED */
|
||||
|
|
@ -25,6 +25,7 @@
|
|||
string orig_label(bool is_orig);
|
||||
void free_X509(void *);
|
||||
X509* d2i_X509_binpac(X509** px, const uint8** in, int len);
|
||||
string handshake_type_label(int type);
|
||||
%}
|
||||
|
||||
%code{
|
||||
|
|
@ -46,6 +47,27 @@ string orig_label(bool is_orig)
|
|||
return d2i_X509(px, (u_char**) in, len);
|
||||
#endif
|
||||
}
|
||||
|
||||
string handshake_type_label(int type)
|
||||
{
|
||||
switch ( type ) {
|
||||
case HELLO_REQUEST: return string("HELLO_REQUEST");
|
||||
case CLIENT_HELLO: return string("CLIENT_HELLO");
|
||||
case SERVER_HELLO: return string("SERVER_HELLO");
|
||||
case SESSION_TICKET: return string("SESSION_TICKET");
|
||||
case CERTIFICATE: return string("CERTIFICATE");
|
||||
case SERVER_KEY_EXCHANGE: return string("SERVER_KEY_EXCHANGE");
|
||||
case CERTIFICATE_REQUEST: return string("CERTIFICATE_REQUEST");
|
||||
case SERVER_HELLO_DONE: return string("SERVER_HELLO_DONE");
|
||||
case CERTIFICATE_VERIFY: return string("CERTIFICATE_VERIFY");
|
||||
case CLIENT_KEY_EXCHANGE: return string("CLIENT_KEY_EXCHANGE");
|
||||
case FINISHED: return string("FINISHED");
|
||||
case CERTIFICATE_URL: return string("CERTIFICATE_URL");
|
||||
case CERTIFICATE_STATUS: return string("CERTIFICATE_STATUS");
|
||||
default: return string(fmt("UNKNOWN (%d)", type));
|
||||
}
|
||||
}
|
||||
|
||||
%}
|
||||
|
||||
|
||||
|
|
@ -88,15 +110,15 @@ refine connection SSL_Conn += {
|
|||
eof=0;
|
||||
%}
|
||||
|
||||
%eof{
|
||||
if ( ! eof &&
|
||||
state_ != STATE_CONN_ESTABLISHED &&
|
||||
state_ != STATE_TRACK_LOST &&
|
||||
state_ != STATE_INITIAL )
|
||||
bro_analyzer()->ProtocolViolation(fmt("unexpected end of connection in state %s",
|
||||
state_label(state_).c_str()));
|
||||
++eof;
|
||||
%}
|
||||
#%eof{
|
||||
# if ( ! eof &&
|
||||
# state_ != STATE_CONN_ESTABLISHED &&
|
||||
# state_ != STATE_TRACK_LOST &&
|
||||
# state_ != STATE_INITIAL )
|
||||
# bro_analyzer()->ProtocolViolation(fmt("unexpected end of connection in state %s",
|
||||
# state_label(state_).c_str()));
|
||||
# ++eof;
|
||||
#%}
|
||||
|
||||
%cleanup{
|
||||
%}
|
||||
|
|
@ -133,11 +155,6 @@ refine connection SSL_Conn += {
|
|||
cipher_suites16 : uint16[],
|
||||
cipher_suites24 : uint24[]) : bool
|
||||
%{
|
||||
if ( state_ == STATE_TRACK_LOST )
|
||||
bro_analyzer()->ProtocolViolation(fmt("unexpected client hello message from %s in state %s",
|
||||
orig_label(${rec.is_orig}).c_str(),
|
||||
state_label(old_state_).c_str()));
|
||||
|
||||
if ( ! version_ok(version) )
|
||||
bro_analyzer()->ProtocolViolation(fmt("unsupported client SSL version 0x%04x", version));
|
||||
|
||||
|
|
@ -175,11 +192,6 @@ refine connection SSL_Conn += {
|
|||
cipher_suites24 : uint24[],
|
||||
comp_method : uint8) : bool
|
||||
%{
|
||||
if ( state_ == STATE_TRACK_LOST )
|
||||
bro_analyzer()->ProtocolViolation(fmt("unexpected server hello message from %s in state %s",
|
||||
orig_label(${rec.is_orig}).c_str(),
|
||||
state_label(old_state_).c_str()));
|
||||
|
||||
if ( ! version_ok(version) )
|
||||
bro_analyzer()->ProtocolViolation(fmt("unsupported server SSL version 0x%04x", version));
|
||||
else
|
||||
|
|
@ -229,11 +241,6 @@ refine connection SSL_Conn += {
|
|||
|
||||
function proc_certificate(rec: SSLRecord, certificates : bytestring[]) : bool
|
||||
%{
|
||||
if ( state_ == STATE_TRACK_LOST )
|
||||
bro_analyzer()->ProtocolViolation(fmt("unexpected certificate message from %s in state %s",
|
||||
orig_label(${rec.is_orig}).c_str(),
|
||||
state_label(old_state_).c_str()));
|
||||
|
||||
if ( certificates->size() == 0 )
|
||||
return true;
|
||||
|
||||
|
|
@ -362,6 +369,7 @@ refine connection SSL_Conn += {
|
|||
handshake_type_label(${hs.msg_type}).c_str(),
|
||||
orig_label(is_orig).c_str(),
|
||||
state_label(old_state_).c_str()));
|
||||
|
||||
return true;
|
||||
%}
|
||||
|
||||
|
|
|
|||
|
|
@ -17,35 +17,6 @@ enum ContentType {
|
|||
UNKNOWN_OR_V2_ENCRYPTED = 400
|
||||
};
|
||||
|
||||
%code{
|
||||
string* record_type_label(int type)
|
||||
{
|
||||
switch ( type ) {
|
||||
case CHANGE_CIPHER_SPEC:
|
||||
return new string("CHANGE_CIPHER_SPEC");
|
||||
case ALERT:
|
||||
return new string("ALERT");
|
||||
case HANDSHAKE:
|
||||
return new string("HANDSHAKE");
|
||||
case APPLICATION_DATA:
|
||||
return new string("APPLICATION_DATA");
|
||||
case V2_ERROR:
|
||||
return new string("V2_ERROR");
|
||||
case V2_CLIENT_HELLO:
|
||||
return new string("V2_CLIENT_HELLO");
|
||||
case V2_CLIENT_MASTER_KEY:
|
||||
return new string("V2_CLIENT_MASTER_KEY");
|
||||
case V2_SERVER_HELLO:
|
||||
return new string("V2_SERVER_HELLO");
|
||||
case UNKNOWN_OR_V2_ENCRYPTED:
|
||||
return new string("UNKNOWN_OR_V2_ENCRYPTED");
|
||||
|
||||
default:
|
||||
return new string(fmt("UNEXPECTED (%d)", type));
|
||||
}
|
||||
}
|
||||
%}
|
||||
|
||||
enum SSLVersions {
|
||||
UNKNOWN_VERSION = 0x0000,
|
||||
SSLv20 = 0x0002,
|
||||
|
|
|
|||
|
|
@ -23,7 +23,6 @@ type uint24 = record {
|
|||
|
||||
string state_label(int state_nr);
|
||||
double get_time_from_asn1(const ASN1_TIME * atime);
|
||||
string handshake_type_label(int type);
|
||||
%}
|
||||
|
||||
extern type to_int;
|
||||
|
|
@ -268,28 +267,6 @@ enum HandshakeType {
|
|||
CERTIFICATE_STATUS = 22, # RFC 3546
|
||||
};
|
||||
|
||||
%code{
|
||||
string handshake_type_label(int type)
|
||||
{
|
||||
switch ( type ) {
|
||||
case HELLO_REQUEST: return string("HELLO_REQUEST");
|
||||
case CLIENT_HELLO: return string("CLIENT_HELLO");
|
||||
case SERVER_HELLO: return string("SERVER_HELLO");
|
||||
case SESSION_TICKET: return string("SESSION_TICKET");
|
||||
case CERTIFICATE: return string("CERTIFICATE");
|
||||
case SERVER_KEY_EXCHANGE: return string("SERVER_KEY_EXCHANGE");
|
||||
case CERTIFICATE_REQUEST: return string("CERTIFICATE_REQUEST");
|
||||
case SERVER_HELLO_DONE: return string("SERVER_HELLO_DONE");
|
||||
case CERTIFICATE_VERIFY: return string("CERTIFICATE_VERIFY");
|
||||
case CLIENT_KEY_EXCHANGE: return string("CLIENT_KEY_EXCHANGE");
|
||||
case FINISHED: return string("FINISHED");
|
||||
case CERTIFICATE_URL: return string("CERTIFICATE_URL");
|
||||
case CERTIFICATE_STATUS: return string("CERTIFICATE_STATUS");
|
||||
default: return string(fmt("UNKNOWN (%d)", type));
|
||||
}
|
||||
}
|
||||
%}
|
||||
|
||||
|
||||
######################################################################
|
||||
# V3 Change Cipher Spec Protocol (7.1.)
|
||||
|
|
@ -425,6 +402,10 @@ type ServerHello(rec: SSLRecord) = record {
|
|||
session_id : uint8[session_len];
|
||||
cipher_suite : uint16[1];
|
||||
compression_method : uint8;
|
||||
# This weirdness is to deal with the possible existence or absence
|
||||
# of the following fields.
|
||||
ext_len: uint16[] &until($element == 0 || $element != 0);
|
||||
extensions : SSLExtension(rec)[] &until($input.length() == 0);
|
||||
} &let {
|
||||
state_changed : bool =
|
||||
$context.connection.transition(STATE_CLIENT_HELLO_RCVD,
|
||||
|
|
|
|||
|
|
@ -81,6 +81,12 @@ public:
|
|||
*/
|
||||
int NumThreads() const { return all_threads.size(); }
|
||||
|
||||
/** Manually triggers processing of any thread input. This can be useful
|
||||
* if the main thread is waiting for a specific message from a child.
|
||||
* Usually, though, one should avoid using it.
|
||||
*/
|
||||
void ForceProcessing() { Process(); }
|
||||
|
||||
protected:
|
||||
friend class BasicThread;
|
||||
friend class MsgThread;
|
||||
|
|
|
|||
|
|
@ -10,13 +10,21 @@ namespace threading {
|
|||
|
||||
////// Messages.
|
||||
|
||||
// Signals child thread to terminate. This is actually a no-op; its only
|
||||
// purpose is unblock the current read operation so that the child's Run()
|
||||
// methods can check the termination status.
|
||||
class TerminateMessage : public InputMessage<MsgThread>
|
||||
// Signals child thread to shutdown operation.
|
||||
class FinishMessage : public InputMessage<MsgThread>
|
||||
{
|
||||
public:
|
||||
TerminateMessage(MsgThread* thread) : InputMessage<MsgThread>("Terminate", thread) { }
|
||||
FinishMessage(MsgThread* thread) : InputMessage<MsgThread>("Finish", thread) { }
|
||||
|
||||
virtual bool Process() { return Object()->DoFinish(); }
|
||||
};
|
||||
|
||||
// A dummy message that's only purpose is unblock the current read operation
|
||||
// so that the child's Run() methods can check the termination status.
|
||||
class UnblockMessage : public InputMessage<MsgThread>
|
||||
{
|
||||
public:
|
||||
UnblockMessage(MsgThread* thread) : InputMessage<MsgThread>("Unblock", thread) { }
|
||||
|
||||
virtual bool Process() { return true; }
|
||||
};
|
||||
|
|
@ -130,13 +138,29 @@ bool ReporterMessage::Process()
|
|||
MsgThread::MsgThread() : BasicThread()
|
||||
{
|
||||
cnt_sent_in = cnt_sent_out = 0;
|
||||
finished = false;
|
||||
thread_mgr->AddMsgThread(this);
|
||||
}
|
||||
|
||||
void MsgThread::OnStop()
|
||||
{
|
||||
// This is to unblock the current queue read operation.
|
||||
SendIn(new TerminateMessage(this), true);
|
||||
// Signal thread to terminate and wait until it has acknowledged.
|
||||
SendIn(new FinishMessage(this), true);
|
||||
|
||||
int cnt = 0;
|
||||
while ( ! finished )
|
||||
{
|
||||
if ( ++cnt > 1000 ) // Insurance against broken threads ...
|
||||
{
|
||||
reporter->Warning("thread %s didn't finish in time", Name().c_str());
|
||||
break;
|
||||
}
|
||||
|
||||
usleep(1000);
|
||||
}
|
||||
|
||||
// One more message to make sure the current queue read operation unblocks.
|
||||
SendIn(new UnblockMessage(this), true);
|
||||
}
|
||||
|
||||
void MsgThread::Heartbeat()
|
||||
|
|
@ -157,6 +181,14 @@ bool MsgThread::DoHeartbeat(double network_time, double current_time)
|
|||
return true;
|
||||
}
|
||||
|
||||
bool MsgThread::DoFinish()
|
||||
{
|
||||
// This is thread-safe "enough", we're the only one ever writing
|
||||
// there.
|
||||
finished = true;
|
||||
return true;
|
||||
}
|
||||
|
||||
void MsgThread::Info(const char* msg)
|
||||
{
|
||||
SendOut(new ReporterMessage(ReporterMessage::INFO, this, msg));
|
||||
|
|
@ -189,7 +221,9 @@ void MsgThread::InternalWarning(const char* msg)
|
|||
|
||||
void MsgThread::InternalError(const char* msg)
|
||||
{
|
||||
SendOut(new ReporterMessage(ReporterMessage::INTERNAL_ERROR, this, msg));
|
||||
// This one aborts immediately.
|
||||
fprintf(stderr, "internal error in thread: %s\n", msg);
|
||||
abort();
|
||||
}
|
||||
|
||||
#ifdef DEBUG
|
||||
|
|
|
|||
|
|
@ -171,6 +171,8 @@ public:
|
|||
protected:
|
||||
friend class Manager;
|
||||
friend class HeartbeatMessage;
|
||||
friend class FinishMessage;
|
||||
friend class FinishedMessage;
|
||||
|
||||
/**
|
||||
* Pops a message sent by the child from the child-to-main queue.
|
||||
|
|
@ -215,6 +217,12 @@ protected:
|
|||
*/
|
||||
virtual bool DoHeartbeat(double network_time, double current_time);
|
||||
|
||||
/** Triggered for execution in the child thread just before shutting threads down.
|
||||
* The child thread should finish its operations and then *must*
|
||||
* call this class' implementation.
|
||||
*/
|
||||
virtual bool DoFinish();
|
||||
|
||||
private:
|
||||
/**
|
||||
* Pops a message sent by the main thread from the main-to-chold
|
||||
|
|
@ -276,6 +284,8 @@ private:
|
|||
|
||||
uint64_t cnt_sent_in; // Counts message sent to child.
|
||||
uint64_t cnt_sent_out; // Counts message sent by child.
|
||||
|
||||
bool finished; // Set to true by Finished message.
|
||||
};
|
||||
|
||||
/**
|
||||
|
|
|
|||
|
|
@ -26,6 +26,20 @@ bool Field::Write(SerializationFormat* fmt) const
|
|||
fmt->Write((int)subtype, "subtype"), fmt->Write(optional, "optional"));
|
||||
}
|
||||
|
||||
string Field::TypeName() const
|
||||
{
|
||||
string n = type_name(type);
|
||||
|
||||
if ( (type == TYPE_TABLE) || (type == TYPE_VECTOR) )
|
||||
{
|
||||
n += "[";
|
||||
n += type_name(subtype);
|
||||
n += "]";
|
||||
}
|
||||
|
||||
return n;
|
||||
}
|
||||
|
||||
Value::~Value()
|
||||
{
|
||||
if ( (type == TYPE_ENUM || type == TYPE_STRING || type == TYPE_FILE || type == TYPE_FUNC)
|
||||
|
|
|
|||
|
|
@ -56,6 +56,12 @@ struct Field {
|
|||
* @return False if an error occured.
|
||||
*/
|
||||
bool Write(SerializationFormat* fmt) const;
|
||||
|
||||
/**
|
||||
* Returns a textual description of the field's type. This method is
|
||||
* thread-safe.
|
||||
*/
|
||||
string TypeName() const;
|
||||
};
|
||||
|
||||
/**
|
||||
|
|
@ -135,8 +141,8 @@ struct Value {
|
|||
|
||||
/**
|
||||
* Returns true if the type can be represented by a Value. If
|
||||
* `atomic_only` is true, will not permit composite types.
|
||||
*/
|
||||
* `atomic_only` is true, will not permit composite types. This
|
||||
* method is thread-safe. */
|
||||
static bool IsCompatibleType(BroType* t, bool atomic_only=false);
|
||||
|
||||
private:
|
||||
|
|
|
|||
|
|
@ -162,6 +162,7 @@ enum Writer %{
|
|||
WRITER_DEFAULT,
|
||||
WRITER_NONE,
|
||||
WRITER_ASCII,
|
||||
WRITER_DATASERIES,
|
||||
%}
|
||||
|
||||
enum ID %{
|
||||
|
|
|
|||
27
src/util.cc
27
src/util.cc
|
|
@ -27,6 +27,8 @@
|
|||
#include <errno.h>
|
||||
#include <signal.h>
|
||||
#include <libgen.h>
|
||||
#include <openssl/md5.h>
|
||||
#include <openssl/sha.h>
|
||||
|
||||
#ifdef HAVE_MALLINFO
|
||||
# include <malloc.h>
|
||||
|
|
@ -35,7 +37,6 @@
|
|||
#include "input.h"
|
||||
#include "util.h"
|
||||
#include "Obj.h"
|
||||
#include "md5.h"
|
||||
#include "Val.h"
|
||||
#include "NetVar.h"
|
||||
#include "Net.h"
|
||||
|
|
@ -546,24 +547,6 @@ bool is_dir(const char* path)
|
|||
return S_ISDIR(st.st_mode);
|
||||
}
|
||||
|
||||
void hash_md5(size_t size, const unsigned char* bytes, unsigned char digest[16])
|
||||
{
|
||||
md5_state_s h;
|
||||
md5_init(&h);
|
||||
md5_append(&h, bytes, size);
|
||||
md5_finish(&h, digest);
|
||||
}
|
||||
|
||||
const char* md5_digest_print(const unsigned char digest[16])
|
||||
{
|
||||
static char digest_print[256];
|
||||
|
||||
for ( int i = 0; i < 16; ++i )
|
||||
snprintf(digest_print + i * 2, 3, "%02x", digest[i]);
|
||||
|
||||
return digest_print;
|
||||
}
|
||||
|
||||
int hmac_key_set = 0;
|
||||
uint8 shared_hmac_md5_key[16];
|
||||
|
||||
|
|
@ -572,12 +555,12 @@ void hmac_md5(size_t size, const unsigned char* bytes, unsigned char digest[16])
|
|||
if ( ! hmac_key_set )
|
||||
reporter->InternalError("HMAC-MD5 invoked before the HMAC key is set");
|
||||
|
||||
hash_md5(size, bytes, digest);
|
||||
MD5(bytes, size, digest);
|
||||
|
||||
for ( int i = 0; i < 16; ++i )
|
||||
digest[i] ^= shared_hmac_md5_key[i];
|
||||
|
||||
hash_md5(16, digest, digest);
|
||||
MD5(digest, 16, digest);
|
||||
}
|
||||
|
||||
static bool read_random_seeds(const char* read_file, uint32* seed,
|
||||
|
|
@ -724,7 +707,7 @@ void init_random_seed(uint32 seed, const char* read_file, const char* write_file
|
|||
|
||||
if ( ! hmac_key_set )
|
||||
{
|
||||
hash_md5(sizeof(buf), (u_char*) buf, shared_hmac_md5_key);
|
||||
MD5((const u_char*) buf, sizeof(buf), shared_hmac_md5_key);
|
||||
hmac_key_set = 1;
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -136,16 +136,12 @@ extern bool ensure_dir(const char *dirname);
|
|||
bool is_dir(const char* path);
|
||||
|
||||
extern uint8 shared_hmac_md5_key[16];
|
||||
extern void hash_md5(size_t size, const unsigned char* bytes,
|
||||
unsigned char digest[16]);
|
||||
|
||||
extern int hmac_key_set;
|
||||
extern unsigned char shared_hmac_md5_key[16];
|
||||
extern void hmac_md5(size_t size, const unsigned char* bytes,
|
||||
unsigned char digest[16]);
|
||||
|
||||
extern const char* md5_digest_print(const unsigned char digest[16]);
|
||||
|
||||
// Initializes RNGs for bro_random() and MD5 usage. If seed is given, then
|
||||
// it is used (to provide determinism). If load_file is given, the seeds
|
||||
// (both random & MD5) are loaded from that file. This takes precedence
|
||||
|
|
|
|||
4
testing/btest/Baseline/bifs.md5/output
Normal file
4
testing/btest/Baseline/bifs.md5/output
Normal file
|
|
@ -0,0 +1,4 @@
|
|||
f97c5d29941bfb1b2fdab0874906ab82
|
||||
7b0391feb2e0cd271f1cf39aafb4376f
|
||||
f97c5d29941bfb1b2fdab0874906ab82
|
||||
7b0391feb2e0cd271f1cf39aafb4376f
|
||||
4
testing/btest/Baseline/bifs.sha1/output
Normal file
4
testing/btest/Baseline/bifs.sha1/output
Normal file
|
|
@ -0,0 +1,4 @@
|
|||
fe05bcdcdc4928012781a5f1a2a77cbb5398e106
|
||||
3e949019500deb1369f13d9644d420d3a920aa5e
|
||||
fe05bcdcdc4928012781a5f1a2a77cbb5398e106
|
||||
3e949019500deb1369f13d9644d420d3a920aa5e
|
||||
4
testing/btest/Baseline/bifs.sha256/output
Normal file
4
testing/btest/Baseline/bifs.sha256/output
Normal file
|
|
@ -0,0 +1,4 @@
|
|||
7692c3ad3540bb803c020b3aee66cd8887123234ea0c6e7143c0add73ff431ed
|
||||
4592092e1061c7ea85af2aed194621cc17a2762bae33a79bf8ce33fd0168b801
|
||||
7692c3ad3540bb803c020b3aee66cd8887123234ea0c6e7143c0add73ff431ed
|
||||
4592092e1061c7ea85af2aed194621cc17a2762bae33a79bf8ce33fd0168b801
|
||||
|
|
@ -1,13 +1,83 @@
|
|||
1332784981.078396 weird: bad_IP_checksum
|
||||
1332784885.686428 weird: bad_TCP_checksum
|
||||
1332784933.501023 weird: bad_UDP_checksum
|
||||
1334075363.536871 weird: bad_ICMP_checksum
|
||||
1332785210.013051 weird: routing0_hdr
|
||||
1332785210.013051 weird: bad_TCP_checksum
|
||||
1332782580.798420 weird: routing0_hdr
|
||||
1332782580.798420 weird: bad_UDP_checksum
|
||||
1334075111.800086 weird: routing0_hdr
|
||||
1334075111.800086 weird: bad_ICMP_checksum
|
||||
1332785250.469132 weird: bad_TCP_checksum
|
||||
1332781342.923813 weird: bad_UDP_checksum
|
||||
1334074939.467194 weird: bad_ICMP_checksum
|
||||
#separator \x09
|
||||
#set_separator ,
|
||||
#empty_field (empty)
|
||||
#unset_field -
|
||||
#path weird
|
||||
#fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p name addl notice peer
|
||||
#types time string addr port addr port string string bool string
|
||||
1332784981.078396 - - - - - bad_IP_checksum - F bro
|
||||
#separator \x09
|
||||
#set_separator ,
|
||||
#empty_field (empty)
|
||||
#unset_field -
|
||||
#path weird
|
||||
#fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p name addl notice peer
|
||||
#types time string addr port addr port string string bool string
|
||||
1332784885.686428 UWkUyAuUGXf 127.0.0.1 30000 127.0.0.1 80 bad_TCP_checksum - F bro
|
||||
#separator \x09
|
||||
#set_separator ,
|
||||
#empty_field (empty)
|
||||
#unset_field -
|
||||
#path weird
|
||||
#fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p name addl notice peer
|
||||
#types time string addr port addr port string string bool string
|
||||
1332784933.501023 UWkUyAuUGXf 127.0.0.1 30000 127.0.0.1 13000 bad_UDP_checksum - F bro
|
||||
#separator \x09
|
||||
#set_separator ,
|
||||
#empty_field (empty)
|
||||
#unset_field -
|
||||
#path weird
|
||||
#fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p name addl notice peer
|
||||
#types time string addr port addr port string string bool string
|
||||
1334075363.536871 UWkUyAuUGXf 192.168.1.100 8 192.168.1.101 0 bad_ICMP_checksum - F bro
|
||||
#separator \x09
|
||||
#set_separator ,
|
||||
#empty_field (empty)
|
||||
#unset_field -
|
||||
#path weird
|
||||
#fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p name addl notice peer
|
||||
#types time string addr port addr port string string bool string
|
||||
1332785210.013051 - - - - - routing0_hdr - F bro
|
||||
1332785210.013051 UWkUyAuUGXf 2001:4f8:4:7:2e0:81ff:fe52:ffff 30000 2001:78:1:32::2 80 bad_TCP_checksum - F bro
|
||||
#separator \x09
|
||||
#set_separator ,
|
||||
#empty_field (empty)
|
||||
#unset_field -
|
||||
#path weird
|
||||
#fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p name addl notice peer
|
||||
#types time string addr port addr port string string bool string
|
||||
1332782580.798420 - - - - - routing0_hdr - F bro
|
||||
1332782580.798420 UWkUyAuUGXf 2001:4f8:4:7:2e0:81ff:fe52:ffff 30000 2001:78:1:32::2 13000 bad_UDP_checksum - F bro
|
||||
#separator \x09
|
||||
#set_separator ,
|
||||
#empty_field (empty)
|
||||
#unset_field -
|
||||
#path weird
|
||||
#fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p name addl notice peer
|
||||
#types time string addr port addr port string string bool string
|
||||
1334075111.800086 - - - - - routing0_hdr - F bro
|
||||
1334075111.800086 UWkUyAuUGXf 2001:4f8:4:7:2e0:81ff:fe52:ffff 128 2001:78:1:32::1 129 bad_ICMP_checksum - F bro
|
||||
#separator \x09
|
||||
#set_separator ,
|
||||
#empty_field (empty)
|
||||
#unset_field -
|
||||
#path weird
|
||||
#fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p name addl notice peer
|
||||
#types time string addr port addr port string string bool string
|
||||
1332785250.469132 UWkUyAuUGXf 2001:4f8:4:7:2e0:81ff:fe52:ffff 30000 2001:4f8:4:7:2e0:81ff:fe52:9a6b 80 bad_TCP_checksum - F bro
|
||||
#separator \x09
|
||||
#set_separator ,
|
||||
#empty_field (empty)
|
||||
#unset_field -
|
||||
#path weird
|
||||
#fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p name addl notice peer
|
||||
#types time string addr port addr port string string bool string
|
||||
1332781342.923813 UWkUyAuUGXf 2001:4f8:4:7:2e0:81ff:fe52:ffff 30000 2001:4f8:4:7:2e0:81ff:fe52:9a6b 13000 bad_UDP_checksum - F bro
|
||||
#separator \x09
|
||||
#set_separator ,
|
||||
#empty_field (empty)
|
||||
#unset_field -
|
||||
#path weird
|
||||
#fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p name addl notice peer
|
||||
#types time string addr port addr port string string bool string
|
||||
1334074939.467194 UWkUyAuUGXf 2001:4f8:4:7:2e0:81ff:fe52:ffff 128 2001:4f8:4:7:2e0:81ff:fe52:9a6b 129 bad_ICMP_checksum - F bro
|
||||
|
|
|
|||
|
|
@ -1,3 +1,56 @@
|
|||
1332785125.596793 weird: routing0_hdr
|
||||
1332782508.592037 weird: routing0_hdr
|
||||
1334075027.053380 weird: routing0_hdr
|
||||
#separator \x09
|
||||
#set_separator ,
|
||||
#empty_field (empty)
|
||||
#unset_field -
|
||||
#path weird
|
||||
#fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p name addl notice peer
|
||||
#types time string addr port addr port string string bool string
|
||||
1334074939.467194 UWkUyAuUGXf 2001:4f8:4:7:2e0:81ff:fe52:ffff 128 2001:4f8:4:7:2e0:81ff:fe52:9a6b 129 bad_ICMP_checksum - F bro
|
||||
#separator \x09
|
||||
#set_separator ,
|
||||
#empty_field (empty)
|
||||
#unset_field -
|
||||
#path weird
|
||||
#fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p name addl notice peer
|
||||
#types time string addr port addr port string string bool string
|
||||
1332785125.596793 - - - - - routing0_hdr - F bro
|
||||
#separator \x09
|
||||
#set_separator ,
|
||||
#empty_field (empty)
|
||||
#unset_field -
|
||||
#path weird
|
||||
#fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p name addl notice peer
|
||||
#types time string addr port addr port string string bool string
|
||||
1332782508.592037 - - - - - routing0_hdr - F bro
|
||||
#separator \x09
|
||||
#set_separator ,
|
||||
#empty_field (empty)
|
||||
#unset_field -
|
||||
#path weird
|
||||
#fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p name addl notice peer
|
||||
#types time string addr port addr port string string bool string
|
||||
1334075027.053380 - - - - - routing0_hdr - F bro
|
||||
#separator \x09
|
||||
#set_separator ,
|
||||
#empty_field (empty)
|
||||
#unset_field -
|
||||
#path weird
|
||||
#fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p name addl notice peer
|
||||
#types time string addr port addr port string string bool string
|
||||
1334075027.053380 - - - - - routing0_hdr - F bro
|
||||
#separator \x09
|
||||
#set_separator ,
|
||||
#empty_field (empty)
|
||||
#unset_field -
|
||||
#path weird
|
||||
#fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p name addl notice peer
|
||||
#types time string addr port addr port string string bool string
|
||||
1334075027.053380 - - - - - routing0_hdr - F bro
|
||||
#separator \x09
|
||||
#set_separator ,
|
||||
#empty_field (empty)
|
||||
#unset_field -
|
||||
#path weird
|
||||
#fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p name addl notice peer
|
||||
#types time string addr port addr port string string bool string
|
||||
1334075027.053380 - - - - - routing0_hdr - F bro
|
||||
|
|
|
|||
|
|
@ -1 +0,0 @@
|
|||
1333663011.602839 weird: unknown_protocol_135
|
||||
|
|
@ -0,0 +1,8 @@
|
|||
#separator \x09
|
||||
#set_separator ,
|
||||
#empty_field (empty)
|
||||
#unset_field -
|
||||
#path weird
|
||||
#fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p name addl notice peer
|
||||
#types time string addr port addr port string string bool string
|
||||
1333663011.602839 - - - - - unknown_protocol_135 - F bro
|
||||
|
|
@ -0,0 +1,4 @@
|
|||
opened
|
||||
write 0
|
||||
write 3
|
||||
write 6
|
||||
|
|
@ -0,0 +1,4 @@
|
|||
opened
|
||||
write 1
|
||||
write 4
|
||||
write 7
|
||||
|
|
@ -0,0 +1,4 @@
|
|||
opened
|
||||
write 2
|
||||
write 5
|
||||
write 8
|
||||
|
|
@ -0,0 +1,6 @@
|
|||
opened
|
||||
write 0
|
||||
opened
|
||||
write 3
|
||||
opened
|
||||
write 6
|
||||
|
|
@ -0,0 +1,6 @@
|
|||
opened
|
||||
write 1
|
||||
opened
|
||||
write 4
|
||||
opened
|
||||
write 7
|
||||
|
|
@ -0,0 +1,6 @@
|
|||
opened
|
||||
write 2
|
||||
opened
|
||||
write 5
|
||||
opened
|
||||
write 8
|
||||
|
|
@ -1,12 +1,12 @@
|
|||
icmp_unreachable (code=0)
|
||||
conn_id: [orig_h=10.0.0.1, orig_p=3/icmp, resp_h=10.0.0.2, resp_p=0/icmp]
|
||||
icmp_conn: [orig_h=10.0.0.1, resp_h=10.0.0.2, itype=3, icode=0, len=0, v6=F]
|
||||
icmp_conn: [orig_h=10.0.0.1, resp_h=10.0.0.2, itype=3, icode=0, len=0, hlim=64, v6=F]
|
||||
icmp_context: [id=[orig_h=::, orig_p=0/unknown, resp_h=::, resp_p=0/unknown], len=0, proto=0, frag_offset=0, bad_hdr_len=T, bad_checksum=F, MF=F, DF=F]
|
||||
icmp_unreachable (code=0)
|
||||
conn_id: [orig_h=10.0.0.1, orig_p=3/icmp, resp_h=10.0.0.2, resp_p=0/icmp]
|
||||
icmp_conn: [orig_h=10.0.0.1, resp_h=10.0.0.2, itype=3, icode=0, len=20, v6=F]
|
||||
icmp_conn: [orig_h=10.0.0.1, resp_h=10.0.0.2, itype=3, icode=0, len=20, hlim=64, v6=F]
|
||||
icmp_context: [id=[orig_h=10.0.0.2, orig_p=0/unknown, resp_h=10.0.0.1, resp_p=0/unknown], len=20, proto=0, frag_offset=0, bad_hdr_len=T, bad_checksum=F, MF=F, DF=F]
|
||||
icmp_unreachable (code=3)
|
||||
conn_id: [orig_h=192.168.1.102, orig_p=3/icmp, resp_h=192.168.1.1, resp_p=3/icmp]
|
||||
icmp_conn: [orig_h=192.168.1.102, resp_h=192.168.1.1, itype=3, icode=3, len=148, v6=F]
|
||||
icmp_conn: [orig_h=192.168.1.102, resp_h=192.168.1.1, itype=3, icode=3, len=148, hlim=128, v6=F]
|
||||
icmp_context: [id=[orig_h=192.168.1.1, orig_p=53/udp, resp_h=192.168.1.102, resp_p=59207/udp], len=163, proto=2, frag_offset=0, bad_hdr_len=F, bad_checksum=F, MF=F, DF=F]
|
||||
|
|
|
|||
Some files were not shown because too many files have changed in this diff Show more
Loading…
Add table
Add a link
Reference in a new issue